Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                

Overloading all attribute lookups: __getattribute__ versus __getattr__ PREMIUM

Trey Hunner smiling in a t-shirt against a yellow wall
Trey Hunner
5 min. read • Python 3.9—3.13 •
Tags

Let's say you'd like to make a Python class that captures every attribute lookup. Is this possible?

Well, Python's unittest.mock.Mock object does this!

>>> from unittest.mock import Mock
>>> mock = Mock()
>>> mock.something
<Mock name='mock.something' id='140543294770640'>
>>> mock.something_else
<Mock name='mock.something_else' id='140543294770688'>
>>> mock.yet_another_attribute
<Mock name='mock.yet_another_attribute' id='140543294681264'>

So yes, it is possible... thanks dunder methods! In fact, there are two different dunder methods that both control attribute lookups.

Let's compare and contrast the two different ways to capture every attribute lookup on a Python object.

Note: if you just need to capture lookups on one specific attribute, use a property instead.

The __getattribute__ method

Python's __getattribute__ method is called for every attribute access.

Here's a class that prints "attribute accessed" and then delegates __getattribute__ in a super class (which will ultimately call object.__getattribute__ and do the usual attribute lookup):

class AttributeWatcher:

    def __init__(self, enabled=True):
        self.enabled = enabled

    def disable(self):
        self.enabled = False

    def enable(self):
        self.enabled = True

    def __getattribute__(self, name):
        """Called for every attribute access."""
        if self.enabled:
            print(f"__getattribute__ called with '{name}'")
        return super().__getattribute__(name)

This __getattribute__ method will be called for any attribute we access:

>>> obj = AttributeWatcher()
>>> obj.x
__getattribute__ called with 'x'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'AttributeWatcher' object has no attribute 'x'

Even method calls will go through __getattribute__:

>>> obj.disable()
__getattribute__ called with 'disable'

This is usually not the dunder method you should implement for capturing all attribute accesses. You'll usually want to implement the __getattr__ method instead.

The __getattr__ method

Python's __getattr__ is called after Python fails to find an attribute through the usual means.

class AttributeWatcher:

    def __init__(self, enabled=True):
        self.enabled = enabled

    def disable(self):
        self.enabled = False

    def enable(self):
        self.enabled = True

    def __getattribute__(self, name):
        """Called for every attribute access."""
        if self.enabled:
            print(f"__getattribute__ called with '{name}'")
        return super().__getattribute__(name)

    def __getattr__(self, name):
        """Called for failed attribute accesses."""
        if self.enabled:
            print(f"__getattr__ called with '{name}'")
        return super().__getattr__(name)

When an attribute exists, Python won't call __getattr__.

>>> obj = AttributeWatcher()
>>> obj.enabled
__getattribute__ called with 'enabled'
True

But when an attribute doesn't exist on an object, Python will call __getattr__:

>>> obj.missing
__getattribute__ called with 'missing'
__getattr__ called with 'missing'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 15, in __getattr__
AttributeError: 'AttributeWatcher' object has no attribute 'missing'

I sometimes wish the __getattr__ method was called something like __missing_attr__ instead.

Implementing __getattr__

Defining your own __getattr__ method may involve checking for specific name patterns before raising an exception:

import re

def to_under(match):
    return "_".join(match.groups()).lower()

class CamelCaseAlso:

    """Missing your camelCase names?"""

    def __getattr__(self, name):
        """Allows camelCase instead of underscore_case."""
        new_name = re.sub(r"([a-z])([A-Z])", to_under", name)
        return getattr(self, new_name)

You can see this class in action here.

When implementing __getattr__, if an attribute should still be considered erroneous after your custom logic, you can either raise an AttributeError or call super().__getattr__(name) to delegate to your parent object's __getattr__. All objects inherit from object and object.__getattr__ will simple raise an AttributeError for all attribute names.

We don't raise an AttributeError explicitly in the above example, but the getattr built-in function will raise an AttributeError if the attribute lookup after our camel-casing logic fails.

The __getattr__ can be fairly simple to implement: it either raises an exception (whether implicitly or explicitly) or it returns an attribute value. That's it!

The __getattribute__ method is a bit trickier to implement.

Implementing __getattribute__

Implementing __getattribute__ takes a bit more care than implementing __getattr__, since all method calls and attribute accesses result in a call to __getattribute__. It's easy to accidentally end up with infinite recursion when implementing __getattribute__.

For example, this implementation of __getattribute__ suffers from infinite recursion:

class AttributeLogger:

    def __init__(self):
        self._attribute_accesses = []

    def __getattribute__(self, name):
        self._attribute_accesses.append(name)
        return super().__getattribute__(name)

    @property
    def attribute_accesses(self):
        return self._attribute_accesses

When we try to access any attribute, the self._attribute_accesses list will be looked up to append to it. That triggers another attribute access though!

So we'll end up calling __getattribute__ again for that attribute access, but that triggers another call to __getattribute__ and so on:

>>> log = AttributeLogger()
>>> log.x
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 5, in __getattribute__
  File "<stdin>", line 5, in __getattribute__
  File "<stdin>", line 5, in __getattribute__
  [Previous line repeated 996 more times]
RecursionError: maximum recursion depth exceeded

To fix this, we need to carefully make exceptions for each attribute that we don't want to trigger an additional __getattribute__ call:

class AttributeLogger:

    def __init__(self):
        self._attribute_accesses = []

    def __getattribute__(self, name):
        if name != "_attribute_accesses":
            self._attribute_accesses.append(name)
        return super().__getattribute__(name)

    @property
    def attribute_accesses(self):
        return self._attribute_accesses

Prefer __getattr__ over __getattribute__

The __getattribute__ method catches all attribute lookups, while the __getattr__ method only catches lookups for non-existent attributes.

Usually implementing __getattr__ is preferable to implementing __getattribute__.

Python's fairly sophisticated unittest.mock.Mock class implements __getattr__ instead of __getattribute__:

class Mock:
    # ...
    def __getattr__(self, name):
        if name in {'_mock_methods', '_mock_unsafe'}:
            raise AttributeError(name)
        elif self._mock_methods is not None:
            if name not in self._mock_methods or name in _all_magics:
                raise AttributeError(name)
        return self._get_child_mock(parent=self, name=name, _new_name=name,
                                    _new_parent=self)

If __getattr__ is good enough for the Mock class, it's probably good enough for your use case too.

The next time you need to capture every attribute access, consider whether you actually only need to capture lookups for missing attributes. If so, use __getattr__ instead of __getattribute__. You'll likely find that __getattr__ is much easier to implement.

This is a free preview of a premium article. You have 2 previews remaining.
Subscribe to read more