I wrote about Python decorators five years ago. [See Python Decorators, part 1 and Python Decorators, part 2] At the time, they were new to me, and I hadn’t thought of good use cases for them. Or really, even just good ways to use them.

But that changed during the last five years as I’ve had occasions to actually use decorators in code. They are extremely handy in certain situations. Today’s post takes a more useful look at Python decorators.

To keep this from being too long, I’m not going to explain decorators in any detail here. The two posts linked to above explain how they work, as does the Python documentation and various tutorials. Very briefly, Python allows decoration of functions, classes, and class methods.

Given these three function definitions:

The above just provides bare-bones do-nothing examples of the three basic types of decorators. We use them to decorate functions, classes, and class methods like this:

The code doesn’t do anything interesting, just demonstrates the syntax of the three decorator types. Note how the decoration — which starts with an at sign (@) — comes on a line before the function, method, or class definition. These decorators end up creating wrappers around the declared function, method, or class. (This is described in detail in the earlier posts.)

Let’s take a closer (but quick) look at each of the three types:

A basic decorator function for decorating functions receives a single parameter — the (now anonymous) function following the decorator. In this template, we grab both the function’s name and type and store them in ftn_name and ftn_type (lines #5 and #6). For illustration purposes, we’ll also print those parameters.

Lines #9 to #15 define the wrapper function that surrounds the client function. We take a generic set of arguments, print them for illustration, and increment a counter that keeps track of how many times the function is called. Note that this counter is defined in line #18 as an attribute of the wrapper function itself.

We use the decorator like this:

Note how (in lines #43 to #45) we access the count attribute of each function — actually of the wrapper function around the client function. This is one key to making productive decorators — they can have accessible attributes.

When run, this prints:

@Decorator: function1 (function)
@Decorator: function2 (function)
@Decorator: function3 (function)

@[function1]: (99, 86, 57, 68) {}
@[function1]: (21, 42, -1, -1) {}
@[function1]: () {'a': 21, 'b': 42, 'c': -1, 'd': -1}
@[function1]: (99, 86) {'d': 57, 'c': 68}

@[function2]: (99, 86, 57, 68) {}
@[function2]: (21, 42) {}
@[function2]: () {'a': 21, 'b': 42}
@[function2]: (99, 86) {'d': 57, 'c': 68}

@[function3]: (99, 86, 57, 68) {}
@[function3]: (21, 42) {}
@[function3]: () {'a': 21, 'b': 42}

function1: 4 calls
function2: 4 calls
function3: 3 calls

Note how the function arguments are printed. At the end, the counters report the number of calls to each function.

Next, let’s look at the class method decorator function:

It’s similar to a function decorator. The main difference is that the wrapper function receives the self parameter, just as an instance method does. Since this is a generic method wrapper that can wrap any class method, we take generic arguments (line #9) and pass them to the method (line #18).

This decorator provides a simple logging feature that records the arguments passed to the method. Rather than keeping the log attribute in the wrapper function (as we did above), we check to see if the object instance has a log attribute and add one if it doesn’t (lines #11 and #12). In line #15, we append to the log attribute.

We use the decorator like this:

Note the use of two decorators in lines #28 and #29 — the built-in property decorator followed by our BasicMethodDecorator. I explained using multiple decorators in the first two posts, and I’ll briefly revisit it below. Suffice here to say that it’s essentially transparent.

We create two class instances (lines #46 and #47), exercise their methods (lines #49 to #67), and then dump their log attributes (lines #69 to #77).

When run, this prints:

@MethodDecorator: point.moveto (function)
@MethodDecorator: point.reset (function)
@MethodDecorator: point.magnitude (function)
@MethodDecorator: point.diff (function)
@MethodDecorator: point.dot (function)

pnt1=[2.718 x 3.142]
pnt1.magnitude=4.1543511992247355
pnt1.dot(pnt1)=17.2586338865
pnt1.dot(pnt2)=-19.6168627509305
pnt1.diff(pnt2)=(9.34435, 3.6525889499999997)

pnt2=[-6.626 x -0.511]
pnt2.magnitude=6.645744771792031
pnt2.dot(pnt1)=-19.6168627509305
pnt2.dot(pnt2)=44.16592357180111
pnt2.diff(pnt1)=(9.34435, 3.6525889499999997)

01: @[point.moveto]: (2.71828, 3.14159) {}
02: @[point.magnitude]: () {}
03: @[point.dot]: ([2.718 x 3.142],) {}
04: @[point.dot]: ([-6.626 x -0.511],) {}
05: @[point.diff]: ([-6.626 x -0.511],) {}

01: @[point.moveto]: (-6.62607, -0.51099895) {}
02: @[point.magnitude]: () {}
03: @[point.dot]: ([2.718 x 3.142],) {}
04: @[point.dot]: ([-6.626 x -0.511],) {}
05: @[point.diff]: ([2.718 x 3.142],) {}

The log lists record each method call and record its arguments.

Lastly, let’s look at a class decorator function:

In this case, the decorator function receives a class object, but otherwise the basics are the same. As with the function decorator above, we create attributes in the wrapper object: a log attribute and a counter attribute. The wrapper function appends to log and increments counter.

We use the decorator like this:

Note we’ve again used the built-in property decorator to convert the abv and ibu methods to (read-only) properties.

When run, this prints:

@ClassDecorator: beer (type)

Yum! Newcastle Brown Ale (4.7/18)
Yum! New Belgium Fat Tire (5.2/15)
Yum! New Belgium Voodoo Ranger (7.0/50)
Yum! Stone IPA (6.9/71)

 0: @[beer]: ('Newcastle', 'Brown Ale') {'abv': 4.7, 'ibu': 18}
 1: @[beer]: ('New Belgium', 'Fat Tire') {'abv': 5.2, 'ibu': 15}
 2: @[beer]: ('New Belgium', 'Voodoo Ranger') {'abv': 7.0, 'ibu': 50}
 3: @[beer]: ('Stone', 'IPA') {'abv': 6.9, 'ibu': 71}

Note how the BasicMethodDecorator attached the log attribute to each instance, whereas the BasicClassDecorator attaches log to the wrapper, which makes it appear as an attribute of the class.

Note an important aspect of using class decorators: Class attributes become essentially inaccessible. Unless you add features to the wrapper to pass through attribute requests, these will fail. For example:

Note how lines #11 and #12 access the class attribute chessboard.Cols and chessboard.Rows. The first one encountered raises an error because Rows and Cols are not attributes of the decorator wrapper function object that’s bound to the name chessboard.

When run, this prints:

@ClassDecorator: chessboard (type)

Traceback (most recent call last):
  File "C:\blog\hcc\fragment.py", line 14, in <module>
    brd = chessboard()
  File "C:\blog\hcc\examples.py", line 104, in wrapper
    return klass(*args, **kwargs)
  File "C:\blog\hcc\fragment.py", line 10, in __init__
    for row in range(chessboard.Rows)]

AttributeError: 'function' object has no attribute 'Rows'

If we instead access the attributes through the instance — by changing chessboard to self — the code works fine. So, one way to access class attributes is through instances of the class, either on the instance itself or on the class through the instance’s __class__ attribute.

It is possible to access the wrapped class through the wrapper but requires digging into the wrapper function:

When this is run, it prints:

@ClassDecorator: chessboard (type)

<__main__.chessboard object at 0x00000228AD99EBA0>

nam = 'chessboard'
cls = <class '__main__.chessboard'>
cls.Rows = 8
cls.Cols = 8

It’s something to keep in mind with class wrappers. Function and method wrappers don’t have the issue, because those objects don’t usually have other attributes of interest. Of course, in cases where they do, steps similar to the above are necessary.

The above should give you a good idea of the possibilities. We’ve already seen how to include useful attributes in the wrapper object. Method decorators provide a means to modify methods (for instance, the built-in method decorators classmethod, staticmethod, and property. We’ll look at more decorators below but first let’s return to the notion of stacking decorators.

There are several instances of it above, and I’ve already said it’s essentially transparent (the exception is when the decorator needs access to the object it thinks it’s decorating rather than another decorator).

We can explore this a little with these two decorator functions:

The second one (Decorator2, lines #27 to #43) is a basic decorator function such seen above. The first one (Decorator1, lines #1 to #25) has a parameterized decorator (discussed in the earlier posts). Parameterized decorators have doubly nested functions. The first, inner (lines #5 to #22), is the actual decorator function. The outer function (Decorator1) must return it after processing the arguments (line #25).

Inside the inner function is the wrapper function (lines #13 to #19), which is similar to previous wrapper functions (and the one in Decorator2, lines #34 to #40).

We use multiple decorators like this:

The decorators can be stacked in any order. When run, this prints:

Decorator1(Agatha)
@Decorator2: function1 (function)
@[Agatha]: Decorator2.<locals>.wrapper (function)
Decorator1(Bertha)
@[Bertha]: function2 (function)
@Decorator2: Decorator1.<locals>.decorator.<locals>.wrapper (function)

[Agatha].args: (21, 42)
[Agatha].kwargs: ['fname', 'fpath']
decorator2.args: (21, 42)
decorator2.kwargs: ['fname', 'fpath']

[Agatha].args: (1, 2, 3, 4, 5)
[Agatha].kwargs: ['x', 'y', 'z']
decorator2.args: (1, 2, 3, 4, 5)
decorator2.kwargs: ['x', 'y', 'z']

decorator2.args: (1, 2, 3, 4, 5)
decorator2.kwargs: []
[Bertha].args: (1, 2, 3, 4, 5)
[Bertha].kwargs: []

decorator2.args: ()
decorator2.kwargs: ['cmd', 'nup', 'pir']
[Bertha].args: ()
[Bertha].kwargs: ['cmd', 'nup', 'pir']

Again, the caveat is the special case where the decorator expects the class (or method or function) and doesn’t expect another decorator. If that matters, the case can be detected and handled (albeit somewhat tediously).

Here’s a function logging decorator that also keeps track of how much time is spent in the function:

This decorator adds a new wrinkle by using a wrapper class rather than a wrapper function. This works fine so long as the class we define implements __call__, because a wrapper must be callable. Using a class just makes it easier to add properties and methods to the wrapper. Note how, in line #45, the decorator returns an instance of the class. The instance is the actual wrapper.

Note also that we’re extending the Python list class, so the wrapper will be a list object containing the log lines along with the count and timer attributes as well as the milliseconds and seconds properties.

We use the decorator like this:

The assumption is that we want to log and time function1 and function2, so we add the FunctionLogger decorator. Then we call both functions a bunch of times, after which we dump the stats.

When run, this prints:

@Logger: function1 (function)
@Logger: function2 (function)

Logger:function1: 6 calls, 2.830552 secs
01: @[           0]: (0, 0, 0, 0) {}
02: @[   354015100]: () {'a': 1, 'b': 2, 'c': 3, 'd': 4}
03: @[   430028000]: (1, -1, 100, -99) {}
04: @[  1115059700]: (21, 42) {'d': 63, 'c': 84}
05: @[  2002259500]: (-9, 99, -999, 9999) {}
06: @[  2078706000]: (9, 99, 999, 9999) {}

Logger:function2: 4 calls, 16.1943 secs
01: @[           0]: (0, 0, 0, 0) {}
02: @[  2353524900]: () {'a': 1, 'b': 2, 'c': 3, 'd': 4}
03: @[  6104884700]: (1, -1, 100, -99) {}
04: @[ 10597902200]: (21, 42, 63, 84) {}

Another useful debugging tool is an error trapper/logger:

To keep things simple, this is a basic function decorator, but what differs is the try-except block in the wrapper. This catches any errors raised by the wrapped function and logs them. Note that the decorator takes an argument — a value to return when the function raises an error — so it has a nested decorator function (lines #4 to #32) with its own nested wrapper function (lines #12 to #24).

We use the decorator like this:

Fifteen calls to the foobar function, which does some math (lines #12 to #26). (Because of the math, certain arguments will cause an exception.) After exercising the function, we print some wrapper stats and then dump any errors logged.

When run, this prints:

@ErrorRecorder: foobar (function)

 1: result=None
 2: result=0.23104906018664842
 3: result=1.617343421306539
 4: result=-0.34969944794454716
 5: result=0.999896315728952
 6: result=0.0
 7: result=None
 8: result=None
 9: result=None
10: result=0.8317766166719343
11: result=None
12: result=None
13: result=0.0
14: result=None
15: result=None

function-name: foobar
calls: 15
errs: 8

errors:
 1: @1 foobar() missing 3 required positional arguments: 'a', 'b', and 'c'
 2: @7 math domain error
 3: @8 foobar() takes 3 positional arguments but 4 were given
 4: @9 foobar() missing 1 required positional argument: 'c'
 5: @11 float division by zero
 6: @12 math domain error
 7: @14 must be real number, not str
 8: @15 unsupported operand type(s) for /: 'float' and 'NoneType'

So, we know that, of the 15 calls, #1, #7–#9, and #11–#15 all raised exceptions, and we can see exactly what those exceptions were.

Python allows the attachment of type annotations to variable and function declarations (for a good overview, see Annotations Best Practices).

Starting with Python 3.10, functions, classes, and modules are guaranteed to have the __annotations__ attribute (a dict) containing any annotations for the declared object. That means we can easily access a function’s annotations (if any):

If a parameter (or the function itself) doesn’t have a type annotation, there is no corresponding entry in the __annotations__ dictionary.

When run, the above prints:

a is an int
b is a float
c is a str
d is a list
return is a tuple

Note how the function’s return value annotation uses the name “return” (a name not allowed for a variable because it’s a Python keyword).

Here’s a decorator function that uses annotations to automatically check the variable types of function arguments:

The structure is similar to other decorator functions but with a bit more work done. Lines #6 to #9 get the __annotations__ attribute of the passed function and list its contents.

The wrapper (lines #11 to #30) uses the annotations to check the function argument types. First it checks any positional arguments (lines #14 to #19). Then it checks any keyword arguments (line #21 to #27). Note that, for demonstrations purposes, the code only lists the input arguments and whether they are “OK” or type violations. A serious implementation would either log the issues or raise an exception.

Note also that we’re not checking the return type. That’s left as a reader exercise.

We use the decorator like this:

When run, this prints:

TypeCheck: foobar(a:int,b:int,c:float):list

foobar: arg[a]:int = 42:int OK
foobar: arg[b]:int = 21:str [str≠int]
foobar: arg[c]:float = 3.1415:float OK

foobar: arg[a]:int = 63:int OK
foobar: kwarg[c]:float = 3:int [int≠float]

foobar: kwarg[b]:int = 21:int OK
foobar: kwarg[a]:int = 4.2:float [float≠int]
foobar: kwarg[c]:float = 3.1415:float OK

foobar: arg[a]:int = 42:str [str≠int]
foobar: arg[b]:int = [21]:list [list≠int]
foobar: arg[c]:float = ('3', '.', '1', '4'):tuple [tuple≠float]

You could combine type-checking, error-trapping, and function timing all into one grand debugging decorator.

That’s probably more than enough for this time. Next time we’ll look at some more involved examples of decorators.

Link: Zip file containing all code fragments used in this post.

∅