Seth Dandridge

Evaluating Boolean Expressions to a Variable

2021-02-16T16:30:00+00:00

Hot off Python inventor Guido van Rossum’s stunning endorsement of last week’s deep dive into function attributes, here’s a little tidbit from an ongoing series of Python minutiae I’m calling Baby Snakes.

(Before continuing, I should disclaim that this is my personal opinion on a matter of style. Coding is an art—even bad code can be art—and one could easily raise practical and aesthetic arguments against my opinions on code style.)

Using Boolean Expressions

I occasionally see inexperienced (and not-so-inexperienced) developers use this pattern:

#> Don't do this:
if some_value > other_value:
  is_some_value_greater = True
else:
  is_some_value_greater = False

What’s wrong with it? Although syntactically and logically correct, it suggests a novice understanding of the nature of boolean expressions.

An expression is a piece of code that evaluates to a value. 1 + 1 is an expression that evaluates to 2. Function calls are expressions since they always evaluate to a value (even if that value is an implicitly returned None). A boolean expression is an expression that evaluates to True or False. In the above snippet, the code between if and : is a boolean expression.

Evaluating Expressions to a Variable

Everyone feels comfortable assigning the result of a mathematical operation or function call to a variable. As noted above, mathematical operations and function calls are both expressions. Why should boolean expressions be treated any differently?

In place of a multi-line if/else statement, assign the result of the boolean expression directly to the variable:

is_some_value_greater = some_value > other_value

This pattern works with any combination of logical, comparison, identity, and membership operators:

is_equal = some_value == other_value
is_dog = type(fido) is Dog
is_clean = isinstance(thing, BadThing) and (thing not in list_of_things)

Boolean Ternary Expressions

An even less stylish pattern than multi-line if/else statements is misuse of conditional expressions, like this funny one-liner:

#> Don't do this:
is_some_value_greater = True if some_value > other_value else False

Conditional expressions evaluate to either the leftmost or rightmost value depending on whether the expression between if and else evaluates to True or False. It’s redundant to wrap a boolean expression in a conditional expression if that conditional expression is merely returning True or False; you can assign the result of the boolean expression directly to a variable and avoid the conditional expression syntax entirely.

This is the third article in my Baby Snakes series, a collection of Python arcana aimed at intermediate and advanced Python developers. For future updates, follow me on Twitter!

Assigning Attributes to Python Functions

2021-02-07T18:59:00+00:00

I was surprised to learn that functions and methods can be assigned attributes, similarly to how one can assign attributes to classes, class instances, and most other objects:

def do_thing(): 
  return
do_thing.whatever = "hi"

print(do_thing.whatever)
#> hi

Wat?

For what reason is this even possible?

You might be thinking:

“Of course this works. Functions are objects. Everything is an object. And therefore it makes sense that functions have assignable attributes. Classes are objects. Classes have assignable attributes. Functions are also objects. Therefore functions must also have assignable attributes.”

Well, just because something is an object doesn’t mean it’s an attribute assignment free-for-all. In fact, Python explicitly forbids attribute assignment in a couple common cases.

For example you can’t assign attributes to built-in functions:

print.some_data = "foo"
#> AttributeError: 'builtin_function_or_method' object has no attribute 'some_data'

And you can’t assign attributes to built-in types:

var = "stringy string"
var.some_data = "foo"
#> AttributeError: 'str' object has no attribute 'some_data'

And yes, both are objects:

isinstance(print, object)
#> True
isinstance(var, object)
#> True

Surprisingly though, Python allows attribute assignment to (non-built-in) functions and methods. And you can put anything you want in there! Check it out:

def do_thing():
  pass
do_thing.sneaky_func = eval
do_thing.sneaky_func("print('O_o')")
#> O_o

Why

This has not always been the case. Functions did not initially support attribute assignment. But despite not supporting attribute assignment, functions were never entirely attribute-less. In even the earlier days of Python, functions had a number of attributes, including a few that were writeable—namely __doc__.

If you’ve ever documented your code, you’re probably familiar with docstrings. Docstrings are syntactic sugar that enable one to write documentation for modules, classes, methods and functions by putting an inline string literal at the top of the object’s definition:

def do_thing():
  """Here's some documentation"""
  return
help(do_thing)
#> Help on function do_thing in module __main__:
#> do_thing()
#>    Here's some documentation

The docstring of a function is stored in its __doc__ attribute:

def do_thing():
  """Here's some documentation"""
  return
print(do_thing.__doc__)
#> Here's some documentation

So what do docstrings have to do with function attributes?

In the earlier days of Python, engineers began hijacking docstrings for untoward purposes—not using the docstring for documentation, but instead for storing other programmatically accessible information about the function.

John Aycock, for example, wrote a clever parsing system that decided whether to apply a method to a target string by evaluating whether that target string matched a regular expression contained in the method’s docstring.

In response to docstring exploitation, Barry Warsaw proposed PEP 232, an extension to the Python language adding support for method/function attributes. It was approved and implemented in Python 2.1+.

When Is It Appropriate To Use Function Attributes?

In my opinion: rarely.

Many experienced Pythonistas aren’t aware that it is even possible to assign function attributes. If you’re doing work-a-day software development, lack of familiarity in the community is usually a red flag for a language feature.

That being said, there are valid reasons one might store metadata on a function.

Function attributes are useful if you are passing functions around and need to a way to graft information onto those functions to use later. Perhaps you’re writing a higher order function—a function that takes a function as an argument. You might want your higher order function to treat the lower order function differently depending on the value stored in an attribute of the function:

def dog_feeder(food_type):
  print(f'feeding {food_type} to the dog')
dog_feeder.species = 'dog'

def cat_feeder(food_type):
  print(f'feeding {food_type} to the cat')
cat_feeder.species = 'cat'

def feed_pet(feed_func):
  # Higher order function that takes a feed function as argument
  if feed_func.species == 'dog':
    feed_func('dog food')
  elif feed_func.species == 'cat':
    pass

feed_pet(dog_feeder)
#> feeding dog food to the dog

feed_pet(cat_feeder)
#>

I should note that in Python 3.4+, the functools.singledispatch decorator better serves the above use case. I’ll talk about singledispatch in a future post. But my point stands: when doing higher-order programming, it can be useful for functions to carry around data about themselves.

PEP 232 and Paul Prescod’s message to the python-dev mailing list describe a few other use cases.

For example, one might want to define different docstring formats in order to accommodate different IDEs or documentation methodologies.

In a similar vein as John Aycock’s parser: if you’re writing a class for traversing a data structure, and you want to associate particular node types with certain methods, the list of node types that a method supports can be stored as an attribute of that method.

When Is It Not Appropriate To Use Function Attributes?

If you need a quick-and-dirty data store and believe dot notation is prettier than dict[key]:

def settings():
  return

settings.host = '0.0.0.0'
settings.port = 8080

app.run(host=settings.host, port=settings.port)

Seriously, don’t do this. If the above code landed on my desk for review, I’d give the engineer a sincere pat on the back for being clever, then politely request they be sent to the gulag for their crimes.

If you need a store of data, use a dictionary, class, or even a module. Anything but a function.

Another deeply inappropriate reason for leveraging function attributes is that you are dissatisfied with off-the-shelf class behavior and want to create functions that sort of behave like classes. You can accomplish that with function attributes and a funky bit of recursion:

def Dog():
  def bark():
    print("bark bark")
  Dog.bark = bark
  return Dog

fido = Dog()
fido.bark()
#> bark bark

Please don’t do this either.

Conclusion

I hope you enjoyed this deep-ish dive into Python function attributes. This is one article in a series on Python arcana I’ll be publishing over the course of the next few months. Check back soon for more!

Defining a Custom Class Constructor in Python

2021-02-01T00:00:00+00:00

Python gives you tremendous visibility into its internal workings. The degree to which you can pop open the hood and modify what’s happening at runtime is powerful and terrifying. The author of the venerable Flask web framework Armin Ronacher gave a fantastic talk on this topic called How Python was Shaped by Leaky Internals.

One way to take advantage of these leaky internals is by overwriting the built-in class constructor. Whenever you define a class, the function __build_class__ is called behind the scenes. __build_class__ is passed the class code (which is actually, suprisingly, a function) and the class name as arguments. The result of executing __build_class__ is your class being created in the appropriate namespace. I’m simplifying, but bear with me.

Investigating __build_class__

To see this in action, let’s define a simple class called Dog and observe __build_class__ do its job.

class Dog:
  def __init__(self):
    print('bark bark')

dog = Dog()
#> bark bark

Now let’s define a custom build class function to see a bit of what’s happening under the hood.

def custom_build_class(*args):
  print(args)

__builtins__.__build_class__ = custom_build_class

class Dog:
  pass
#> (<function Dog at 0x10f6ff6a8>, 'Dog')

You can see the two arguments that are passed to __build_class__ are the Dog function (the code of the Dog class) and the name of the class, 'Dog'. Because we don’t ever call the “real” __build_class__, the class is never defined in the namespace. Curiously, however, accessing the variable Dog doesn’t throw NameError. Instead, it’s justNone.

print(Dog)
#> None

Not sure why this is—evidently there’s something outside of __build_class__ that’s modifying the namespace. Worth further investigation.

Redefining __build_class__

Now let’s redefine the built-in class constructor so that no matter what class you define, the __build_class__ function assigns the code for Dog to that class name instead of the code of the class you’re defining.

class Dog:
  def __init__(self):
    print('woof woof')

def custom_build_class(*args):
  return Dog

__builtins__.__build_class__ = custom_build_class

class Cat:
  def __init__(self):
    print('meow meow')

cat = Cat()
#> bark bark

print(type(cat))
#> <class '__main__.Dog'>

Chaos reigns!

Conclusion

Aside from impressing your friends and coworkers with verboten Python magic, why would you ever want to do any of this? I actually can’t think of a legitimate reason. Perhaps you need a hook that registers every class upon its creation? However, this case would probably be better served with a metaclass. If you can think of a legitimate real-world scenario for overriding __build_class__, let me know.

This is the second in a series of blog entries I’m calling “baby snakes”, bits of Python arcana aimed at intermediate and advanced Python developers. For a similar exercise in Python’s built-in abuse, see Redefining Python’s Print() Function and check back soon for even more!

Redefining Python’s Print() Function

2021-01-25T14:27:00+00:00

Every novice Pythonista is familiar with the trusty built-in function print(). Its job is simple: output the string representation of an object or expression.

print('the answer is', 1 + 2)
'the answer is 3'

But let’s say you want to change the behavior of print. Instead of sending its output to the terminal, let’s send that output to a file on the disk. We can accomplish this by redefining the print variable.

Printing to a File

The full function signature of print is:

print(*objects, sep=' ', end='\n', file=sys.stdout, flush=False)

Note the third keyword argument, file. The default value of file is sys.stdout, which represents the terminal output buffer. The only real requirement for the object passed as file is that it has a .write() method. You know what has a write method? File handlers.

with open('log.txt', 'a+') as f: # f is a file handler
    f.write('the answer is', 1 + 2)

Let’s redefine the print variable to instead be a wrapper function around the vanilla built-in print(). The wrapped print() needs to be referenced through the builtins module so that we don’t define a recursive function.

import builtins

def print(*args, **kwargs):
    with open('log.txt', 'a+') as f:
        return builtins.print(*args, file=f, **kwargs)

print('the answer is', 1 + 2)

Now, when we call print(), no output is displayed in the terminal. However, if we read the contents of log.txt, we can see that our redefined print() has written to the file.

$ cat log.txt
the answer is 3

Print as a Buffer Object

Let’s make it weird. We can actually redefine print() to be a buffer object that stores all the strings that are passed to it for printing. Instead of a function, print will be an instance of a class that inherits io.StringIO.

StringIO is a in-memory text buffer that works similarly to a file handler. In fact, file handlers and StringIO both implement the abstract base class IOBase, which means they have many of the same methods defined.

Here’s a simple example of how to use StringIO:

import io

buffer = io.StringIO()
buffer.write('here is some input')
buffer.getvalue()
'here is some input'

Let’s define a new subclass of StringIO called PrintWrapper. We’re also going to define a __call__ magic method on this new class. __call__ allows you to define how an instance of a class behaves when it is called like a function.

import builtins
import io

class PrintWrapper(io.StringIO):
  def __call__(self, *args, **kwargs):
    # Pass the object instance (self) as the file
    return builtins.print(*args, file=self, **kwargs)

print = PrintWrapper() # print is now the printer AND a buffer
print('this was added to buffer')  # triggers print.__call__()
print('this was also added to the buffer')
print.getvalue()
'this was added to buffer\nthis was also added to buffer\n'

Strings that you have printed are seperated by \n when you call print.getvalue(), but you can define a different seperator by passing it as a string to the end= keyword argument of builtins.print().

Conclusion

Why would you ever want to do any of this? Let’s say you are building a web-based programming environment like CoderPad. When a user calls print(), you wouldn’t want the Python interpreter running on the server to send the printed statement to stdout. You want it outputted to a handler that transmits it to the user’s browser over the Internet. There are probably other ways to accomplish this, but redefining print() behind the scenes is one possibility.

Edit: As Reddit user Dasher38 points out, there’s a context manager in the standard library contextlib.redirect_stdout that is better suited for redirecting the output of a web-based programming environment backend. redirect_stdout redirects everything sent to sys.stdout (including printed statements) to a specified file-like object.

This is the first of a series of blog entries I’m calling “baby snakes”, bits of Python arcana aimed at intermediate and advanced Python developers. Check back soon for more!

Hello World!

2021-01-20T18:06:17+00:00

Welcome to my new blog! Over the past few years I’ve collected a number of writings to share. These mostly focus on the Python programming language and other software engineering topics, but occasionally there will be a few random side-obsessions thrown in to keep things spicy.

For the time being, this site is powered by the static site generator Jekyll and a lightly modified version of its default theme, Minima.

A few other options were considered before I settled on Jekyll:

Write my own static site generator in Python
Write my own dynamic content management system from scratch using Flask
Use an existing Python-based static site generator like Pelican

However, I ultimately decided on Jekyll. Even though it’s written in Ruby, with which I have next to nil experience, I liked the default theme and it was a much faster route to getting words on the screen than something written from scratch.

Anyway, happy reading and I hope you enjoy my musings.