Just Send The Message

Note: this post is presented as a Socratic dialog because I'm feeling experimental. This conversation is a generalization of conversations I've had in the past about the use of class hierarchies in Python.

Early Career Engineer (ECE): Hey, you said in that code review comment that you wanted to chat about some revisions offline, what's up?

Me: Yeah, thanks for following up. Sometimes these discussions benefit from higher bandwidth communication than going back and forth on GitHub.

ECE: No problem. What did you want to discuss?

Me: I noticed a particular pattern in your code that I thought would be worth talking about in a bit more depth. It's one that I've seen a lot of folks write when they're new to Python, and you mentioned in the PR description that that's the case for you. Thanks for noting that by the way, it's super helpful context! So what language are you more familiar with?

ECE: I used Java at my last job. This is my first time working in a large Python codebase.

Me: Ah, I thought that might be the case. So the thing I noticed, and the reason I thought you might have a Java background, is the use of an Abstract Base Class (ABC) that the two new classes you wrote inherit from. Could you explain your thought process around that inheritance hierarchy?

ECE: Sure. I had an inuition that that those two classes had a lot in common, so I started by identifying all of the common characteristics and putting them into a superclass. I realized that that superclass doesn't really have a default implementatation, so I made it abstract.

Me: Ok, cool, that makes sense. So first off, your PR had a lot going for it. But I think there's a bit of advice here that will make you a more effective Python programmer.

ECE: Ok, what is it?

Me: You don't need ABCs. Just send the message.

ECE: Cool... What does that mean?

Me: Let's use a simpler example than your PR in case this conversation is ever recorded and put on the internet. Say we have this code:

from abc import ABC, abstractmethod

class Vehicle(ABC):
    @abstractmethod
    def number_wheels():
        pass

    @abstractmethod
    def number_doors():
        pass


class Car(Vehicle):
    def number_wheels():
        return 4

    def number_doors():
        return 4


class Motorcycle(Vehicle):
    def number_wheels():
        return 2

    def number_doors():
        return 0

ECE: Ok, looks straightforward enough.

Me: Right, but there's a few issues with it. First, we designed our class hierarchy up front, which despite its prevalence as a pattern is almost always a bad idea.

ECE: Why's that?

Me: Starting with a class hierarchy almost always bakes in assumptions that turn out to be wrong. In our example, we're just assuming that all cars have four doors. But what about sports cars? The first time our programmer decides to trade in their modest sedan for a two-door speedster we'll have to change our class hierarchy.

ECE: Right, I see. We're also assuming that all motorcycles have two wheels, but I guess that there are those silly three-wheel motorcycles.

Me: Exactly. Those ridiculous grown up tricycles totally break our class hierarchy.

ECE: So what should we do instead? Some classes are bound to have common methods.

Me: In most languages that support class hierarchies, it's almost always preferable to extract a superclass than to start with one.

ECE: Got it. Start with Motorcycle and Car, use them in calling code, and extract a superclass if we need to abstract over the two?

Me: Correct.

ECE: You said "in most langugages." Is Python one of them?

Me: Not really, and that has to do with Python's dynamic type system. In Java, any method that takes a superclass as an argument will take its subclasses as arguments too, right?

ECE: Right, that's one reason that we use them.

Me: Well Python does that without declaring class hierarchies. All function and method arguments are basically just "object" as far as the type checker is concerned. So any object passed into a function is valid. Calling a method on an object is sometimes called "sending a message" to that object, so when I say "just send the message," I mean that it's the responsibility of the consuming code to ensure that they're only sending messages that your object responds to. We don't need a class hierarchy, we can just write two classes that have methods with the same names.

ECE: So wait, why do ABCs exist in Python at all?

Me: The abc module provides some functionality that's useful in situations where you need to do metaprogramming. For instance, marking a method as abstractmethod will raise a TypeError if a subclass doesn't override it, and ABCs can dynamically insert themselves into an existing class' inheritance tree. That can be really useful if you're interacting with user supplied code that you can't change, for instance if you're writing a framework. 99% of the time though, you don't really need that kind of functionality.

ECE: Got it. So wait, if we're saying that it's the responsibility of the caller to supply a valid object, how do we have any faith in our code? I know that we have good engineers here, but everyone makes mistakes.

Me: That's where tests come in. Generally we assume that unit test coverage in dynamically typed languages replaces some of the verification you get with a static type system. That's why the unit test suites in dynamically typed languages tends to be larger than in statically typed languages. Unit tests are also great for driving design, documenting the behavior of your objects, etc. They're really useful in statically typed languages, but they're indispensible in dynamically typed languages.

ECE: But isn't that pushing the burden of verifying correctness onto a human? I thought we wanted to push errors as far back in the software development lifecycle as they'll go, and to make invalid states unrepresentable. Isn't a compiler error better than a test failure for that?

Me: Absolutely, and that's one of the reasons that static type systems are becoming popular again after having a bit of a lull in the industry. Even dynmically typed languages are starting to add on static type systems. Look at Sorbet for Ruby, or MyPy for Python.

ECE: Ok, I have to run to a meeting. Just to recap, ABCs aren't really useful in Python the way they are in Java, and we most likely don't need them. We should just be sending messages to objects and verifying with unit tests?

Me: Yup, and do check out MyPy. I think you'll really like it.

ECE: Thank you, this has been helpful. I'll make some revisions and update you on the PR.

Me: Great, I'll look forward to reading it.