Simplifying Codebases

Param’s just a Python library, and so anything you can do with Param you can do “manually”. So, why use Param?

The most immediate benefit to using Param is that it allows you to greatly simplify your codebases, making them much more clear, readable, and maintainable, while simultaneously providing robust handling against error conditions.

Param does this by letting a programmer explicitly declare the types and values of parameters accepted by the code. Param then ensures that only suitable values of those parameters ever make it through to the underlying code, removing the need to handle any of those conditions explicitly.

To see how this works, let’s create a Python class with some attributes without using Param:

class OrdinaryClass(object):
    def __init__(self, a=2, b=3, title="sum"):
        self.a = a
        self.b = b
        self.title = title
    
    def __call__(self):
        return self.title + ": " + str(self.a + self.b)

As this is just standard Python, we can of course instantiate this class, modify its variables, and call it:

o1 = OrdinaryClass(b=4, title="Sum")
o1.a=4
o1()

'Sum: 8'

The same code written using Param would look like:

import param
        
class ParamClass(param.Parameterized):
    a = param.Integer(2, bounds=(0,1000), doc="First addend")
    b = param.Integer(3, bounds=(0,1000), doc="Second addend")
    title = param.String(default="sum", doc="Title for the result")
    
    def __call__(self):
        return self.title + ": " + str(self.a + self.b)

o2 = ParamClass(b=4, title="Sum")
o2()

'Sum: 6'

As you can see, the Parameters here are used precisely like normal attributes once they are defined, so the code for __call__ and for invoking the constructor are the same in both cases. It’s thus generally quite straightforward to migrate an existing class into Param. So, why do that?

Well, with fewer lines of code than the ordinary class, you’ve now unlocked a whole wealth of features and better behavior! For instance, what happens if a user tries to supply some inappropriate data? With Param, such errors will be caught immediately:

with param.exceptions_summarized():    
    o3 = ParamClass()
    o3.b = -5

ValueError: Integer parameter 'ParamClass.b' must be at least 0, not -5.

Of course, you could always add more code to an ordinary Python class to check for errors like that, but it quickly gets unwieldy:

class OrdinaryClass2(object):
    def __init__(self, a=2, b=3, title="sum"):
        if type(a) is not int:
            raise ValueError("'a' must be an integer")
        if type(b) is not int:
            raise ValueError("'b' must be an integer")
        if a<0:
            raise ValueError("'a' must be at least `0`")
        if b<0:
            raise ValueError("'b' must be at least `0`")
        if type(title) is not str:
            raise ValueError("'title' must be a string")          
        
        self.a = a
        self.b = b
        self.title = title
    
    def __call__(self):
        return self.title + ": " + str(self.a + self.b)

with param.exceptions_summarized():    
    OrdinaryClass2(a="f")

ValueError: 'a' must be an integer

Unfortunately, catching errors in the constructor like that won’t help if someone modifies the attribute directly, which won’t be detected as an error:

o4 = OrdinaryClass2()
o4.a = "four"

Python will happily accept this incorrect value and will continue processing. It may only be much later, in a very different part of your code, that you see a mysterious error message that’s then very difficult to relate back to the actual problem you need to fix:

with param.exceptions_summarized():    
    o4()

TypeError: can only concatenate str (not "int") to str

Here there’s no problem with the code in the cell above; o4() is fully valid Python; the real problem is in the preceding cell, which could have been in a completely different file or library. The error message is also obscure and confusing at this level, because the user of o4 may have no idea why strings and integers are getting concatenated.

To get a better error message, you could move those checks into the __call__ method, which would make sure that errors are always eventually detected:

class OrdinaryClass3(object):
    def __init__(self, a=2, b=3, title="sum"):        
        self.a = a
        self.b = b
        self.title = title
    
    def __call__(self):
        if type(self.a) is not int:
            raise ValueError("'a' must be an integer")
        if type(self.b) is not int:
            raise ValueError("'b' must be an integer")
        if self.a<0:
            raise ValueError("'a' must be at least `0`")
        if self.b<0:
            raise ValueError("'b' must be at least `0`")
        if type(self.title) is not str:
            raise ValueError("'title' must be a string")          

        return self.title + ": " + str(self.a + self.b)

o5 = OrdinaryClass3()
o5.a = "four"

with param.exceptions_summarized():    
    o5()

ValueError: 'a' must be an integer

But you’d now have to check for errors in every single method that might use those parameters. Worse, you still only detect the problem very late, far from where it was first introduced. Any distance between the error and the error report makes it much more difficult to address, as the user then has to track down where in the code a might have gotten set to a non-integer.

With Param you can catch such problems at their start, as soon as an incorrect value is provided, when it is still simple to detect and correct it. To get those same features in hand-written Python code, you would need to provide explicit getters and setters, which is made easier with Python properties and decorators, but is still quite unwieldy:

class OrdinaryClass4(object):
    def __init__(self, a=2, b=3, title="sum"):
        self.a = a
        self.b = b
        self.title = title
    
    @property
    def a(self): return self.__a
    @a.setter
    def a(self, a):
        if type(a) is not int:
            raise ValueError("'a' must be an integer")
        if a < 0:
            raise ValueError("'a' must be at least `0`")
        self.__a = a
        
    @property
    def b(self): return self.__b
    @b.setter
    def b(self, b):
        if type(b) is not int:
            raise ValueError("'a' must be an integer")
        if b < 0:
            raise ValueError("'a' must be at least `0`")
        self.__b = b

    @property
    def title(self): return self.__title
    def title(self, b):
        if type(title) is not string:
            raise ValueError("'title' must be a string")
        self.__title = title

    def __call__(self):
        return self.title + ": " + str(self.a + self.b)

o5=OrdinaryClass4()
o5()

'sum: 5'

with param.exceptions_summarized():    
    o5=OrdinaryClass4()
    o5.b=-6

ValueError: 'a' must be at least `0`

Note that this code has an easily overlooked mistake in it, reporting a rather than b as the problem. This sort of error is extremely common in copy-pasted validation code of this type, because tests rarely exercise all of the error conditions involved.

As you can see, even getting close to the automatic validation already provided by Param requires 8 methods and >30 highly repetitive lines of code, even when using relatively esoteric Python features like properties and decorators, and still doesn’t yet implement other Param features like automatic documentation, attribute inheritance, or dynamic values. With Param, the corresponding ParamClass code only requires 6 lines and no fancy techniques beyond Python classes. Most importantly, the Param version lets readers and program authors focus directly on what this code actually does, which is to compute a function from three provided parameters:

class ParamClass(param.Parameterized):
    a = param.Integer(2, bounds=(0,1000), doc="First addend")
    b = param.Integer(3, bounds=(0,1000), doc="Second addend")
    title = param.String(default="sum", doc="Title for the result")
    
    def __call__(self):
        return self.title + ": " + str(self.a + self.b)

Even a quick skim of this code reveals what parameters are available, what values they will accept, what the default values are, and how those parameters will be used in the method. Plus the actual code of the method stands out immediately, as all the code is either parameters or actual functionality. In contrast, users of OrdinaryClass3 will have to read through dozens of lines of code to discern even basic information about usage, or else authors of the code will need to create and maintain docstrings that may or may not match the actual code over time and will further increase the amount of text to write and maintain.

Programming contracts

If you think about the examples above, you can see how Param makes it simple for programmers to make a contract with their users, being explicit and clear what will be accepted and rejected, while also allowing programmers to make safe assumptions about what inputs the code may ever receive. There is no need for __call__ ever to check for the type of one of its parameters, whether it’s in the range allowed, or any other property that can be enforced by Param. Your custom code can then be much more linear and straightforward, getting right to work with the actual task at hand, without having to have reams of if statements and asserts() that disrupt the flow of the source file and make the reader get sidetracked in error-handling code. Param lets you once and for all declare what this code accepts, which is both clear documentation to the user and a guarantee that the programmer can forget about any other possible value a user might someday supply.

Crucially, these contracts apply not just between the user and a given piece of code, but also between components of the system itself. When validation code is expensive, as in ordinary Python, programmers will typically do it only at the edges of the system, where input from the user is accepted. But expressing types and ranges is so easy in Param, it can be done for any major component in the system. The Parameter list declares very clearly what that component accepts, which lets the code for that component ignore all potential inputs that are disallowed by the Parameter specifications, while correctly advertising to the rest of the codebase what inputs are allowed. Programmers can thus focus on their particular components of interest, knowing precisely what inputs will ever be let through, without having to reason about the flow of configuration and data throughout the whole system.

Without Param, you should expect Python code to be full of confusing error checking and handling of different input types, while still only catching a small fraction of the possible incorrect inputs that could be provided. But Param-based code should be dramatically easier to read, easier to maintain, easier to develop, and nearly bulletproof against mistaken or even malicious usage.