In Python, functions are first-class objects.
First-Class functions: "A programming language is said to have first class functions if it treats functions as first-class citizens."
First-Class Citizen (Programming): "A first-class citizen (sometimes calles first-class object) is an entity which supports all the operations generally available to other entities, such as being being passes as an argument, returned from a function, and assigned to a variable."
Assign function to variables
def square(x):
return x * x
f = square(5) # Assign function *output* to variable
f = square # Assign function to variable (no (), as this triggers execution)
print(square)
print(f) # f points to same object as square
print(f.__name__)
f(5)
<function square at 0x106f968b0> <function square at 0x106f968b0> square
25
Use function as argument in function
print(map(lambda x: x ** 2, [1, 2, 3]))
filter(lambda x: x % 2 == 0, [1, 2, 3])
<map object at 0x106fd07c0>
<filter at 0x106f9ca90>
def do_twice(fn, *args):
fn(*args)
fn(*args)
do_twice(print, "Hello world!")
Hello world! Hello world!
Return function from function
def create_adder(n):
def adder(m):
return n + m
return adder
add_5 = create_adder(5)
print(add_5.__name__)
add_5(10)
adder
15
def greeter(text="Hello"):
"""Print greeting text."""
print(text)
def printer(func):
print("Docstring:", func.__doc__)
print("Name:", func.__name__)
print("Attributes:", func.__dict__)
print("Code:", func.__code__)
print("Defaults:", func.__defaults__)
print("Globals:", func.__globals__.keys())
print("Closures:", func.__closure__)
printer(greeter)
Docstring: Print greeting text. Name: greeter Attributes: {} Code: <code object greeter at 0x7fbc814a3ea0, file "<ipython-input-28-dd4fd47dbd24>", line 1> Defaults: ('Hello',) Globals: dict_keys(['__name__', '__doc__', '__package__', '__loader__', '__spec__', '__builtin__', '__builtins__', '_ih', '_oh', '_dh', 'In', 'Out', 'get_ipython', 'exit', 'quit', '_', '__', '___', '_i', '_ii', '_iii', '_i1', 'greeter', 'printer', '_i2', '_i3', '_i4', '_i5', '_5', '_i6', '_6', '_i7', '_7', '_i8', 'c', '_i9', '_i10', '_i11', '_i12', '_i13', '_i14', '_i15', '_i16', '_16', '_i17', '_i18', '_18', '_i19', '_19', '_i20', '_20', '_i21', '_21', '_i22', '_i23', '_23', '_i24', '_i25', '_i26', '_26', '_i27', '_i28']) Closures: None
Some of these attributes return objects that have attributes themselves:
print(greeter.__doc__.upper())
print(greeter.__doc__.splitlines())
PRINT GREETING TEXT. ['Print greeting text.', ' ']
print(greeter.__code__.co_argcount)
print(greeter.__code__.co_code)
1 b't\x00|\x00\x83\x01\x01\x00d\x01S\x00'
global
declaration.nonlocal
declaration. global
¶Because a
is assigned inside the function body, it is a local variable in the local namespace of the f
function and a completely separate object from the variable a
defined in the global namespace.
a = 5
def f():
a = 10
f()
a
5
global
declares the a
variable created inside the function body to be a global variable, making it reassing the previously declared global variable of the same name.
a = 5
def f():
global a
a = 10
f()
a
10
nonlocal
:¶Because n
reassigned inside decrement
, it is a local variable of that function's namespace. But because n -= 1
is the same as n = n - 1
and Python first runs the right hand side of assignment statements, when running n - 1
throws an error because there it is asked to reference the variable n
that hasn't been assigned yet.
def countdown(n):
def display():
print(n)
def decrement():
n -= 1
while n:
display()
decrement()
countdown(3)
3
--------------------------------------------------------------------------- UnboundLocalError Traceback (most recent call last) <ipython-input-21-473dc28d8ecc> in <module> 12 13 ---> 14 countdown(3) <ipython-input-21-473dc28d8ecc> in countdown(n) 9 while n: 10 display() ---> 11 decrement() 12 13 <ipython-input-21-473dc28d8ecc> in decrement() 5 6 def decrement(): ----> 7 n -= 1 8 9 while n: UnboundLocalError: local variable 'n' referenced before assignment
nonlocal
declares n
to be a variable in the scope of an enclosing function of decrement
.
def countdown(n):
def display():
print(n)
def decrement():
nonlocal n
n -= 1
while n:
display()
decrement()
countdown(3)
3 2 1
A closure is the object that results from packaging up the statements that make up a function with the environment in which it executes. David Beazley, Python Essential Reference
A closure is a function that has access to variables that are neither global nor local (i.e. not defined in the function's body). Luciano Ramalho, Fluent Python
A closure is an inner function that remembers and has access to variables in the local scope in which it was created, even after the outer function has finished executing. - 'A closure closes over the free variables from the environment in which they were creates'. - A free variable is a variable defined inside the scope of the outer function that is not an argument in the inner function. Below, the 'free variable' is message. Stanford CS course
How I think of them (not sure that's correct):
The closure attribute of a user-defined function is non-empty if the function is defined inside the scope of an outer function and references a local variable defined in the scope of the outer function. From the point of view of the inner function, said variable is a nonlocal variable.
A user-defined function is a closure if its closure attribute is non-empty.
In the first case below, the inner function doesn't reference a nonlocal variable and thus isn't a closure. In the second case, it does reference a nonlocal variable, which gets stored in its closure
attribute, which makes inner
a closure.
def outer():
x = 5
def inner():
print("Hello")
return inner
def outer2():
x = 5
def inner():
print(x)
return inner
g = outer()
print(g.__closure__)
g = outer2()
print(g.__closure__[0].cell_contents)
None 5
Notes based on chapter 8 in Fluent Python.
Functions that take mutable objects as arguments require caution, because function arguments are aliases for the passed arguments (i.e. they refer to the original object). This can cause unintended behaviour in two types of situations:
class HauntedBus:
def __init__(self, passengers=[]):
self.passengers = passengers
def pick(self, name):
self.passengers.append(name)
def drop(self, name):
self.passengers.remove(name)
bus1 = HauntedBus(["pete", "lara", "nick"])
bus1.drop("nick")
print(bus1.passengers)
bus2 = HauntedBus()
bus2.pick("heather")
print(bus2.passengers)
bus3 = HauntedBus()
bus3.pick("katy")
print(bus3.passengers)
['pete', 'lara'] ['heather'] ['heather', 'katy']
Between bus 1 and 2, all works as intended, since we passed our own list when creating bus 1. Then things get a bit weird, though: how did Heather get into bus 3? When we define the HauntedBus
class, we create a single empty list that is kept in the background and will be used whenever we instantiate a new bus without a custom passenger list. Importantly, all such buses will operate on the same list. We can see this by checking the object ids of the three buses' passenger lists:
assert bus1.passengers is not bus2.passengers
assert bus2.passengers is bus3.passengers
This shows that while the passenger list of bus 1 and 2 are not the same object, the lists of bus 2 and 3 are. Once we know that, the above behaviour makes sense: all passenger list operations on buses without a custom list operate on the same list. Anohter way of seeing this by inspecting the default dict of HauntedBus
after our operations abve.
HauntedBus.__init__.__defaults__
(['heather', 'katy'],)
The above shows that after the bus3.pick('katy')
call above, the default list is now changed, and will be inherited by future instances of HauntedBus
.
bus4 = HauntedBus()
bus4.passengers
['heather', 'katy']
This behaviour is an example of why it matters whether we think of variables as boxes or labels. If we think that variables are boxes, then the above bevaviour doesn't make sense, since each passenger list would be its own box with its own content. But when we think of variables as labels -- the correct way to think about them in Python -- then the behaviour makes complete sense: each time we instantiate a bus without a custom passenger list, we create a new label -- of the form name-of-bus.passengers
-- for the empty list we created when we loaded or created HauntedBus
.
What to do to avoid the unwanted behaviour? The solution is to create a new empty list each time no list is provided.
class Bus:
def __init__(self, passengers=None):
if passengers is None:
self.passengers = []
else:
self.passengers = list(passengers)
def pick(self, name):
self.passengers.append(name)
def drop(self, name):
self.passengers.remove(name)
bus1 = Bus()
bus1.pick("tim")
bus2 = Bus()
bus2.passengers
[]
The init
method of the above class copies the passed passenger list by calling list(passengers)
. This is critical. If, instead of copying we alias the passed list, we change lists defined outside the function that are passed as arguments, which is probably not what we want.
class Bus:
def __init__(self, passengers=None):
if passengers is None:
self.passengers = []
else:
self.passengers = passengers
def pick(self, name):
self.passengers.append(name)
def drop(self, name):
self.passengers.remove(name)
team = ["hugh", "lisa", "gerd", "adam", "emily"]
bus = Bus(team)
bus.drop("hugh")
team
['lisa', 'gerd', 'adam', 'emily']
Again, the reason for this is that self.passengers
is an alias for passengers
, which is itself an alias for team
, so that all operations we perfom on self.passengers
are actually performed on team
. The identity check below shows what the passengers attribute of bus
is indeed the same object as the team list.
bus.passengers is team
True
To summarise: unless there is a good reason for an exception, for functions that take mutable objects as arguments do the following:
Create a new object each time a class is instantiated by using None as the default parameter, rather than creating the object at the time of the function definition.
Make a copy of the mutable object for processing inside the function to leave the original object unchanged.
A programming paradigm built on the idea of composing programs of functions, rather than sequential steps of execution.
Advantages of functional programming:
Memory and speed considerations:
See here for more.
Mapping is the process of applying a function elementwise to an array and storing the result. Apply the len
function to each item in a
and return a list of lengths.
a = ["a", "ab", "abc", "abcd"]
Procedural approach.
def lengths(items):
result = []
for item in items:
result.append(len(item))
return result
lengths(a)
[1, 2, 3, 4]
List comprehension.
def lengths(items):
return [len(item) for item in items]
lengths(a)
[1, 2, 3, 4]
Functional approach.
def lengths(items):
return list(map(len, items))
lengths(a)
[1, 2, 3, 4]
Filtering is the process of extracting items from an iterable which satisfy a certain condition. Filter all elements of even length from a
.
Procedural approach.
def even_lengths(items):
result = []
for item in items:
if len(item) % 2 == 0:
result.append(item)
return result
even_lengths(a)
['ab', 'abcd']
List comprehension.
def even_lengths(items):
return [item for item in items if len(item) % 2 == 0]
even_lengths(a)
['ab', 'abcd']
Functional approach.
def even_lengths(items):
return list(filter(lambda x: len(x) % 2 == 0, items))
even_lengths(a)
['ab', 'abcd']