Dive into Functions

Dive into Functions

Functions are a way to package functionalities. There are 4 kind of functions in Python:

  • global functions
  • local functions
  • lambda functions
  • methods

Global functions are created with the keyword def and take a name and an optional list of parameters. They are accessible to any code in the same module in which it is created. They can also be accessible from other modules with a mechanism of import.

Local are created as globals functions but defined inside other functions (also called nested functions). These functions are visible only to the function where they are defined.

Lambda functions are expressions, so they can be created at their point of use. however they are much more limited than normal functions.

Methods are functions that are bound to a particular data type and can be used only in conjunction with this data type.

global functions, local functions and method are created with the keyword def and return a value. To return a value we explicitly use the keyword return if we do not do that None is return automatically by python. We can leave a function at any point by using the return statement (the yield can be used also but will not cover here). We can call functions by appending parenthesis to the function name.

>>> def global_func():
      return "global_func is a global function"

>>> print global_func()
   "global_func is a global function"

Names and Docstrings

see Names and Docstrings

functions are objects

see functions are objects

nested functions

It is useful to have helper function inside a function. To do this we simply define a function inside the definition of an existing function. Such function are often called nested functions or locals functions.

def outer():
    x = 1
    def inner():
        return 2
    return x + inner()

outer()
3

function argument vs parameters

These two terms parameter and argument are sometimes loosely used interchangeably, and the context is used to distinguish the meaning. The term parameter (sometimes called formal parameter) is often used to refer to the variable as found in the function definition, while argument (sometimes called actual parameter) refers to the actual value passed. To avoid confusion, it is common to view a parameter as a variable, and an argument as a value. Python allow us to pass arguments to functions. The parameter names become local variable of our function [parameters_and_arguments]. If there is more parameters than one, they are written as a sequence of comma separated identifiers, or as sequence of identifier = value pair. For instance, here is a function that calculates the area of a triangle using Heron’s formula:

def heron(a, b, c):
   s = (a + b + c) / 2
   return math.sqrt(s * (s - a) * (s - b) * (s - c))

Inside the function each parameter, a, b, c, is initialized with the corresponding value that was passed as an argument. When the function is called, we must supply all arguments, for example, heron(3, 4, 5). If we give too few or too many arguments, a TypeError exception will be raised.

When we do a call like this we said to be using a positional arguments, because each argument passed is set as the value of parameter in the corresponding position. So in this case, a is set to 3, b to 4, and c to 5, when the function is called.

Some functions have parameters for which there can be sensible default.

Arguments and Parameters

Python has different ways to define function parameters and pass arguments to them. Function parameters can be either

  • positional parameters that are mandatory or named,
  • keyword parameters that provide a default value.

The parameter syntax does not permit us to follow parameters with default value with parameters that don’t have default value. So def bad(a, b = 1, c) wont work. We are not force to pass our arguments in the order they appear in the function’s definition, instead, we can use keyword arguments, passing each argument in the form name = value.

def argument(a, b, c = 3, d = 4):
   print '{} {} {} {}'.format(a,b,c,d)

argument(1,2)
1 2 3 4

argument(a = 1, b = 2)
1 2 3 4

argument(b = 2, a = 1)
1 2 3 4

Warning

When default values are given they are created at the time of the def statement is executed (i.e. when the function is created), not when the function is called. For immutable arguments like numbers or strings this doesn’t make any difference, but for mutable arguments a subtle trap is lurking.

def app(x, lst = []):
    #id print the memory adress of the object
    print( id(lst) )
    lst.append(x)
    return lst

#the default value of the function app is an empty list
app.func_defaults
 ([],)


app(1)
 #the memory adress of lst is 21665048
 21665048
[1]

app.func_defaults
([1],)
# now the default value of the app function is list [1]
# the first call to app have a side effect

app(2)
 # the memory adress does not change (this is the same object than at the first call
 # the list was created at app function create time.
 21665048
[1, 2]

app.func_defaults
([1, 2],)

Here at the list lst was created at function creation time, at each call Python reuse the same list to add new element. This induce a big and dangerous side effect, and usually it’s not the desired behavior. Here a new version without side effect:

def app(x, lst = None):
   if lst is None:
     lst = []
   #id print the memory adress of the object
   print( id(lst) )
   lst.append(x)
   return lst

Variable number of parameters

A function can take additional optional arguments by prefixing the last parameter with an * (asterix). Optional arguments are then available in the tuple referenced by this parameter. Optional variables can also by passed as keywords, if the last parameter is preceded by **. In this case, the optional variables are available within the function as a dictionary. The operation consisting to get the arguments passed as sequence is call argument unpacking. Let look how it work, especially there are significant differences between python 2 and 3.

Sequence unpacking

We can unpack any iterables (list, tuples, …) with the operator *. When used with two or more variables on the left-hand side of an assignment, one of which is preceded by *, items are assigned to the variables, with all those left over assigned to the stared variables.

>>> first, *rest = [1,2,3,4]
>>> first
1
>>> rest
[2, 3, 4]
>>>
>>> first, *mid, last = [1,2,3,4]
>>> first
1
>>> mid
[2, 3]
>>> last
4

Note

The unpacking operator does not exist in Python 2

Argument unpacking

Unpacking list, tuple

We can use the sequence unpacking operator in a function’s parameter list. | This useful when we want to create functions that can take a variable number of positional arguments. Here a product() function [prog_in_python3] .

>>> def product(*args):                                                                                                                                                                   |
...     result = 1                                                                                                                                                                        |
...     for arg in args:                                                                                                                                                                  |
...             result *= arg                                                                                                                                                             |
...     return result                                                                                                                                                                     |
...                                                                                                                                                                                       |
>>> product(1, 2, 3, 4)                                                                                                                                                                   |
24                                                                                                                                                                                        |
>>>                                                                                                                                                                                       |
>>> product(2, 3)                                                                                                                                                                         |
6                                                                                                                                                                                         |

Python3 support keywords arguments following positional arguments, even if it’s an unpacking sequence argument.

>>> def func( *arg, arg2 = None):
...     print(arg)
...     print(arg2)
...
>>> func([1,2,3])
([1, 2, 3],)
None
>>>
>>> func([1,2,3] , arg2='a')
([1, 2, 3],)
a
Unpacking mapping

Just as we can unpack a sequence to populate a function’s positional arguments, we can unpack a mapping using the mapping unpacking operator ** . We can use ** to pass a dictionary to a argument.

Here the options dictionary’s key-value pairs are unpackecd with each key’s value being assigned to the parameter whose name is the same as the key. If the dictionnary contains a key for wich there is no corresponding parameter, a TypeError is raised. Any argument for which the dictionary has no corresponding item is set at this default value.

>>> def func(a = 2, b = 3):
...     print(a, b)
...
>>> func(**{'a':4,'b':5})

>>>
>>> func(**{'a':4,'c':5})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: func() got an unexpected keyword argument 'c'
>>>
>>> func(**{'a':4})
>>>

We can also use mapping unpacking operator with parameter.

In this case, the ** operator must be the last argument.

>>> def func(a = 2, b = 3,**kwargs):
...     print(a)
...     print(b)
...     print(kwargs)
...
>>> def func(a = 2, b = 3, **kwargs, d = 4):
File "<stdin>", line 1
def func(a = 2, b = 3, **kwargs, d = 4):
^
SyntaxError: invalid syntax
>>>
>>> def func(*arg, **kwarg):
...     print(arg)
...     print(kwarg)
...
>>> func(1, 2, 3)
arg = (1, 2, 3)
kwatg = {}
>>>
>>> func([1, 2, 3], a= 'A', b = 'B')
arg = ([1, 2, 3],)
kwarg = {'a': 'A', 'b': 'B'}
>>>
>>> func([1, 2, 3],{'a':'A', 'b':'B'})
arg = ([1, 2, 3], {'a': 'A', 'b': 'B'})
kwarg = {}
>>> l = [1, 2, 3]
>>> d = {'a':'A', 'b':'B'}
>>> func(*l, **d)
arg = (1, 2, 3)
kwarg = {'a': 'A', 'b': 'B'}

Note

In python2 the sequence unpacking oeprator can be used in functions parameter’s list But we cannot have arguments after unpacking sequence.

>>> def func(*arg, arg2 = None):                                                                                     |                                                                    |
File "<stdin>", line 1                                                                                               | >>> def func( *arg, arg2 = None):                                  |
def func(*arg, arg2 = None):                                                                                         | ...     print(arg)                                                 |
                 ^                                                                                                    | ...     print(arg2)                                                |
SyntaxError: invalid syntax

Scope of variables

For variables, Python has function scope, module scope, and global scope (in python the term of namespaces is often used) [Franklin]. Names enter scope at the start of a context (function, module, or globally), and exit scope when a non-nested function is called or the context ends.

If a name is used prior to variable initialization, this raises a syntax error.

Variable resolution rules

Although scopes are determined statically, they are used dynamically. At any time during execution, there are at least three nested scopes whose namespaces are directly accessible:

  1. the innermost scope, which is searched first, contains the local names
  2. the scopes of any enclosing functions, which are searched starting with the nearest enclosing scope, contains non-local, but also non-global names
  3. the next-to-last scope contains the current module’s global names
  4. the outermost scope (searched last) is the namespace containing built-in name

If a variable is simply accessed (not assigned to) in a context, name resolution follows the LEGB rule (Local, Enclosing, Global, Built-in). However, if a variable is assigned to, it defaults to creating a local variable, which is in scope for the entire context. Both these rules can be overridden with a global or nonlocal (nonlocal does not exists in Python 2) declaration prior to use, which allows accessing global variables even if there is an intervening nonlocal variable, and assigning to global or nonlocal variables [scope] .

functions are object
G = 4
I = 12
def func(p):
   I = 5
   res = p + I - G
   return res
We first defined 2 objects references G and I which refer respectively to integers 4 and 12 then we create a new object reference func which refer to the function code (I remember you that in Python everything is an object)
_images/spacer.png
functions are object
y = func(3)
  1. When we call the function func with argument 3, Python create a namespace local to the function,
    with a first reference object “p” which refer to an integer object with the value 3.
  2. Then the code of the function is executed, a variable “I” is assigned to, so Python creates a new local reference.
  3. I show with small arrows how python resolve the variables to compute the statement
  4. then a reference “res” is created which point to the result of the statement “p + I - G”
_images/spacer.png
functions are object
  1. a new reference call “y” to the integer object 4 is created in the global namespace.
  2. the local namespace relative to the function execution is tagged to be removed by the garbage collector. As the int object with 4 as value have another reference (y) it will not be destroyed.
_images/spacer.png

We can see this mechanism in action as in Python we can view the content of the local the global namespace via two built-in functions locals and globals

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
   def outer_func():
      x = 'outer'
      print('outer locals = ', locals())
      print(x)

      def inner_func():
         nonlocal x
         print('inner locals = ', locals())
         x = 'inner'
         print('inner locals = ', locals())
         print(x)

      inner_func()
      print('outer locals = ', locals())

   outer_func()
when we use the nonlocals keywords the variable find in the outer scope is seen as it belong to the local scope. We can manipulate it as a local variable. If we reassign a new value to this reference, the outer reference is also modified.

Note

This piece of code illustrate the globals and locals namespaces. Although this code is writen in python3 the concepts are the same in python2. But the keywords nonlocals is python3 specific. In python2, we can refer to a non local variable, but we cannot assign a new value to a non local variable, when we try to assign a new value, a new local object reference is created.

namespaces in python 3
3. outer locals = {‘x’: ‘outer’}
4. outer
7. inner locals = {‘x’: ‘outer’}
9. inner locals = {‘x’: ‘inner’}
10. inner
12. outer locals = {‘x’: ‘inner’, ‘inner_func’: <function outer_func.<locals>.inner_func at 0x7f19d8d965f0>}
_images/spacer.png

Variable lifetime

It’s also important to note that not only do variables live inside a namespace, they also have lifetimes. Consider

>>> def foo():
...     x = 1
>>> foo()
>>> print(x) # 1
Traceback (most recent call last):
  ...
NameError: name 'x' is not defined

It isn’t just scope rules at point #1 that cause a problem (although that’s why we have a NameError) it also has to do with how function calls are implemented in Python and many other languages. There isn’t any syntax we can use to get the value of the variable x at this point - it literally doesn’t exist! The namespace created for our function foo is created from scratch each time the function is called and it is destroyed when the function ends [Franklin].

Lambda functions

In addition the def statement, Python also provides an expression form that generates function objects. The lambda’s syntax is keyword lambda, followed by one or more arguments (exactly like the arguments list you enclose in parentheses in a def header), followed by an expression after a colon:

lambda argument1, argument2,… argumentN : expression using arguments

Function objects returned by running lambda expressions work exactly the same as those created and assigned by defs, but there are a few differences that make lambdas useful in specialized roles:

  • lambda is an expression, not a statement. Then a lambda can appear in places a def is not allowed by Python’s syntax:
    • inside a list literal
    • or a function call’s arguments, for example. As an expression, lambda returns a value (a new function) that can optionally be assigned a name. In contrast, the def statement always assigns the new function to the name in the header, instead of returning it as a result.
  • lambda’s body is a single expression, not a block of statements.
    The lambda’s body is similar to what you’d put in a def body’s return statement; you simply type the result as a naked expression, instead of explicitly returning it. Because it is limited to an expression, a lambda is less general than a def you can only squeeze so much logic into a lambda body without using statements such as if. This is by design, to limit program nesting: lambda is designed for coding simple functions, and def handles larger tasks.

Apart from those distinctions, defs and lambdas do the same sort of work:

>>> def func(x, y, z): return x + y + z
...
>>> func(2, 3, 4)
9

>>> f = lambda x, y, z: x + y + z
>>> f(2, 3, 4)
9

We can use default arguments or tuple or dict argument like *args or **kwargs exactly as in def functions. The code in a lambda body also follows the same scope lookup rules (LGEB) as code inside a def. lambda expressions introduce a local scope much like a nested def, which automatically sees names in enclosing functions, the module, and the built-in scope.

Why Use lambda?

Generally speaking, lambdas come in handy as a sort of function shorthand that allows you to embed a function’s definition within the code that uses it. They are entirely optional (you can always use defs instead), but they tend to be simpler coding constructs in scenarios where you just need to embed small bits of executable code.

For instance, it will very often use in function that take a function as parameter, as sort, filter, …

from collections import namedtuple
Sequence = namedtuple("Sequence", "id comment sequence")
sequences = [
   Sequence('abcd3_rat', '',
            'MAAFSKYLTARNSSLAGAAFLLFCLLHKRRRALGLHGKKSGKPPLQNNEKEGKKERAVVDKVFLSRLSQILKI'),
   Sequence('il2_human_matured', 'matured sequence of il2_human',
            'APTSSSTKKTQLQLEHLLLDLQMILNGINNYKNPKLTRMLTFKFYMPKKATELKHLQCLEEELKPLEEV'),
   Sequence('il2_human', '',
            'MYRMQLLSCIALSLALVTNSAPTSSSTKKTQLQLEHLLLDLQMILNGINNYKNPKLTRMLTFKFYMPKKATELKHLQCLE'),
   Sequence('TRYP_PIG', '' ,
            'FPTDDDDKIVGGYTCAANSIPYQVSLNSGSHFCGGSLINSQWVVSAAHCYKSRIQVRLGE')]

filter(lambda seq: seq.sequence.startswith('M'), sequences)

[Sequence(id='il2_human', comment='',
         sequence='MYRMQLLSCIALSLALVTNSAPTSSSTKKTQLQLEHLLLDLQMILNGINNYKNPKLTRMLTFKFYMPKKATELKHLQCLE'),
 Sequence(id='abcd3_rat', comment='',
         sequence='MAAFSKYLTARNSSLAGAAFLLFCLLHKRRRALGLHGKKSGKPPLQNNEKEGKKERAVVDKVFLSRLSQILKIMVPRTFC')]

sequences.sort(lambda seq1, seq2: len(seq2.sequence)- len(seq1.sequence))

[ (seq.id, len(seq.sequence)) for seq in sequences]
[('il2_human', 80), ('abcd3_rat', 73), ('il2_human_matured', 69), ('TRYP_PIG', 60)]

Without due care, they can lead to unreadable (a.k.a. obfuscated) Python code. In general, simple is better than complex, explicit is better than implicit, and full statements are better than arcane expressions. That’s why lambda is limited to expressions. If you have larger logic to code, use def; lambda is for small pieces of inline code. On the other hand, you may find these techniques useful in moderation.

Exercises

Exercise

Without executing the code in Python interpreter, can you determine what the code above print out. help you by drawing diagram.

Hint locals print a dictionary with local variable as keys and their respective values.

x = 4

def func():
   y  = 5
   print(locals())

func()
print(x)

Exercise

Without executing the code in Python interpreter, can you determine what the code above print out. help you by drawing diagram.

Hint locals print a dictionary with local variable as keys and their respective values.

x = 4

def func():
   y = 5
   x = 8
   print(locals())
   x = x + 2

y = func()

print(y)
print(x)

Exercise

Without executing the code in Python interpreter, can you determine what the code above print out. help you by drawing diagram.

Hint locals print a dictionary with local variable as keys and their respective values.

x = 4

def func(a):
   y = x + 2
   print(locals())
   x = y
   return y

y = func(x)

print(y)
print(y == x)

Exercise

Without executing the code in Python interpreter, can you determine what the code above print out. help you by drawing diagram.

Hint locals print a dictionary with local variable as keys and their respective values.

x = 4

def func(a):
   x = x + 2
   print(locals())
   return x

y = func(x)

print(y)
print(y == x)

Exercise

Without executing the code in Python interpreter, can you determine what the code above print out. help you by drawing diagram.

x = 4

def func(x):
   x = x + 2
   return x

y = func(x)

print(x)
print(y)

Exercise

Without executing the code in Python interpreter, can you determine what the code above print out. help you by drawing diagram.

def func():
   y = x
   return y

x = 4
z = func()

print(x)
print(z)
print(id(z) == id(x))

Exercise

Without executing the code in Python interpreter, can you determine what the code above print out. help you by drawing diagram.

x = 4

def func(x = 5):
   x = x + 2
   return x

y = func()

print(x)
print(y)

Exercise

Without executing the code in Python interpreter, can you determine what the code above print out. help you by drawing diagram.

Hint locals print a dictionary with local variable as keys and their respective values.

x = 4

def func(a):
   global x
   def func2():
      print(locals())
      y = x + 4
      return y
   z = func2()
   return z

y = func(x)

print(x)
print(y)

Exercise

Without executing the code in Python interpreter, can you determine what the code above print out. help you by drawing diagram.

x = {'a' : 4}

def func(a):
   a['b'] = 5
   return a

y = func(x)

print(x)
print(y)
print(x is y)

Exercise

Without executing the code in Python interpreter, can you determine what the code above print out. help you by drawing diagram.

x = {'a' : 4}

def func(a):
   a['b'] = 5

y = func(x)

print(x)
print(y)

Exercise

Without executing the code in Python interpreter, can you determine what the code above print out. help you by drawing diagram.

x = {'a' : 4}

def func(a):
   x['b'] = 5
   def func2():
      a['b'] = 6
   return a

y = func(x)

print(x)
print(y)

Exercise

Without executing the code in Python interpreter, can you determine what the code below print out. help you by drawing diagram.

x = {'a' : 4}

def func(a):
   x['b'] = 5
   def func2():
      a['b'] = 6
   func2()
   return a

y = func(x)

print(x)

Exercise

Without executing the code in Python interpreter, can you determine what the code below print out. help you by drawing diagram.

x = {'a' : 4}

def func(a):
   x['b'] = 5
   def func2(x):
      x['b'] = 6
   func2(a.copy())
   return a

y = func(x)
print(x)

Exercise

Write a function translate that have a nucleic sequence as parameter, and return the translated sequence. We give you a genetic code :

code = {  'ttt': 'F', 'tct': 'S', 'tat': 'Y', 'tgt': 'C',
           'ttc': 'F', 'tcc': 'S', 'tac': 'Y', 'tgc': 'C',
           'tta': 'L', 'tca': 'S', 'taa': '*', 'tga': '*',
           'ttg': 'L', 'tcg': 'S', 'tag': '*', 'tgg': 'W',
           'ctt': 'L', 'cct': 'P', 'cat': 'H', 'cgt': 'R',
           'ctc': 'L', 'ccc': 'P', 'cac': 'H', 'cgc': 'R',
           'cta': 'L', 'cca': 'P', 'caa': 'Q', 'cga': 'R',
           'ctg': 'L', 'ccg': 'P', 'cag': 'Q', 'cgg': 'R',
           'att': 'I', 'act': 'T', 'aat': 'N', 'agt': 'S',
           'atc': 'I', 'acc': 'T', 'aac': 'N', 'agc': 'S',
           'ata': 'I', 'aca': 'T', 'aaa': 'K', 'aga': 'R',
           'atg': 'M', 'acg': 'T', 'aag': 'K', 'agg': 'R',
           'gtt': 'V', 'gct': 'A', 'gat': 'D', 'ggt': 'G',
           'gtc': 'V', 'gcc': 'A', 'gac': 'D', 'ggc': 'G',
           'gta': 'V', 'gca': 'A', 'gaa': 'E', 'gga': 'G',
           'gtg': 'V', 'gcg': 'A', 'gag': 'E', 'ggg': 'G'
      }

bonus

This function can to take the phase as parameter

Exercise

In a new file named matrix.py Implement a matrix and functions to handle it. choose the data structure of your choice. The API (Application Programming Interface) to implement is the following:

maker

have parameter:

  • the number of rows
  • the number of columns
  • a default value to fill the matrix

and return a matrix of rows_num x col_num

size

have parameter:

  • a matrix

return the number of rows, and number of columns

get_cell

have parameter:

  • a matrix
  • the number of rows
  • the number of columns

the content of cell corresponding to row numberx col number

set_cell

have parameter:

  • a matrix
  • the row number of cell to set
  • the column number of cell to set
  • the value to set in cell

set the value val in cell sepcified by row number x column number

to_str

have parameter:

  • a matrix

return a string representation of the matrix

mult

have parameter:

  • a matrix
  • val the value to multiply the matrix with

return a new matrix which is the scalar product of matrix x val

get_row

have parameter:

  • a matrix
  • the number of rows

return a copy of the row corresponding to row number

set_row

have parameter:

  • a matrix
  • the row number
  • the value to put in cells of the row

set value in each cells of the row specify by the row number

get_col

have parameter:

  • a matrix
  • the column number

return a copy of the column corresponding to the column number

set_col

have parameter:

  • a matrix
  • the column number
  • the value to put in cells

set all cells of a column with value

replace_col

have parameter:

  • a matrix
  • the column number to replace
  • the list of values to use as replacement of column

replace a column col_no with list of values

replace_row

have parameter:

  • a matrix
  • the row number to replace
  • the list of values to use as replacement of row

replace a row row_no with list of values