\chapter{Functional Programming in Python} 
\label{chap:fun}

Python includes some functional programming features, used heavily by
the ``Pythonistas.''  You should find them useful too.

These features provide concise ways of doing things which, though
certainly doable via more basic constructs, compactify your code and
thus make it easier to write and read.  They may also make your code run
much faster.  Moreover, it may help us avoid bugs, since a lot of the
infracture we'd need to write ourselves, which would be bug-prone, is
automatically taken care of us by the functional programming constructs.

Except for the first feature here (lambda functions), these features
eliminate the need for explicit loops and explicit references to list
elements.  As mentioned in Section \ref{idiom}, this makes for cleaner,
clearer code.

\section{Lambda Functions}
\label{lambda}

{\bf Lambda functions} provide a way of defining short functions.  They
help you avoid cluttering up your code with a lot of definitions of
``one-liner'' functions that are called only once.  For example:

\begin{Verbatim}[fontsize=\relsize{-2}]
>>> g = lambda u:u*u
>>> g(4)
16
\end{Verbatim}

Note carefully that this is NOT a typical usage of lambda functions; it
was only to illustrate the syntax.  Usually a lambda functions would not
be defined in a free-standing manner as above; instead, it would be
defined inside other functions, as seen next.

Here is a more realistic illustration, redoing the sort example from
Section \ref{sorting}:

\begin{Verbatim}[fontsize=\relsize{-2}]
>>> x = [[1,4],[5,2]]
>>> x
[[1, 4], [5, 2]]
>>> x.sort()
>>> x
[[1, 4], [5, 2]]
>>> x.sort(lambda u,v: u[1]-v[1])
>>> x
[[5, 2], [1, 4]]
\end{Verbatim}

A bit of explanation is necessary.  If you look at the online help for
{\bf sort()}, you'll find that the definition to be

\begin{Verbatim}[fontsize=\relsize{-2}]
sort(...)
    L.sort(cmp=None, key=None, reverse=False) -- stable sort *IN PLACE*;
    cmp(x, y) -> -1, 0, 1
\end{Verbatim}

You see that the first argument is a named argument (recall Section
\ref{namedargs}), {\bf cmp}.  That is our compare function, which we
defined above by writing \verb=lambda u,v: u[1]-v[1]=.

At any rate, the point is that using a lambda function above made the
code more compact and readable.

The general form of a lambda function is

\begin{Verbatim}[fontsize=\relsize{-2}]
lambda arg 1, arg 2, ...: expression
\end{Verbatim}

So, multiple arguments are permissible, but the function body itself
must be an expression.  

\section{Mapping}

The {\bf map()} function converts one sequence to another, by applying
the same function to each element of the sequence.  For example:

\begin{Verbatim}[fontsize=\relsize{-2}]
>>> z = map(len,["abc","clouds","rain"])
>>> z
[3, 6, 4]
\end{Verbatim}

So, we have avoided writing an explicit {\bf for} loop, resulting in
code which is a little cleaner, easier to write and read.  In a large
setting, it may give us a good speed increase too.

In the example above we used a built-in function, {\bf len()}.  We could
also use our own functions; frequently these are conveniently expressed
as lambda functions, e.g.:

\begin{Verbatim}[fontsize=\relsize{-2}]
>>> x = [1,2,3]
>>> y = map(lambda z: z*z, x)
>>> y
[1, 4, 9]
\end{Verbatim}

The condition that a lambda function's body consist only of an
expression is rather limiting, for instance not allowing if-then-else
constructs.  If you really wish to have the latter, you could use a
workaround.  For example, to implement something like

\begin{Verbatim}[fontsize=\relsize{-2}]
if u > 2: u = 5
\end{Verbatim}

we could work as follows:

\begin{Verbatim}[fontsize=\relsize{-2}]
>>> x = [1,2,3]
>>> g = lambda u: (u > 2) * 5 + (u <= 2) * u
>>> map(g,x)
[1, 2, 5]
\end{Verbatim}

Or, a little fancier:

\begin{Verbatim}[fontsize=\relsize{-2}]
>>> g = lambda u: (u > 2) * 5 or u
>>> map(g,x)
[1, 2, 5]
\end{Verbatim}

Clearly, this is not feasible except for simple situations.  For more
complex cases, we would use a non-lambda function.  

You can use {\bf map()} with more than one data argument. For instance
: 
\begin{Verbatim}[fontsize=\relsize{-2},numbers=left]
>>> map(max,(5,12,13),range(7,10))
[7, 12, 13]
\end{Verbatim}

Here I used Python's built-in {\bf max()} function, but of course you
can write your own functions to use in {\bf map()}. And the function
need not be scalar-valued. For example:

\begin{Verbatim}[fontsize=\relsize{-2},numbers=left]
>>> map(lambda u,v: (min(u,v),max(u,v)),(5,12,13),range(7,10))
[(5, 7), (8, 12), (9, 13)]
\end{Verbatim}

\section{Filtering}

The {\bf filter()} function works like {\bf map()}, except that it culls
out the sequence elements which satisfy a certain condition.  The
function which {\bf filter()} is applied to must be boolean-valued, i.e.
return the desired true or false value.  For example:

\begin{Verbatim}[fontsize=\relsize{-2}]
>>> x = [5,12,-2,13]
>>> y = filter(lambda z: z > 0, x)
>>> y
[5, 12, 13]
\end{Verbatim}

Again, this allows us to avoid writing a {\bf for} loop and an {\bf if}
statement.

\section{Reduction}

The {\bf reduce()} function is used for applying the sum or other
arithmetic-like operation to a list.  For example,

\begin{Verbatim}[fontsize=\relsize{-2}]
>>> x = reduce(lambda x,y: x+y, range(5))
>>> x
10
\end{Verbatim}

Here {\bf range(5)} is of course [0,1,2,3,4].  What {\bf reduce()} does
is it first adds the first two elements of [0,1,2,3,4], i.e. with 0
playing the role of {\bf x} and 1 playing the role of {\bf y}.  That
gives a sum of 1.  Then that sum, 1, plays the role of {\bf x} and 
the next element of [0,1,2,3,4], 2, plays the role of {\bf y}, yielding
a sum of 3, etc.  Eventually {\bf reduce()} finishes its work and
returns a value of 10.

Once again, this allowed us to avoid a {\bf for} loop, plus a statement
in which we initialize {\bf x} to 0 before the {\bf for} loop.

The {\bf reduce()} function has an optional third argument, which is the
intial value to be used in the reduction process.  By default 0 is used
(which might be, say, False in a boolean addition).

\section{List Comprehension}
\label{listcomps}

This allows you to compactify a {\bf for} loop that produces a list.
For example:

\begin{Verbatim}[fontsize=\relsize{-2}]
>>> x = [(1,-1), (12,5), (8,16)]
>>> y = [(v,u) for (u,v) in x]
>>> y
[(-1, 1), (5, 12), (16, 8)]
\end{Verbatim}

This is more compact than first initializing {\bf y} to [], then having
a {\bf for} loop in which we call {\bf y.append()}.

It gets even more compact when done in nested form.  Say for instance we
have a list of lists which we want to concatenate together, ignoring the
first element in each.  Here's how we could do it using list
comprehensions:

\begin{Verbatim}[fontsize=\relsize{-2}]
>>> y
[[0, 2, 22], [1, 5, 12], [2, 3, 33]]
>>> [a for b in y for a in b[1:]]
[2, 22, 5, 12, 3, 33]
\end{Verbatim}

Here is pseudocode for what is going on:

\begin{Verbatim}[fontsize=\relsize{-2}]
for b = [0, 2, 22],  [1, 5, 12],  [2, 3, 33]
   for a in b[1:]
      emit a
\end{Verbatim}

Is that compactness worth the loss of readability?  Only you can decide.
It is amusing that the official Python documentation says, ``If you’ve
got the stomach for it, list comprehensions can be nested''
(\url{http://docs.python.org/tutorial/datastructures.html}).

\section{Example:  Textfile Class Revisited}

Here is the text file example from Section \ref{tfe} again, now redone
with functional programming features:

\begin{lstlisting}[numbers=left]
class textfile:
   ntfiles = 0  # count of number of textfile objects
   def __init__(self,fname):
      textfile.ntfiles += 1
      self.name = fname  # name 
      self.fh = open(fname)  # handle for the file
      self.lines = self.fh.readlines()
      self.nlines = len(self.lines)  # number of lines 
      self.nwords = 0  # number of words
      self.wordcount()

   def wordcount(self):
      "finds the number of words in the file"
      self.nwords = \
         reduce(lambda x,y: x+y, \
            map(lambda line: len(line.split()), \
                  self.lines))
   def grep(self,target):
      "prints out all lines containing target"
      lines = filter(lambda line: line.find(target) >= 0,self.lines)
      print lines

a = textfile('x')
b = textfile('y')
print "the number of text files open is", textfile.ntfiles
print "here is some information about them (name, lines, words):"
for f in [a,b]:
   print f.name,f.nlines,f.nwords
a.grep('bc')
\end{lstlisting}

\section{Example:  Prime Factorization}

The function {\bf primefact()} below finds the prime factorization of
number, relative to the given primes.  For example, the call {\bf
primefact([2,3,5,7,11],24)} would return [ 2,3], [3,1] ], meaning that
$24 = 2^3 3^1$.  (It is assumed that the prime factorization of {\bf n}
does indeed exist for the numbers in {\bf primes}.)

\begin{lstlisting}[numbers=left]
def dividetomax(p,m):
   k = 0
   while True:
      if m % p != 0: return (p,k)
      k += 1
      m /= p

def primefact(primes,n):
   def divmax(p): return dividetomax(p,n)  # note: n in scope
   tmp = map(divmax,primes)
   tmp = filter(lambda u: u[1] > 0,tmp) # a factor m^0 = 1 not of interest
   return tmp
\end{lstlisting}

\section{Example: a Function to Combine the Map and Filter Operations}

Say we wish to computer a list consisting of double the positive values
in a numeric list.  We could apply \textbf{filter()} to cull out the
positive ones, then call \textbf{map()}.  But that would mean two passes
at the data, inefficient if the list is very large.

It would thus be nice to have a function that combines the two
operations.  Below is code to implement this.  Of course, it would need
to be done in C for real speed.

\begin{lstlisting}
def filterMap(f,g,x):
   y = []
   for xi in x:
      if f(xi):
         y.append(g(xi))
   return y

# example
u = [5,-12,13,-8]
filterMap(lambda x: x > 0,lambda x: 2*x,u)
\end{lstlisting}

\textbf{reduce()}) to total those numbers