\chapter{Introduction} \label{chap:intro} So, let's get started with programming right away. \section{A 5-Minute Introductory Example} \subsection{Example Program Code} \label{veryfirst} Here is a simple, quick example. Suppose I wish to find the value of $$ g(x) = \frac{x}{1-x^2} $$ for x = 0.0, 0.1, ..., 0.9. I could find these numbers by placing the following code, \begin{samepage} \begin{Verbatim}[fontsize=\relsize{-2}] for i in range(10): x = 0.1*i print x print x/(1-x*x) \end{Verbatim} \end{samepage} in a file, say {\bf fme.py}, and then running the program by typing \begin{Verbatim}[fontsize=\relsize{-2}] python fme.py \end{Verbatim} at the command-line prompt. The output will look like this: \begin{Verbatim}[fontsize=\relsize{-2}] 0.0 0.0 0.1 0.10101010101 0.2 0.208333333333 0.3 0.32967032967 0.4 0.47619047619 0.5 0.666666666667 0.6 0.9375 0.7 1.37254901961 0.8 2.22222222222 0.9 4.73684210526 \end{Verbatim} \subsection{Python Lists} How does the program work? First, Python's {\bf range()} function is an example of the use of {\bf lists}, i.e. Python arrays,\footnote{I loosely speak of them as ``arrays'' here, but as you will see, they are more flexible than arrays in C/C++. On the other hand, true arrays can be accessed more quickly. In C/C++, the $i^{th}$ element of an array {\bf X} is i words past the beginning of the array, so we can go right to it. This is not possible with Python lists, so the latter are slower to access. The NumPy add-on package for Python offers true arrays.} even though not quite explicitly. Lists are absolutely fundamental to Python, so watch out in what follows for instances of the word ``list''; resist the temptation to treat it as the English word ``list,'' instead always thinking about the Python construct {\bf list}. Python's {\bf range()} function returns a list of consecutive integers, in this case the list [0,1,2,3,4,5,6,7,8,9]. Note that this is official Python notation for lists---a sequence of objects (these could be all kinds of things, not necessarily numbers), separated by commas and enclosed by brackets. \subsection{Loops} So, the {\bf for} statement above is equivalent to: \begin{Verbatim}[fontsize=\relsize{-2}] for i in [0,1,2,3,4,5,6,7,8,9]: \end{Verbatim} As you can guess, this will result in 10 iterations of the loop, with {\bf i} first being 0, then 1, etc. The code \begin{Verbatim}[fontsize=\relsize{-2}] for i in [2,3,6]: \end{Verbatim} would give us three iterations, with {\bf i} taking on the values 2, 3 and 6. Python has a {\bf while} construct too (though not an {\bf until}). There is also a {\bf break} statement like that of C/C++, used to leave loops ``prematurely.'' For example: \begin{Verbatim}[fontsize=\relsize{-2}] x = 5 while 1: x += 1 if x == 8: print x break \end{Verbatim} Also very useful is the {\bf continue} statement, which instructs the Python interpreter to skip the remainder of the current iteration of a loop. For instance, running the code \begin{Verbatim}[fontsize=\relsize{-2}] sum = 0 for i in [5,12,13]: if i < 10: continue sum += i print sum \end{Verbatim} prints out 12+13, i.e. 25. The {\bf pass} statement is a ``no-op,'' doing nothing. \subsection{Python Block Definition} Now focus your attention on that innocuous-looking colon at the end of the {\bf for} line above, which defines the start of a block. Unlike languages like C/C++ or even Perl, which use braces to define blocks, Python uses a combination of a colon and indenting to define a block. I am using the colon to say to the Python interpreter, \begin{quote} Hi, Python interpreter, how are you? I just wanted to let you know, by inserting this colon, that a block begins on the next line. I've indented that line, and the two lines following it, further right than the current line, in order to tell you those three lines form a block. \end{quote} I chose 3-space indenting, but the amount wouldn't matter as long as I am consistent. If for example I were to write\footnote{Here {\bf g()} is a function I defined earlier, not shown.} \begin{Verbatim}[fontsize=\relsize{-2}] for i in range(10): print 0.1*i print g(0.1*i) \end{Verbatim} the Python interpreter would give me an error message, telling me that I have a syntax error.\footnote{Keep this in mind. New Python users are often baffled by a syntax error arising in this situation.} I am only allowed to indent further-right within a given block if I have a sub-block within that block, e.g. \begin{Verbatim}[fontsize=\relsize{-2}] for i in range(10): if i%2 == 1: print 0.1*i print g(0.1*i) \end{Verbatim} Here I am printing out only the cases in which the variable {\bf i} is an odd number; \% is the ``mod'' operator as in C/C++. Again, note the colon at the end of the {\bf if} line, and the fact that the two {\bf print} lines are indented further right than the {\bf if} line. Note also that, again unlike C/C++/Perl, there are no semicolons at the end of Python source code statements. A new line means a new statement. If you need a very long line, you can use the backslash character for continuation, e.g. \begin{Verbatim}[fontsize=\relsize{-2}] x = y + \ z \end{Verbatim} Most of the usual C operators are in Python, including the relational ones such as the {\bf ==} seen here. The {\bf 0x} notation for hex is there, as is the FORTRAN {\bf **} for exponentiation. Also, the {\bf if} construct can be paired with {\bf else} as usual, and you can abbreviate {\bf else if} as {\bf elif}. \begin{Verbatim}[fontsize=\relsize{-2}] >> def f(x): ... if x > 0: return 1 ... else: return 0 ... >>> f(2) 1 >>> f(-1) 0 \end{Verbatim} The boolean operators are {\bf and}, {\bf or} and {\bf not}. In addition to the constants {\bf True} and {\bf False}, these are also represented numerically by nonzero and zero values, respectively. The value {\bf None} is treated as {\bf False}. You'll see examples as we move along. By the way, watch out for Python statements like {\bf print a or b or c}, in which the first true (i.e. nonzero) expression is printed and the others ignored; this is a common Python idiom. \subsection{Python Also Offers an Interactive Mode} \label{interactive} A really nice feature of Python is its ability to run in interactive mode. You usually won't do this, but it's a great way to do a quick tryout of some feature, to really see how it works. Whenever you're not sure whether something works, your motto should be, ``When in doubt, try it out!'', and interactive mode makes this quick and easy. We'll also be doing a lot of that in this tutorial, with interactive mode being an easy way to do a quick illustration of a feature. Instead of executing this program from the command line in {\bf batch} mode as we did above, we could enter and run the code in {\bf interactive} mode: \begin{Verbatim}[fontsize=\relsize{-2}] % python >>> for i in range(10): ... x = 0.1*i ... print x ... print x/(1-x*x) ... 0.0 0.0 0.1 0.10101010101 0.2 0.208333333333 0.3 0.32967032967 0.4 0.47619047619 0.5 0.666666666667 0.6 0.9375 0.7 1.37254901961 0.8 2.22222222222 0.9 4.73684210526 >>> \end{Verbatim} Here I started Python, and it gave me its $>>>$ interactive prompt. Then I just started typing in the code, line by line. Whenever I was inside a block, it gave me a special prompt, ``...'', for that purpose. When I typed a blank line at the end of my code, the Python interpreter realized I was done, and ran the code.\footnote{Interactive mode allows us to execute only single Python statements or evaluate single Python expressions. In our case here, we typed in and executed a single {\bf for} statement. Interactive mode is not designed for us to type in an entire program. Technically we could work around this by beginning with something like "if 1:", making our program one large {\bf if} statement, but of course it would not be convenient to type in a long program anyway.} While in interactive mode, one can go up and down the command history by using the arrow keys, thus saving typing. To exit interactive Python, hit ctrl-d. {\bf Automatic printing:} By the way, in interactive mode, just referencing or producing an object, or even an expression, without assigning it, will cause its value to print out, even without a {\bf print} statement. For example: \begin{Verbatim}[fontsize=\relsize{-2}] >>> for i in range(4): ... 3*i ... 0 3 6 9 \end{Verbatim} Again, this is true for general objects, not just expressions, e.g.: \begin{Verbatim}[fontsize=\relsize{-2}] >>> open('x') \end{Verbatim} Here we opened the file {\bf x}, which produces a file object. Since we did not assign to a variable, say {\bf f}, for reference later in the code, i.e. we did not do the more typical \begin{Verbatim}[fontsize=\relsize{-2}] f = open('x') \end{Verbatim} the object was printed out. We'd get that same information this way: \begin{Verbatim}[fontsize=\relsize{-2}] >>> f = open('x') >>> f \end{Verbatim} \subsection{Python As a Calculator} Among other things, this means you can use Python as a quick calculator (which I do a lot). If for example I needed to know what 5\% above \$88.88 is, I could type \begin{Verbatim}[fontsize=\relsize{-2}] % python >>> 1.05*88.88 93.323999999999998 \end{Verbatim} Among other things, one can do quick conversions between decimal and hex: \begin{Verbatim}[fontsize=\relsize{-2}] >>> 0x12 18 >>> hex(18) '0x12' \end{Verbatim} One can also look up ASCII codes: \begin{lstlisting} >>> ord('A') 65 \end{lstlisting} If I need math functions, I must {\bf import} the Python math library first. This is analogous to what we do in C/C++, where we must have a {\bf \#include} line for the library in our source code and must link in the machine code for the library. We must refer to imported functions in the context of the library, in this case the math library. For example, the functions {\bf sqrt()} and {\bf sin()} must be prefixed by {\bf math}:\footnote{A method for avoiding the prefix is shown in Sec. \ref{import}.} \begin{Verbatim}[fontsize=\relsize{-2}] >>> import math >>> math.sqrt(88) 9.3808315196468595 >>> math.sin(2.5) 0.59847214410395655 \end{Verbatim} \section{A 10-Minute Introductory Example} \label{tenmin} \subsection{Example Program Code} \label{tme} This program reads a text file, specified on the command line, and prints out the number of lines and words in the file: \begin{Verbatim}[fontsize=\relsize{-2},numbers=left] # reads in the text file whose name is specified on the command line, # and reports the number of lines and words import sys def checkline(): global l global wordcount w = l.split() wordcount += len(w) wordcount = 0 f = open(sys.argv[1]) flines = f.readlines() linecount = len(flines) for l in flines: checkline() print linecount, wordcount \end{Verbatim} Say for example the program is in the file {\bf tme.py}, and we have a text file {\bf x} with contents \begin{Verbatim}[fontsize=\relsize{-2}] This is an example of a text file. \end{Verbatim} (There are five lines in all, the first and last of which are blank.) If we run this program on this file, the result is: \begin{Verbatim}[fontsize=\relsize{-2}] python tme.py x 5 8 \end{Verbatim} On the surface, the layout of the code here looks like that of a C/C++ program: First an {\bf import} statement, analogous to {\bf \#include} (with the corresponding linking at compile time) as stated above; second the definition of a function; and then the ``main'' program. This is basically a good way to look at it, but keep in mind that the Python interpreter will execute everything in order, starting at the top. In executing the {\bf import} statement, for instance, that might actually result in some code being executed, if the module being imported has some free-standing code rather than just function definitions. More on this later. Execution of the {\bf def} statement won't execute any code for now, but the act of defining the function is considered execution. Here are some features in this program which were not in the first example: \begin{itemize} \item use of command-line arguments \item file-manipulation mechanisms \item more on lists \item function definition \item library importation \item introduction to scope \end{itemize} I will discuss these features in the next few sections. \subsection{Command-Line Arguments} First, let's explain {\bf sys.argv}. Python includes a {\bf module} (i.e. library) named {\bf sys}, one of whose member variables is {\bf argv}. The latter is a Python list, analogous to {\bf argv} in C/C++.\footnote{There is no need for an analog of {\bf argc}, though. Python, being an object-oriented language, treats lists as objects, The length of a list is thus incorporated into that object. So, if we need to know the number of elements in {\bf argv}, we can get it via {\bf len(argv)}.} Element 0 of the list is the script name, in this case {\bf tme.py}, and so on, just as in C/C++. In our example here, in which we run our program on the file {\bf x}, {\bf sys.argv[1]} will be the string 'x' (strings in Python are generally specified with single quote marks). Since {\bf sys} is not loaded automatically, we needed the {\bf import} line. Both in C/C++ and Python, those command-line arguments are of course strings. If those strings are supposed to represent numbers, we could convert them. If we had, say, an integer argument, in C/C++ we would do the conversion using {\bf atoi()}; in Python, we'd use {\bf int()}. For floating-point, in Python we'd use {\bf float()}.\footnote{In C/C++, we could use {\bf atof()} if it were available, or {\bf sscanf()}.} \subsection{Introduction to File Manipulation} \label{files} The function {\bf open()} is similar to the one in C/C++. Our line \begin{Verbatim}[fontsize=\relsize{-2}] f = open(sys.argv[1]) \end{Verbatim} created an object of {\bf file} class, and assigned it to {\bf f} . The {\bf readlines()} function of the {\bf file} class returns a list (keep in mind, ``list'' is an official Python term) consisting of the lines in the file. Each line is a string, and that string is one element of the list. Since the file here consisted of five lines, the value returned by calling {\bf readlines()} is the five-element list \begin{Verbatim}[fontsize=\relsize{-2}] ['','This is an','example of a','text file',''] \end{Verbatim} (Though not visible here, there is an end-of-line character in each string.) \subsection{Lack of Declaration} \label{decl} Variables are not declared in Python. A variable is created when the first assignment to it is executed. For example, in the program {\bf tme.py} above, the variable {\bf flines} does not exist until the statement \begin{Verbatim}[fontsize=\relsize{-2}] flines = f.readlines() \end{Verbatim} is executed. By the way, a variable which has not been assigned a value yet, such as {\bf wordcount} at first above, has the value {\bf None}. And this can be assigned to a variable, tested for in an {\bf if} statement, etc. \subsection{Locals Vs. Globals} Python does not really have global variables in the sense of C/C++, in which the scope of a variable is an entire program. We will discuss this further in Section \ref{unctuous}, but for now assume our source code consists of just a single {\bf .py} file; in that case, Python does have global variables pretty much like in C/C++ (though with important differences). Python tries to infer the scope of a variable from its position in the code. If a function includes any code which assigns to a variable, then that variable is assumed to be local, unless we use the {\bf global} keyword. So, in the code for {\bf checkline()}, Python would assume that {\bf l} and {\bf wordcount} are local to {\bf checkline()} if we had not specified {\bf global}. Use of global variables simplifies the presentation here, and I personally believe that the unctuous criticism of global variables is unwarranted. (See \url{http://heather.cs.ucdavis.edu/~matloff/globals.html}.) In fact, in one of the major types of programming, {\bf threads}, use of globals is basically {\it mandatory}. You may wish, however, to at least group together all your globals into a class, as I do. See Appendix \ref{glbls}. \subsection{A Couple of Built-In Functions} The function {\bf len()} returns the number of elements in a list. In the {\bf tme.py} example above, we used this to find the number of lines in the file, since {\bf readlines()} returned a list in which each element consisted of one line of the file. The method {\bf split()} is a member of the {\bf string} class.\footnote{Member functions of classes are referred to via {\bf methods}.} It splits a string into a list of words, for example.\footnote{The default is to use blank characters as the splitting criterion, but other characters or strings can be used.} So, for instance, in {\bf checkline()} when {\bf l} is 'This is an' then the list {\bf w} will be equal to ['This','is','an']. (In the case of the first line, which is blank, {\bf w} will be equal to the empty list, [].) \section{Types of Variables/Values} As is typical in scripting languages, type in the sense of C/C++ {\bf int} or {\bf float} is not declared in Python. However, the Python interpreter does internally keep track of the type of all objects. Thus Python variables don't have types, but their values do. In other words, a variable {\bf X} might be bound to (i.e. point to) an integer in one place in your program and then be rebound to a class instance at another point. Python's types include notions of scalars, {\bf sequences} (lists or {\bf tuples}) and dictionaries (associative arrays, discussed in Sec. \ref{hashes}), classes, function, etc. \section{String Versus Numerical Values} Unlike Perl, Python does distinguish between numbers and their string representations. The functions {\bf eval()} and {\bf str()} can be used to convert back and forth. For example: \begin{Verbatim}[fontsize=\relsize{-2}] >>> 2 + '1.5' Traceback (most recent call last): File "", line 1, in ? TypeError: unsupported operand type(s) for +: 'int' and 'str' >>> 2 + eval('1.5') 3.5 >>> str(2 + eval('1.5')) '3.5' \end{Verbatim} There are also {\bf int()} to convert from strings to integers, and {\bf float()}, to convert from strings to floating-point values: \begin{Verbatim}[fontsize=\relsize{-2}] >>> n = int('32') >>> n 32 >>> x = float('5.28') >>> x 5.2800000000000002 \end{Verbatim} See also Section \ref{formatted}. \section{Sequences} Lists are actually special cases of {\bf sequences}, which are all array-like but with some differences. Note though, the commonalities; all of the following (some to be explained below) apply to any sequence type: \begin{itemize} \item the use of brackets to denote individual elements (e.g. {\bf x[i]}) \item the built-in {\bf len()} function to give the number of elements in the sequence\footnote{This function is applicable to dictionaries too.} \item {\bf slicing} operations, i.e. the extraction of subsequences \item use of {\bf +} and {\bf *} operators for concatenation and replication \end{itemize} \subsection{Lists (Quasi-Arrays)} As stated earlier, lists are denoted by brackets and commas. For instance, the statement \begin{Verbatim}[fontsize=\relsize{-2}] x = [4,5,12] \end{Verbatim} would set {\bf x} to the specified 3-element array. A number of other operations are available for lists, a few of which are illustrated in the following code: \begin{Verbatim}[fontsize=\relsize{-2},numbers=left] >>> x = [5,12,13,200] >>> x [5, 12, 13, 200] >>> x.append(-2) >>> x [5, 12, 13, 200, -2] >>> del x[2] >>> x [5, 12, 200, -2] >>> z = x[1:3] # array "slicing": elements 1 through 3-1 = 2 >>> z [12, 200] >>> yy = [3,4,5,12,13] >>> yy[3:] # all elements starting with index 3 [12, 13] >>> yy[:3] # all elements up to but excluding index 3 [3, 4, 5] >>> yy[-1] # means "1 item from the right end" 13 >>> x.insert(2,28) # insert 28 at position 2 >>> x [5, 12, 28, 200, -2] >>> 28 in x # tests for membership; 1 for true, 0 for false 1 >>> 13 in x 0 >>> x.index(28) # finds the index within the list of the given value 2 >>> x.remove(200) # not same as del(), indexed by value; removes 1st only >>> x [5, 12, 28, -2] >>> w = x + [1,"ghi"] # concatenation of two or more lists >>> w [5, 12, 28, -2, 1, 'ghi'] >>> qz = 3*[1,2,3] # list replication >>> qz [1, 2, 3, 1, 2, 3, 1, 2, 3] >>> x = [1,2,3] >>> x.extend([4,5]) >>> x [1, 2, 3, 4, 5] >>> g = [1,2,3] >>> g.append([4,5]) >>> g [1, 2, 3, [4, 5]] >>> y = x.pop(0) # deletes and returns 0th element >>> y 1 >>> x [2, 3, 4, 5] >>> t = [5,12,13] >>> t.reverse() >>> t [13, 12, 5] \end{Verbatim} We also saw the {\bf in} operator in an earlier example, used in a {\bf for} loop. A list could include mixed elements of different types, including other lists themselves. Lists may grow dynamically, using the {\bf list} class' {\bf append()} or {\bf extend()} functions. For example, if after the above statement we were to execute \begin{Verbatim}[fontsize=\relsize{-2}] x.append(-2) \end{Verbatim} {\bf x} would now be equal to [4,5,12,-2]. The difference between the two is that {\bf append()} adds one element to a list, while {\bf extend} does concatenation of lists. This can be subtle: \begin{lstlisting} >>> y = [1,2,3] >>> y.extend([8,88]) >>> y [1, 2, 3, 8, 88] >>> y.append([8,88]) >>> y [1, 2, 3, 8, 88, [8, 88]] \end{lstlisting} The Python idiom includes a number of common ``Python tricks'' involving sequences, e.g. the following quick, elegant way to swap two variables {\bf x} and {\bf y}: \begin{Verbatim}[fontsize=\relsize{-2}] >>> x = 5 >>> y = 12 >>> [x,y] = [y,x] >>> x 12 >>> y 5 \end{Verbatim} Note that the elements of the list may be mixed types, e.g. \begin{Verbatim}[fontsize=\relsize{-2}] >>> z = [1,'a'] >>> z [1, 'a'] \end{Verbatim} Multidimensional lists can be implemented as lists of lists. For example: \begin{Verbatim}[fontsize=\relsize{-2}] >>> x = [] >>> x.append([1,2]) >>> x [[1, 2]] >>> x.append([3,4]) >>> x [[1, 2], [3, 4]] >>> x[1][1] 4 \end{Verbatim} But be careful! Look what can go wrong: \begin{Verbatim}[fontsize=\relsize{-2}] >>> x = 4*[0] >>> y = 4*[x] >>> y [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]] >>> y[0][2] 0 >>> y[0][2] = 1 >>> y [[0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 1, 0], [0, 0, 1, 0]] \end{Verbatim} The problem is that that assignment to {\bf y} was really a list of four references to the same thing ({\bf x}). When the object pointed to by {\bf x} changed, then all four rows of {\bf y} changed. The Python Wikibook (\url{http://en.wikibooks.org/wiki/Python_Programming/Lists}) suggests a solution, in the form of {\bf list comprehensions}, which we cover in Section \ref{listcomps}: \begin{Verbatim}[fontsize=\relsize{-2}] >>> z = [[0]*4 for i in range(5)] >>> z [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]] >>> z[0][2] = 1 >>> z [[0, 0, 1, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]] \end{Verbatim} \subsection{Tuples} {\bf Tuples} are like lists, but are {\bf immutable}, i.e. unchangeable. They are enclosed by parentheses or nothing at all, rather than brackets. The parentheses are mandatory if there is an ambiguity without them, e.g. in function arguments. A comma must be used in the case of empty or single tuple, e.g. {\bf (,)} and {\bf (5,)}. Since a tuple can't grow or shrink, the Python interpreter can store it in a contiguous block of memory, thus create efficient code. The same operations can be used, except those which would change the tuple. So for example \begin{Verbatim}[fontsize=\relsize{-2}] x = (1,2,'abc') print x[1] # prints 2 print len(x) # prints 3 x.pop() # illegal, due to immutability \end{Verbatim} A nice function is {\bf zip()}, which strings together corresponding components of several lists, producing tuples, e.g. \begin{Verbatim}[fontsize=\relsize{-2}] >>> zip([1,2],['a','b'],[168,168]) [(1, 'a', 168), (2, 'b', 168)] \end{Verbatim} \subsection{Strings} \label{stringsec} Strings are essentially tuples of character elements. But they are quoted instead of surrounded by parentheses, and have more flexibility than tuples of character elements would have. Some subtleties can occur, e.g. \begin{lstlisting} >>> z = [1,2,3] >>> z.extend('abc') >>> z [1, 2, 3, 'a', 'b', 'c'] \end{lstlisting} The fact that 'abc' is considered a 3-element tuple, rather than a 1-element scalar, came into play here. \subsubsection{Strings As Turbocharged Tuples} Let's see some examples of string operations: \begin{Verbatim}[fontsize=\relsize{-2},numbers=left] >>> x = 'abcde' >>> x[2] 'c' >>> x[2] = 'q' # illegal, since strings are immmutable Traceback (most recent call last): File "", line 1, in ? TypeError: object doesn't support item assignment >>> x = x[0:2] + 'q' + x[3:5] >>> x 'abqde' \end{Verbatim} (You may wonder why that last assignment \begin{Verbatim}[fontsize=\relsize{-2}] >>> x = x[0:2] + 'q' + x[3:5] \end{Verbatim} does not violate immmutability. The reason is that {\bf x} is really a pointer, and we are simply pointing it to a new string created from old ones. See Section \ref{effects}.) As noted, strings are more than simply tuples of characters: \begin{Verbatim}[fontsize=\relsize{-2}] >>> x.index('d') # as expected 3 >>> 'd' in x # as expected 1 >>> x.index('de') # pleasant surprise 3 \end{Verbatim} As can be seen, the {\bf index()} function from the {\bf str} class has been overloaded, making it more flexible. Thus a list of character strings is a two-dimensional array: \begin{lstlisting} >>> x = ['abc','de','f'] >>> x[1] 'de' >>> x[1][1] 'e' \end{lstlisting} There are many other handy functions in the {\bf str} class. For example, we saw the {\bf split()} function earlier. The opposite of this function is {\bf join()}. One applies it to a string, with a sequence of strings as an argument. The result is the concatenation of the strings in the sequence, with the original string between each of them:\footnote{The example here shows the ``new'' usage of {\bf join()}, now that string methods are built-in to Python. See discussion of ``new'' versus ``old'' below.} \begin{Verbatim}[fontsize=\relsize{-2}] >>> '---'.join(['abc','de','xyz']) 'abc---de---xyz' >>> q = '\n'.join(('abc','de','xyz')) >>> q 'abc\nde\nxyz' >>> print q abc de xyz \end{Verbatim} Here are some more: \begin{Verbatim}[fontsize=\relsize{-2}] >>> x = 'abc' >>> x.upper() 'ABC' >>> 'abc'.upper() 'ABC' >>> 'abc'.center(5) # center the string within a 5-character set ' abc ' >>> 'abc de f'.replace(' ','+') 'abc+de+f' >>> x = 'abc123' >>> x.find('c1') # find index of first occurrence of 'c1' in x 2 >>> x.find('3') 5 >>> x.find('1a') -1 \end{Verbatim} A very rich set of functions for string manipulation is also available in the {\bf re} (``regular expression'') module. % The {\bf str} class is built-in for newer versions of Python. With % an older version, you will need a statement % % \begin{Verbatim}[fontsize=\relsize{-2}] % import string % \end{Verbatim} % % That latter class does still exist, and the newer {\bf str} class does % not quite duplicate it. \subsubsection{Formatted String Manipulation} \label{formatted} String manipulation is useful in lots of settings, one of which is in conjunction with Python's {\bf print} command. For example, \begin{Verbatim}[fontsize=\relsize{-2}] print "the factors of 15 are %d and %d" % (3,5) \end{Verbatim} prints out \begin{Verbatim}[fontsize=\relsize{-2}] the factors of 15 are 3 and 5 \end{Verbatim} The {\bf \%d} of course is the integer format familiar from C/C++. But actually, the above action is a string issue, not a print issue. Let's see why. In \begin{Verbatim}[fontsize=\relsize{-2}] print "the factors of 15 are %d and %d" % (3,5) \end{Verbatim} the portion \begin{Verbatim}[fontsize=\relsize{-2}] "the factors of 15 are %d and %d" % (3,5) \end{Verbatim} is a string operation, producing a new string; the {\bf print} simply prints that new string. For example: \begin{Verbatim}[fontsize=\relsize{-2}] >>> x = "%d years old" % 12 \end{Verbatim} The variable {\bf x} now is the string '12 years old'. This is another very common idiom, quite powerful.\footnote{Some C/C++ programmers might recognize the similarity to {\bf sprintf()} from the C library.} Note the importance above of writing '(3,5)' rather than '3,5'. In the latter case, the {\bf \%} operator would think that its operand was merely 3, whereas it needs a 2-element tuple. Recall that parentheses enclosing a tuple can be omitted as long as there is no ambiguity, but that is not the case here. \subsection{Sorting Sequences} \label{sorting} The Python function {\bf sort()} can be applied to any sequence. For nonscalars, one provides a ``compare'' function, which returns a negative, zero or positive value, signfiying $<$, $=$ or $>$. As an illustration, let's sort an array of arrays, using the second elements as keys: \begin{Verbatim}[fontsize=\relsize{-2}] >>> x = [[1,4],[5,2]] >>> x [[1, 4], [5, 2]] >>> x.sort() >>> x [[1, 4], [5, 2]] >>> def g(u,v): ... return u[1]-v[1] ... >>> x.sort(g) >>> x [[5, 2], [1, 4]] \end{Verbatim} (This would be more easily done using ``lambda'' functions. See Section \ref{lambda}.) There is a Python library module, {\bf bisect}, which does binary search and related sorting. \section{Determining Object Types} You can use {\bf type()} to determine object type: \begin{lstlisting} >>> x = [1,2,3] >>> type(x) >>> type(x) is list True \end{lstlisting} \section{Dictionaries (Hashes)} \label{hashes} Dictionaries are {\bf associative arrays}. The technical meaning of this will be discussed below, but from a pure programming point of view, this means that one can set up arrays with non-integer indices. The statement \begin{Verbatim}[fontsize=\relsize{-2}] x = {'abc':12,'sailing':'away'} \end{Verbatim} sets {\bf x} to what amounts to a 2-element array with {\bf x['abc']} being 12 and {\bf x['sailing']} equal to 'away'. We say that {\bf 'abc'} and {\bf 'sailing'} are {\bf keys}, and 12 and 'away' are {\bf values}. Keys can be any immmutable object, i.e. numbers, tuples or strings.\footnote{Here we see another reason why Python distinguishes between tuples and lists. Allowing mutable keys would be an implementation nightmare, and probably lead to error-prone programming.} Use of tuples as keys is quite common in Python applications, and you should keep in mind that this valuable tool is available. Internally, {\bf x} here would be stored as a 4-element array, and the execution of a statement like \begin{Verbatim}[fontsize=\relsize{-2}] w = x['sailing'] \end{Verbatim} would require the Python interpreter to search through that array for the key 'sailing'. A linear search would be slow, so internal storage is organized as a hash table. This is why Perl's analog of Python's dictionary concept is actually called a {\bf hash}. Here are examples of usage of some of the member functions of the {\bf dictionary} class: \begin{Verbatim}[fontsize=\relsize{-2},numbers=left] >>> x = {'abc':12,'sailing':'away'} >>> x['abc'] 12 >>> y = x.keys() >>> y ['abc', 'sailing'] >>> z = x.values() >>> z [12, 'away'] x['uv'] = 2 >>> x {'abc': 12, 'uv': 2, 'sailing': 'away'} \end{Verbatim} Note how we added a new element to {\bf x}. The keys need not be tuples. For example: \begin{Verbatim}[fontsize=\relsize{-2}] >>> x {'abc': 12, 'uv': 2, 'sailing': 'away'} >>> f = open('z') >>> x[f] = 88 >>> x {: 88, 'abc': 12, 'uv': 2, 'sailing': 'away'} \end{Verbatim} Deletion of an element from a dictionary can be done via {\bf pop()}, e.g. \begin{Verbatim}[fontsize=\relsize{-2}] >>> x.pop('abc') 12 >>> x {: 88, 'uv': 2, 'sailing': 'away'} \end{Verbatim} The {\bf in} operator works on dictionary keys, e.g. \begin{Verbatim}[fontsize=\relsize{-2}] >>> x = {'abc': 12, 'uv': 2, 'sailing': 'away'} >>> 'uv' in x True >>> 2 in x False \end{Verbatim} \section{Function Definition} Obviously the keyword {\bf def} is used to define a function. Note once again that the colon and indenting are used to define a block which serves as the function body. A function can return a value, using the {\bf return} keyword, e.g. \begin{Verbatim}[fontsize=\relsize{-2}] return 8888 \end{Verbatim} However, the function does not have a type even if it does return something, and the object returned could be anything---an integer, a list, or whatever. Functions are {\bf first-class objects}, i.e. can be assigned just like variables. Function names {\it are} variables; we just temporarily assign a set of code to a name. Consider: \begin{Verbatim}[fontsize=\relsize{-2}] >>> def square(x): # define code, and point the variable square to it ... return x*x ... >>> square(3) 9 >>> gy = square # now gy points to that code too >>> gy(3) 9 >>> def cube(x): ... return x**3 ... >>> cube(3) 27 >>> square = cube # point the variable square to the cubing code >>> square(3) 27 >>> square = 8.8 >>> square 8.8000000000000007 # don't be shocked by the 7 >>> gy(3) # gy still points to the squaring code 9 \end{Verbatim} \section{Use of {\bf \_\_name\_\_}} \label{namemain} In some cases, it is important to know whether a module is being executed on its own, or via {\bf import}. This can be determined through Python's built-in variable {\bf \_\_name\_\_}, as follows. Whatever the Python interpreter is running is called the {\bf top-level program}. If for instance you type \begin{Verbatim}[fontsize=\relsize{-2}] % python x.py \end{Verbatim} then the code in {\bf x.py} is the top-level program. If you are running Python interactively, then the code you type in is the top-level program. The top-level program is known to the interpreter as {\bf \_\_main\_\_}, and the module currently being run is referred to as {\bf \_\_name\_\_}. So, to test whether a given module is running on its own, versus having been imported by other code, we check whether {\bf \_\_name\_\_} is {\bf \_\_main\_\_}. If the answer is yes, you are in the top level, and your code was not imported; otherwise it was. For example, let's add a statement \begin{Verbatim}[fontsize=\relsize{-2}] print __name__ \end{Verbatim} to our very first code example, from Section \ref{veryfirst}, in the file {\bf fme.py}: \begin{Verbatim}[fontsize=\relsize{-2}] print __name__ for i in range(10): x = 0.1*i print x print x/(1-x*x) \end{Verbatim} Let's run the program twice. First, we run it on its own: \begin{Verbatim}[fontsize=\relsize{-2}] % python fme.py __main__ 0.0 0.0 0.1 0.10101010101 0.2 0.208333333333 0.3 0.32967032967 ... [remainder of output not shown] \end{Verbatim} Now look what happens if we run it from within Python's interactive interpreter: \begin{Verbatim}[fontsize=\relsize{-2}] >>> import fme fme 0.0 0.0 0.1 0.10101010101 0.2 0.208333333333 0.3 0.32967032967 ... [remainder of output not shown] \end{Verbatim} Our module's statement \begin{Verbatim}[fontsize=\relsize{-2}] print __name__ \end{Verbatim} printed out {\bf \_\_main\_\_} the first time, but printed out {\bf fme} the second time. Here's what happened: In the first run, the Python interpreter was running {\bf fme.py}, while in the second one it was running {\bf import fme}. The latter of course resulting in the {\bf fme.py} code running, but that code was now second-level. It is customary to collect one's ``main program'' (in the C sense) into a function, typically named {\bf main()}. So, let's change our example above to {\bf fme2.py}: \begin{Verbatim}[fontsize=\relsize{-2}] def main(): for i in range(10): x = 0.1*i print x print x/(1-x*x) if __name__ == '__main__': main() \end{Verbatim} The advantage of this is that when we import this module, the code won't be executed right away. Instead, {\bf fme2.main()} must be called, either by the importing module or by the interactive Python interpreter. Here is an example of the latter: \begin{Verbatim}[fontsize=\relsize{-2}] >>> import fme2 >>> fme2.main() 0.0 0.0 0.1 0.10101010101 0.2 0.208333333333 0.3 0.32967032967 0.4 0.47619047619 ... \end{Verbatim} One of the uses of this involves executable code in imported modules. Say you have a module includes a statement \begin{lstlisting} z = 2 \end{lstlisting} that is freestanding, i.e. NOT part of some function. You may want that statement to execute if the module is run directly, but not if it is imported. The above test using {\bf \_\_name\_\_} would enable you to distinguish between the two cases. Among other things, this will be a vital point in using debugging tools (Section \ref{chap:dbg}). So get in the habit of always setting up access to {\bf main()} in this manner in your programs. \section{Example: Computing Final Grades} \begin{Verbatim}[fontsize=\relsize{-2},numbers=left] # computes and records final grades # input line format: # name and misc. info, e.g. class level # Final Report grade # Midterm grade # Quiz grades # Homework grades # e.g. # # John Paul Jones SR LCSI 70 A 50 B 69 B+ B+ A 3.52 B+ # comment lines, beginning with #, are ignored for computation but are # printed out; thus various notes can be put in comment lines; e.g. # notes on missed or makeup exams # usage: # python FinalGrades.py input_file nq nqd nh wts # where there are nq Quizzes, the lowest nqd of which will be # deleted; nh Homework assignments; and wts is the set of weights # for Final Report, Midterm, Quizzes and Homework # outputs to stdout the input file with final course grades appended; # the latter are numerical only, allowing for personal inspection of # "close" cases, etc. import sys def convertltr(lg): # converts letter grade lg to 4-point-scale if lg == 'F': return 0 base = lg[0] olg = ord(base) if len(lg) > 2 or olg < ord('A') or olg > ord('D'): print lg, 'is not a letter grade' sys.exit(1) grade = 4 - (olg-ord('A')) if len(lg) == 2: if lg[1] == '+': grade += 0.3 elif lg[1] == '-': grade -= 0.3 else: print lg, 'is not a letter grade' sys.exit(1) return grade def avg(x,ndrop): tmp = [] for xi in x: tmp.append(convertltr(xi)) tmp.sort() tmp = tmp[ndrop:] return float(sum(tmp))/len(tmp) def main(): infile = open(sys.argv[1]) nq = int(sys.argv[2]) nqd = int(sys.argv[3]) nh = int(sys.argv[4]) wts = [] for i in range(4): wts.append(float(sys.argv[5+i])) for line in infile.readlines(): toks = line.split() if toks[0] != '#': lw = len(toks) startpos = lw - nq - nh - 3 # Final Report frgrade = convertltr(toks[startpos]) # Midterm letter grade (skip over numerical grade) mtgrade = convertltr(toks[startpos+2]) startquizzes = startpos + 3 qgrade = avg(toks[startquizzes:startquizzes+nq],nqd) starthomework = startquizzes + nq hgrade = avg(toks[starthomework:starthomework+nh],0) coursegrade = 0.0 coursegrade += wts[0] * frgrade coursegrade += wts[1] * mtgrade coursegrade += wts[2] * qgrade coursegrade += wts[3] * hgrade print line[:len(line)-1], coursegrade else: print line[:len(line)-1] if __name__ == '__main__': main() \end{Verbatim} \section{Object-Oriented Programming} \label{oop} In contrast to Perl, Python has been object-oriented from the beginning, and thus has a much nicer, cleaner, clearer interface for OOP. \subsection{Example: Text File} \label{tfe} As an illustration, we will develop a class which deals with text files. Here are the contents of the file {\bf tfe.py}: \begin{samepage} \begin{Verbatim}[fontsize=\relsize{-2},numbers=left] class textfile: ntfiles = 0 # count of number of textfile objects def __init__(self,fname): textfile.ntfiles += 1 self.name = fname # name self.fh = open(fname) # handle for the file self.lines = self.fh.readlines() self.nlines = len(self.lines) # number of lines self.nwords = 0 # number of words self.wordcount() def wordcount(self): "finds the number of words in the file" for l in self.lines: w = l.split() self.nwords += len(w) def grep(self,target): "prints out all lines containing target" for l in self.lines: if l.find(target) >= 0: print l a = textfile('x') b = textfile('y') print "the number of text files open is", textfile.ntfiles print "here is some information about them (name, lines, words):" for f in [a,b]: print f.name,f.nlines,f.nwords a.grep('example') \end{Verbatim} \end{samepage} (By the way, note the {\bf docstrings}, the double-quoted, comment-like lines in {\bf wordcount()} and {\bf grep()}. These are ``supercomments,'' explained in Section \ref{docstrings}.) In addition to the file {\bf x} I used in Section \ref{tenmin} above, I had the 2-line file {\bf y}. Here is what happened when I ran the program: \begin{Verbatim}[fontsize=\relsize{-2}] % python tfe.py the number of text files opened is 2 here is some information about them (name, lines, words): x 5 8 y 2 5 example of a \end{Verbatim} \subsection{The Objects} In this code, we created two {\bf objects}, which we named {\bf a} and {\bf b}. Both were {\bf instances} of the class {\bf textfile}. \subsection{Constructors and Destructors} Technically, an instance of a class is created at the time the class name is invoked as a function, as we did above in the line \begin{Verbatim}[fontsize=\relsize{-2}] a = textfile('x') \end{Verbatim} So, one might say that the class name, called in functional form, is the constructor. However, we'll think of the constructor as being the {\bf \_\_init()\_\_} method. It is a built-in method in any class, but is typically overridden by our own definition, as we did above. (See Section \ref{glbls} for an example in which we do not override {\bf \_\_init()\_\_}.) The first argument of {\bf \_\_init()\_\_} is mandatory, and almost every Python programmer chooses to name it {\bf self}, which C++/Java programmers will recognize as the analog of {\bf this} in those languages. Actually {\bf self} is not a keyword. Unlike the {\bf this} keyword in C++/Java, you do not HAVE TO call this variable {\bf self}. Whatever you place in that first argument of {\bf \_\_init()\_\_} will be used by Python's interpreter as a pointer to the current instance of the class. If in your definition of {\bf \_\_init()\_\_} you were to name the first argument {\bf me}, and then write ``me'' instead of ``self'' throughout the definition of the class, that would work fine. However, you would invoke the wrath of purist pythonistas all over the world. So don't do it. Often {\bf \_\_init()\_\_} will have additional arguments, as in this case with a filename. The destructor is {\bf \_\_del()\_\_}. Note that it is only invoked when garbage collection is done, i.e.\ when all variables pointing to the object are gone. \subsection{Instance Variables} In general OOP terminology, an {\bf instance variable} of a class is a member variable for which each instance of the class has a separate value of that variable. In the example above, the instance variable {\bf fname} has the value 'x' in object {\bf a}, but that same variable has the value 'y' in object {\bf b}. In the C++ or Java world, you know this as a variable which is not declared {\bf static}. The term {\it instance variable} is the generic OOP term, non-language specific. \subsection{Class Variables} A {\bf class variable} is one that is associated with the class itself, not with instances of the class. Again in the C++ or Java world, you know this as a {\bf static} variable. It is designated as such by having some reference to {\bf v} in code which is in the class but not in any method of the class. An example is the code \begin{Verbatim}[fontsize=\relsize{-2}] ntfiles = 0 # count of number of textfile objects \end{Verbatim} above.\footnote{By the way, though we placed that code at the beginning of the class, it could be at the end of the class, or between two methods, as long as it is not inside a method. In the latter situation {\bf ntfiles} would be considered a local variable in the method, not what we want at all.} Note that a class variable {\bf v} of a class {\bf u} is referred to as {\bf u.v} within methods of the class and in code outside the class. For code inside the class but not within a method, it is referred to as simply {\bf v}. Take a moment now to go through our example program above, and see examples of this with our {\bf ntfiles} variable. \subsection{Instance Methods} The method {\bf wordcount()} is an {\bf instance method}, i.e. it applies specifically to the given object of this class. Again, in C++/Java terminology, this is a non-{\bf static} method. Unlike C++ and Java, where {\bf this} is an implicit argument to instance methods, Python wisely makes the relation explicit; the argument {\bf self} is required. The method {\bf grep()} is another instance method, this one with an argument besides {\bf self}. Note also that {\bf grep()} makes use of one of Python's many string operations, {\bf find()}. It searches for the argument string within the object string, returning the index of the first occurrence of the argument string within the object string, or returning -1 if none is found.\footnote{Strings are also treatable as lists of characters. For example, 'geometry' can be treated as an 8-element list, and applying {\bf find()} for the substring 'met' would return 3.} \subsection{Class Methods} \label{cm} A {\bf class method} is associated with the class itself. It does not have {\bf self} as an argument. Python has two (slightly differing) ways to designate a function as a class method, via the functions {\bf staticmethod()} and {\bf classmethod()}. We will use only the former.\footnote{See also Section \ref{decorators}.} As our first example, consider following enhancement to the code in within the class {\bf textfile} above: \begin{Verbatim}[fontsize=\relsize{-2}] class textfile: ... def totfiles(): print "the total number of text files is", textfile.ntfiles totfiles = staticmethod(totfiles) ... # here we are in "main" ... textfile.totfiles() ... \end{Verbatim} Note that {\bf staticmethod()} is indeed a function, as the above syntax would imply. It takes one function as input, and outputs another function. One can also designate a function as a class method this way: \begin{Verbatim}[fontsize=\relsize{-2}] class textfile: ... @staticmethod def totfiles(): print "the total number of text files is", textfile.ntfiles \end{Verbatim} A class method can be called even if there are not yet any instances of the class, say {\bf textfile} in this example. Here, 0 would be printed out, since no files had yet been counted. Note carefully that this is different from the Python value {\bf None}. Even if we have not yet created instances of the class {\bf textfile}, the code \begin{Verbatim}[fontsize=\relsize{-2}] ntfiles = 0 \end{Verbatim} would still have been executed when we first started execution of the program. As mentioned earlier, the Python interpreter executes the file from the first line onward. When it reaches the line \begin{Verbatim}[fontsize=\relsize{-2}] class textfile: \end{Verbatim} it then executes any free-standing code in the definition of the class. \subsection{Example: Vending Machine Collection} Here we will deal with a class representing a vending machine. It will give us more practice both classes and dictionaries. Each object of this class represents one machine, but all the machines carry the same items (though the current size of the stock of a given item may vary from machine to machine). The {\bf inventory} variable will be a dictionary with keys being item names and values being the current stocks of those items, e.g.\ \lstinline{'Kit Kat':8} signifying that this machine currently holds a stock of 8 Kit Kat bars. The method {\bf newstock()} adds to the stocks of the given items; e.g.\ \lstinline{m.newstock({'Kit Kat':3,'Sun Chips':2}) would record that the stocks of Kit Kat bars and bags of Sun Chips at machine {\bf m} have been replenished by 3 bars and 2 bags, respectively. \begin{lstlisting}[numbers=left] class machine: itemnames = [] def __init__(self): # in (itemname, stock) form self.inventory = {} for nm in machine.itemnames: self.inventory[nm] = 0 # adds the new stock to inventory; items is in dictionary form, # (itemname, newstock form) def newstock(self,newitems): for itm in newitems.keys(): self.inventory[itm] += newitems[itm] \end{lstlisting} Of course, many more methods could be added. \subsection{Derived Classes} Inheritance is very much a part of the Python philosophy. A statement like \begin{Verbatim}[fontsize=\relsize{-2}] class b(a): \end{Verbatim} starts the definition of a subclass {\bf b} of a class {\bf a}. Multiple inheritance, etc. can also be done. Otherwise everything is the same, but note that when the constructor for a derived class is called, the constructor for the base class is \underline{not} automatically called. If you wish the latter constructor to be invoked, you must invoke it yourself, e.g. \begin{Verbatim}[fontsize=\relsize{-2}] class b(a): def __init__(self,xinit): # constructor for class b self.x = xinit # define and initialize an instance variable x a.__init__(self) # call base class constructor \end{Verbatim} \subsection{A Word on Class Implementation} A Python class instance is implemented internally as a dictionary. For example, in our program {\bf tfe.py} above, the object {\bf b} is implemented as a dictionary. Among other things, this means that you can add member variables to an instance of a class ``on the fly,'' long after the instance is created. We are simply adding another key and value to the dictionary. For example, say we place the following code line 23, Section \ref{tfe}: \begin{lstlisting} a.u = 3 # legal! textfile.u = 3 # legal too! a.u # prints 3 b.u # error textfile.u # prints 3 a.__dict__ # prints instance variables, incl.u b.__dict__ # prints instance variables, no u textfile.__dict__ # some surprises \end{lstlisting} \section{Importance of Understanding Object References} \label{effects} A variable which has been assigned a mutable value is actually a pointer to the given object. For example, consider this code: \begin{Verbatim}[fontsize=\relsize{-2}] >>> x = [1,2,3] # x is mutable >>> y = x # x and y now both point to [1,2,3] >>> x[2] = 5 # the mutable object pointed to by x now "mutes" >>> y[2] # this means y[2] changes to 5 too! 5 >>> x = [1,2] >>> y = x >>> y [1, 2] >>> x = [3,4] >>> y [1, 2] \end{Verbatim} In the first few lines, {\bf x} and {\bf y} are references to a list, a mutable object. The statement \begin{Verbatim}[fontsize=\relsize{-2}] x[2] = 5 \end{Verbatim} then changes one aspect of that object, but {\bf x} still points to that object. On the other hand, the code \begin{Verbatim}[fontsize=\relsize{-2}] x = [3,4] \end{Verbatim} now changes {\bf x} itself, having it point to a different object, while {\bf y} is still pointing to the first object. If in the above example we wished to simply copy the list referenced by {\bf x} to {\bf y}, we could use slicing, e.g. \begin{Verbatim}[fontsize=\relsize{-2}] y = x[:] \end{Verbatim} Then {\bf y} and {\bf x} would point to different objects; {\bf x} would point to the same object as before, but the statement for {\bf y} would create a new object, which {\bf y} would point to. Even though those two objects have the same values for the time being, if the object pointed to by {\bf x} changes, {\bf y}'s object won't change. As you can imagine, this gets delicate when we have complex objects. See Python's {\bf copy} module for functions that will do object copying to various depths. % For more on this point, consider the following example: % % \begin{Verbatim}[fontsize=\relsize{-2}] % >>> for i in x: % ... if i == 12: x.remove(i) % ... % >>> i % 13 % \end{Verbatim} % % We probably were expecting the output to be % % \begin{Verbatim}[fontsize=\relsize{-2}] % % \end{Verbatim} An important similar issue arises with arguments in function calls. Any argument which is a variable which points to a \underline{mutable} object can change the value of that object from within the function, e.g.: \label{sideeffectpage} \begin{Verbatim}[fontsize=\relsize{-2}] >>> def f(a): ... a = 2*a # numbers are immutable ... >>> x = 5 >>> f(x) >>> x # x doesn't change 5 >>> def g(a): ... a[0] = 2*a[0] # lists are mutable ... >>> y = [5] >>> g(y) >>> y # y changes! [10] \end{Verbatim} Function names are references to objects too. What we think of as the name of the function is actually just a pointer---a mutable one---to the code for that function. For example, \begin{Verbatim}[fontsize=\relsize{-2}] >>> def f(): ... print 1 ... >>> def g(): ... print 2 ... >>> f() 1 >>> g() 2 >>> [f,g] = [g,f] >>> f() 2 >>> g() 1 \end{Verbatim} \section{Object Deletion} Objects can be deleted from Python's memory by using {\bf del}, e.g. \begin{Verbatim}[fontsize=\relsize{-2}] >>> del x \end{Verbatim} {\bf NOTE CAREFULLY THAT THIS IS DIFFERENT FROM DELETION FROM A LIST OR DICTIONARY}. If you use {\bf remove()} or {\bf pop()}, for instance, you are simply removing the pointer to the object from the given data structure, but as long as there is at least one {\bf reference}, i.e. a pointer, to an object, that object still takes up space in memory. This can be a major issue in long-running programs. If you are not careful to delete objects, or if they are not simply garbage-collected when their scope disappears, you can accumulate more and more of them, and have a very serious memory problem. If you see your machine running ever more slowly while a program executes, you should immediately suspect this. \section{Object Comparison} \label{cmp} One can use the $<$ operator to compare sequences, e.g. \begin{Verbatim}[fontsize=\relsize{-2}] if x < y: \end{Verbatim} for lists {\bf x} and {\bf y}. The comparison is {\bf lexicographic}. This ``dictionary'' ordering first compares the first element of one sequence to the first element of the other. If they aren't equal, we're done. If not, we compare the second elements, etc. For example, \begin{Verbatim}[fontsize=\relsize{-2}] >>> [12,16] < [12,88] # 12 = 12 but 16 < 88 True >>> [5,88] > [12,16] # 5 is not > 12 (even though 88 > 16) False >>> [5,88] < [5,89] True \end{Verbatim} Of course, since strings are sequences, we can compare them too: \begin{Verbatim}[fontsize=\relsize{-2}] >>> 'abc' < 'tuv' True >>> 'xyz' < 'tuv' False >>> 'xyz' != 'tuv' True \end{Verbatim} Note the effects of this on, for example, the {\bf max()} function, since it depends on operators such as $<$: \begin{Verbatim}[fontsize=\relsize{-2}] >>> max([[1, 2], [0], [12, 15], [3, 4, 5], [8, 72]]) [12, 15] >>> max([8,72]) 72 \end{Verbatim} We can set up comparisons for non-sequence objects, e.g. class instances, by defining a {\bf \_\_cmp()\_\_} function in the class. The definition starts with \begin{Verbatim}[fontsize=\relsize{-2}] def __cmp__(self,other): \end{Verbatim} It must be defined to return a negative, zero or positive value, depending on whether {\bf self} is less than, equal to or greater than {\bf other}. Very sophisticated sorting can be done if one combines Python's {\bf sort()} function with a specialized {\bf \_\_cmp()\_\_} function. \section{Modules } You've often heard that it is good software engineering practice to write your code in ``modular'' fashion, i.e. to break it up into components, top-down style, and to make your code ``reusable,'' i.e. to write it in such generality that you or someone else might make use of it in some other programs. Unlike a lot of follow-like-sheep software engineering shiboleths, this one is actually correct! :-) A {\bf module} is a set of classes, library functions and so on, all in one file. Unlike Perl, there are no special actions to be taken to make a file a module. Any file whose name has a {\bf .py} suffix is a module!\footnote{Make sure the base part of the file name begins with a letter, not, say, a digit.} \subsection{Example Program Code} \label{tfmodexample} As our illustration, let's take the {\bf textfile} class from our example above. We could place it in a {\it separate} file {\bf tf.py}, with contents \begin{samepage} \begin{Verbatim}[fontsize=\relsize{-2},numbers=left] # file tf.py class textfile: ntfiles = 0 # count of number of textfile objects def __init__(self,fname): textfile.ntfiles += 1 self.name = fname # name self.fh = open(fname) # handle for the file self.lines = self.fh.readlines() self.nlines = len(self.lines) # number of lines self.nwords = 0 # number of words self.wordcount() def wordcount(self): "finds the number of words in the file" for l in self.lines: w = l.split() self.nwords += len(w) def grep(self,target): "prints out all lines containing target" for l in self.lines: if l.find(target) >= 0: print l \end{Verbatim} \end{samepage} Note that even though our module here consists of just a single class, we could have several classes, plus global variables,\footnote{Though they would be global only to the module, not to a program which imports the module. See Section \ref{unctuous}.} executable code not part of any function, etc.) Our test program file, {\bf tftest.py}, might now look like this: \begin{Verbatim}[fontsize=\relsize{-2},numbers=left] # file tftest.py import tf a = tf.textfile('x') b = tf.textfile('y') print "the number of text files open is", tf.textfile.ntfiles print "here is some information about them (name, lines, words):" for f in [a,b]: print f.name,f.nlines,f.nwords a.grep('example') \end{Verbatim} \subsection{How {\tt import} Works} \label{import} The Python interpreter, upon seeing the statement {\bf import tf}, would load the contents of the file {\bf tf.py}.\footnote{In our context here, we would probably place the two files in the same directory, but we will address the issue of search path later.} Any executable code in {\bf tf.py} is then executed, in this case \begin{Verbatim}[fontsize=\relsize{-2}] ntfiles = 0 # count of number of textfile objects \end{Verbatim} (The module's executable code might not only be within classes.) Later, when the interpreter sees the reference to {\bf tf.textfile}, it would look for an item named {\bf textfile} within the module {\bf tf}, i.e. within the file {\bf tf.py}, and then proceed accordingly. An alternative approach would be: \begin{Verbatim}[fontsize=\relsize{-2},numbers=left] from tf import textfile a = textfile('x') b = textfile('y') print "the number of text files open is", textfile.ntfiles print "here is some information about them (name, lines, words):" for f in [a,b]: print f.name,f.nlines,f.nwords a.grep('example') \end{Verbatim} This saves typing, since we type only ``textfile'' instead of ``tf.textfile,'' making for less cluttered code. But arguably it is less safe (what if {\bf tftest.py} were to have some other item named {\bf textfile}?) and less clear ({\bf textfile}'s origin in {\bf tf} might serve to clarify things in large programs). The statement \begin{Verbatim}[fontsize=\relsize{-2}] from tf import * \end{Verbatim} would import everything in {\bf tf.py} in this manner. In any event, by separating out the {\bf textfile} class, we have helped to modularize our code, and possibly set it up for reuse. \subsection{Using reload() to Renew an Import} Say you are using Python in interactive mode, and are doing code development in a text editor at the same time. If you change the module, simply running {\bf import} again won't bring you the next version. Use {\bf reload()} to get the latter, e.g. \begin{Verbatim}[fontsize=\relsize{-2}] reload(tf) \end{Verbatim} \subsection{``Compiled'' Code} Like the case of Java, the Python interpreter compiles any code it executes to {\bf byte code} for the Python virtual machine. If the code is imported, then the compiled code is saved in a file with suffix {\bf .pyc}, so it won't have to be recompiled again later. Running byte code is faster, since the interpreter doesn't need to translate the Python syntax anymore. Since modules are objects, the names of the variables, functions, classes etc. of a module are attributes of that module. Thus they are retained in the {\bf .pyc} file, and will be visible, for instance, when you run the {\bf dir()} function on that module (Section \ref{dirftn}). \subsection{Miscellaneous} A module's (free-standing, i.e. not part of a function) code executes immediately when the module is imported. Modules are objects. They can be used as arguments to functions, return values from functions, etc. The list {\bf sys.modules} shows all modules ever imported into the currently running program. \subsection{A Note on Global Variables Within Modules} \label{unctuous} Python does not truly allow global variables in the sense that C/C++ do. An imported Python module will not have direct access to the globals in the module which imports it, nor vice versa. For instance, consider these two files, {\bf x.py}, \begin{Verbatim}[fontsize=\relsize{-2}] # x.py import y def f(): global x x = 6 def main(): global x x = 3 f() y.g() if __name__ == '__main__': main() \end{Verbatim} and {\bf y.py}: \begin{Verbatim}[fontsize=\relsize{-2}] # y.py def g(): global x x += 1 \end{Verbatim} The variable {\bf x} in {\bf x.py} is visible throughout the module {\bf x.py}, but not in {\bf y.py}. In fact, execution of the line \begin{Verbatim}[fontsize=\relsize{-2}] x += 1 \end{Verbatim} in the latter will cause an error message to appear, ``global name 'x' is not defined.'' Let's see why. The line above the one generating the error message, \begin{Verbatim}[fontsize=\relsize{-2}] global x \end{Verbatim} is telling the Python interpreter that there will be a global variable {\bf x} {\it in this module}. But when the interpreter gets to the next line, \begin{Verbatim}[fontsize=\relsize{-2}] x += 1 \end{Verbatim} the interpreter says, ``Hey, wait a minute! You can't assign to {\bf x} its old value plus 1. It doesn't have an old value! It hasn't been assigned to yet!'' In other words, the interpreter isn't treating the {\bf x} in the module {\bf y.py} to be the same as the one in {\bf x.py}. You can, however, refer to the {\bf x} in {\bf y.py} while you are in {\bf x.py}, as {\bf y.x}. % Indeed, a global variable in a module is merely an attribute (i.e. a % member entity) of that module, similar to a class variable's role within % a class. When module B is imported by module A, B's namespace is copied % to A's. If module B has a global variable {\bf X}, then module A will % create a variable of that name, whose initial value is whatever module B % had for its variable of that name at the time of importing. But changes % to {\bf X} in one of the modules will NOT be reflected in the other. % % Say {\bf X} does change in B, but we want code in A to be able to get % the latest value of {\bf X} in B. We can do that by including a % function, say named {\bf GetX()} in B. Assuming that A imported % everything from B, then A will get a function {\bf GetX()} which is a % copy of B's function of that name, and whose sole purpose is to return % the value of {\bf X}. Unless B changes that function (which {\it is} % possible, e.g. functions may be assigned), the functions in the two % modules will always be the same, and thus A can use its function to get % the value of {\bf X} in B. \subsection{Data Hiding} Python has no strong form of data hiding comparable to the {\bf private} and other such constructs in C++. It does offer a small provision of this sort, though: If you prepend an underscore to a variable's name in a module, it will not be imported if the {\bf from} form of {\bf import} is used. For example, if in the module {\bf tf.py} in Section \ref{tfmodexample} were to contain a variable {\bf z}, then a statement \begin{Verbatim}[fontsize=\relsize{-2}] from tf import * \end{Verbatim} would mean that {\bf z} is accesible as just {\bf z} rather than {\bf tf.z}. If on the other hand we named this variable {\bf \_z}, then the above statement would not make this variable accessible as {\bf \_z}; we would need to use {\bf tf.\_z}. Of course, the variable would still be visible from outside the module, but by requiring the {\bf tf.} prefix we would avoid confusion with similarly-named variables in the importing module. A double underscore results in mangling, with another underscore plus the name of the module prepended. \section{Packages} As mentioned earlier, one might place more than one class in a given module, if the classes are closely related. A generalization of this arises when one has several modules that are related. Their contents may not be so closely related that we would simply pool them all into one giant module, but still they may have a close enough relationship that you want to group them in some other way. This is where the notion of a {\bf package} comes in. For instance, you may write some libraries dealing with some Internet software you've written. You might have one module {\bf web.py} with classes you've written for programs which do Web access, and another module {\bf em.py} which is for e-mail software. Instead of combining them into one big module, you could keep them as separate files put in the same directory, say {\bf net}. To make this directory a package, simply place a file {\bf \_\_init\_\_.py} in that directory. The file can be blank, or in more sophisticated usage can be used for some startup operations. In order to import these modules, you would use statements like \begin{Verbatim}[fontsize=\relsize{-2}] import net.web \end{Verbatim} This tells the Python interpreter to look for a file {\bf web.py} within a directory {\bf net}. The latter, or more precisely, the parent of the latter, must be in your Python search path, which is a collection of directories in which the interpreter will look for modules. If for example the full path name for {\bf net} were \begin{Verbatim}[fontsize=\relsize{-2}] /u/v/net \end{Verbatim} then the directory {\bf /u/v} would need to be in your Python search path. The Python search path is stored in an environment variable for your operating system. If you are on a Linux system, for example, and are using the C shell, you could type \begin{Verbatim}[fontsize=\relsize{-2}] setenv PYTHONPATH /u/v \end{Verbatim} If you have several special directories like this, string them all together, using colons as delimiters: \begin{Verbatim}[fontsize=\relsize{-2}] setenv PYTHONPATH /u/v:/aa/bb/cc \end{Verbatim} You can access the current path from within a Python program in the variable {\bf sys.path}. It consists of a list of strings, one string for each directory, separated by colons. It can be printed out or changed by your code, just like any other variable.\footnote{Remember, you do have to import {\bf sys} first.} Package directories often have subdirectories, subsubdirectories and so on. Each one must contain a {\bf \_\_init\_\_.py} file. \section{Exception Handling (Not Just for Exceptions!)} \label{except} By the way, Python's built-in and library functions have no C-style error return code to check to see whether they succeeded. Instead, you use Python's {\bf try/except} exception-handling mechanism, e.g. \begin{Verbatim}[fontsize=\relsize{-2}] try: f = open(sys.argv[1]) except: print 'open failed:',sys.argv[1] \end{Verbatim} Here's another example: \begin{Verbatim}[fontsize=\relsize{-2}] try: i = 5 y = x[i] except: print 'no such index:', i \end{Verbatim} But the Python idiom also uses this for code which is not acting in an exception context. Say for example we want to find the index of the number 8 in the list {\bf z}, with the provision that if there is no such number, to first add it to the list. The ``ordinary'' way would be something like this: \begin{Verbatim}[fontsize=\relsize{-2}] # return first index of n in x; if n is not in x, then append it first def where(x,n): if n in x: return x.index(n) x.append(n) return len(x) - 1 \end{Verbatim} Let's try it: \begin{Verbatim}[fontsize=\relsize{-2}] >>> x = [5,12,13] >>> where(x,12) 1 >>> where(x,88) 3 >>> x [5, 12, 13, 88] \end{Verbatim} But we could also do it with {\bf try/except}: \begin{Verbatim}[fontsize=\relsize{-2}] def where1(x,n): try: return x.index(n) except: x.append(n) return len(x) - 1 \end{Verbatim} As seen above, you use {\bf try} to check for an exception; you use {\bf raise} to raise one: \begin{lstlisting} >>> def f(s,t): ... if s > t: raise ValueError('s must be <= t') ... return s+t ... >>> f(2,5) 7 >>> f(5,2) Traceback (most recent call last): File "", line 1, in File "", line 2, in f ValueError: s must be <= t \end{lstlisting} \section{Docstrings} \label{docstrings} There is a double-quoted string, ``finds the number of words in the file'', at the beginning of {\bf wordcount()} in the code in Section \ref{tfe}. This is called a {\bf docstring}. It serves as a kind of comment, but at runtime, so that it can be used by debuggers and the like. Also, it enables users who have only the compiled form of the method, say as a commercial product, access to a ``comment.'' Here is an example of how to access it, using {\bf tf.py} from above: \begin{Verbatim}[fontsize=\relsize{-2}] >>> import tf >>> tf.textfile.wordcount.__doc__ 'finds the number of words in the file' \end{Verbatim} A docstring typically spans several lines. To create this kind of string, use triple quote marks. By the way, did you notice above how the docstring is actually an attribute of the function, this case {\bf tf.textfile.wordcount.\_\_doc\_\_}? Try typing \begin{Verbatim}[fontsize=\relsize{-2}] >>> dir(tf.textfile.wordcount.__doc__) \end{Verbatim} to see the others. You can call {\bf help()} on any of them to see what they do. \section{Named Arguments in Functions} \label{namedargs} Consider this little example: \begin{Verbatim}[fontsize=\relsize{-2},numbers=left] def f(u,v=2): return u+v def main(): x = 2; y = 3; print f(x,y) # prints 5 print f(x,v=y) # prints 5, clearer code print f(x) # prints 4 if __name__ == '__main__': main() \end{Verbatim} Here, the argument {\bf v} is called a {\bf named argument}, with {\bf default value} 2. The ``ordinary'' argument {\bf u} is called a {\bf mandatory argument}, as it must be specified while {\bf v} need not be. Another term for {\bf u} is {\bf positional argument}, as its value is inferred by its position in the order of declaration of the function's arguments. Mandatory arguments must be declared before named arguments. \section{Terminal I/O Issues} \subsection{Keyboard Input} \label{key} The {\bf raw\_input()} function will display a prompt and read in what is typed. For example, \begin{Verbatim}[fontsize=\relsize{-2}] name = raw_input('enter a name: ') \end{Verbatim} would display ``enter a name:'', then read in a response, then store that response in {\bf name}. Note that the user input is returned in string form, and needs to be converted if the input consists of numbers. If you don't want the prompt, don't specify one: \begin{Verbatim}[fontsize=\relsize{-2}] >>> y = raw_input() 3 >>> y '3' \end{Verbatim} Alternatively, you can directly specify {\bf stdin}: \begin{Verbatim}[fontsize=\relsize{-2}] >>> import sys >>> z = sys.stdin.readlines() abc de f >>> z ['abc\n', 'de\n', 'f\n'] \end{Verbatim} After typing `f', I hit ctrl-d to close the {\bf stdin} file.) \subsection{Printing Without a Newline or Blanks} A {\bf print} statement automatically prints a newline character. To suppress it, add a trailing comma. For example: \begin{Verbatim}[fontsize=\relsize{-2}] print 5, # nothing printed out yet print 12 # '5 12' now printed out, with end-of-line \end{Verbatim} The {\bf print} statement automatically separates items with blanks. To suppress blanks, use the string-concatenation operator, {\bf +}, and possibly the {\bf str()} function, e.g. \begin{Verbatim}[fontsize=\relsize{-2}] x = 'a' y = 3 print x+str(y) # prints 'a3' \end{Verbatim} By the way, {\bf str(None)} is {\bf None}. \section{Example: Creating Linked Data Structures in Python} Below is a Python class for implementing a binary tree. The comments should make the program self-explanatory (no pun intended).\footnote{But did you get the pun?} \begin{Verbatim}[fontsize=\relsize{-2},numbers=left] # bintree.py, a module for handling sorted binary trees; values to be # stored can be general, not just numeric, as long as an ordering # relation exists # here, only have routines to insert and print, but could add delete, # etc. class treenode: # create a 1-node tree, storing value v; later this node will be # added to some tree def __init__(self,v): self.value = v; self.left = None; self.right = None; # inserts the node nd into tree rooted at self; typically that tree # will be a subtree of a larger tree def ins(self,nd): m = nd.value if m < self.value: if self.left == None: self.left = nd else: self.left.ins(nd) else: if self.right == None: self.right = nd else: self.right.ins(nd) # finds max value in the subtree rooted at self def max(self): # no name conflict s = self.value if self.right != None: s = max(s,self.right.max()) return s def prnt(self): # prints the subtree rooted at self if self.value == None: return if self.left != None: self.left.prnt() print self.value if self.right != None: self.right.prnt() class tree: def __init__(self): # tree starts out empty, no nodes self.root = None # create a new tree node, contents m, and add it to the specified # tree def insrt(self,m): newnode = treenode(m) if self.root == None: self.root = newnode return self.root.ins(newnode) \end{Verbatim} And here is a test: \begin{Verbatim}[fontsize=\relsize{-2},numbers=left] # trybt1.py: test of bintree.py # usage: python trybt.py numbers_to_insert import sys import bintree def main(): tr = bintree.tree() for n in sys.argv[1:]: tr.insrt(int(n)) tr.root.prnt() if __name__ == '__main__': main() \end{Verbatim} The good thing about Python is that we can use the same code again for nonnumerical objects, as long as they are comparable. (Recall Section \ref{cmp}.) So, we can do the same thing with strings, using the {\bf tree} and {\bf treenode} classes AS IS, NO CHANGE, e.g. \begin{Verbatim}[fontsize=\relsize{-2}] # trybt2.py: test of bintree.py # usage: python trybt.py strings_to_insert import sys import bintree def main(): tr = bintree.tree() for s in sys.argv[1:]: tr.insrt(s) tr.root.prnt() if __name__ == '__main__': main() \end{Verbatim} \begin{Verbatim}[fontsize=\relsize{-2}] % python trybt2.py abc tuv 12 12 abc tuv \end{Verbatim} Or even \begin{Verbatim}[fontsize=\relsize{-2}] # trybt3.py: test of bintree.py import bintree def main(): tr = bintree.tree() tr.insrt([12,'xyz']) tr.insrt([15,'xyz']) tr.insrt([12,'tuv']) tr.insrt([2,'y']) tr.insrt([20,'aaa']) tr.root.prnt() if __name__ == '__main__': main() \end{Verbatim} \begin{Verbatim}[fontsize=\relsize{-2}] % python trybt3.py [2, 'y'] [12, 'tuv'] [12, 'xyz'] [15, 'xyz'] [20, 'aaa'] \end{Verbatim} \section{Making Use of Python Idioms} \label{idiom} In the example in Section \ref{tfe}, it is worth calling special attention to the line \begin{Verbatim}[fontsize=\relsize{-2}] for f in [a,b]: \end{Verbatim} where {\bf a} and {\bf b} are objects of type {\bf textfile}. This illustrates the fact that the elements within a list do not have to be scalars, and that we can loop through a nonscalar list. Much more importantly, it illustrates that really effective use of Python means staying away from classic C-style loops and expressions with array elements. This is what makes for much cleaner, clearer and elegant code. It is where Python really shines. You should almost never use C/C++ style {\bf for} loops---i.e. where an index (say {\bf j}), is tested against an upper bound (say {\bf j $<$ 10}), and incremented at the end of each iteration ({say \bf j++}). Indeed, you can often avoid explicit loops, and should do so whenever possible. For example, the code \begin{Verbatim}[fontsize=\relsize{-2}] self.lines = self.fh.readlines() self.nlines = len(self.lines) \end{Verbatim} in that same program is much cleaner than what we would have in, say, C. In the latter, we would need to set up a loop, which would read in the file one line at a time, incrementing a variable {\bf nlines} in each iteration of the loop.\footnote{By the way, note the reference to an object within an object, {\bf self.fh}.} Another great way to avoid loops is to use Python's {\bf functional programming} features, described in Chapter \ref{chap:fun}. Making use of Python idioms is often referred to by the {\it pythonistas} as the {\it pythonic} way to do things. \section{Decorators} \label{decorators} Recall our example in Section \ref{cm} of how to designate a method as a class method: \begin{Verbatim}[fontsize=\relsize{-2}] def totfiles(): print "the total number of text files is", textfile.ntfiles totfiles = staticmethod(totfiles) \end{Verbatim} That third line does the designation. But isn't it kind of late? It comes as a surprise, thus making the code more difficult to read. Wouldn't it be better to warn the reader ahead of time that this is going to be a class method? We can do this with a {\bf decorator}: \begin{Verbatim}[fontsize=\relsize{-2}] @staticmethod def totfiles(): print "the total number of text files is", textfile.ntfiles \end{Verbatim} Here we are telling the Python interpreter, ``OK, here's what we're going to do. I'm going to define a function {\bf totfiles()}, and then, interpreter, I want you to use that function as input to {\bf staticmethod()}, and then reassign the output back to {\bf totfiles()}. So, we are really doing the same thing, but in a syntactically more readable manner. You can do this in general, feeding one function into another via decorators. This enables some very fancy, elegant ways to produce code, somewhat like macros in C/C++. However, we will not pursue that here. \section{Online Documentation} \label{online} \subsection{The dir() Function} \label{dirftn} There is a very handy function {\bf dir()} which can be used to get a quick review of what a given object or function is composed of. You should use it often. To illustrate, in the example in Section \ref{tfe} suppose we stop at the line \begin{Verbatim}[fontsize=\relsize{-2}] print "the number of text files open is", textfile.ntfiles \end{Verbatim} Then we might check a couple of things with {\bf dir()}, say: \begin{Verbatim}[fontsize=\relsize{-2}] (Pdb) dir() ['a', 'b'] (Pdb) dir(textfile) ['__doc__', '__init__', '__module__', 'grep', 'wordcount', 'ntfiles'] \end{Verbatim} When you first start up Python, various items are loaded. Let's see: \begin{Verbatim}[fontsize=\relsize{-2}] >>> dir() ['__builtins__', '__doc__', '__name__'] >>> dir(__builtins__) ['ArithmeticError', 'AssertionError', 'AttributeError', 'DeprecationWarning', 'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 'False', 'FloatingPointError', 'FutureWarning', 'IOError', 'ImportError', 'IndentationError', 'IndexError', 'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 'NameError', 'None', 'NotImplemented', 'NotImplementedError', 'OSError', 'OverflowError', 'OverflowWarning', 'PendingDeprecationWarning', 'ReferenceError', 'RuntimeError', 'RuntimeWarning', 'StandardError', 'StopIteration', 'SyntaxError', 'SyntaxWarning', 'SystemError', 'SystemExit', 'TabError', 'True', 'TypeError', 'UnboundLocalError', 'UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeError', 'UnicodeTranslateError', 'UserWarning', 'ValueError', 'Warning', 'ZeroDivisionError', '_', '__debug__', '__doc__', '__import__', '__name__', 'abs', 'apply', 'basestring', 'bool', 'buffer', 'callable', 'chr', 'classmethod', 'cmp', 'coerce', 'compile', 'complex', 'copyright', 'credits', 'delattr', 'dict', 'dir', 'divmod', 'enumerate', 'eval', 'execfile', 'exit', 'file', 'filter', 'float', 'frozenset', 'getattr', 'globals', 'hasattr', 'hash', 'help', 'hex', 'id', 'input', 'int', 'intern', 'isinstance', 'issubclass', 'iter', 'len', 'license', 'list', 'locals', 'long', 'map', 'max', 'min', 'object', 'oct', 'open', 'ord', 'pow', 'property', 'quit', 'range', 'raw_input', 'reduce', 'reload', 'repr', 'reversed', 'round', 'set', 'setattr', 'slice', 'sorted', 'staticmethod', 'str', 'sum', 'super', 'tuple', 'type', 'unichr', 'unicode', 'vars', 'xrange', 'zip'] \end{Verbatim} Well, there is a list of all the builtin functions and other attributes for you! Want to know what functions and other attributes are associated with dictionaries? \begin{Verbatim}[fontsize=\relsize{-2}] >>> dir(dict) ['__class__', '__cmp__', '__contains__', '__delattr__', '__delitem__', '__doc__', '__eq__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__str__', 'clear', 'copy', 'fromkeys', 'get', 'has_key', 'items', 'iteritems', 'iterkeys', 'itervalues', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values'] \end{Verbatim} Suppose we want to find out what methods and attributes are associated with strings. As mentioned in Section \ref{stringsec}, strings are now a built-in class in Python, so we can't just type \begin{Verbatim}[fontsize=\relsize{-2}] >>> dir(string) \end{Verbatim} But we can use any string object: \begin{Verbatim}[fontsize=\relsize{-2}] >>> dir('') ['__add__', '__class__', '__contains__', '__delattr__', '__doc__', '__eq__', '__ge__', '__getattribute__', '__getitem__', '__getnewargs__', '__getslice__', '__gt__', '__hash__', '__init__', '__le__', '__len__', '__lt__', '__mod__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__setattr__', '__str__', 'capitalize', 'center', 'count', 'decode', 'encode', 'endswith', 'expandtabs', 'find', 'index', 'isalnum', 'isalpha', 'isdigit', 'islower', 'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip', 'replace', 'rfind', 'rindex', 'rjust', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill'] \end{Verbatim} \subsection{The help() Function} For example, let's find out about the {\bf pop()} method for lists: \begin{Verbatim}[fontsize=\relsize{-2}] >>> help(list.pop) Help on method_descriptor: pop(...) L.pop([index]) -> item -- remove and return item at index (default last) (END) \end{Verbatim} And the {\bf center()} method for strings: \begin{Verbatim}[fontsize=\relsize{-2}] >>> help(''.center) Help on function center: center(s, width) center(s, width) -> string Return a center version of s, in a field of the specified width. padded with spaces as needed. The string is never truncated. \end{Verbatim} Hit 'q' to exit the help pager. You can also get information by using {\bf pydoc} at the Linux command line, e.g. \begin{Verbatim}[fontsize=\relsize{-2}] % pydoc string.center [...same as above] \end{Verbatim} \subsection{PyDoc} The above methods of obtaining help were for use in Python's interactive mode. Outside of that mode, in an OS shell, you can get the same information from PyDoc. For example, \begin{Verbatim}[fontsize=\relsize{-2}] pydoc sys \end{Verbatim} will give you all the information about the {\bf sys} module. For modules outside the ordinary Python distribution, make sure they are in your Python search path, and be sure show the ``dot'' sequence, e.g. \begin{Verbatim}[fontsize=\relsize{-2}] pydoc u.v \end{Verbatim} \section{Putting All Globals into a Class} \label{glbls} As mentioned in Section \ref{decl}, instead of using the keyword {\bf global}, we may find it clearer or more organized to group all our global variables into a class. Here, in the file {\bf tmeg.py}, is how we would do this to modify the example in that section, {\bf tme.py}: \begin{Verbatim}[fontsize=\relsize{-2},numbers=left] # reads in the text file whose name is specified on the command line, # and reports the number of lines and words import sys def checkline(): glb.linecount += 1 w = glb.l.split() glb.wordcount += len(w) class glb: linecount = 0 wordcount = 0 l = [] f = open(sys.argv[1]) for glb.l in f.readlines(): checkline() print glb.linecount, glb.wordcount \end{Verbatim} Note that when the program is first loaded, the class {\bf glb} will be executed, even before {\bf main()} starts. \section{Looking at the Python Virtual Machine} One can inspect the Python virtual machine code for a program. For the program {\bf srvr.py} in Chapter \ref{chap:pythr}, I once did the following: Running Python in interactive mode, I first imported the module {\bf dis} (``disassembler''). I then imported the program, by typing \begin{lstlisting} import dis import srvr \end{lstlisting} (I first needed to add the usual {\bf if \_\_name\_ == '\_\_main\_\_'} code, so that the program wouldn't execute upon being imported.) I then ran \begin{Verbatim}[fontsize=\relsize{-2}] >>> dis.dis(srvr) \end{Verbatim} How do you read the code? You can get a list of Python virtual machine instructions in {\it Python: the Complete Reference}, by Martin C. Brown, pub. by Osborne, 2001. But if you have background in assembly language, you can probably guess what the code is doing anyway. \section{Running Python Scripts Without Explicitly Invoking the Interpreter} Say you have a Python script {\bf x.py}. So far, we have discussed running it via the command\footnote{This section will be Linux-specific.} \begin{Verbatim}[fontsize=\relsize{-2}] % python x.py \end{Verbatim} or by importing {\bf x.py} while in interactive mode. But if you state the location of the Python interpreter in the first line of {\bf x.py}, e.g. \begin{Verbatim}[fontsize=\relsize{-2}] #! /usr/bin/python \end{Verbatim} and use the Linux {\bf chmod} command to make {\bf x.py} executable, then you can run {\bf x.py} by merely typing \begin{Verbatim}[fontsize=\relsize{-2}] % x.py \end{Verbatim} This is necessary, for instance, if you are invoking the program from a Web page. Better yet, you can have Linux search your environment for the location of Python, by putting this as your first line in {\bf x.py}: \begin{Verbatim}[fontsize=\relsize{-2}] #! /usr/bin/env python \end{Verbatim} This is more portable, as different platforms may place Python in different directories.