\chapter{Miscellaneous Systems Issues}
\label{chap:os} 

The material in this appendix pops up often throughout the book, so it's
worth the time spent.  For further details, see my computer systems
book, \url{http://heather.cs.ucdavis.edu/~matloff/50/PLN/CompSystsBook.pdf}.

\section{Timesharing}
\label{processes}

\subsection{Many Processes, Taking Turns}

Suppose you and someone else are both using the computer {\bf pc12} in
our lab, one of you at the console and the other logged in remotely.
Suppose further that the other person's program will run for five hours!
You don't want to wait five hours for the other person's program to end.
So, the OS arranges things so that the two programs will take turns
running, neither of them running to completion all at once.  It won't be
visible to you, but that is what happens.

\textbf{Timesharing} involves having several programs running in what
\underbar{appears} to be a simultaneous manner.   (These programs could
be from different users or the same user; in our case with threaded
code, several processes actually come from a single invocation of a
program.) If the system has only one CPU, which we'll assume
temporarily, this simultaneity is of course only an illusion, since only
one program can run at any given time, but it is a worthwhile illusion,
as we will see.

First of all, how is this illusion attained? The answer is that we have
the programs all take turns running, with each turn---called a
\textbf{quantum} or \textbf{timeslice}---being of very short duration,
for example 50 milliseconds.  (We'll continue to assume 50 ms quanta
below.)

Say we have four programs, {\bf u}, {\bf v}, {\bf x} and {\bf y},
running currently. What will happen is that first {\bf u} runs for 50
milliseconds, then {\bf u} is suspended and {\bf v} runs for 50
milliseconds, then {\bf v}  is suspended and {\bf x} runs for 50
milliseconds, and so on.  After {\bf y} gets its turn, then {\bf u} gets
a second turn, etc.  Since the turn-switching, formally known as
\textbf{context-switching},\footnote{We are switching from the
``context'' of one program to another.} is happening so fast (every 50
milliseconds), it appears to us humans that each program is running
continuously (though at one-fourth speed), rather than on and off, on
and off, etc.\footnote{Think of a light bulb turning on and off
extremely rapidly, with half the time on and half off.  If it is
blinking rapidly enough, you won't see it go on and off.  You'll simply
see it as shining steadily at half brightness.}

But how can the OS enforce these quanta? For example, how can the OS
force the program {\bf u} above to stop after 50 milliseconds?  The
answer is, {}``It can't! The OS is dead while {\bf u} is running.{}''
Instead, the turns are implemented via a timing device, which emits a
hardware interrupt at the proper time. For example, we could set the
timer to emit an interrupt every 50 milliseconds.  When the timer goes
off, it sends a pulse of current (the interrupt) to the CPU, which is
wired up to suspend its current process and jump to the driver of the
interrupting device (here, the timer).  Since the driver is in the OS,
the OS is now running!

We will make such an assumption here. However, what is more common is to
have the timer interrupt more frequently than the desired quantum size.
On a PC, the 8253 timer interrupts 100 times per second.  Every sixth
interrupt, the OS will perform a context switch.  That results in
a quantum size of 60 milliseconds.  But this can be changed, simply by
changing the count of interrupts needed to trigger a context switch.

The timer device driver saves all {\bf u}'s current register values,
including its Program Counter value (the address of the current
instruciton) and the value in its EFLAGS register (flags that record,
for instance, whether the last instruction produced a 0 result). Later,
when {\bf u}'s next turn comes, those values will be restored, and {\bf
u} including its PC value and the value in its EFLAGS register. Later,
when {\bf u}'s next turn comes, those values will be restored, and {\bf
u} will resume execution as if nothing ever happened. For now, though,
the OS routine will restore {\bf v}'s's previously-saved register
values, making sure to restore the PC value last of all. That last
action forces a jump from the OS to {\bf v}, right at the spot in {\bf
v}  where {\bf v} was suspended at the end of its last quantum. (Again,
the CPU just {}``minds its own business,{}'' and does not {}``know{}''
that one program, the OS, has handed over control to another, {\bf v};
the CPU just keeps performing its fetch/execute cycle, fetching whatever
the PC points to, oblivious to which process is running.)

A process' turn can end early, if the current process voluntarily gives
us control of the CPU.  Say the process reaches a point at which it is
supposed to read from the keyboard, with the source code calling, say,
{\bf scanf()} or {\bf cin}.  The process will make a systems call to do
this (the compiler placed that there), which means the OS will now be
running!  The OS will mark this process as being in Sleep state, meaning
that it's waiting for some action.  Later, when the user for that
process hits a key, it will cause an interrupt, and since the OS
contains the keyboard device driver, this means the OS will then start
running.  The OS will change the process' entry in the process table
from Sleep to Run---meaning only that it is ready to be given a turn.
Eventually, after some other process' turn ends, the OS will give this
process its next turn.

On a multicore machine, several processes can be physically running at
the same time, but the operation is the same as above.  On a Linux
system, to see all the currently running threads, type

\begin{lstlisting}
ps -eLf
\end{lstlisting}

\section{Memory Hierarchies}

\subsection{Cache Memory}
\label{cache}

Memory (RAM) is usually not on the processor chip, which makes it ``far
away.''  Signals must go through thicker wires than the tiny ones inside
the chip, which slows things down.  And of course the signal does have
to travel further.  All this still occurs quite quickly by human
standards, but not relative to the blinding speeds of modern CPUs.

Accordingly, a section of the CPU chip is reserved for a {\bf cache},
which at any given time contains a copy of part of memory.  If the
requested item (say {\bf x} above) is found in the cache, the CPU is in
luck, and access is quick; this is called a {\bf cache hit}.  If the CPU
is not lucky (a {\bf cache miss}), it must bring in the requested item
from memory.

Caches organize memory by chunks called {\bf blocks}.  When a cache miss
occurs, the entire block containing the requested item is brought into
the cache.  Typically a block currently in the cache must be {\bf
evicted} to make room for the new one.  

\subsection{Virtual Memory}
\label{vm}

Most modern processor chips have virtual memory (VM) capability, and
most general-purpose OSs make use of it.  

\subsubsection{\label{goals}Make Sure You Understand the Goals}

VM has the following basic goals: 

\begin{itemize}

\item {\bf Overcome limitations on memory size:}

We want to be able to run a program, or collectively several programs,
whose memory needs are larger than the amount of physical memory
available.

\item {\bf Relieve the compiler and linker of having to deal with
real addresses}

We want to facilitate \textbf{relocation} of programs, meaning that the
compiler and linker do not have to worry about where in memory a
program will be loaded when it is run.  They can, say, arrange for every
program to be loaded into memory starting at address 20200, without fear
of conflict, as the actualy address will be different.

\item {\bf Enable security:}

We want to ensure that one program will not accidentally (or
intentionally) harm another program's operation by writing to the
latter's area of memory, read or write to another program's I/O streams,
etc.

\end{itemize}

\subsubsection{How It Works}
\label{howvmworks}

Suppose a variable {\bf x} has the virtual address 1288, i.e. {\bf \&x =
1288} in a C/C++ program.  But, when the OS loads the program into
memory for execution, it rearranges everything, and the actual physical
address of {\bf x} may be, say, 5088.  

The high-order bits of an address are considered to be the {\bf page
number} of that address, with the lower bits being the {\bf offset}
within the page.  For any given item such as {\bf x}, the offset is the
same in both its virtual and physical addresses, but the page number
differs.

To illustrate this simply, suppose that our machine uses base-10
numbering instead of base-2, and that page size is 100 bytes.  Then {\bf
x} above would be in offset 88 of virtual page 12.  Its physical page
would be 50, with the same offset.  In other words, {\bf x} is stored 88
bytes past the beginning of page 50 in memory.

The correspondences between virtual and physical page numbers is given
in the {\bf page table}, which is simply an array in the OS.  The OS
will set up this array at the time it loads the program into memory, so
that the virtual-to-physical address translations can be done.

Those translations are done by the hardware.  When the CPU executes a
machine instruction that specifies access to 1288, the CPU will do a
lookup on the {\bf page table}, in the entry for virtual page 12, and
find that the actual page is 50.  The CPU will then know that 
the true location is 5088, and it would place 5088 in the address lines
in the system bus to then access 5088.  

On the other hand, {\bf x} may not currently be {\bf resident} in memory
at all, in which case the page table will mark it as such.  If the CPU
finds that page 12 is nonresident, we say a {\bf page fault} occurs, and
this will cause an internal interrupt, which in turn will cause a jump
to the operating system (OS).  The OS will then read the page containing
{\bf x} in from disk, place it somewhere in memory, and then update the
page table to show that virtual page 12 is now in some physical page  in
memory.  The OS will then execute an interrupt return instruction, and
the CPU will restart the instruction which triggered the page fault.

\subsection{Performance Issues}

Upon a cache miss, the hardware will need to read an entire block from
memory, and if an eviction is involved, an entire block will be written
as well, assuming a {\bf write-back} policy.  (See the reference at the
beginning of this appendix.)  All this is obviously slow.

The cache\footnote{Or caches, plural, as there are often multiple levels
of caches in today's machines.} is quite small relative to memory, so
you might guess that cache misses are very frequent.  Actually, though,
they aren't, due to something called {\bf locality of reference}.  This
term refers to the fact that most programs tend to either access the
same memory item repeatedly within short time periods ({\bf temporal}
locality), and/or access items within the same block often during short
periods ({\bf spatial} locality).  Hit rates are typically well above
90\%.  Part of this depends on having a good {\bf block replacement
policy}, which decides which block to evict (hopefully one that won't
be needed again soon!).

A page fault is pretty catastrophic in performance terms.  Remember, the
disk speed is on a mechanical scale, not an electronic one, so it will
take quite a while for the OS to service a page fault, much worse than
for a cache miss.  So the page replacement policy is even more important
as well.

On Unix-family machines, the {\bf time} command not only tells how long
your program ran, but also how many page faults it caused.  Note that
since the OS runs every time a page fault occurs, it can keep track of
the number of faults.  This is very different from a cache miss, which
although seems similar to a page fault in many ways, the key point is
that a cache miss is handled solely in hardware, so no program can count
the number of misses.\footnote{Note by the way that cache misses, though
harmful to program speed, aren't as catastrophic as page faults, as the
disk is not involved.}

Note that in a VM system each memory access becomes two memory
accesses---the page table read and the memory access itself.  This would
kill performance, so there is a special cache just for the page table,
called the Translation Lookaside Buffer.

\section{Array Issues}

\subsection{Storage}
\label{arraystorage}

It is important to understand how compilers store arrays in memory, an
overview of which will now be presented.

Consider the array declaration

\begin{lstlisting}
int y[100];
\end{lstlisting}

The compiler will store this in 100 consecutive words of memory.  You
may recall that in C/C++, an expression consisting of an array name, no
subscript, is a pointer to the array.  Well, more specifically, it is
the address of the first element of the array.

An array element, say \lstinline{y[8]} actually means the same as the
C/C++ pointer expression \lstinline{y+8}, which in turn means ``the word 8
{\bf int}s past the beginning of {\bf y}.''

Two-dimensional arrays, say

\begin{lstlisting}
int z[3][10];
\end{lstlisting}

exist only in our imagination.  They are actually treated as
one-dimensional arrays, in the above case consisting of $3 \times 10 =
30$ elements.  C/C++ arranges this in {\bf row-major} order, meaning
that all of row 0 comes first, then all of row 1 and so on.  So for
instance \lstinline{z[2][5]} is stored in element $10 + 10 + 5$ of {\bf
z}, and we could for example set that element to 8 with the code

\begin{lstlisting}
z[25] = 8;
\end{lstlisting}

or

\begin{lstlisting}
*(z+25) = 8;
\end{lstlisting}

Note that if we have a c-column two-dimensional array, element (i,j) is
stored in the word $i \times c + j$ of the array.  You'll see this fact
used a lot in this book, and in general in code written in the parallel
processing community.

\subsection{Subarrays}

The considerations in the last section can be used to access subarrays.
For example, here is code to find the sum of a {\bf float} array of
length k:

\begin{lstlisting}[numbers=left]
float sum(float x,int k)
{  float s = 0.0; int i;
   for (i = 0; i < k; i++) s += x[i];
   return s;
}
\end{lstlisting}

Quite ordinary, but suppose we wish to find the sum in row 2 of the
two-dimensional array {\bf z} above.  We could do this as
\lstinline{sum(z+20,10)}.

\subsection{Memory Allocation}

Very often one needs to set up an array whose size is not known at
compile time.  You are probably accustomed to doing this via {\bf
malloc()} or {\bf new}.  However, in large parallel programs, this
approach may be quite slow.

With an array with size known at compile time, and which is declared
local to some function, it will be allocated on the stack and you might
run out of stack space.  The easiest solution is probably to make the
array global, of fixed size.

To accommodate larger arrays under {\bf gcc} on a 64-bit system, use the
{\bf -mcmodel=medium} command line option.