\chapter{Overview of Functions of an Operating System:  Memory and I/O}
\label{chap:osmemio}

\section{Virtual Memory} 

\subsection{\label{goals}Make Sure You Understand the Goals}

Now let us add in the effect of virtual memory (VM). VM has the
following basic
goals, which are so important to understand that I am describing in 
separate short sub-subsections:

\subsubsection{Overcome Limitations on Memory Size}

We want to be able to run a program, or collectively several programs,
whose memory needs are larger than the amount of physical memory
available.

\subsubsection{Relieve the Compiler and Linker of Having to Deal with
Real Addresses}

We want to facilitate \textbf{relocation} of programs, meaning that the
compiler and linker do not have to worry about where in memory a
program will be loaded when it is run. 

\subsubsection{Enable Security}

We want to ensure that one program will not accidentally (or
intentionally) harm another program's operation by writing to the
latter's area of memory, read or write to another program's I/O streams,
etc.

\subsection{The Virtual Nature of Addresses}

The word \textit{virtual} means {}``apparent.{}'' It will
\underbar{appear} that a program resides entirely in main memory, when
in fact only part of it is there.  It will \underbar{appear} that your
program variables are stored at the places the linker assigned to
them,\footnote{You can determine these by running {\bf readelf} on your
executable file, with the {\bf -s} option.} when in fact they actually
will be stored somewhere else.

(For the time being, it will be easier to understand VM by assuming
there is no cache.  We will return to this in Section \ref{cacherole}.)

To make this more concrete, suppose our C source file 
includes the following statements:

\begin{Verbatim}[fontsize=\relsize{-2}]
int x;
...
x = 8;
printf("%d",&x);
\end{Verbatim}

Recall that addresses of variables are set in the following manner.  Say
{\bf x} is global.  Then the compiler will generate a {\bf .comm} line
in assembly language it produces.\footnote{If {\bf x} had been declared
in initialzed form, e.g. {\bf int x = 28;} then it would have been in a
{\bf .data} section.} The latter will be translated by the assembler,
and then the linker will deal with the machine code.  Among other
things, the linker will assign an address for {\bf x}.  It will be this
address which is printed out above.

If {\bf x} had been local, it would be stored on the stack, but that is
still memory of course, and thus it would have an address too.  Again,
that would be printed out.  

In any case, either of these addresses would be fake.  Let's see what
this really means.  Suppose that {\bf x} is global, and that the
compiler and linker assign it address 200, i.e. {\bf \&x} is 200.  Then
the above code 

\begin{Verbatim}[fontsize=\relsize{-2}]
x = 8;
\end{Verbatim}

would correspond to an instruction in {\bf a.out} something like

\begin{Verbatim}[fontsize=\relsize{-2}]
movl $8, 200
\end{Verbatim}

At the time \textbf{a.out} is loaded by the OS into memory, the OS will
divide both the {\bf .text} (instructions) and data portions of \textbf{a.out}
({\bf .data}, {\bf .bss}, stack, heap, etc.) into chunks, and find
unused places in memory at which to place these chunks. The chunks are
called \textbf{pages} of the program, and the same-sized places in
memory in which the OS puts them are called pages of memory.\footnote{As
with the stack, etc., keep in mind that these pages are only conceptual;
there is no ``fence'' or any other physical demarcation between one page
in memory and the next.  A page is simply a unit of measurement, like
acres and liters.} The OS sets up a \textbf{page table} for this
process, which is an array which is maintained by the OS, in which the
OS records the correspondences, i.e. lists which page of the process is
stored in which page of memory.  Remember, the OS is a program, and the
page table is an array in that program, and thus it too is in memory.

What appears to be in word 200 in memory from the program code above may
actually be in, say, word 1204. At the time the CPU executes that instruction,
the CPU will determine where `Word 200'' really is by doing a lookup
in the page table. In our example here, the table will show that the item we
want is actually in word 1204, and the CPU will then write to that location.
The hardware would put 1204 on the address bus, 8 on the data bus, and
would assert the Memory Write line in the control bus.

\checkpoint

In this example, we say the \textbf{virtual address} is 200, and the
\textbf{physical address} is 1204.

Note that while virtual addressing is enabled (it is turned off in
Kernel Mode), all addresses seen by the hardware will be virtual.  For
example, the contents of ESP will be a virtual address, with the real
top-of-stack address being something else, not what ESP says.  A similar
statement holds for the PC, etc.

\subsection{Overview of How the Goals Are Achieved}

Let's look at our stated goals in Section \ref{goals} above, and take a
first glance at how they are achieved (details will be covered later):

\subsubsection{Overcoming Limitations on Memory Size}

To conserve memory space, the OS will initially load only part of
\textbf{a.out} into memory, with the remainder being left back on disk.
The pages of the program which are not loaded will be marked in the page
table as currently being {\bf nonresident}, meaning not in memory, and
their locations on disk will be shown in the table. During execution of
the program, if the program needs one of the nonresident pages, the CPU
will notice that the requested page is nonresident.  This is called a
{\bf page fault}, and it causes an internal interrupt.  The interrupt
number is 0xe, i.e. entry 0xe in the IDT.  That causes a jump to the OS,
which will bring in that page from disk, and then jump back to the
program, which will resume at the instruction which accessed the missing
page.  

Note that pages will often go back and forth between disk and memory in this
manner. Each time the program needs a missing page, that page is brought in
from disk and a page which had been resident is written back to disk in order
to make room for the new one.   Note also that a given virtual page
might be in different physical pages at different times.

A big issue is the algorithm the OS uses to decide which page to move
back to disk (i.e. which page to replace) whenever it brings a page from
disk after a page fault.  Due to the huge difference in CPU and disk
speeds, a page fault is a catastrophic even in terms of program speed.
We hope to have as few page faults as possible when our program runs.

So, we want to check the pages to evict very carefully.  If we often
evict a page which is needed again very soon, our program's performance
will really suffer.  This is called {\bf thrashing}.  Details of page
replacement policies are beyond the scope of this document here, but one
point to notice is that the policy will be chosen so as to work well on
``most'' programs.  For any given policy, it will do well on some
programs (i.e. produce few page faults) while doing poorly on some other
programs (produce many page faults).  

So, what they try to do is come up with a policy which works reasonably
well on a reasonably broad variety of programs.  Most policies are some
variation on the Least Recently Used policy common for associative
caches.

\subsubsection{Relieving the Compiler and Linker of Having to Deal with
Real Addresses}

This is clear from the example above, where the location {}``200{}''
which the compiler and linker set up for {\bf x} was in effect changed by the
OS to 1204 at the time the program was loaded. The OS recorded this in
the page table, and then during execution of the program, the VM
hardware in the CPU does lookups in the page table to get the correct
addresses.  The point is that the compiler and linker can assign {\bf x} 
to location 200 without having to worry whether location is actually
available at the time the program will be run, because that variable
actually {\bf won't} be at 200.

\subsubsection{Enabling Security}

The page table will consist of one entry per page. That entry will, as
noted earlier, include information as to where in memory that page of
the program currently resides, of if currently nonresident, where on
disk the page is stored.  But in addition, the entry will also list the
permissions the program has to access this particular page---read,
write, execute---in a manner analogous to file-access permissions.  If .
an {\bf access violation} occurs, the VM hardware in the CPU will cause
an internal interrupt (again it's interrupt number 0xe, as for page
faults), causing the OS to run.  The OS will then kill the process, i.e.
remove it from the process table. 

Normally data-related sections such as {\bf .data} and the stack will be
the only ones with write permission.  However, you can also arrange for
the {\bf .text} section to be writable, via the {\bf -N} option to {\bf
ld}.

You might wonder why execute permission is included.  One situation in
which this would be a useful check is that in which we have a pointer to
a function.  If for example we forgot to initialize the pointer, a
violation will be detected.

Suppose for instance your program tries to read from virtual page number
40000 but that virtual address is out of the ranges of addresses which
the program is allocated.  The hardware will check the entry for 40000
in the page table, and find that not only is that page not resident, but
also it isn't on disk either.  Instead, the entry will say that there is
no virtual page 40000 among the pages allocated to this program.

We don't want an ordinary user programs to be able to, say maliciously, 
access the I/O streams of other programs.  To accomplish this, we want
to forbid a user program from accessing the I/O buffer space of another
program, which is easy to to accomplish since that space is in memory;
we simply have the OS set up the page table for that program
accordingly.  But we also need to forbid a user program from directly
performing I/O, e.g. the INB and OUTB instructions on Intel machines.

This latter goal is achieved by exploiting the fact that most modern
CPUs run in two or more \textbf{privilege levels}.  The CPU is designed
so that certain instructions, for example those which perform I/O such
as INB and OUTB, can be executed only at higher privilege levels, say
Kernel Mode. (The term \textbf{kernel} refers to the OS.)

But wait!  Since the OS is a program too, how does {\it it} get into
Kernel Mode?  Obviously there can't be some instruction to do this;
if there were, then ordinary user programs could use it to put
themselves into Kernel Mode.  So, how is it done?

When the machine boots up, it starts out in Kernel Mode.  Thus the OS
starts out in Kernel Mode.  Among other things, the interrupts can be
configured to change the privileges level of the CPU.  Recall that the
OS starts a turn for a user program (including that program's first
turn) by executing an IRET instruction, a software interrupt.  So, the
OS can arrange both for user programs to run in non-Kernel Mode and for
Kernel Mode to be restored when an interrupt comes during a user program
is running.  So for example, an interrupt from the timer will not only
end a user program's turn, but also will place the CPU in Kernel Mode,
thus having the OS run in that mode. 

\subsubsection{Is the Hardware Support Needed?}

It is instructive to think about the role of the VM hardware in
achieving the three goals above---being able to run larger programs than
our RAM size; being able to shift memory addresses of programs; enabling
security.  In the first two cases, we could accomplish our goal without
the hardware, even though it would be inconvenient; but in the third
case, the hardware is really necesary.

In the pre-VM days, programmers used {\bf overlays}.  A program might
need a very large array, but not have enough RAM.  If so, the programmer
could insert code to move portions of the array in from disk when they
were needed, and write them back to disk when they are no longer needed.
Of course, this was a nuisance for the programmer, but anyway the goal
could be accomplished.

Similarly, an executable file could be rewritten just before loading it
into memory for execution, so that it could be stored at a different
address than originally planned.  Depending on the archicture of the
machine, there may be some register who usage in effect shifted
everything to another place in memory.

But for the security goal, there really isn't anything we can do.  We
need something to check every memory access, and unless we run
interpreted code, the hardware is needed.

\subsection{Who Does What When?}

Note carefully the roles of the players here: It is the software, the
OS, that creates and maintains the page table, but it is the hardware
that actually uses the page table to generate physical addresses, check
page residency and check security.  In the event of a page fault or
security violation, the hardware will cause a jump to the OS, which
actually responds to those events.

The OS writes to the page table (including creating it in the first
place), and the hardware reads it.

\checkpoint

The hardware will have a special Page Table Register (PTR) to point to
the page table of the current process. When the OS starts a turn for a
process, it will restore the previously-saved value of the PTR, and thus
this process' page table will now be in effect.

On the Pentium, the name of the PTR is CR3.  Actually, the Pentium uses
a two-level hierarchy for its page tables, but we will not pursue that
point here, and thus simply refer to the register as PTR.

\subsection{Details on Usage of the Page Table}

\subsubsection{\label{trans}Virtual-to-Physical Address Translation,
Page Table Lookup}

Whenever the running program generates an address---either the address
of an instruction, as will be the case for an instruction fetch, or the
address of data, as will be the case during the execution of instructions
which have memory operands---this address is only virtual. It must be
translated to the physical address at which the requested item actually
resides.  The circuitry in the CPU is designed to do this translation by
performing a lookup in the page table.

For convenience, say the page size is 4096 bytes, which is the case for
Pentium CPUs.  (The treatment here is somewhat simplified, though.)
Both the virtual and physical address spaces are broken
into pages.  For example, consider virtual address 8195.  Since 

\begin{equation}
\label{ex8195}
8195 = 2 \times 4096 + 3
\end{equation}

that address would be in virtual page 2 (the first page is page 0).  We
can then speak of how far into a page a given address is.  Here, because
of the remainder 3 in Equation (\ref{ex8195}), we see that virtual
address 8195 is byte 3 in virtual page 2.  We refer to the 3 as the 
{\bf offset} within the page, i.e. its distance from the beginning of
the page.  

You can see that for any virtual address, the virtual page number is
equal to the address divided by the page size, 4096, and its offset
within that page is the address mod 4096.  Using our knowledge of the
properties of powers of 2 and the fact that $4096 = 2^{12}$, this means
that for a 32-bit address (which we'll assume throughout), the upper 20
bits contain the page number and the lower 12 bits contain the offset.

The page/offset description of the position of a byte is quite analogous
to a feet/inches description of distance.  We could measure everything
in inches, but we choose to use a larger unit, the foot, to give a rough
idea of the distance, and then use inches to describe the remainder.
The concept of offset will be very important in what follows.

Now, to see how the page table is used to convert virtual addresses to
physical ones,  consider for example the Intel instruction

\begin{Verbatim}[fontsize=\relsize{-2}]
movl $6, 8195
\end{Verbatim}

This would copy the constant 6 to location 8195.  Remember, this is page
2, offset 3.  However, it is a virtual address.  The hardware would see
the 8195,\footnote{Remember, it will be embedded within the instruction
itself, as this is direct addressing mode.}  The hardware knows that any
address given to it is a virtual one, which must be converted to a
physical one.  So, the hardware would look at the entry for virtual page
2, and find what physical page that is; suppose it is 5.  Then that page
starts at physical memory location $5 \times 4096 = 20480$.  

What about the offset within that physical page 5?  The custom is that an
item will have the same offset, no matter whether we are talking about
its virtual address or its physical one.  So, the offset of our
destination in physical page 5 would be 3.  In other words, the
physical address is

\begin{equation}
5 \times 4096 + 3 = 20483
\end{equation}

The CPU would now be ready to execute the instruction, which, to refresh
your memory, was

\begin{Verbatim}[fontsize=\relsize{-2}]
movl $6, 8195
\end{Verbatim}

The CPU knows that the real location of the destination is 20483, not
8195.  It would put 20483 on the address bus, put 6 on the data bus, and
assert the Memory Write line in the control bus, and the instruction
would be done. 

Of course, the same would occur with the instruction

\begin{Verbatim}[fontsize=\relsize{-2}]
movl $6, (%eax)
\end{Verbatim}

if c(EAX) = 8195.

\checkpoint

\subsubsection{Layout of the Page Table}

Suppose the entries in our page table are 32 bits wide, i.e. one word
per entry.\footnote{If we were to look at the source code for the OS, we
would probably see that the page table is stored as a very long array of
type \textbf{unsigned int}, with each array element being one page table
entry.  } Let's label the bits of an entry 31 to 0, where Bit 31 is in
the most-significant (i.e. leftmost) position and Bit 0 is in the least
significant (i.e. rightmost) place. Suppose the format of an entry is as
follows:

\begin{itemize}
\item Bits 31-12: physical page number if resident, disk location if not 
\item Bit 11: 1 if page is resident, 0 if not 
\item Bit 10: 1 if have read permission, 0 if not 
\item Bit 9: 1 if have write permission, 0 if not 
\item Bit 8: 1 if have execute permission, 0 if not 
\item Bit 7: 1 if page is ``dirty,'' 0 if not (see below)
\item Bits 6-0: other information, not discussed here 
\end{itemize}

Now, here is what will happen when the CPU executes the instruction

\begin{Verbatim}[fontsize=\relsize{-2}]
movl $6, 8195
\end{Verbatim}

above:

\begin{itemize}

\item The CPU does the computation in Equation (\ref{ex8195}), and finds
that the requested address is in virtual page 2, offset 3.

\item Since we are dealing with virtual page 2, the CPU will need to go
to get the entry for that virtual page in the page table, as follows.
Suppose the contents of the PTR is 5000.  Then since each entry is 4
bytes long, the table entry of interest here, i.e. the entry for virtual
page 2, is at location 

\begin{equation}
5000 + 4 \times 2 = 5008
\end{equation}

The CPU will read the desired entry from that location, getting, say,
0x000005e0. 

\item The CPU looks at Bits 11-8 of that entry, getting 0xe, finding
that the page is resident (Bit 11 is 1) and that the program has read
and write permission (Bits 10 and 9 are 1) but no execute permission
(Bit 8 is 0). The permission requested was write, so this is OK. 

\item The CPU looks at Bits 31-12, getting 5, so the hardware would know
that virtual page 2 is actually physical page 5.
The virtual offset,
which we found earlier to be 3, is always retained, so the CPU now
knows that the physical address of the virtual location 8195 is

\begin{equation}
5 \times 4096 + 3 = 20483
\end{equation}

\item  The CPU puts the latter on the address bus, puts 6 on the data
bus, and asserts the Write line in the bus. This writes 6 to memory
location 20483, and we are done. 

\checkpoint

\end{itemize}

By the way, all this was for Step C of the above MOV instruction.  The
same actions would take place in Step A.  The value in the PC would be
broken down into a virtual page number and an offset; the virtual page
number would be used as an index into the page table; Bits 10 and 8 in
the page table element would be checked to see whether we have
permission to read and execute that instruction; assuming the
permissions are all right, the physical page number would be obtained
from Bits 31-12 of the page table element; the physical page number
would be combined with the offset to form the physical address; and the
physical address would be placed in the MAR and the instruction fetched.

Recall from above that the upper 20 bits of an address form the page
number, and the lower 12 bits form the offset.  A similar statement
holds for physical addresses and physical page numbers.  So, all the
hardware need do is:  use the upper 20 bits of the virtual address as an
index in the page table (i.e. multiply this by 4 and add to c(PTR); take
bits 31-12 of from the table entry reached in this manner, to get the
physical page number; and finally, concatenate this physical page number
with the lower 12 bits of the original virtual address.  Then the
hardware has the physical address, which it places on the address bus. 

\subsubsection{Page Faults}

Suppose in our example above Bit 11 of the page table entry had been 0,
indicating that the requested page was not in memory.  As mentioned
earlier, this event is known as a \textbf{page fault}. If that occurs,
the CPU will perform an internal interrupt, and will also record the PC
value of the instruction which caused the page fault, so that that
instruction can be restarted after the page fault is processed.  In
Pentium CPUs, the CR2 register is used to store this PC value.  This
will force a jump to the OS. 

The OS will first decide which currently-resident page to replace, then
will write that page back to disk, if the Dirty Bit is set (see below).
The OS will then bring in the requested page from disk.  The OS would
then update two entries in the page table: (a) it would change the entry
for the page which was replaced, changing Bit 11 to 0 to indicate the
page is not resident, and changing Bits 31-12 and possible Bit 7; and
(b) the OS would update the page table entry of the new item's page, to
indicate that the new item is resident now in memory (setting Bit 11 to
1), show where it resides (by filling in Bits 31-12), and setting Bit 7
to 0.

The role of the Dirty Bit is as follows:  When a page is brought into
memory from disk, this bit will be set to 0.  Subsequently, if the page
is written to, the bit will be changed to 1.  So, when it comes time to
evict the page from memory, the Dirty Bit will tell us whether there is
any discrepancy between the contents of this page in memory and its copy
on disk.  If there is a difference, the OS must write the new contents
back to disk.  That means all 4096 bytes of the page.  We must
write back the whole page, because we don't know what bytes in the page
have changed.  The Dirty Bit only tells us that there has been some
change(s), not where the change(s) are.  So, if the Dirty Bit is 0, we
avoid a time-consuming disk write.

\checkpoint

Since accessing the disk is far, far slower than accessing memory, a program
will run quite slowly if it has too many page faults. If for example your PC
at home does not have enough memory, you will find that you often have to wait
while a large application program is loading, during which time you can hear
the disk drive doing a lot of work, as the OS ejects many currently-resident
pages to bring in the new application.


\subsubsection{Access Violations}

If on the other hand an access violation occurs, the OS will announce an
error---in Unix (Linux, MacOS etc.), referred to as a
\textbf{segmentation fault}---and kill the process, i.e. remove it from
the process table.

For example, considering the following code:

\begin{Verbatim}[fontsize=\relsize{-2},numbers=left]
int q[200];

main()

{  int i;

   for (i = 0; i < 2000; i++) {
      q[i] = i;
   }

}
\end{Verbatim}

Notice that the programmer has apparently made an error in the loop, setting
up 2000 iterations instead of 200. The C compiler will not catch this at compile
time, nor will the machine code generated by the compiler check that the array
index is out of bounds at execution time.

If this program is run on a non-VM platform,\footnote{ Recall that
{}``VM platform{}'' requires both that our CPU has VM capability, and
that our OS uses this capability.  } then it will merrily execute
without any apparent error. It will simply write to the 1800 words which
follow the end of the array {\bf q}. This may or may not be harmful,
depending on what those words had been used for.

But on a VM platform, in our case Unix, an error will indeed be
reported, with a {}``Segmentation fault{}'' message.\footnote{On
Microsoft Windows systems, it's called a ``general protection error.''}
However, as we look into how this comes about, the timing of the error
may surprise you. The error is not likely to occur when {\bf i} = 200;
it is likely to be much later than that.

To illustrate this, I ran this program under {\bf gdb} so that I could
take a look at the address of {\bf q[199]}.\footnote{Or I could have
added a {\bf printf()} statement to get such information.  Note by the
way that either running under {\bf gdb} or adding {\bf printf()}
statement will change the load locations of the program, and thus
affect the results.}  After running this program, I found that the seg
fault occurred not at {\bf i} = 200, but actually at {\bf i} = 728.
Let's see why.

From queries to {\bf gdb} I found that the array {\bf q} ended at
0x080497bf, i.e. the last byte of {\bf q[199]} was at that address.  On
Intel machines, the page size is 4096 bytes, so a virtual address breaks
down into a 20-bit page number and a 12-bit offset, just as in Section
\ref{trans} above. In our case here, {\bf q} ends in virtual page number
0x8049 = 32841, offset 0x7bf = 1983.  So, after {\bf q[199]}, there are
still 4096-1984 = 2112 bytes left in the page.  That amount of space
holds 2112/4 = 528 {\bf int} variables, i.e. elements ``200'' through
``727'' of {\bf q}.  Those elements of {\bf q} don't exist, of course,
but as discussed in Chapter \ref{chap:datarep} the compiler will not complain.
Neither will the hardware, as we will be writing to a page for which we
do have write permission.  But when {\bf i} becomes 728, that will take
us to a new page, one for which we don't have write (or any other)
permission; the hardware will detect this and trigger the seg fault.

\checkpoint

We could get a seg fault not only by accessing off-limits data items,
but also by trying to execute code at an off-limits location.  For
example, suppose in the above example {\bf q} had been local instead of
global.  Then it would be on the stack.  As we go past the end of {\bf
q}, we would go deeper and deeper into the stack.  This may not directly
cause a seg fault, if the stack already starts out fairly large and is
stored in physically contiguous pages in memory.  But we would overwrite
all the preceding stack frames, including return addresses.  When we
tried to ``return'' to those addresses,\footnote{Recall that {\bf
main()} is indeed called by other code, as explained in Chapter
\ref{chap:sub}.} we would likely attempt to execute in a page for which
we do not have execute permission, thus causing a seg fault.

As another example of violating execute permission, consider the
following code, with a pointer to a function:\footnote{If your classes
on C/C++ did not cover this important topic of pointers to functions,
the comments in the code below should be enough to introduce it to you.
Pointers to functions are used in many applications, such as threaded
programs, as mentioned earlier.}

\begin{Verbatim}[fontsize=\relsize{-2},numbers=left]
int f(int x)
{ return x*x; }

// review of pointers to functions in C/C++: below p is a pointer to a
// function; the first int means that whatever function p points to, it
// returns an int value; the second int means that whatever function p
// points to, it has one argument, an int
int (*p)(int);

main()
{  int u;

   p = f;  // point p to f
   u = (*p)(5);  // call f with argument 5 
   printf("%d\n",u);  // prints 25
}
\end{Verbatim}

If we were to forget to include the line

\begin{verbatim}
   p = f;  // point p to f
\end{verbatim}

then the variable {\bf p} would not point to a function, and we would
attempt to execute ``code'' in a location off limits to us when we tried

\begin{verbatim}
   u = (*p)(5);
\end{verbatim}

A seg fault would result.

\checkpoint

\subsection{VM and Context Switches}

A context switch will be accompanied by a switch to a new page table.

Say for example Process A has been running, but its turn ends and
Process B is given a turn.  Of course we must now use B's page table.
So, before starting B's turn, the OS must change the PTR to point to B's
page table. 

Thus, there is actually no such thing as ``the'' page table.  There are
many page tables.  The term ``the'' page table merely means the page
table which PTR currently points to.

\subsection{Improving Performance---TLBs}

Virtual memory comes at a big cost, in the form of overhead incurred by
accessing the page tables. For this reason, the hardware will also
typically include a \textbf{translation lookaside buffer} (TLB). This is
a special cache to keep a copy of part of the page table in the CPU, to
reduce the number of times one must access memory, where the page table
resides.

One might ask, why not store the entire page table in the TLB?  First of
all, the page table theoretically consists of $2^{32}/2^{12}$ entries.
This is somewhat large. Though there are ways around this, but we would
need to change this large amount of data at each context switch, which
would really slow down context switching.  Note by the way that we would
need to have special instructions which the OS could use to write to the
TLB.  That's OK, though, and there are other special instructions for
the OS already.

% \subsection{Intel Page Tables}
% 
% On Intel machines, each process actually has many tables, not just one. And
% some of them may even be nonresident currently; in other words, even the page
% tables are paged!
%
% There is a {}``page table table,{}'' which serves as directory of page
% tables for the current process. A special register, CR3, points to the
% page table table.
%
% A virtual address is broken down into three fields, rather than the two in our
% descriptions above:
% 
% \begin{itemize}
% \item Bits 31-22: i, the index into the page table table 
% \item Bits 21-12: j, index into some page table 
% \item Bits 11-0: the offset 
% \end{itemize}
% 
% So, in processing a virtual address, the CPU will first fetch from the
% address c+4{*}i, where c is the current contents of CR3. That fetched
% word w will tell the CPU whether the page table for this virtual address
% is currently resident in memory. If that table is resident, w will show
% the address of the table, d, while if it is nonresident, it will show
% where on disk to get it. The CPU will bring in the table from disk in
% the latter case. Once the CPU has d, it will then fetch from the address
% d+4{*}j, yielding the page table entry for the item our running program
% has requested. At this point, the operations are the same as described
% earlier.

\subsection{The Role of Caches in VM Systems}
\label{cacherole}

Up to now, we have been assuming that the machine doesn't have a
cache.\footnote{Except for a TLB, which is of course very specialized.}
But in fact most machines which use VM also have caches, and in such
cases, what roles do the caches play?

The central point is that speed is still an issue.  The CPU will look
for an item in its cache first, since the cache is internal to (or at
least near) the CPU.  If there is a cache miss, the CPU will go to
memory for the item.  If the item is resident in memory, the entire
block containing the item will be copied to the cache.  If the item is
not resident, then we have a page fault, and the page must be copied to
memory from disk, a very slow process, before the processing of the
cache miss can occur.

\subsubsection{Addressing}

An issue that arises is whether the cache is the CPU will {\bf virtually
addressed} or {\bf physically addressed}.  Suppose for instance the
instruction being executed reads from virtual address 200.  If the cache
is virtually addressed, then the CPU would do its cache lookup using 200
as its index.  On the other hand, if the cache is physically addressed,
then the CPU would first convert the 200 to its physical address (by
checking the TLB, and then the page table if need be), and then do the
cache lookup based on that address.

A possible advantage of virtual addressing of caches would be that if we
have a cache hit, we eliminate the time delay needed for the
virtual-to-physical address translation.  Moreover, since we usually
will have a hit, we can afford to put the TLB in not-so-fast memory
external to the CPU, which is done in some MIPS models.

On the other hand, with physical cache addressing, two different
processes could both have separate, unrelated copies of different blocks
in the cache at the same time, but with the same virtual addresses.
They could coexist in the cache since their physical addresses would be
different.  That would mean we would not have to flush the cache at each
new timeslice, possible increasing performance. 

\checkpoint

\subsubsection{Hardware Vs. Software}

Note that cache design is entirely a hardware issue.  The cache lookup
and the block replacement upon a miss is hard-wired into the circuitry.
By contrast, in the VM case it is a mixture of hardware and software.
The hardware does the page table lookup, and checks page residency and
access permissions.  But it is the software---the OS---which creates and
maintains the page table.  Moreover, when a page fault occurs, it is the
OS that does the page replacement and updates the page table.

So, you could have two different versions of Unix, say,\footnote{Or,
Linux versus Windows, etc.} running on the same machine, using the same
compiler, etc. and yet they may have quite different page fault rates.
The page replacement policy for one OS may work out better for this
particular program than does the one of the other OS.  

Note that the OS can tell you how many page faults your program has had
(see the {\bf time} command below); each page fault causes the OS to run,
so the OS can keep track of how many page faults your program had
incurred.  By contrast, the OS can NOT keep track of how many cache
misses your program has had, since the OS is not involved in handling
cache misses; it is done entirely by the hardware.

\subsection{Making These Concepts Concrete: Commands You Can Try Yourself}

The Unix \textbf{time} command will report how much time your program
takes to run, how many page faults it generated, etc. Place it just
before your program's name on the command line. (This program could be
either one you wrote, or something like, say, \textbf{gcc}.) For
example, if you have a program \textbf{x} with argument 12, type

\begin{verbatim}
% time x 12
\end{verbatim}

instead of

\begin{verbatim}
% x 12
\end{verbatim}

In fact, trying running this several times in quick succession.  You
main find a reduction in page faults, since the required pages which
caused page faults in the first run might be resident in later runs.

Also, the {\bf top} program is very good, displaying lots of good
information on the memory usage of each process.

\section{A Bit More on System Calls}

Recall that the OS makes available to application programs services such
as I/, process management, etc.  It does this by making available
functions that application programs can call.  Since these are function
in the OS, calls to them are known as {\bf system calls}.

When you call {\bf printf()}, for instance, it is just in the C library,
not the OS, but it in turn calls {\bf write()} which \textit{is} in the
OS.\footnote{There is a slight complication here.  Calling {\bf write()}
from C will take us to a function in the C library of that name.  It in
turn will call the OS function.  We will ignore this distinction here.} 
The call to {\bf write()} is a system call.  

Or you can make system calls in your own code.  For example, try
compiling and running this code:

\begin{Verbatim}[fontsize=\relsize{-2}]
main()
{ write(1,"abc\n",4); }  
\end{Verbatim}

The function {\bf write()} takes three arguments:  the file handle
(here 1, for the Unix ``file'' {\bf stdout}, i.e. the screen); a pointer
to the array of characters to be written [note that NULL characters mean
nothing to {\bf write()}]; and the number of characters to be
written.  

Similarly, executing a {\bf cout} statement in C++ ultimately results in
a call to {\bf write()} too.  In fact, the G++ compiler in GCC
translates {\bf cout} statements to calls to {\bf printf()}, which as we
saw calls {\bf write()}.  Check this by writing a small C++ program with
a {\bf cout} in it, and then running the compiled code under {\bf
strace}.

A non-I/O example familiar to you is the {\bf execve()} service is used
by one program to start the execution of another.  Another non-I/O
example is {\bf getpid()}, which will return the process number of the
program which calls it. 

Calling {\bf write()} means the OS is now running, and {\bf write()}
will ultimately result in the OS running code which uses the OUT machine
instruction.  Recall, though, that we want to arrange things so that
\underline{only} can execute instructions like Intel's IN and OUT.  So
the hardware is designed so that these instructions can be executed only
in Kernel Mode.

For this reason, one usually cannot implement a system call using an
ordinary subroutine CALL instruction, because we need to have a
mechanism that will change the machine to Kernel Mode. (Clearly, we
cannot just have an instruction to do this, since ordinary user programs
could execute this instruction and thus get into Kernel Mode themselves,
wreaking all kinds of havoc!) Another problem is that the linker will
not know where in the OS the desired subroutine resides.

Instead, system calls are implemented via an instruction type which is
called a \textbf{software interrupt}. On Intel machines, this takes the
form of the INT instruction, which has one operand.

We will assume Linux in the remainder of this subsection, in which case
the operand is 0x80.\footnote{Windows uses 0x21.} In other words, the
call to {\bf write()} in your C program (or in {\bf printf()} or {\bf
cout}, which call {\bf write()} will execute

\begin{verbatim}
...  # code to put parameters values into designated registers
int $0x80
\end{verbatim}

The INT instruction works like a hardware interrupt, in the sense
that it will force a jump to the OS, and change the privilege level to
Kernel Mode, enabling the OS to execute the privileged instructions it
needs. You should keep in mind, though, that here the {}``interrupt{}''
is caused deliberately by the program which gets {}``interrupted,{}''
via an INT instruction. This is much different from the case of a
hardware interrupt, which is an action totally unrelated to the program
which is interrupted.

The operand, 0x80 above, is the analog of the device number in the case
of hardware interrupts. The CPU will jump to the location indicated by
the vector at c(IDT)+8{*}0x80.\footnote{By the way, users could cause
mischief by changing this area of memory, so the OS would set up their
page tables to place it off limits.}

When the OS is done, it will execute an IRET instruction to return
to the application program which made the system call. The {\bf iret}
also makes a change back to User Mode.

\checkpoint

As indicated above, a system call generally has parameters, just as
ordinary subroutine calls do. One parameter is common to all the
services---the service number, which is passed to the OS via the EAX
register. Other registers may be used too, depending on the service.

As an example, the following Intel Linux assembly language program
writes the string ``ABC'' and an end-of-line to the screen, and then
exits:

\begin{Verbatim}[fontsize=\relsize{-2}]
.text
hi:  .string "ABC\n"
.globl _start
_start:

      # write "ABC\n" to the screen
      movl $4, %eax     # the write() system call, number 4 obtained
                        # from /usr/include/asm/unistd.h
      movl $1, %ebx     # 1 = file handle for stdout
      movl $hi, %ecx    # write from where
      movl $4, %edx     # write how many bytes
      int $0x80         # system call

      # call exit()
      movl $1, %eax     # exit() is system call number 1
      int $0x80         # system call
\end{Verbatim}

For this particular OS service, {\bf write()}, the parameters are passed in
the registers EBX, ECX and EDX (and, as mentioned before, with EAX
specifying which service we want).  Whoever wrote {\bf write()} has
forced us to use those registers for those purposes.

Note that we do indeed need to tell {\bf write()} how many bytes to
write.  It will NOT stop if it encounters a null character.

Here are some examples of the numbers (to be placed in EAX before
calling {\bf int \$0x80}) of other system services:

\begin{verbatim}
read              3
file open         5
execve           11
chdir            12
kill             37
\end{verbatim}

\section{OS File Management}

The OS will maintain a table showing the starting sectors of all files
on the disk.  (The table itself is on the disk.)  The reason that the
table need store only the starting sector of a given file is that the
various sectors of the file can be linked together in ``linked-list''
fashion.  In other words, at the end of the first sector of a file, the
OS will store information stating the track and sector numbers of the
next sector of the file.

The OS must also maintain a table showing unused sectors.  When a
user creates a new file, the OS checks this table to find space to put
the file.  Of course, the OS must then update its file table
accordingly.

If a user deletes a file, the OS will update both tables, removing the
file's entry in the file table, and adding the former file's space to
the unused sector table.\footnote{However, the file contents are still
there.  This is how ``undelete'' programs work, by attempting to recover
the sectors liberated when the user (accidentally) deleted the file.}

As files get created and deleted throughout the usage of a machine, the
set of unused sectors typically becomes like a patchwork quilt, with the
unused sectors dispersed at random places all around the disk.  This
means that when a new file is created, then {\it its} sectors will be
dispersed all around the disk.  This has a negative impact on
performance, especially seek time.  Thus OSs will often have some kind
of {\bf defragmenting} program, which will rearrange the positioning of
files on the disk, so that the sectors of each individual file are
physically close to each other on the disk.

\section{To Learn More}

There are several books on the Linux OS kernel.  Before you go into such
detail, though, you may wish to have a look at the Web pages
\url{http://www.tldp.org/LDP/tlk/tlk.html} and
\url{http://learnlinux.tsf.org.za/courses/build/internals/}, which
appear to be excellent overviews.  

\section{Intel Pentium Architecture}
\label{pentium}

We will assume Pentia with 32-bit word and address size.  On Linux the
CPU runs in {\bf flat mode}, meaning that the entire address space of
$2^{32}$ bytes is available to the programmer.

The main registers of interest to us here are named EAX, EBX, ECX, EDX,
ESI, EDI, EBP and ESP.\footnote{We will use the all-caps notation, EAX,
EBX, etc. to discuss registers in the text, even though in program code
they appear as {\bf \%eax}, {\bf \%ebx}, ...}  ESP is used as the stack
pointer.  The program counter is EIP.  There is also a condition codes
register EFLAGS, which is used to record whether the results of
operations are zero, negative and so on.

In this document we use AT\&T syntax for assembly code.  Here is some
sample code, which adds 4 elements of an array pointed to by ECX, with
the sum being stored in EBX:

\begin{Verbatim}[fontsize=\relsize{-2},numbers=left]
      movl $4, %eax  # copy the number 4 to EAX
      movl $0, %ebx  # copy the number 0 to EBX
      movl $x, %ecx  # copy the address of the memory location
                     # whose label is x to ECX
top:  addl (%ecx), %ebx  # add the memory word pointed to by ECS to EBX
      addl $4, %ecx  # add the number 4 to ECS
      decl %eax  # decrement EAX by 1
      jnz top  # if EAX not 0, then jump to top
\end{Verbatim}

Pentia use vector tables for interrupt handlers.  Typical Pentium-based
computers include one or more Intel 8259A chips for controlling and
prioritizing bus access by input/output devices. 

As with most modern processor chips, Pentia are highly pipelined and
superscalar, the latter term meaning muliple ALU compoents, thus
allowing more than one instruction to execute per clock cycle.

For more information see the various PDF files in
\url{http://heather.cs.ucdavis.edu/~matloff/50/PLN}.