\documentclass[11pt]{article} \setlength{\oddsidemargin}{0in} \setlength{\evensidemargin}{0in} \setlength{\topmargin}{0.0in} \setlength{\headheight}{0in} \setlength{\headsep}{0.5in} \setlength{\textwidth}{6.5in} \setlength{\textheight}{8.5in} \setlength{\parindent}{0in} \setlength{\parskip}{0.1in} \usepackage{times} \usepackage{psfig} \usepackage{fancyheadings} \begin{document} \pagestyle{fancy} \cfoot{Pipelining: \thepage} \title{Pipelining} \author{N.S. Matloff \\ University of California at Davis} \date{February 23, 1999} \maketitle \tableofcontents \section{General Issues} In spite of advances in chip technology, there are still real limits to the degree of speed increase one can attain in a CPU, due to physical constraints, e.g. the speed of light, heat dissipation, and so on. So, a crucial source of speed in modern systems---and even more so for systems of the future---is to do operations in parallel. One type of parallel processing is {\bf pipelining}. Pipelining is fairly analogous to an assembly line, say in an automobile factory. The process of building a car is broken down into a number of stages, each one of approximately equal duration, and at any given time different cars are at different stages of production---but all simultaneously, the key to parallelism. As a quick example, imagine a widget factory, in which the construction of a widget is broken down into three separate stages, called stage A, stage B and stage C, with each stage taking one minute of time. Suppose we have a continuing supply of raw materials for widget-making entering the factory, and that at time 0, our first widget goes into production. It will be completely finished at time 3, but the point is that once first widget leaves stage A at time 1, the second widget goes into production. While the first widget is at stage B, the second widget is at stage A. When the first widget is at stage C, the second one is at B, and a third one now starts at A. So, what is the rate of widget production? Without pipelining, the rate would be one widget produced every 3 minutes, but with pipelining the rate is one widget produced every 1 minute! For example, the second widget will exit stage C one minute after the first widget, the third widget will exit one minute after the second one, etc. Every minute a newly-completed widget will exit the pipe. But all of this depends on having the continuing supply of widget-making raw materials to enter the pipe, with one raw widget entering stage A per minute. If, for example, purchase orders for widgets come in only sporadically, then there will rarely be more than one widget in production at any given time, and the benefits of pipelining will be lost. \subsection{Example: Instruction Prefetch} This is an example of pipelining with which you are already familiar, at least to some degree. The processor will try to fetch the next instruction while processing the current one. As a simple example consider a 3-stage instruction pipeline, with stage A being the instruction fetch, stage B being the decode and stage C being the execution. Assume that the three stages have equal times.\footnote{Typically the instruction fetch will take more time, but we might have an instruction cache to make this assumption reasonable. Note, by the way, that if the stage times \underline {are} unequal, we have to set the clock to the time of the longest stage.} Consider instructions at addresses 100, 101 and 102. Then we could have the following sequence of events: \begin{verbatim} time 0: start fetch of I100 time 1: start decode of I100, start fetch of I101 time 2: start execute of I100, start decode of I101, start fetch of I102 \end{verbatim} and so on. But what is the instruction-fetch analog of a ``lack of widget-making materials''? The answer is branch (jump) instructions! Suppose, for example, that I100 is ``if negative then jump to 132.'' We won't know whether the jump is actually taken until the end of time slot 2, in which case the remainder of the pipe (I101, I102) will be useless. The instruction fetch for I132 won't be started until time 3, and its execution thus won't finish until the end of time 5. In other words, instead of ``widgets'' exiting the pipe on every clock, i.e. at the ends of clock 2, clock 3, clock 4, clock 5, and so on, we will have them exiting at clocks 2 and 5, missing clocks 3 and 4. We have two wasted time slots. \subsection{A More Elaborate Pipeline Example} Consider a polynomial of degree 3: \begin{equation} a x ^ 3 + b x ^2 + c x + d \end{equation} Suppose that for fixed a, b, c and d, we wish to repeatedly evaluate this polynomial, for different values of x, say because we are graphing the polynomial. Say we are designing a special-purpose chip to evaluate third-degree polynomials. Since speed is undoubtedly an issue (otherwise, we would not be designing a special-purpose chip), we should investigate pipelined designs. \subsubsection{A First Try} We could set up the following pipeline: \vspace{0.2in} \par \psfig{file=PolyPipe.eps} \par Here the A's and M's are adders and multipliers, respectively. Though not shown, there are also registers below each M, which hold the value of x, which is passed in from the previous stage of the pipeline, and registers just to the left of the M's, to hold the value passed on by the previous stage.\footnote{We would not need a register to the left of the M in the first stage, since its input never changes. Make sure you understand this. Ask if you do not completely grasp it.} Also not shown is the clock, which would feed into each register. The way the system works is as follows. First, rewrite the polynomial in the form \begin{equation} x \{ x [ x(a) + b ] + c \} + d \end{equation} Then at stage 0, M inputs a and x, and outputs x(a) to A, which then forms x(a)+b and sends this value to M at stage 1. The latter then forms x[x(a)+b] and sends it to A, which forms x[x(a)+b]+c, and so on. Note carefully that at any given time, there will be three values of x in this circuit, a different x at each of the three stages. Say for example that a = 2, b = 5 and c = 1, and that the stream of x values entering the pipe is 6, 10, 4, 29, ... If the 6 enters at time 0, then at time 2, then the x at the third (i.e. rightmost) stage will be 6; at the second stage the x value will be 10; and at the first stage the x value will be 4. Moreover, at that time, the other input to the M unit at the second stage will be $2 \times 10 + 5$, which is the output of the first stage for x = 10. In other words, both inputs to M at the second stage will correspond to x = 10. Similarly, at that time both inputs to M at the third stage will correspond to x = 6. Note that the clock here will have to be set to the maximum M time plus the maximum A time. This combined time might be long. \subsubsection{A More Refined Approach} We could split each stage above into two stages: \vspace{0.2in} \par \psfig{file=PolyPipe2.eps} \par Here again there would be registers: to the left of each M (except for the first); to the left of each A, to hold incoming results from the previous stage; one beneath each M; and one \underline{between} each pair of consecutive Ms, in order to force each x value to take two clock cycles (instead of one) to get from one M to the next. Again, the clock lines are not shown either. How much faster would this new pipe be? The answer depends on the relative times of the M and A units. An M unit is likely to need much more time than an A unit, since the M unit itself probably does some additions. Thus the M's will be the ``weak link in the chain,'' and the clock cycle time will have to be set to the maximum time needed to do a multiplication. If that time is, say, 4 times longer than the maximum time for an addition, then, this new pipe will have a cycle time only 20\% shorter than the original one, and thus a throughput rate only about 20\% higher.\footnote{Again, make sure you understand this.} Note that even though this new pipe has twice as many stages as the original design, for any given x, the computation time will not double. It will increase somewhat, though, owing to the fact that the A time is less than the M time. \subsubsection{A Feedback Version} If we were short on space but willing to sacrifice some speed, we could have a feedback version of the polynomial pipe: \vspace{0.2in} \par \psfig{file=PolyPipe3.eps} \par Again the registers and clock lines are not shown. Note, though, that the clock line will feed into not only the registers but also the box labeled ``mod 3'' and to the bottom MUX. Here we save space by having only one M and only one A. The intermediate values are repeatedly fed back into the M/A unit. The clock time, mod 3, controls the multiplexer, so that the proper one of b, c and d is input to the adder. In this design, the computation for only one x can be handled at any given time. However, a refinement could be made in which we handled two x computations at a time; here, the M unit would repeatedly input the two x's in alternative clock cycles, requiring another MUX. When the addition for one x is being done, the multiplication for the other x is being done. \subsection{Example: Floating-Point Addition} \subsubsection{Review} {\bf Floating-point} numbers, as in for example the type {\bf float} in the C language, store numbers corresponding to ``scientific notation'' such as $7.2 \times 10^5$. In the latter number, 7.2 is called the {\bf mantissa} and 5 is the {\bf exponent}. If for instance we have the C code \begin{verbatim} float X; \end{verbatim} Then on a 32-bit machine, the compiler would allocate one 32-bit word for X (just as it would if X had been of type {\bf int}). Within that word, a certain set of the 32 bits would be used for the mantissa and the rest for the exponent. (Both of these values would be signed, i.e. could take on either positive or negative values.) To make this concrete, here is a simplified example. Suppose our word size is 6 bits, and that we only wish to store nonnegative numbers. The leftmost (i.e. most significant) 4 bits form the mantissa m and the last 2 bits comprise the exponent e, which is going to serve on a base of 2. The m field is to be considered to be an unsigned integer, while e is a 2s-complement signed integer. The number stored is $m \times 2^e$. As an example, verify for yourself that \begin{verbatim} 001110 + 010111 = 110110 \end{verbatim} \subsubsection{The Pipeline} A floating-point adder could be implemented as a four-stage pipe as follows: \begin{itemize} \item [S1.] Compare exponents. \item [S2.] Shift the mantissa paired with the lesser exponent so that two exponents match, adjusting the paired exponent accordingly. \item [S3.] Add mantissas (updating exponents, if carry-out occurs). \item [S4.] Normalize to standard form. \end{itemize} The standard form referred to in S4 involves making sure that the first bit in the mantissa is a 1, except if the mantissa is 0. (We did not do this in our simplified example above.) To normalize to standard form, we shift the mantissa left and subtract from the exponent. Once again, pipelining will be useful only if we have a continuing supply of tasks entering the pipe. The example in the next section will provide such a situation. \subsection{Memory Considerations} As stressed earlier, the key to the success of a pipelined design is to keep the pipe filled. If the pipe is part of a CPU, this may be difficult, as it may be hard to bring in numbers from memory quickly enough. Consider the case of {\it vector processors}, so called because their instruction sets include operations whose operands are vectors rather than scalars. As our running example here, consider an instruction VADD X,Y,N where X and Y are arrays of type {\bf float}, and N is an {\bf int} denoting the array length. The instruction will calculate X[0]+Y[0] and put the result back in X[0], put X[1]+Y[1] in X[1], and so on. Please note carefully that VADD would be one single instruction, doing the work normally done by a whole {\bf for} loop of instructions. It should run faster than the loop, for at least two reasons : \begin{itemize} \item Fewer instruction fetches. \item Pipelined operation. \end{itemize} For the ALU, we will assume that the floating-point addition unit is the four-stage pipe described earlier. So, in the first time period, the exponents of X[0] and Y[0] will be compared. In the second time period, one of the mantissas of X[0] and Y[0] will be shifted, \underline{and} the exponents of X[1] and Y[1] compared, etc. The problem is fetching the components of X and Y fast enough so that one pair can enter the pipe in each time period. In order to do this, we can use {\it low-order memory interleaving}. Consider the following setup: \vspace{0.2in} \par \psfig{file=Interleaving.eps} \par For simplicity, we are assuming 16-bit word and address widths. The notation A[m:n] means address bits m through n (15 being the most significant), with a similar notation D[m:n] for data bits. Again for simplicity, suppose that each Mj stores 16K words; that would mean that each chip has 14 address pins, attached to the lines labeled A[15:2], and 16 data pins, leading to the MUX.\footnote{We might have all the Mj hanging from a single data bus leading to the CPU, with M0 sending a value first, then M1 immediately afterward, then M2, etc. In that case, we could even describe the memory as being pipelined.} Suppose that individual bytes within a word are not addressable, so that consecutive words have consecutive addresses. Using low-order interleaving, word i will be stored in Mj, where j = i mod 4. In other words, word 0 is in M0, word 1 is in M1, word 2 is in M2, word 3 is in M3, word 4 is back in M0, word 5 is in M1, and so on. In other words, word m within Mn plays the role of word 4m+n in the CPU's address space. (Make absolutely sure that you understand all of these numbers and relations before continuing. You may wish to review the material on memory chips and systems in the digital logic handout, {\it Digital Logic---the Bare Minimum}.) Let us see how this setup can be used to read a vector, say X, from memory in a very rapid manner.\footnote{Our example is of course not quite that simple. We are actually reading two vectors, X and Y, and in addition are writing to one of them, X. One possible solution would be to use {\it multiport} memory, which allows more than one access to a memory chip at a time. But to keep things simple, suppose we are just reading X.} Suppose the CPU needs to read word 26, which is 0000000000011010 in binary. Then it puts 00000000000110 on the upper address lines in the picture, and all four Mj respond: M0, M1, M2 and M3 send words 24, 25, 26 and 27, respectively, which will be saved in latches at the entrance to the MUX (not shown). The CPU then puts the lower two bits of the address, 10, on the lower address lines in the picture, which are attached to the selector inputs of the MUX, with the result that the MUX selects whatever M2 sent it (since 10 binary is 2 decimal); this is word 26, just what the CPU wanted. The point is that the CPU can now pick up words 24, 25 and 27 ``for free,'' i.e. with almost no extra time spent, if it needs them.\footnote{The word ``almost'' refers to the fact that although there is some amount of gate delay in the MUX, that delay should be much smaller than the delay in an Mj.} For instance, if N = 2 above and X starts at word 26, the CPU will now put 11 on the lower address lines which lead to the MUX, and thus pick up word 27. The result is that the CPU has read two words in just a little more time than it takes to read one word. In fact, if it so happens that the application program running on the CPU needed to read a four-element array beginning at word 24, we would be able to read four words in the time normally needed to read just one. In other words, the setup here can increase memory access speed by a factor of nearly four. Of course, the speedup would occur because we happen to need to read four consecutive words of memory. But even that condition is not sufficient for achieving a four-fold speedup. For example, suppose we need to read the four words 26, 27, 28 and 29. We would be able to quickly get the first two of these, words 26 and 27. But after getting them we would have to issue a separate ``major memory request'' for words 28 and 29, by putting a new value, 00000000000111, on the upper address lines in the picture (and then issuing the values 00 and 01 to the MUX). So, here we would have a speedup of only about two, not four. The applications which run on vector machines, tend to be heavily mathematical, such as matrix operations. In this setting, to write efficient software, one must pay attention to the internal storage of arrays. One-dimensional arrays are stored contiguously, but higher-dimensional arrays are more complicated. In C/C++, two-dimensional arrays are stored in {\it row-major order}. What this means is that the first part of the storage consists of all of row 0, right after which is stored all of row 1, and so on. For example, suppose we have a matrix declared as \begin{verbatim} int z[5][5]; \end{verbatim} and again for simplicity, assume that individual bytes are not addressable on our machine. Say that z begins at word 520. Then z[0][0] is in word 520, z[0][1] in 521, ..., z[0][4] in 524, z[1][0] is in word 525, z[1][1] in 526, and so on. Thus, while a matrix row will be stored in contiguous locations, a matrix column will not. In the case of z here, for instance, consecutive elements within a matrix column will be spaced exactly five words apart. Thus we will be able to get the benefit of memory ``pipelining'' for rows but not columns. Accordingly, our program might run much faster if we work with matrix transposes in some cases. \section{Scheduling of Feedback Pipes} As noted earlier, in some pipes a task may need to visit some pipe stages more than once, forming a feedback structure. However, in most such cases, not all stages are in use during all time slots while a task is being processed.\footnote{I am using the term {\it time slot} here to mean one stage time.} This suggests the possibility of having more than one task in the pipe at the same time. Note that in a nonfeedback pipe, we always have more than one task in the pipe at one time (assuming we have a ready supply of tasks waiting to be processed), but this may not be possible in a feedback pipe, since separate tasks may need to use the same stage during the same time slot. Consider an example in {\it Computer Architecture and Organization}, by John Hayes (McGraw-Hill, 1988). We start with the same four-stage pipe for floating-point addition in our examples above, but with a different type of task. In the VADD example, the task was to add a pair of numbers, and the pipe was nonfeedback. In our example here, though, our task will be to find the sum of eight numbers, $b_1+b_2+...+b_8$. The question then becomes, Can we have more than one eight-number set in the pipe at the same time, and if so, at what times should the different sets be started? Let us first look in more detail at how \underline{one} task, i.e. the summing of eight numbers, is accomplished: Roughly speaking, first the sums $b_1+b_5$, $b_2+b_6$, $b_3+b_7$ and $b_4+b_8$ will be computed by running the corresponding pairs of numbers through the pipeline. Then the sums $b_1+b_2+b_5+b_6$ and $b_3+b_4+b_7+b_8$ will be computed, and so. Here is how it will work: Say there are ``left'' and ``right'' inputs at the entrance to the pipe, with MUXs at each. The MUXs will select either a new input or the feedback from the exit of the pipe. In time slot 1, $b_1$ and 0.0 enter the pipe, in preparation to compute $b_1+0.0$. It is assumed here that the $b_i$ are arriving one at a time from memory. Thus $b_5$ is not available yet, so we cannot start computing $b_1+b_5$ yet, and we have to settle for computing the ``sum'' $b_1+0.0$. Say $b_1$ enters the left input of the pipe, and the constant 0.0 enters at the right. In time slot 2, $b_2$ enters the left input from memory, and 0.0 enters the right input. In time slot 3, $b_3$ and 0.0 enter, and so on. At the end of time slot 4, $b_1+0.0$ finishes at S4, and is recycled back to S1, say to the left MUX. It is joined by $b_5$ entering the right input from memory, and the computation of $b_1+b_5$ begins in time slot 5. That sum exits S4 at the end of time slot 8, and is held in a register for one more time slot, after which $b_2+b_6$ exits that stage at the end of time slot 9. Both of these sums are then recycled back to S1, so that the computation of $b_1+b_2+b_5+b_6$ can be begun in time slot 10, and so on. The entire process turns out to require 19 time slots, i.e. $b_1+b_2+b_3+b_4+b_5+b_6+b_7+b_8$ is finally ready at the end of time slot 19. That is already a big improvement, because without pipelining the sum of eight numbers would take (4+2+1)x4 = 28 time slots, much longer than 19.\footnote{Each sum operation takes 4 time slots to perform. If you look at which sums are performed to get the final grand total of eight numbers above, you will see that there are 4+2+1 such sums to be done.} But we want to be even greedier, and have more than one set of eight numbers being in the pipe at the same time. In other words, we want to start the computation of some sum $c_1+c_2+...+c_8$ while the computation of $b_1+b_2+...+b_8$ is still in progress. Then the following {\it reservation table} shows which stages are busy at which time slots for the computation of $b_1+b_2+...+b_8$: \newpage \begin{verbatim} 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 S1 x x x x x x x x x x x S2 x x x x x x x x x x x S3 x x x x x x x x x x x S4 x x x x x x x x x x x \end{verbatim} Now consider what happens if we try to have more than one eight-number set in the pipe at a time. Suppose that at time slot, say, 527, the pipe is empty and we start an eight-number set's computation. Let us call this set Set I. Note that all times will be relative. For that set, ``time slot 1'' in the reservation table for this first set is absolute time 527, time slot 2 is absolute time 528, and so on. This set will finish at the end of absolute time slot 545 (527+19-1). Now, when can we start the computation of a second set of eight numbers, which we will call Set II? For instance, could we do so, say, in absolute time slot 530? The answer is no, for the following reason: If we were to start Set II at 530, then its ``time slot 1'' would be at time 530, at which time it would use S1, as can be seen from the reservation table. But absolute time slot 530 is relative time slot 4 for Set I, and from column 4 in the reservation table we can see that Set I is using S1 at that time. In other words, if we were to start Set II at 530, it would immediately {\it collide} with Set I. The reservation table does show that S1 is idle at relative time slot 9. So, how about starting Set II at absolute time 527+9-1=535? Set I will not be using S1 at that time (again, 535 would be relative time slot 9 for Set I), so it seems that this might be a feasible time to start Set II. But it turns out that collisions would occur later on, for instance at time 536: That time is relative time slot 10 for Set I, and would be relative time slot 2 for Set II. As can be seen from the reservation table, S1 is in use both in time slots 2 and 10. The same reasoning shows that if two x's in some row of the reservation table are a distance k apart, we cannot start Set II at time 527+k. In the S1 row of our reservation table here, there are x's in time slots 2 and 10, corresponding to k=8, so we cannot start Set II at time 527+8 = 535, as we discovered in the last paragraph. We define the {\it forbidden list} to be the set of all such values of k, i.e. all the values of k such that in some row of the reservation table there are two x's a distance k apart. In this case, the forbidden list will be \begin{verbatim} 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 \end{verbatim} We can start Set II at any time 527+j for which j is not in the forbidden list. Clearly, that would be true for any j larger than 19, because Set I will have left the pipe after 19 time slots, but we can do better than that, such as j=16. In other words, we could start Set II at time 527+16=543, without causing a collision. However, what about Set III? In determining times at which we can start Set III, we now have to worry about possible collisions with both Set I and Set II, so the forbidden list is not enough. We need a more sophisticated analytical method. To explain this method, let us look at a simpler example, a four-stage pipe. Say it makes widgets.\footnote{By the way, though most of the example pipes we have seen here have four stages, the reader should of course not conclude that most pipes have that number.} with reservation table as follows: \begin{verbatim} 1 2 3 4 5 6 7 S1 x x S2 x x S3 x x S4 x \end{verbatim} This yields a forbidden list of \begin{verbatim} 2,3,6 \end{verbatim} Now suppose that the pipe is empty before time, say, 34, and that at that time we start Widget I. Based on the numbers in the forbidden list, we can't start Widget II at 2, 3 or 6 time slots later (i.e. at absolute times 36, 37 or 40). But we can start Widget II at any other time, i.e. 1, 4 or 5 slots after Widget I (or 7 or more time slots, since Widget I will be out of the system by then). We can summarize the findings of the previous paragraph by saying that the {\it collision vector} for deciding when to start Widget II is $(c_1,c_2,c_3,c_4,c_5,c_6)$ = (0,1,1,0,0,1), where $c_i = 1$ means that we cannot start Widget II in the time slot which is i slots after the one in which Widget I was started. In other words, the collision vector (0,1,1,0,0,1) is just a compact way of saying that we can't start Widget II 2, 3 or 6 slots after Widget I, but all other slots are all right. Assume we choose to start Widget II 5 slots after Widget I, i.e. in absolute time slot 39. Now then, what about Widget III? For the same reasons, in order to avoid colliding with Widget II, we cannot start Widget III 2, 3 or 6 slots past the slot in which Widget II started (i.e. at absolute time slots 41, 42 or 45). So, this again gives us a collision vector of (0,1,1,0,0,1) for Widget III, at least in terms of avoiding collisions with Widget II. But we also have to make sure Widget III doesn't collide with Widget I. For instance, Widget II had to avoid starting 6 slots after Widget I, because otherwise they would collide at S1. So now, 5 time slots later, Widget III has to refrain from starting 6-5=1 slots from now. In other words, the 1 which was in the $c_6$ position of Widget II's collision vector now becomes a 1 in the $c_1$ position of Widget III's collision vector. To reflect this, we take Widget II's collision vector and shift it to the left 5 positions. That is, we shift (0,1,1,0,0,1) left five positions, yielding (1,0,0,0,0,0). What that 1 means in the first component position is that we cannot start Widget III in the first time slot after Widget II, because to do so would collide with Widget I. Starting in any other time slot, though, will not produce a collision with Widget I. The point is that whatever restrictions Widget I caused for Widget II now must be updated to list the restrictions Widget I is causing for Widget III. Since 5 time units have passed since Widget II listed its restrictions, we must shift that list left by five positions. To illustrate the principle another way, suppose Mary wants to avoid bumping into Jim at the student union. Say 2 hours ago, Mary sent John e-mail saying, "John will be there 6 hours from now, so I won't go there then.'' Suppose John wants to avoid Jim too. That means that John should avoid going to the student union 6-2 = 4 hours from now. To summarize what we have found, Widget III's collision vector for avoiding collisions with Widget II is (0,1,1,0,0,1), and Widget III's collision vector for avoiding collisions with Widget I is (1,0,0,0,0,0). So, the full collision vector for Widget III is obtained by taking the ``bitwise'' (i.e. componentwise) logical-OR of these two vectors, which is (1,1,1,0,0,1). We see, then, that Widget III cannot be started 1, 2, 3 or 6 time slots after Widget II (i.e. in absolute slots 40, 41, 42 or 45). Any other time is all right. Here, then, is the general procedure: We first find the {\it initial collision vector}, CV0, from the forbidden list, as we did for Widget II. This will consist of an m-component vector, where m is the largest value in the forbidden list. CV0 will be the collision vector for the second task in the system, i.e. it will say how long after the first task we can initiate the second one. For each subsequent task, the collision vector is obtained by OR-ing the \underline{shifted} collision vector for the previous task with CV0, where the amount of shifting corresponds to our choice of waiting time for that previous task. For example, we obtained Widget III's collision vector by shifting Widget II's collision vector by 5 positions, since we had decided to start Widget II 5 slots after Widget I, and then OR-ing that shifted vector with CV0. This gives rise to the {\it state diagram} for this pipe: \vspace{0.2in} \par \psfig{file=States.eps} \par The numbers which label the arcs denote the waiting time before starting the task. For instance, look at the collision vector (1,1,1,0,0,1). Since there are 0's at positions 4 and 5, we can start our next task either 4 or 5 slots after the last task. If we choose to wait 4 slots, the collision vector for the following task will be (0,1,1,0,0,1). (Make sure to compute this yourself to check.) If on the other hand we wait 5 slots, our new collision vector will be (1,1,1,0,0,1) again. Now, what is the optimal schedule? What we want is to minimize the \underline{average} time between successive task initiations; this is equivalent to maximizing the throughput rate of widgets in this pipe.\footnote{This is actually a {\it long-run} average, since we may visit a few other states when the pipe first starts up, after which we settle down to a set pattern of state visits.} So, what schedule will have minimal average latency (MAL)? Inspection of the state diagram reveals the MAL schedule to be the one that cycles repeatedly as follows: (0,1,1,0,0,1), (1,1,1,0,1,1), (1,1,1,0,0,1), (0,1,1,0,0,1), (1,1,1,0,1,1), ... In other words, we schedule Widget II 1 slot after Widget I, Widget III 4 slots after Widget II, Widget IV 4 slots after Widget III, Widget V 1 slot after Widget IV, etc. In terms of absolute times, Widgets I, II, III, IV, V, ... will start at slots 39, 40, 44, 48, 49, ... The MAL will be (1+4+4)/3 = 3, i.e. on the average we will start a task every 3 slots; this corresponds to a throughput of 1/3 task per slot. By the way, at any state we can always wait 7 or more slots, since all previous tasks will have left the system by then (one task in this pipe takes 7 time units to complete), but for clarity we have not drawn arcs for them. If we did, all such arcs would lead to CV0, (0,1,1,0,0,1). The throughput rate for a pipe can sometimes be increased by adding {\bf delay stages}, that is, extra stages that do nothing. This seems strange at first, but these stages can help avoid collisions, and thus improve throughput. We will not pursue this further here, though. \section{Pipelining and RISC Architecture} \subsection{History} The term {\bf RISC} is an acronym for \underline{r}educed \underline{i}nstruction \underline{s}et \underline{c}omputer, the antonym being {\bf CISC}, for \underline{c}omplex \underline{i}nstruction \underline{s}et \underline{c}omputer. During the 1970s, computers became more and more CISC-like, with richer and richer instruction sets. The most cited example is that of the VAX family, whose architecture includes 304 instructions, including such highly specialized instructions as POLY, which evaluates polynomials. By contrast, the UC Berkeley RISC I architecture, which later became the ``rough draft'' for what is now the commercial SPARC chip, which is used in the ACS SPARCstations, had only 32 instructions. At the same time, the single-chip CPU was starting out on its path to dominance of the market. The more components a computer has, the more expensive it is to manufacture, so single-chip CPUs became very attractive.\footnote{And there is a performance gain as well. Multi-chip CPUs suffer from the problem that off-chip communication is slower than within-chip.} But the problem then became that the very limited space on a chip---this space is colloquially called ``real estate''---made it very difficult, if not impossible, to fit a CISC architecture onto a single chip.\footnote{It would seem that the solution is to simply make larger chips. However, this idea has not yet worked. For example, larger chips have lower {\bf yield} rates, i.e. lower rates of flaw-free chips. Of course, one can try to make chips denser as well, but there are limits here too.} The VAX was a clear casualty of this. When the Digital Equipment Corporation (DEC) tried to produce a ``VAX-on-a-chip,'' they found that they could not fit the entire instruction set on one chip. They were then forced to implement the remaining instructions in software. For example, when the CPU encountered a POLY instruction, that would cause an ``illegal op code'' condition, which triggered a {\bf trap} (i.e. internal interrupt); the trap-service routine would then be a procedure consisting of ordinary ADD and MUL instructions which compute the polynomial. At this point, some researchers at IBM\footnote{Thoug Professor Patterson at UC Berkeley is the one who really popularized the RISC idea, it was actually invented at IBM first. Actually, the old CDC machines designed by Seymour Cray had many RISC characteristics, i.e. small instruction sets and load/store architecture.} decided to take a fresh look at the whole question of instruction set formulation. The main point is optimal use of real estate. In a CISC CPU chip, a large portion of the chip is devoted to the microcode---and yet much of that microcode is rarely, if ever used. For example, most users of VAX machines do not make use of the POLY instruction, and yet the microcode for that instruction would occupy valuable-but-wasted real estate in the chips comprising a VAX CPU. Maybe, for example, an on-chip cache would make better use of that space; at least the cache would adapt itself to the characteristics of whatever program is currently running, quite unlike the the case of the POLY microcode. True, for those who do use the POLY instruction, execution speed will increase somewhat,\footnote{You should think this. First, POLY, being a single instruction, would require only one instruction fetch, compared to many fetches which would be needed if one wrote a function consisting of many instructions which would collectively perform the work that POLY does. Second, on the microcode level, we might be able to perform more than one register transfer at once.} but for most programs the POLY instruction is just getting in the way of fast performance. Other factors came into play. For example, the richer the instruction set, the longer the decode portion of the instruction cycle would take (whether for a microprogrammed \underline{or} a hardwired machine). Even more important is the issue of pipelining. Unless we are willing to make pipeline stage times to be of variable length, we then have to make each stage have time equal to that of the worst possible case (e.g. the longest possible decode time in the example above), or add stages to the pipe. The former of course slows things down a lot whenever we don't have the worst-case situations, and the latter has real problems when branches occur. For this reason, the IBM people asked, ``Why do we need the microcode at all? Why not just make the microengine itself our machine?'' They didn't literally mean a microengine, with different bit fields directly controlling various CPU components, but instead meant a CPU with a very simple, limited instruction set, eliminating the need for all that real-estate-consuming microcode, and one in which most instructions would---like microengine instructions---execute in a single clock cycle, so that a smooth pipeline could be built. Unfortunately, IBM scrapped their RISC project, but later David Patterson of the Division of Computer Science at UC Berkeley became interested in the idea, and developed two RISC CPU chips, RISC I and RISC II. These two chips will be our focus here. Patterson's team did not invent the RISC concept, nor did they invent some of the features of their design such as delayed branch, register windows, and so on. Nevertheless, these people singlehandedly changed the thinking of the entire computer industry, by showing that RISC could really work well. The team considered the problem as a whole, investigating all aspects, ranging from the questions of chip layout and VLSI electronics technology to issues of compiler writing, and through this integrated approach were able to make a good case for the RISC concept. Manolis Katevenis, a Ph.D. student who wrote his dissertation on RISC under Professor Patterson, worked on everything from the software analysis to the actual internal design of the CPU chip, and won the annual ACM for the best dissertation in Computer Science.\footnote{Published as {\it Reduced Instruction Set Computer Architectures for VLSI}, M. Katevenis, MIT Press, 1984.} Today virtually every computer manufacturer has placed a major emphasis on RISC. Sun Microsystems has the SPARC chip, DEC uses MIPS and also their own new Alpha chip, IBM and Motorola have the Power PC chip series, and so on. (On the other hand, this does not imply that RISC is inherently ``superior,'' and as electronic technologies advance in the future, there may well be some resurgence of CISC ideas, at least to some degree.) The Patterson effort is often cited today as an example of the contributions which can be made by university Computer Science Departments to the computer industry. \subsection{The Role of the RISC Concept in Achieving a Smooth Pipeline} It is of absolutely central importance that you keep in mind when reading the material that follows that the ``RISC-ness'' of the RISC I/II instruction set is of tremendous help in achieving a smooth pipeline. First, all RISC I/II instructions are four bytes in length. This compares, for instance, to the CISC architecture in the Intel 80x86 family, in which instruction length can range from one to six bytes. An 80x86 pipeline would have to either allow for the worst case in the instruction-fetch stage, thus possibly making the common stage time for all states longer than necessary, or would have to undergo a vast increase in pipeline complexity, having to allow for many different special cases. Similarly, the Intel family also has variable-length op codes, so the decoding portion of the pipe\footnote{Which in the case of RISC I/II is so simple that we have incorporated it into the instruction-fetch stages in the pictures above.} becomes more complex too, with the same kinds of negative implications for the pipe. Second, except for load/store instructions, all RISC I/II instructions execute in a single clock cycle. By contrast, Intel instruction times have a wide range. For instance, an ADD instruction could execute in as few as 2 clock cycles, while a DIV could take as many as 25 cycles. Again, there are profound implications of this on the complexity of the pipes in the Intel chips. \subsection{A Definition of RISC} There is no universally-agreed-upon definition of RISC, but most people would agree at least to the following characteristics: \begin{itemize} \item Most instructions execute in a single clock cycle. \item ``Load/store architecture''---the only instructions which access memory are of the LOAD and STORE type; e.g. memory-to-register adds like \begin{verbatim} ADD R2,X \end{verbatim} are not possible. \item Every instruction has the same length, e.g. 4 bytes. By contrast, Intel instructions range in length from 1 to 6 bytes. \item ``Orthogonal architecture''---any operation can use any operand. This is in contrast, for example, to the Intel 80x86-family architecture, in which for example, MUL and DIV require that one of their operands be in a specific register, AX. \item The instruction set is limited to only those instructions which are truly justified in terms of performance/space/pipeline tradeoffs. The SPARC chip, for instance, does not even have a MUL instruction. And the ``VAX on a chip'' ended up doing this kind of thing anyway, albeit not intentionally. \item Hardwired implementation, i.e. not microcoded. \end{itemize} All but the last of the traits listed above deal with pipelining. Concerning hardwired implementation, this actually follows from the others. In the old days, the rationale for microcoded implementation of a CPU was that it made the CPU development faster, and the CPU debugging faster. It simplified a very complex job. But if the CPU is very simple anyway, that rationale disappears. \subsection{Effects on Compiler Writers} Computer architects used to partly justify including instructions like POLY in the instruction set by saying that this made the job of compiler writers easier. However, it was discovered that this was not the case, for several reasons. \begin{itemize} \item Compiler writers found it difficult to write compilers which would automatically recognize situations in (say) C source code in which instructions like POLY could be used. \item Compiler writers found that some specialized instructions that were actually motivated by high-level language constructs did not match the latter well. For example, the VAX had a CASE instruction, obviously motivated by the Pascal {\bf case} and C {\bf switch} constructs; yet it turned out to be too restrictive to use well for compiling those constructs. \item CISC generally led to very nonorthogonal architectures, which made the job of compiler writers quite difficult. For instance, the requirement that the AX register be used in MUL and DIV, \underline{plus} the requirement that it be used in other instructions, implies that compiler writers ``don't have any spare registers to work with,'' and thus they must have the compiler generate code to save and restore ``popular'' registers such as AX; this is a headache for the compiler writer. \end{itemize} Most RISC machines do ask that the compiler writers do additional work of other kinds, though, in that they have to try to fill {\bf delay slots}; more on this below. \subsection{Dealing With the Branch Problem} As you know, a major problem with pipelined CPUs is that the pipeline must be drained when a branch is taken. If for example there are n stages to the pipe, and the instruction is a conditional branch (so that we won't know until the last stage whether the branch will be taken), we will have n-1 empty clock cycles in which no instruction is executed. \subsubsection{Branch Delay Slots} Many RISC processors use something called a {\bf delayed branch} to deal with the branch problem. The term means that the execution of the {\bf branch target}, i.e. the place to which we jump on the branch, is delayed by one cycle. This is accomplished in an odd but actually quite clever way: Regardless of whether a branch is taken or not, we still execute the instruction following the branch. For example, suppose we have a 2-stage pipe, with Stage 0 being the instruction fetch and Stage 1 being decode and execute; while one instruction is executing, the next one is being fetched. Note that this implicitly means that we are assuming that it takes one pipe-stage time to read memory.\footnote{Or possibly to read from a cache.} Now consider the following generic (not RISC I/II) code: \begin{verbatim} ADD R5,R3 ADD R2,R4 JNEG T n.s.i. \end{verbatim} Here n.s.i. stands for ``next sequential instruction,'' i.e. the instruction following JNEG in memory, which will be executed only if the branch is not taken. Using the normal scheme, if the condition in JNEG is satisfied and we jump to T, we will have one empty clock cycle in which no instruction is executed, because we will have to fetch the instruction at T (which had not been prefetched). Under the delayed-branch scheme, the n.s.i. will be executed regardless of whether we jump to T! By the time we start execution of the n.s.i., we will know whether the condition in the JNEG was satisfied, and so during this time we either fetch from T (if it was satisfied) or from the location following the n.s.i. (if it was not satisfied). In either case, we will have no empty clock cycles. A potential problem arises in that we may not want to have the n.s.i. executed. Thus the compiler (or assembly-language programmer) would first tentatively produce code like this \begin{verbatim} ADD R5,R3 ADD R2,R4 JNEG T NOP n.s.i. \end{verbatim} so that the delayed-branch mechanism will not harm correct execution of the program. (Recall that most machines have an NOP instruction, pronounced ``no-opp,'' which does literally that, i.e. no operation at all.) This code would work properly, but it would not exploit the parallelism idea of delayed branches. Instead, what we would like to do is change the order of the instructions: \begin{verbatim} ADD R2,R4 JNEG T ADD R5,R3 n.s.i. \end{verbatim} Here the ADD to R5 has been moved to follow the JNEG, and this ADD will be done right after the JNEG. The change in order won't affect the correct execution of the code,\footnote{If we had moved the other ADD, there would have been an effect.}, but the key point is that in the case the branch is taken, we save one cycle! The time that would have been wasted fetching the instruction at T is now usefully employed by execution of the ADD at R5 at the same time. And of course if the branch is \underline{not} taken, no harm is done. The instruction position following a branch is called a {\bf delay slot}. Machines such as MIPS and SPARC have just one delay slot, because their design fulfills an instruction-fetch request in just one clock cycle. (This assumes a cache hit; in the case of a miss, the pipe needs to be suspended for quite some time.) But some other machines have more than one delay slot. Clearly, the more delay slots there are, the harder it is to ``fill'' them, i.e. to find instructions to move to put in the slots, and thus avoid simply putting wasteful NOPs in them. How hard is it to fill even one delay slot? The Katevenis dissertation on RISC says: \begin{quote} Measurements have shown [citation given] that the [compiler] optimizer is able to remove about 90\% of the no-ops following unconditional transfers and 40\% to 60\% of those following conditional branches. The unconditional and conditional transfer-instructions each represent approximately 10\% of all executed instructions (20\% total). Thus, while a conventional pipeline would lose $\approx$ 20\% of the cycles, optimized RISC code only loses about 6\% of them. \end{quote} \subsubsection{Branch Prediction} Most modern CPUs use {\bf branch prediction} as a way to avoid pipeline delays due to branches (either in addition to delayed branch or instead of it). Here is an overview of how it might work. In the decoding stage of the pipeline, suppose the CPU discovers an instruction like \begin{verbatim} JNEG T \end{verbatim} and suppose that T is a location with a lower memory address than that of the instruction, i.e. this is a backward branch. Usually this will mean that the instruction is the last one in a loop. Since it has been found empirically that most loops iterate more than once, that means that backward branches are usually taken. So, the CPU circuitry can be built so that the branch target, say T above, is always prefetched. The reasoning above says that we will usually be correct, and thus the branch will usually cause us no delays. Of course, sometimes this guess will be incorrect, in which case we do incur a delay. Another approach to branch prediction is to maintain a {\bf branch history table}. This is a small cache inside the CPU that records the last action (taken/not taken) for each of several recently-executed branch instructions.\footnote{More complicated histories could be maintained instead.} We would predict a given branch to be taken or not, depending on what the last action of that instruction was. The table entry for a given instruction might consist of two fields, first the instruction's address and second the history bit. In a more ambitious version, the second field might actually be the target instruction itself (i.e. the instruction at T in the above example), so that the prefetching would actually be done on the table entry. Note that we might incur more of a penalty for incorrectly guessing a branch to be taken than for wrongly guessing it to not be taken. This is because the target T might be far enough away from the current location that prefetching the target causes a cache miss, or even worse a virtual memory page fault. \subsection{Register Windows} Though register windows are not considered to be part of the definition of RISC, the RISC I/II machines did use them, and a number of the earlier commercial RISC processors---notably SPARC---followed suit. The motivation underlying the register window concept is to reduce traffic from the CPU to memory, since such access is much slower than CPU speeds. Let us first recall the notion of a stack frame. \subsubsection{Stack Frames} In most modern machine/compiler combinations, function calls are handled via the stack. Not only does a machine CALL instruction save the return program counter value on the stack, but also the compiler produces code to push function parameters on the stack, and also to allocate stack space for local variables in the called function. For example, suppose we have a C function whose first few lines are \begin{verbatim} x(a,b) int a,b; { int c,d,e; \end{verbatim} and that there is a call to the function like this: \begin{verbatim} x(m,n); \end{verbatim} The compiler will translate the call to something like \begin{verbatim} PUSH N PUSH M CALL X \end{verbatim} and will translate the beginning of the function itself to something like \begin{verbatim} SUB SP,3 \end{verbatim} (assuming successive words differ in address by 1, and assuming that the stack grows toward 0), in order to allocate three words on the stack for x's local variables c, d and e. In other words, for a given function call, the portion of the stack will be devoted to that call will look like this: \begin{verbatim} local variables return address parameters \end{verbatim} This is called the {\bf stack frame} for that call. \subsubsection{Stack Frames on RISC I and II} RISC I and II store the top few frames of the stack in registers within the CPU. When the stack grows to the point at which all these registers are occupied, a stack overflow occurs, and some of the lower part of the stack is copied to memory, to make room for the newly-pushed part of the stack. Later, when the stack is popped enough to reach the part of the stack which was shunted out to memory, a stack underflow occurs, and that data will be copied back to these registers in the CPU. Remember, due to top-down software design, typically we will have function f1 call function f2, f2 then in turn calling f3, and so on. This leads to the notion of {\bf parent} and {\bf child} in the calling chain. For example, in the chain above, f2 would have f1 as its parent and f3 as its child. This structure is then reflected in the register windows. What is normally called a stack frame for other machines is called a {\bf register window} here. Each window consists of 32 registers, called r31 through r0. Of these, r15-r10 are used by the function to receive parameters from the parent, r25-r16 are used to store the function's local variables, r31-r26 are used to send parameters to the child, and r9-r0 are used to store global variables (but r0 has the value 0 hardwired into it). The RISC II CPU chip contains a total of 138 registers, which I will denote here by R137-R0. These registers then form 8 windows, numbered 0 through 7. Each window register ri ($0 <= i <= 31$) maps to some Rj, in the following way. \begin{itemize} \item [1.] R9-R0 are held in common by all windows, so that a reference to ri, $0 <= i <= 9$, will map to Ri. Again, except for r0/R0, which stores the constant 0, these registers store global variables, accessible to all functions. \item [2.] Registers r31-r26 of a window will be physically identical to r15-r10 of the child. It is this overlap which allows parameter passing between functions, as well as passing the return program counter value if this function in turns calls a child. \item [3.] Registers r25-r16 are used to store local variables. \end{itemize} Below are the correspondences between the ``virtual'' register numbers ri and the ``physical'' register numbers Rj. Note that 1r11, for instance, means r11 in window 1. \begin{verbatim} ... ... ... R122 6r26 7r10 ... ... ... R47 1r31 2r15 ... ... ... R43 1r27 2r11 R42 1r26 2r10 R41 1r25 ... ... R32 1r16 R31 0r31 1r15 ... ... ... R27 0r27 1r11 R26 0r26 1r10 R25 0r25 ... ... R16 0r16 R15 0r15 ... ... R11 0r11 R10 0r10 R9 0r9 1r9 2r9 ... 6r9 7r9 ... ... ... ... ... ... R1 0r1 1r1 2r1 ... 6r1 7r1 R0 0r0 1r0 2r0 ..1 6r0 7r0 \end{verbatim} The second column, labled `0', corresponds to main(). If main() then, say, calls x() which in turns calls y(), then the column labeled `1' would be for x(), column `2' would be for y() and so on. A special Current Window Pointer register points to r16 of the current window. As an example, consider the C code presented above. Assuming that m and n were global variables, the compiler might have them stored in r1 and r2, and might translate the call to the following RISC II instructions:\footnote{This is somewhat simplified from the real RISC II.} \begin{verbatim} add r0,r1,r26 add r0,r2,r27 call x,r28 \end{verbatim} Since r0 contains the constant 0, the add instructions here simply copy r1 and r2 to r26 and r26, respectively.\footnote{By this device, there is no need to have a ``move'' instruction in the RISC I/II architecture, and there is none.} Note that the return address is saved in the window too, in this case in r28. The code produced by the compiler for the function x itself would access the formal parameters m and n via registers r10 and r11; remember, these registers are physically the same as r26 and r27 of the parent. The local variables c, d and e would go in r16-r18. Finally, the last instruction in x would be \begin{verbatim} ret r12 \end{verbatim} (Again, you should verify all of these statements.) If a function has more than six parameters, there is not enough room in a window to pass them. Thus even though the idea of register windows is to replace the concept of a stack, the excess parameters in this situation must be passed through a stack after all, and compilers for machines having register windows do set up stacks. The stacks are needed for another reason as well. Consider the following code: \begin{verbatim} w(s) int *s; {...} main() { int z; ... w(&z); ... } \end{verbatim} Since z is local to main(), we would normally store it in a window, somewhere in r25-r16. But here that would be impossible, since the expression \begin{verbatim} &z \end{verbatim} would be meaningless. So here too z would need to be stored in a stack. \subsubsection{Performance} The Katevenis thesis reports the following findings, in a comparison of words of memory access for data in two sample programs: \begin{verbatim} machine puzzle quicksort ------- ------ --------- VAX-11/780 440K 700K RISC 8K 4K \end{verbatim} So, the use of register windows does appear to achieve a dramatic reduction in memory traffic, extremely valuable in light of the slowness of memory. Anti-register-window advocates argue that a cache might do as well or better (the VAX did have a cache, but not as big a cache as it would have today). Katevenis, Patterson {\it et al} disagree, but the question would be hard to resolve completely to everyone's satisfaction. \subsection{SPARC Example: Delayed Branch} \begin{verbatim} 1 Script started on Sun Mar 10 16:19:06 1996 2 taco 1: m a.c 3 4 int a,b; 5 6 main(argc,argv) 7 int argc; char **argv; 8 9 { int x,y; 10 11 12 a = atoi(argv[1]); 13 b = atoi(argv[2]); 14 if (a < b) x = a; 15 else y = a+b; 16 } 17 18 taco 2: 19 taco 2: 20 taco 2: 21 taco 2: cc -S a.c 22 taco 3: m a.s 23 LL0: 24 .seg "data" 25 .common _a,0x4,"data" 26 .common _b,0x4,"data" 27 .seg "text" 28 .proc 04 29 .global _main 30 _main: 31 !#PROLOGUE# 0 32 sethi %hi(LF14),%g1 33 add %g1,%lo(LF14),%g1 34 save %sp,%g1,%sp 35 !#PROLOGUE# 1 36 st %i0,[%fp+0x44] 37 st %i1,[%fp+0x48] 38 ld [%fp+0x48],%o0 39 ld [%o0+0x4],%o0 40 call _atoi,1 41 nop 42 sethi %hi(_a),%o1 43 st %o0,[%o1+%lo(_a)] 44 ld [%fp+0x48],%o2 45 ld [%o2+0x8],%o0 46 call _atoi,1 47 nop 48 sethi %hi(_b),%o3 49 st %o0,[%o3+%lo(_b)] 50 sethi %hi(_a),%o4 51 ld [%o4+%lo(_a)],%o4 52 sethi %hi(_b),%o5 53 ld [%o5+%lo(_b)],%o5 54 cmp %o4,%o5 55 bge L17 56 nop 57 sethi %hi(_a),%o7 58 ld [%o7+%lo(_a)],%o7 59 st %o7,[%fp+-0x4] 60 b L18 61 nop 62 L17: 63 sethi %hi(_a),%l0 64 ld [%l0+%lo(_a)],%l0 65 sethi %hi(_b),%l1 66 ld [%l1+%lo(_b)],%l1 67 add %l0,%l1,%l2 68 st %l2,[%fp+-0x8] 69 L18: 70 LE14: 71 ret 72 restore 73 LF14 = -104 74 LP14 = 96 75 LST14 = 96 76 LT14 = 96 77 .seg "data" 78 taco 4: 79 80 81 taco 4: 82 taco 4: 83 taco 4: cc -S -O1 a.c 84 taco 5: m a.s 85 .seg "text" ! [internal] 86 .proc 4 87 .global _main 88 _main: 89 !#PROLOGUE# 0 90 !#PROLOGUE# 1 91 save %sp,-104,%sp 92 st %i0,[%fp+68] 93 st %i1,[%fp+72] 94 call _atoi,1 95 ld [%i1+4],%o0 96 sethi %hi(_a),%l0 ! [internal] 97 st %o0,[%l0+%lo(_a)] 98 ld [%fp+72],%o0 99 call _atoi,1 100 ld [%o0+8],%o0 101 sethi %hi(_b),%o1 ! [internal] 102 st %o0,[%o1+%lo(_b)] 103 ld [%l0+%lo(_a)],%l0 104 ld [%o1+%lo(_b)],%o1 105 cmp %l0,%o1 106 bge,a LY1 107 sethi %hi(_a),%l0 108 sethi %hi(_a),%o7 109 ld [%o7+%lo(_a)],%o7 110 b LE14 111 st %o7,[%fp-4] 112 LY1: ! [internal] 113 ld [%l0+%lo(_a)],%l0 114 sethi %hi(_b),%l1 115 ld [%l1+%lo(_b)],%l1 116 add %l0,%l1,%l0 117 st %l0,[%fp-8] 118 LE14: 119 ret 120 restore 121 .seg "data" ! [internal] 122 .common _a,4,"data" 123 .common _b,4,"data" 124 taco 6: 125 taco 6: 126 taco 6: e 127 exit 128 129 script done on Sun Mar 10 16:20:11 1996 \end{verbatim} In lines 3-17, we have a simple C source file a.c, which, in line 21 we compile, using the -S option. This option produces a file a.s, which shows us the assembly language produced in the compilation. We look at that file starting at line 22 (in my .cshrc file, I have aliased `m' to `more'). There are a number of assembler directives which have been inserted by the compiler, such as those seen in lines 24-29, but they do not concern us here. Recall that the SPARC was modeled on RISC I/II. In particular, it uses register windows. Remember that registers r31-r26 in RISC I/II were used to send parameters to the child function, i.e. as ``output'' to the called function. Similarly, in SPARC the ``output'' registers are denoted \%o0, \%o1,..., \%o5, and have virtual register numbers r8-r13. The registers r15-r10 in RISC I/II are for receiving parameters (``input'') from the function which called the current one, and the analogs in SPARC are denoted \%i0, \%i1,..., \%i5, and in virtual terms are r24-r29. The SPARC registers r16-r23, denoted \%l0-\%l7, are intended for storing local variables. The SPARC registers named \%gn are analogous to RISC I/II's global registers r9-r0; \%g1-\%g7 are virtual registers r1-r7. As in RISC I/II, register r0 (denoted \%g0) has the value 0 hardwired into it. Here are the rest: r14 is the stack pointer, denoted by sp; r15 is for temporary use, denoted \%o7; r30 is the frame pointer, fp; and r31, \%i7, is for the return address. Registers r8-r15 of a parent window are physically identical to r24-r31 of the child. The \%fp register is the frame pointer, which serves a second copy of the stack pointer. So the expression [\%fp+0x48] in line 38 refers to the location 0x48 bytes into the stack frame, for instance. The brackets notation ``[x]'' means ``contents of memory location x.'' Note that the addressing here is similar to local addressing in Mac-1 (though on SPARC any register can be used, not just the stack pointer). The assembly-language notation lists the destination operand last. For example, in line 39, the destination operand is \%o0. Let's go right to line 36, which (together with the lines following it) is the result of compiling line 12, the call to atoi(). The call occurs at line 40, but first we have to get the parameter ready. This occurs in lines 36-39: Keep in mind that main() is {\it itself} a function, whose parameters are argc and argv. So, argc and argv will be in some \%i registers, apparently \%i0 and \%i1, which are copied to the stack frame in lines 36 and 37. Recall (line 7) that argv is a pointer-to-a-pointer. After line 38 is executed, \%o0 now points to argv[0], \%o0+4 points to argv[1], \%o0+8 points to argv[2], and so on. This is reflected in lines 39 and 45. After line 39 sets up the parameter, line 40 then is the actual call to the function. (The operand 1 indicates that there is 1 parameter in the call.) Of particular interest to us here is that the compiler has placed a {\bf nop} in line 41, because after all, the {\bf call} in line 40 is a form of branch. Since we did not request any optimization in our command line (line 21), the compiler has made no attempt to fill this branch-delay slot. The function's return value will be in \%o0. We now have to store it in the variable `a', which is done in lines 42 and 43: The {\bf sethi} (``set high'') instruction (line 42) puts a value into the high 22 bits of the destination register. Typically the value consists of the high bits of an address, and {\bf sethi} is executed in preparation for a load or store to that address, in this case the {\bf st} in line 43. Here are the details of line 43. The high 22 bits of the address of `a' already have been copied (by line 42) to the high 22 bits of the register \%o1. By adding the low 10 bits of the address of `a', we then get the full address of `a'. Line 43 therefore stores to `a'. Note, by the way, that the reason we must resort to this piecemeal way to set up the address of `a' is that under the RISC philosophy all instructions ought to be the same length, in this case 32 bits. Since the address of `a' is already 32 bits, we could not fit both it \underline{and} an op code into one 32-bit instruction! Now, in line 83, we have recompiled the program, using first-level optimization (-O1), to force the compiler to attempt to fill the branch-delay slots. To see an example of how the code has changed, note that the {\bf ld} which used to be just before the call, in line 39, now appears just after the call, in line 95. (It also now has slightly different form, because the compiler has done some other optimizations in addition to trying to fill delay slots.) Now look at lines 106-107. The {\bf bge} instruction in line 55 has been changed to {\bf bge,a,} which means the following: If the branch in line 55 is taken, do execute line 56; if the branch is not taken, then do not execute line 56. The basic motivation is this: Suppose we have a branch to a target, T, but cannot find an instruction to move and replace the {\bf nop} which had been tentatively placed in the delay slot. An alternative would be to move T itself into the delay slot (assuming T is not also the target for some other branch). If the branch is taken, this saves us a clock cycle. But if the branch is not taken, we don't want T executed. The SPARC architecture is set up to deal with this by the ``,a'' appendage, where `a' stands for ``annul.'' It says that the instruction in the delay slot should be executed if the branch is taken but is skipped otherwise. We still lose a cycle in the ``not taken'' case, since we will not have prefetched the instruction following the delay slot, but in the ``taken'' case we do save a cycle. \subsection{Analysis of the UCB RISC I/II Pipelines} \subsubsection{RISC I} RISC I had a two-stage pipeline, and three internal buses, the first two of which serve to copy register values into the ALU, with the third copying the output of the ALU back to a register. Here are diagrams of the RISC I timing, both in its pipeline and its within-cycle performance: \vspace{0.2in} \par \psfig{file=IPipe.eps} \par The upper half of the picture gives an overview of the pipeline. The first stage is the instruction fetch, and the second is the execution of the instruction. Here I1, I2 and I3 are three instructions which are executed consecutively.\footnote{As opposed to being consecutive in memory. E.g. I1 might be a conditional branch, in which case I2 would be consecutive to I1 in memory, but I3 would be consecutive in memory to neither.} The fetch of I2 is done during execution of I1, etc. The second half of the picture gives the details of the breakdown of one cycle into three subcycles, for each of the major components, the three buses and the ALU (as well as the outputs of the latter).\footnote{The term {\bf precharge} here refers to a process necessitated by the NMOS electronics technology, and the modified six-transistor register cell design, used in the RISC I and II chips. The details may be found in Katevenis' thesis and the references it cites, but do not worry about this for the purposes of our course.} Consider, for example, the instruction \begin{verbatim} add r2,r12,r13 \end{verbatim} which puts the sum of registers r2 and r12 into r13. In RISC I, registers r2 and r12 would be put onto buses 1 and 2 during subcycle 1, the sum would be formed during subcycle 2, and the sum would be written back to bus 3 and register r13 during subcycle 3. \subsubsection{RISC II} RISC II's design is more complicated, but more efficient, using only two internal buses: \vspace{0.2in} \par \psfig{file=IIPipe.eps} \par Here the instruction add r2,r12,r13 would execute in the following way: As before registers r2 and r12 would be read onto buses 1 and 2 during subcycle 1. The sum would be computed in subcycles 2 and 3, and then the sum would be written back to bus 1 and r13 during subcyle 2 of the \underline{next} cycle.\footnote{And thus subcycle 2 of the \underline{current} cycle would do the write-back of the result from the \underline{previous} cycle.} This design is clearly more efficient. The register file is kept busy all the time (as opposed to being idle 1/3 of the time in the earlier design), and precious chip real estate is saved: With this two-bus design, they were able to put 75\% more registers into a 25\% smaller area. But what if the following instruction were, say, \begin{verbatim} add r4,r13,r1 \end{verbatim} This wouldn't work. During subcycle 1 for this instruction, r13 won't be ready for copying onto bus 2, since the previous instruction, which changes r13, has not done its write-back yet. To solve this problem, the designers included an {\bf internal forwarding} feature, which sends to result of the previous instruction directly to the ALU, even though the write-back for that instruction has not been done yet. Indicator bits in the ALU will be set or cleared, depending on whether the instruction that follows needs to use the internally-forwarded value. Note again that a three-stage pipe is no slower than a two-stage, as long as we keep the pipe filled. Of course, whenever we have a taken branch, that condition fails, but if the compiler optimizer has been successful in filling most delay slots, the problem will arise only rarely. \subsubsection{Taking Memory Access Into Account} RISC I/II were designed on the assumption that memory (or cache) access time would be approximately the same as the time to do one ALU computation. The pipelines described above work that way, with an instruction fetch being done during an ALU operation. But what happens when a load or store instruction is encountered? If we only have one path to memory---i.e. we have a cacheless system with only one system bus, or we have a combined instruction/data cache---then a load/store can't be done at the same time as an instruction fetch. When a load/store instruction is encountered, the pipeline must be suspended. This is how RISC I/II are actually implemented. Another alternative, though, would be to have two paths to memory. For example, there could be separate instruction and data caches, or two system buses, both connected to a {\bf dual-port} memory. In such case, a four-stage pipe such as the following would be appropriate: \vspace{0.2in} \par \psfig{file=DualPipe.eps} \par The boxes labeled `comp' indicate a computation, either that as in \begin{verbatim} add r4,r13,r1 \end{verbatim} or possibly in a memory load/store instruction such as \begin{verbatim} ldxw r12,30000,r13 \end{verbatim} The latter instruction is of interest to us now. It uses indexed addressing mode; it loads the contents of the memory word which is c(r12)\footnote{The notation c( ) means ``contents of.''} bytes past location 30000 into r13. Thus the addition 30000+c(r12) must be performed, so that even in the case of load/store instructions, the labels `comp' in the boxes above do make sense. The labels `mem' mean that \underline{if} a data memory access is needed for this instruction---i.e. if the instruction type is load or store---then the memory access will be done at this stage. Otherwise, this stage of the pipe becomes a dummy stage. Note that double internal forwarding is used here, since the register write-back occurs two time slots after the computation, rather than one. (In the picture the internal forwarding from memI1 to compI3 genuinely comes after a memory access if I1 is a store instruction. Otherwise, the forwarding is done from this dummy stage, and the forwarded value came from compI1.) In addition, no dependencies are allowed between a load and the instruction which follows it. If for example I1 is the ldxw instruction shown above, I2 is not allowed to use r13 as a source register, since compI2 occurs at the same time as memI1. The compiler must avoid writing such code, resorting to inserting a nop instruction if necessary. As before, the addition of one more pipe stage (from three to four), even a stage for which non-load/store instructions do nothing, does not hurt us at all---until we encounter a branch instruction. So again, the efficiency of the delayed-branch mechanism is vital. \subsection{So, How Well Does RISC Work?} Some statistics from Katevenis: \begin{itemize} \item Due to the simple, orthogonal instruction set, the load/store architecture, etc., the C compiler for the machine was easy to write, taking only 6 person-months, including the optimization components. \item Though RISC's simpler instruction set means we sometimes need to use more instructions to accomplish a given task, the RISC I/II code was typically no more than 50\% larger than that of the VAX. \item The ratio of execution times of hand-coded assembly language to C code was about 90\% for RISC I/II, but around 45\% for the VAX. In other words, on a VAX you pay a much higher speed penalty for the convenience of coding in a high-level language. \item The {\bf control area} of the RISC II chip consists just of the opcode decoder, and occupied only 0.5\% of the chip. By contrast, in the Motorola 68000,\footnote{The chip family formerly used in Macintoshes.} the control area, in this case meaning the microcode, occupied 68\% of the chip. \item The chip layout design for RISC II was done by students in 12 person-months. For the 68000 the process was done by experienced professionals, and took 70 person-months. \item RISC II and a PDP-11/70, having the same clock speed and the same register-to-register addition time, were compared over 11 programs. RISC II was on the average 2.1 times faster. \end{itemize} \subsection{What to Do With All That Extra Space?} Now that RISC has liberated all this real estate, what might we do with it? \begin{itemize} \item cache \item more registers \item on-chip stack \item another CPU! \end{itemize} \end{document}