\documentclass[11pt]{article}

\setlength{\oddsidemargin}{0.0in}
\setlength{\evensidemargin}{0.0in}
\setlength{\topmargin}{0.0in}
\setlength{\headheight}{0in}
\setlength{\headsep}{0in}
\setlength{\textwidth}{6.5in}
\setlength{\textheight}{9.0in}
\setlength{\parindent}{0in}
\setlength{\parskip}{0.1in}

\usepackage{times}
\usepackage{fancyvrb}
\usepackage{relsize}
\usepackage{hyperref}
\usepackage{graphicx}

\begin{document}

\title{Statistical Inference on Simulation Output}

\author{Norm Matloff}

\date{March 16, 2008 \\
\copyright{}2006-8, N.S. Matloff} 

\maketitle

\tableofcontents 

\newpage

\section{Review of Confidence Intervals}

\subsection{Example}

\label{bus}

Suppose buses arrive at a certain bus stop at random times, with
interarrival times being independent exponentially distributed random
variables with mean 10 minutes.  You arrive at the bus stop every day at
a certain time, say 90 minutes after the buses start their morning run.
What is your mean wait for the next bus?  One can show mathematically
that it is 10, but suppose we didn't know that, and wished to find the
answer via simulation.  We could write a program:

\begin{Verbatim}[fontsize=\relsize{-2},numbers=left]
# Bus.py

import random,sys

rnd = random.Random(12345)

def doexpt(opt):
   arvtm = 0.0
   nbus = 0
   while arvtm < opt:
      nbus += 1
      arvtm += rnd.expovariate(0.1)
   return arvtm-opt

def main():
   observationpoint = float(sys.argv[1])
   nreps = int(sys.argv[2])
   sum = 0.0
   for i in range(nreps):
      wait = doexpt(observationpoint)
      sum += wait
   print 'the estimated mean wait is', sum/nreps

if __name__ == '__main__': main()
\end{Verbatim}

Running the program yields

\begin{Verbatim}[fontsize=\relsize{-2}]
% python Bus.py 90 1000
the estimated mean wait is 10.2182290246
\end{Verbatim}

Was 1000 iterations enough?  How close is this value 10.2182290246 to
the true expected value of waiting time?\footnote{Of course, we'll
continue to ignore the fact that we know that this value is 10.0.  What
we're trying to do here is figure out how to answer ``how close is it''
questions in general, when we don't know the true mean.}

What we would like to do is something like what the pollsters do during
presidential elections, when they say ``Ms. X is supported by 62\% of
the likely voters, with a margin of error of 4\%.''  In fact, we will do
exactly this, in the next section.

\subsection{Confidence Intervals for Means}
\label{cim}

In our example in Section \ref{bus}, let $W$ denote the random time one
waits for a bus in this situation.  We are using the program to estimate
the mean of $W$, $E(W)$, which we will denote by $\mu$.  While we're at
it, let's denote the variance of $W$, $Var(W)$ by $\sigma^2$.

Let $W_i$ denote the $i^{th}$ waiting time, i = 1,2,... and let
$\bar{W}$ denote the sample mean among our n = {\bf nreps} simulated
wait times:

\begin{equation}
\bar{W} = \frac{W_1+...W_n}{n}
\end{equation}

Note that $\bar{W}$ is what the program prints out. 

The key points are that

\begin{itemize}

\item the random variables $W_i$ have the same distribution; for
example, we are just as likely to wait more than 12.6 minutes in our
first simulated wait as in our second 

\item the random variables $W_i$ are independent, e.g. our second
simulated wait is independent of the first

\end{itemize}

Let's estimate $\sigma^2$ by taking the sample analogs of the population
variance,

\begin{equation}
\label{s2}
s^2 = \frac{1}{n} \sum_{i=1}^{n} (W_i-\bar{W})^2  =
\frac{1}{n} \sum_{i=1}^{n} W_i^2 - \bar{W}^2
\end{equation}

We will thus take our estimate of $\sigma$ to be $s$, the square root of
that quantity.\footnote{It is customary to divide by n-1 instead of n,
for reasons that are largely theoretical.  But when n is large, which we
are assuming for the Central Limit Theorem, it doesn't make any
appreciable difference.}

One can use the Central Limit Theorem to show that 

\begin{equation}
0.95 \approx P \left ( \bar{W} - 1.96 \frac{s}{\sqrt{n}} < \mu
< \bar{W} + 1.96 \frac{s}{\sqrt{n}} \right ) 
\end{equation}

In other words, we are about 95\% sure that the interval 

\begin{equation}
\label{meanci}
(\bar{W} - 1.96 \frac{s}{\sqrt{n}}, \bar{W} + 1.96 \frac{s}{\sqrt{n}})
\end{equation}

contains $\mu$.  This is called a 95\% {\bf confidence interval} for $\mu$.

We could add this feature to our program:

\begin{Verbatim}[fontsize=\relsize{-2},numbers=left]
# Bus2.py, illustration of the bus paradox

import random,sys
from math import sqrt

rnd = random.Random(12345)

def doexpt(opt):
   arvtm = 0.0
   nbus = 0
   while arvtm < opt:
      nbus += 1
      arvtm += rnd.expovariate(0.1)
   return arvtm-opt

def main():
   observationpoint = float(sys.argv[1])
   nreps = int(sys.argv[2])
   sum = 0.0
   sum2 = 0.0
   for i in range(nreps):
      wait = doexpt(observationpoint)
      sum += wait
      sum2 += wait**2
   wbar = sum/nreps
   wbar2 = sum2/nreps
   s = sqrt(wbar2-wbar**2)
   print 'the estimated mean wait is', wbar
   print 'with a 95% margin of error of', 1.96*s/sqrt(nreps)

if __name__ == '__main__': main()
\end{Verbatim}

In our example above, the interval would work out to (9.58,10.85).  We
would then say, ``We are about 95\% confident that the true mean wait
time until the next bus is between 9.58 and 10.85.''

{\bf What does this really mean?}  This question is of the utmost
importance.  We will devote the next section to it.

\subsection{Meaning of Confidence Intervals}

\subsubsection{A Weight Survey in Davis}

Let's forget simulation for a minute, and consider the question of
measuring the mean weight $\mu$ of all adults in the city of Davis.  Say
we sample 1000 people at random, and record their weights, with $W_i$
being the weight of the $i^{th}$ person in our sample.\footnote{Do you
like our statistical pun here?  Typically an example like this would
concern people's heights, not weights.  But it would be nice to use the
same letter for random variables as in Section \ref{cim}, i.e. the
letter W, so we'll have our example involve peoples weights instead of
heights.  It works out neatly, because the word {\it weight} has the
same sound as {\it wait}.}

Say our interval turns out to be (142.6,158.8).  We say that we are
about 95\% confident that the mean weight $\mu$ of all adults in Davis
is contained in this interval.  {\bf What does this mean?}  Say we were
to perform this experiment many, many times:  We'd sample 1000 people at
random, then record our interval $(\bar{W} - 1.96 \frac{s}{\sqrt{n}},
\bar{W} + 1.96 \frac{s}{\sqrt{n}})$; then we'd sample another 1000
people at random, and record what interval we got that time (it would
have a different center and a different radius); then we'd do this a
third time, a fourth, a fifth and so on.  We would get a large number of
intervals---and approximately 95\% of them would contain $\mu$, the mean
weight in the entire adult population of Davis.

Well, in our simulation case, it is {\it exactly the same situation}.
Simulation is a sampling process.  Our $\mu$ is the mean in the
``population'' of all bus waits, while $\bar{W}$ is the mean in our
sample of 1000 waits.  This is not mere analogy; mathematically the two
situations are completely identical.

Let's now admit that we actually know that from previous theory that $\mu$
is 10.  Then another way to see how to interpret confidence intervals is
to say that the output of the following program should be about 0.95:

\begin{Verbatim}[fontsize=\relsize{-2},numbers=left]
# Bus3.py, illustration of the bus paradox

import random,sys
from math import sqrt

rnd = random.Random(12345)

def doexpt(opt):
   arvtm = 0.0
   nbus = 0
   while arvtm < opt:
      nbus += 1
      arvtm += rnd.expovariate(0.1)
   return arvtm-opt

def main():
   observationpoint = float(sys.argv[1])
   nreps = int(sys.argv[2])
   count = 0
   for i in range(10000):
      sum = 0.0
      sum2 = 0.0
      for i in range(nreps):
         wait = doexpt(observationpoint)
         sum += wait
         sum2 += wait**2
      wbar = sum/nreps
      wbar2 = sum2/nreps
      s = sqrt(wbar2-wbar**2)
      if abs(wbar-10.0) < 1.96*s/sqrt(nreps): count += 1
   print count/10000.0

if __name__ == '__main__': main()
\end{Verbatim}

In fact, the output of that program was 0.9483, sure enough about
95\%.\footnote{Why not exactly 0.95?  Well, first of all, we only did
10000 intervals.  But second, remember, the Central Limit Theorem is
only approximate, though for n = 1000 here it should be quite close.}

\subsubsection{One More Point About Interpretation}
\label{craziness}

Some statistics instructors give students the odd warning, ``You can't
say that the probability is 95\% that $\mu$ is IN the interval; you can
only say that the probability is 95\% confident that the interval
CONTAINS $\mu$.'' This of course is nonsense.  Of course saying $\mu$ is
in the interval is equivalent to saying that the inteval contains $\mu$!

Where did this craziness come from?  Well, way back in the early days of
statistics, some instructor was afraid that a statement like ``The
probability is 95\% that $\mu$ is in the interval'' would make it sound
like $\mu$ is a random variable.  That was a legitimate fear, because
$\mu$ is not a random variable.  The random entity is the interval, not
$\mu$.  This is clear in our program {\bf Bus3.py} above---the 10 is
constant, while {\bf wbar} and {\bf s} vary from interval to interval.  

So, it was reasonable for teachers to warn students not to think $\mu$
is a random variable.  But later on, some statistics teachers just used
it as an unthinking rule, not understanding what the original concern
was, and as a result made foolish statements.

\subsection{Confidence Intervals for Proportions}
\label{propcis}

In our bus example above, suppose we also want our simulation to print
out the (estimated) probability that one must wait longer than 6.2
minutes.  We could add code for this in our simulation:

\begin{Verbatim}[fontsize=\relsize{-2},numbers=left]
# Bus4.py, illustration of the bus paradox

import random,sys
from math import sqrt

rnd = random.Random(12345)

def doexpt(opt):
   arvtm = 0.0
   nbus = 0
   while arvtm < opt:
      nbus += 1
      arvtm += rnd.expovariate(0.1)
   return arvtm-opt

def main():
   observationpoint = float(sys.argv[1])
   nreps = int(sys.argv[2])
   sum = 0.0
   sum2 = 0.0
   gt62 = 0  # count of the number of waits > 6.2 minutes
   for i in range(nreps):
      wait = doexpt(observationpoint)
      sum += wait
      sum2 += wait**2
      if wait > 6.2: gt62 += 1
   wbar = sum/nreps
   wbar2 = sum2/nreps
   s = sqrt(wbar2-wbar**2)
   print 'the estimated mean wait is', wbar
   print 'with a 95% margin of error of', 1.96*s/sqrt(nreps)
   p62 = float(gt62)/nreps
   print 'the estimated probability of waiting longer than 6.2 is', p62

if __name__ == '__main__': main()
\end{Verbatim}

The value printed out for the probability is 0.544.  We again ask the
question, how can we determine how close this is to the true population
probability?

It turns out that we already have our answer, because a probability is
just a special case of a mean.  To see this, let

\begin{equation}
Y =
\cases{
   1, & if $W > 6.2$\cr
   0, & otherwise \cr
}
\end{equation}

Then

\begin{equation}
E(Y) = 1 \cdot P(Y = 1) + 0 \cdot P(Y = 0) = P(W > 6.2)
\end{equation}

Let $p$ denote the probability we are trying to estimate, and let
$\widehat{p}$ denote our estimate of it; $\widehat{p}$ is our {\bf p62} in the
program.\footnote{The quantity is pronounced ``p-hat.''  The ``hat''
notation is traditional for ``estimate of.''}  Then one can show that

\begin{equation}
s^2 = \widehat{p} (1-\widehat{p})
\end{equation}

Equation (\ref{meanci}) becomes

\begin{equation}
\label{propci}
\left ( 
\widehat{p} 
- 1.96 \sqrt{\widehat{p} (1-\widehat{p})/n},
\widehat{p}
+ 1.96 \sqrt{\widehat{p} (1-\widehat{p})/n}
\right ) 
\end{equation}

We incorporate that into our program:

\begin{Verbatim}[fontsize=\relsize{-2},numbers=left]
# Bus5.py, illustration of the bus paradox

import random,sys
from math import sqrt

rnd = random.Random(12345)

def doexpt(opt):
   arvtm = 0.0
   nbus = 0
   while arvtm < opt:
      nbus += 1
      arvtm += rnd.expovariate(0.1)
   return arvtm-opt

def main():
   observationpoint = float(sys.argv[1])
   nreps = int(sys.argv[2])
   sum = 0.0
   sum2 = 0.0
   gt62 = 0  # count of the number of waits > 6.2 minutes
   for i in range(nreps):
      wait = doexpt(observationpoint)
      sum += wait
      sum2 += wait**2
      if wait > 6.2: gt62 += 1
   wbar = sum/nreps
   wbar2 = sum2/nreps
   s = sqrt(wbar2-wbar**2)
   print 'the estimated mean wait is', wbar
   print 'with a 95% margin of error of', 1.96*s/sqrt(nreps)
   p62 = float(gt62)/nreps
   print 'the estimated probability of waiting longer than 6.2 is', p62
   perr = 1.96*sqrt(p62*(1-p62)/nreps)
   print 'with a 95% margin of error of', perr

if __name__ == '__main__': main()
\end{Verbatim}

In this case, we get an interval of (0.51,0.57).

Note again that this uses the same principles as our Davis weights
example.  Suppose we were interested in estimating the proportion of
adults in Davis who weigh more than 150 pounds.  Suppose that proportion
is 0.45 in our sample of 1000 people.  This would be our estimate
$\widehat{p}$ for the population proportion $p$, and an approximate 95\%
confident interval (\ref{propci})for the population proportion would be
(0.42,0.48).

Note also that although we've used the word {\it proportion} in the
Davis weights example instead of {\it probability}, they are the same.
If I choose an adult at random from the population, the probability that
his/her weight is more than 150 is equal to the proportion of adults in
the population who have weights of more than 150.

And the same principles are used in opinion polls during presidential
elections.  Here $p$ is the proportion of people who plan to vote for
the given candidate.  We again use (\ref{propci}).

Note that in both the Davis and election examples, it doesn't matter
what the size of the population is.  The distribution of $\widehat{p}$, and
thus the accuracy of $\widehat{p}$, depends only on $p$ and $n$, the latter
being the number of people we sample.\footnote{Implicit in our
assumption that the $W_i$ are independent is that we are sampling {\bf
with replacement}, which means it's possible that our random sampling
process might choose the same person twice.  But except for cases in
which our sample size is a substantial fraction of the population size,
the probability of getting the same person twice would be very low, so
it doesn't matter.} So if you've ever wondered how a nationwide election
poll can get by with sampling only 1200 people, which is a common
number, you now know the answer.

In fact, since the maximum possible value of $\widehat{p} (1-\widehat{p})$ is
0.25,\footnote{Use calculus to find the maximum value of f(x) = x(1-x).}
the pollsters know that their margin of error with n = 1200 will be at
most $1.96 \times 0.5/\sqrt{1200}$, or about 3\%. 

\subsection{Conditional Confidence Intervals}

Often in simulations we will be interested in conditional quantities.
In a queuing model, for instance, we might wish to estimate
the mean wait time of jobs which arrive when the server is busy.
We still use the usual formulas, e.g. (\ref{meanci}) and (\ref{propci}),
but keep in mind that the value of n in those formulas, even though we
might have run the simulation for a long time and the n for
unconditional quantities would be large.

\section{Statistical Inference in Infinite Time Horizon Settings}

\subsection{Illustrative Example}

Recall our example program {\bf Aloha2.py} in our introductory unit on
simulation, \url{http://heather.cs.ucdavis.edu/~matloff/156/PLN/SimIntro.pdf}.  
Here network nodes would generate messages to send at random times, but also
randomly refrain from sending, in order to avoid collisions.

Let $Y_i$ denote the number of active nodes at epoch i.  We are
interested in what happens as time goes to infinity.  But what does that
really mean?

Intuitively, we feel that this system will converge to steady state in
some sense.  This certainly doesn't mean in the deterministic sense.
The latter would mean that 

\begin{equation}
\lim_{i \rightarrow \infty} Y_i
\end{equation}

exists, which cannot be true; the $Y_i$ will continue to fluctuate.  But
we do feel that they will have a long-run average, i.e. that

\begin{equation}
\label{cesaro}
\lim_{i \rightarrow \infty} \frac{Y_1+...+Y_i}{i}
\end{equation}

will exist as some constant $\nu$ and that 

\begin{equation}
\label{nulim}
\lim_{i \rightarrow \infty} E(Y_i) = \nu
\end{equation}

These things can be proven under certain reasonable mathematical
conditions.  In fact, under these conditions one can show that 

\begin{equation}
\label{pyik}
\lim_{i \rightarrow} P(Y_i = k)
\end{equation}

exists for each k.  For instance, the probability that there are, say, 3
nodes active at time i will converge as i gets large.

The questions of interest here will be

\begin{itemize}

\item How long do we need to run the simulation to get a reasonably
accurate estimate of $\nu$?

\item How can we find a valid confidence interval for $\nu$?

\end{itemize} 

\subsection{Mathematical Statement of the Problem}

This task would be straightforward in the case of estimating $\nu_8 =
E(Y_8)$.  We could simulate, say, 1000 replications of the system
through time 8.  Letting $Y_{8j}$ denote the $j^{th}$ of these
replications, we would these random variables would be {\it independent}
and {\it identically distributed} with mean $\nu_8$.  Equation
(\ref{meanci}) would apply perfectly.  The width of the confidence
interval would then give us a good idea as to the accuracy of our
estimate of $\nu$.

The situation in (\ref{cesaro}) is not nearly so straightforward.  There
the $Y_i$ are {\it neither} independent {\it nor} identically
distributed.  Let's look at this a bit more:

\begin{itemize} 

\item The $Y_i$ are not independent.  The number of active nodes at time
28, say, will have some effect on the number at time 29.

\item The $Y_i$ are not identically distributed.  The process does
converge to a steady state, but it has a different distribution at
each finite time.  For example, $Y_1 = 0$ with certainty, while $\lim_{i
\rightarrow \infty} P(Y_i = 0)$ will be quite different.

\end{itemize}

The problem of not being identically distributed can be phrased as
saying that near the beginning of the simulation, we are in a {\bf
transient period}, during which the process is settling down to steady
state.  Of course, technically speaking, the transient period is
infinite, since we ``reach'' steady state only after an infinite amount
of time.  But in practical terms, we come reasonably close enough to
steady state after a finite amount of time.  In (\ref{pyik}), for
instance, the limiting probability for k = 2 might be, say, 0.64, but
after a certain number of epochs it will be within 0.02 of that.

\subsection{Determining the Transient Period}

The problem of the $Y_i$ not being identically distributed is somewhat
easier to deal with.  To discuss that, let's start by looking at
(\ref{cesaro}): 

\begin{Verbatim}[fontsize=\relsize{-2},numbers=left]
# AlohaC2.py, form of slotted ALOHA

import random, sys 

class node:  # one object of this class models one network node
   s = 4  # number of nodes
   b = 0.3  # backoff parameter; refrain from sending with probability b
   q = 0.2  # msg creation probability
   activeset = []  # which nodes are active now
   inactiveset = []  # which nodes are inactive now
   r = random.Random(98765)  
   def __init__(self):  
      node.inactiveset.append(self)
   def checkgoactive():  # generate nodes which change to active
      for n in node.inactiveset:
         if node.r.uniform(0,1) < node.q:
            node.inactiveset.remove(n)
            node.activeset.append(n)
   checkgoactive = staticmethod(checkgoactive)  
   def trysend():  # the active nodes try to send, or not
      numnodestried = 0  
      whotriedlast = None  
      for n in node.activeset:
         if node.r.uniform(0,1) > node.b:
            whotriedlast = n
            numnodestried += 1
      if numnodestried == 1:
         node.activeset.remove(whotriedlast)
         node.inactiveset.append(whotriedlast)
   trysend = staticmethod(trysend)  
   def reset():  
      for n in node.activeset:
         node.activeset.remove(n)
         node.inactiveset.append(n)
   reset = staticmethod(reset)  
            
def main():
   for i in range(node.s): node()
   sum = 0  
   for epoch in range(10000):  # simulate until time 9999
      node.checkgoactive()
      node.trysend()
      sum += len(node.activeset)  # add Yi to total
      print epoch, float(sum)/(epoch+1)  # to be redirected to a file

if __name__ == '__main__': main()
\end{Verbatim}

Here we've adapted the program to look only at a specific value of {\bf
b}, and to look at many epochs instead of replications for epoch 8.
Most importantly, we are printing out the values of (\ref{cesaro}) as we
go along.  Figure \ref{meanyi} shows the graph of these values. 

\begin{figure}[tb]
\centerline{
% \includegraphics[width=4.0in]{Cesaro2.pdf}
\includegraphics{Cesaro2.pdf}
}
\caption{Mean Yi}
\label{meanyi}
\end{figure}

Convergence seems to occur after about 1000 epochs.  What that says is
that to deal with the problem of bias during the transient period, we
can just throw out the first 1000 epochs of our simulation.  Instead of
computing

\begin{equation}
\label{ces1}
\frac{Y_1+...+Y_{10000}}{10000}
\end{equation}

we would compute

\begin{equation}
\label{ces2}
\frac{Y_{1001}+...+Y_{10000}}{9000}
\end{equation}

Actually, even discarding 1000 observation is much too conservative.
The expected values in (\ref{nulim}) converge much faster than that.
The slowness of convergence here is due to the fact that we have only
one observation at each epoch.  If we were to do many replications at
each epoch (i.e.  run the above code, with 10000 epochs, in many
replications), we'd see much faster convergence.  Short of doing that,
though, we can take the conservative route.

\subsection{Has the Simulation Run Long Enough?}

We see that (\ref{cesaro}) has settled down to around 2.95.  For
informal purposes, this may be good enough.

But in many applications of simulation, such as writing research
publications and government reports, we must present not only the sample
estimate but also a margin of error.  So, again we must ask how to find
a confidence inteval for $\nu$.

We saw above how to deal with the fact that our observations $Y_i$ are
not identically distributed.  What should we do about their lack of
independence?  There are three main methods for handling this, which we
will now describe.

\subsection{The Replications Method}

Here we would run, say, 1000 replications of the simulation up to time
10000, getting 1000 values of (\ref{ces2}).  These values would be
independent and identically distributed, and thus we could form a
confidence interval for $\nu$ using (\ref{meanci}).

This is a clean way to solve the problem, but obviously with
considerable extra run time.

\subsection{The Batch Means Method}

Though our $Y_i$ are not independent, they are approximately independent
when spaced far enough apart.  For instance, $Y_{26}$ and $Y_{39}$
should have almost nothing to do with each other.

So, we could take our 9000 observations $Y_{1001},...,Y_{10000}$ and
divide them into batches of size, say, 50.  $Y_{1001},...,Y_{1050}$
would be the first batch, $Y_{1051},...,Y_{1100}$ would be the second
batch, and so on.  Let $B_k$ denote the mean Y value in the $k^{th}$
batch.  Then the $B_k$ would be approximately independent, by virtue of
the spacing, and approximately identically distributed, by virtue of
their having occurred past the transient period.  Thus we could use them
in (\ref{meanci}), with 9000/50 = 180 in place of n,
$\sum_{k=1}^{180}B_k/180$ in place of $\bar{W}$, and with s calculated
from (\ref{s2}) for the $B_k$. 

This is probably the most commonly-used method.

\subsection{The Regenerative Process Method}
\label{regen}

This method is the most elegant of the three, though it is not always
feasible.  The key to this method is to find a state of the system in
which ``time starts over.''  Some readers may find this reminiscent of
the fact that exponential distributions are memoryless, and what we are
discussing here is similar,  but much broader.

{\bf Example 1:}

Consider an example from our introductory unit on SimPy, at
\url{http://heather.cs.ucdavis.edu/~matloff/156/PLN/DESimIntro.pdf}.  We
have two machines, whose up times are exponentially distributed with
intensity 1.0, and whose repair times are exponentially distributed with
intensity 0.5.  The repairperson is not summoned until both machines are
down.  We are interested in the long-run proportion $\rho$ of the time
in which both machines are up simultaneously.

Consider the times at which the system enters the state in which both
machines are up, which we will call the ``full-capacity state.''  Our
simulation has them start in this state.  After a while, one of the
machines will go down, then the other.  Then one of the machines, say
machine B, will begin repair, and after that is done, the other machine,
A, will begin repair.  Actually, while A is being repaired, B may go
down again and so on, but eventually we will reach a situation in which
we have entered into the full-capacity state for the second time.  This
is our first ``time starts over'' point, which is called a {\bf
regeneration point}.  The subsequent regeneration points are the
subsequent times at which we enter full capacity state.

We must ask the question, does ``time start over'' at our proposed
regeneration times?  Probabilistically speaking, is the distribution of
the process from such a time onward the same as it was starting at time
0.0?  The answer is yes, but only because up times are exponentially
distributed and thus memoryless.  If up times were to have some other
distribution, things would not work.  

For instance, say at certain regeneration point is triggered by a repair
of A, with B having already been up for a while. If up time is not
memoryless, the fact that B has been up for a while changes the
probability that it will go down soon.  So, without the exponential
distribution here, time would NOT ``start over'' when A comes up.

But if up times are exponentially distributed, the times at which we
enter the full-capacity state are indeed times in which the system
``starts over,'' and acts independently from the past.  In fact, we
would not even need to have repair times be exponentially distributed.
distribution.\footnote{\label{indp}Note that we have also tacitly
assumed that the two machines act independently of each other, that for
a given machine its successive up times are independent and so on.} 

There are other sets of regeneration points in this process as well.
One such set consists of the times at which the repairperson is
summoned.  This set is especially important, because it does not depend
on the assumption of exponential distributions for either up time or
down time.  

Note carefully that we continue to make the independence assumption for
the set of up and down times of the machines.  In other words, if say
Machine A has a long up time in its first up period, we are assuming
that this does not make the machine any more or less likely to have a
long up time in its second up period, etc.

{\bf Example 2:}

Consider the G/G/1 queue, where the 1 means there is one server and
the two Gs stand for ``general,'' meaning that the interarrival times
and service times have general distributions, not necessarily
exponential.  Though one might at first think that the lack of
exponential distributions would imply lack of regeneration points, this
is definitely not the case.  In particular, the points in time in which
an arriving job encounters an idle server are indeed regeneration
points.

Let us suppose that in this example we are interested in $\mu$, the
long-run average queue length.

\subsubsection{What We'll Be Doing Here}

Keep in mind that in both these examples, there is no problem in
estimating the quantity of interest, $\rho$ and $\mu$, respectively.
The question at hand is how to find valid confidence intervals for these
quantities.  It will happen, though, that we get a new estimator in the
process as well.

Let's call the periods between successive regeneration points {\bf
regeneration cycles}.  The basic idea is that the activities of our
process in one cycle is independent of, and identically distributed
with, the corresponding activities in the other cycles.  For instance in
Example 1, the total time in which the system is in full capacity during
the second such period is independent of, and has the same distribution
as, the corresponding time in the first period.\footnote{You might
object to the assertion here that the full-capacity times for the two
periods have the same distribution.  You may ask, ``What if the second
period is longer?  Won't that mean that its full-capacity time will tend
to be longer?'' That is true, but it merely says that, {\it conditioned
on the length of the period}, the full-capacity times will not have the
same distribution.  But unconditionally, the two distributions will be
identical.}  

\subsubsection{Notation}

Let $V_t$ denote the state of our system at time t.  $V_t$ is a vector
of quantities that fully describe the current state of the system.  For
instance, in our machine repair example above, $V_t$ would consist of
the following quantities at time t:

\begin{itemize}

\item 1 or 0, according to whether machine A is up 

\item 1 or 0, according to whether machine B is up 

\item time A has been up, if it's up

\item time B has been up, if it's up

\item 1 or 0, according to whether machine A is being repaired 

\item 1 or 0, according to whether machine B is being repaired 

\item time A has been under repair, if it's being repaired

\item time B has been under repair, if it's being repaired

\end{itemize}

We need the population quantity $\gamma$ that we are trying to
estimate to be of the form 

\begin{equation}
\label{gammalim}
\gamma = \lim_{t \rightarrow \infty} \frac{1}{t} \int_{0}^{t} h(V_u) ~ du
\end{equation}

for some function h.  

For instance, in our machine repair example we had

\begin{equation} h(t) = \cases{ 1, & if full capacity at time t\cr 0, &
otherwise \cr } \end{equation}

Here, the integral in (\ref{gammalim}) is the total time we are in full
capacity during (0,t).  Dividing that integral by t gives us the
proportion of time we are in full capacity during (0,t).

In our queuing system example, $h(V_t)$ would simply be the length of
the queue at time t.  In this case, its integral from 0 to t is the sum
of the queue lengths during that time, multiplied by the amount of time
each length occurred.

Let $T_i$ denote the $i^{th}$ regeneration point, with $T_0 = 0.0$.  Let
$D_i = T_i - T_{i-1}$ represent the duration of the $i^{th}$
regeneration cycle.  We run the simulation for n cycles.

Now set

\begin{equation}
C_i = \int_{T_{i-1}}^{T_i} h(V_u) ~ du
\end{equation}

so that for instance in Example 1 above, $C_5$ is the amount of time the
system spends in the full-capacity state during the fifth regeneration
cycle.

As usual, define sample means:

\begin{equation}
\overline{C} = \frac{1}{n} \sum_{i=1}^n C_i
\end{equation}

\begin{equation}
\overline{D} = \frac{1}{n} \sum_{i=1}^n D_i
\end{equation}

\subsubsection{Estimator and Confidence Interval}

Renewal theory (a topic treated in our unit at
\url{http://heather.cs.ucdavis.edu/~matloff/256/PLN/Renewal.pdf}) can be
used to prove, and it is intuitive, that $\rho$ is equal to the ratio
between the mean time in full capacity per cycle divided by the mean
cycle time,

\begin{equation}
\label{iglehart}
\gamma = \frac{E(C)}{E(D)}
\end{equation}

where $C$ and $D$ are random variables having the common distribution of
the $C_i$ and $D_i$, respectively.  Then the natural estimate of
$\gamma$ based on our simulation output is

\begin{equation}
\widehat{\gamma} =
\frac
{\overline{C}}  % \bar{} wasn't showing up well, so used \overline{}
{\overline{D}}
\end{equation}

Now consider the random variables $X_i = C_i - \gamma D_i$.  Remember, we
don't know the value of $\gamma$, but these random variables exist.  

Because of the nature of the regeneration cycles, the $X_i$ are
independent and identically distributed.  From (\ref{iglehart}), we know
that they have mean 0.  

Thus the Central Limit Theorem (CLT) tells us that the random variable

\begin{equation}
\label{regenclt}
\frac
{\overline{X}}
{\sqrt{Var(X)/n}}
\end{equation}

is approximately distributed N(0,1) when n is large, where
$\overline{X} (X_1+...X_n)/n = \overline{C} - \gamma \overline{D}$. 

For a generic random variable $X$ having the distribution of the $X_i$,

\begin{equation}
\label{varcov}
Var(X) = Var(C) + \gamma^2 Var(D) - 2\gamma Cov(C,D)
\end{equation}

Replacing by sample analogs in (\ref{varcov}) gives us (\ref{weirds2}).
It can be shown that the CLT is still valid if we make this substitution
in (\ref{regenclt}).  So, the quantity

\begin{equation}
\label{regenclt2}
\frac
{\overline{X}}
{s_X/n^{0.5}}
\end{equation}

has an approximate N(0,1) distribution.  So,

\begin{eqnarray}
0.95 &\approx& P \left ( - 1.96 \frac{s_X}{\sqrt{n}} < 
\overline{C} - \gamma \overline{D} <
1.96 \frac{s_X}{\sqrt{n}} \right ) \\
&=& \left ( - 1.96 \frac{s_X}{\overline{D}\sqrt{n}} < 
\widehat{\gamma} - \gamma <
1.96 \frac{s_X}{\overline{D}\sqrt{n}} \right ) 
\end{eqnarray}

Thus an approximate 95\% confidence interval for $\gamma$ is

\begin{equation}
\label{gammaci}
(\widehat{\gamma} - 1.96 \frac{s_X}{\overline{D} n^{0.5}},
\widehat{\gamma} + 1.96 \frac{s_X}{\overline{D} n^{0.5}})
\end{equation}

where

\begin{equation}
\label{weirds2}
s^2_X = \frac{1}{n} \sum_{i=1}^n (C_i - \bar{C})^2 +
\widehat{\gamma}^2  \frac{1}{n} \sum_{i=1}^n (D_i - \bar{D})^2 -
2 \widehat{\gamma} \frac{1}{n} \sum_{i=1}^n (C_i - \bar{C}) (D_i - \bar{D})
\end{equation}

\subsubsection{Example:  Machine Repair Simulation}

Let's take our earlier machine repair example from Section \ref{regen}.
We'll use the first set of regeneration points described there, the
times at which we enter full-capacity state.  This requires us to assume
exponential distributions for the up times (though not the repair
times), but it makes the illustration of the formation of a confidence
interval easier.  

Here is the code:

\begin{Verbatim}[fontsize=\relsize{-2},numbers=left]
#!/usr/bin/env python

# MachRep3Regen.py

# SimPy example:  Variation of Mach1.py, Mach2.py.  Two machines, but
# sometimes break down.  Up time is exponentially distributed with mean
# 1.0, and repair time is exponentially distributed with mean 0.5.  In
# this example,there is only one repairperson, and she is not summoned
# until both machines are down.  We find the proportion of time that
# both machines are up simultaneously (mathematically this can be shown
# to be 0.181818...), extending our point estimate for that proportion
# to an approximate 90% regenerative confidence interval.  We choose as our
# regeneration points the times at which we enter the "full capacity"
# state, i.e. both machines up (valid only if up time is exponentially
# distributed).

from SimPy.Simulation import *
from random import Random,expovariate
from math import *

class G:  # globals
   Rnd = Random(12345)
   RepairPerson = Resource(1)

class MachineClass(Process):
   MachineList = []  # list of all objects of this class
   UpRate = 1/1.0
   RepairRate = 1/0.5
   TotalUpTime = 0.0  # total up time for all machines
   NextID = 0  # next available ID number for MachineClass objects
   NUp = 0  # number of machines currently up
   CMon = Monitor() 
   DMon = Monitor()
   # these concern the regeneration cycles
   StartCycle = 0.0  # time the current cycle started
   BeginningOfCycle = True  # both machines are up now; once one machine
                            # goes down, they will never both be up
                            # again during this cycle
   BothUpThisCycle = None  # amount of time both machines up in this cycle
   def __init__(self):
      Process.__init__(self)  
      self.StartUpTime = None  # time the current up period started
      self.ID = MachineClass.NextID   # ID for this MachineClass object
      MachineClass.NextID += 1
      MachineClass.MachineList.append(self)
      MachineClass.NUp += 1  # start in up mode
   def Run(self):
      while 1:
         self.StartUpTime = now()  
         yield hold,self,G.Rnd.expovariate(MachineClass.UpRate)
         UpTime = now() - self.StartUpTime
         MachineClass.TotalUpTime += UpTime
         MachineClass.NUp -= 1
         if MachineClass.BeginningOfCycle:
            MachineClass.BothUpThisCycle = now() - MachineClass.StartCycle
            MachineClass.BeginningOfCycle = False
         # update number of up machines
         # if only one machine down, then wait for the other to go down
         if MachineClass.NUp == 1:
            yield passivate,self
         # here is the case in which we are the second machine down;
         # either (a) the other machine was waiting for this machine to 
         # go down, or (b) the other machine is in the process of being
         # repaired 
         elif G.RepairPerson.n == 1:
            reactivate(MachineClass.MachineList[1-self.ID])
         # now go to repair
         yield request,self,G.RepairPerson
         yield hold,self,G.Rnd.expovariate(MachineClass.RepairRate)
         MachineClass.NUp += 1
         if MachineClass.NUp == 2:
            MachineClass.CMon.observe(MachineClass.BothUpThisCycle)
            MachineClass.DMon.observe(now()-MachineClass.StartCycle)
            MachineClass.StartCycle = now()
            MachineClass.BeginningOfCycle = True
         yield release,self,G.RepairPerson

def Regen(CMon,DMon,Z):
   CBar = CMon.mean()
   DBar = DMon.mean()
   GammaHat = CBar/DBar
   CDBar = 0.0
   N = len(CMon)
   for i in range(N):
      CDBar += CMon[i][1] * DMon[i][1]
   CDBar /= N
   CDBar -= CBar * DBar
   S2 = CMon.var() + GammaHat**2 * DMon.var() - 2 * GammaHat * CDBar
   Radius = Z * sqrt(S2) / (DBar*sqrt(N)) 
   return GammaHat,Radius

def main():
   initialize()  
   for I in range(2):
      M = MachineClass()
      activate(M,M.Run())
   MaxSimtime = 1000.0
   simulate(until=MaxSimtime)
   # for a 90% CI, use 1.65
   (Center,Radius) = Regen(MachineClass.CMon,MachineClass.DMon,1.65)
   print 'center, radius of CI:', Center, Radius

if __name__ == '__main__':  main()

\end{Verbatim}

\subsubsection{Feasibility}

Note that in a simulation with a very complicated state vector, a
regeneration point may be difficult to find.  Moreover, the regeneration
cycles may be extremely long.  These considerations make this method
difficult to use in some applications. 

\section{Learning More}

The use of statistics is widespread throughout the world.
Unfortunately, much of it is {\it mis}use, even by professional
statisticians.  We saw an example in Section \ref{craziness}.

For a careful look at the foundations and applications of statistics,
see my online course, at
\url{http://heather.cs.ucdavis.edu/~matloff/probstatbook.html}.


\end{document}