===============================================================
Lect 18 - November 25, 2008 - ECS 20 - Fall 2008 - Phil Rogaway
===============================================================
Today: o Probability
Announcements
- Said one more quiz, but probably scuttle that: too few remaining
class meetings and now too close to the final exam.
----------------------------------------------------------------------------
1. Basic definitions / theory
----------------------------------------------------------------------------
Schaum's, chapt 7.
DEF: A [*finite*] *probability space* is a finite set S ("the sample space")
together with a function P: S-> [0,1] (the *probability measure*)
that
\sum P(x) = 1 More often: omega, mu, Omega
x \in S for x P S
In general, whenever you hear "probability" make sure that you are clear
WHAT is the probability space and WHAT is the event in question.
DEF: Let (S, P) be a probability space.
An *event* is a subset of S.
An *outcome* is a point in S.
DEF: Let A be an event of probability space (\Omega, \mu).
P(A) = \sum P(a) (used to using Pr, will probability slip)
a \in A
"The probability of event A"
DEF: The *uniform* distribution is the one where P(a) = 1/|S| -- all points
equiprobable.
DEF: Events A and B are *independent* if P(A\cap B) = P(A) P(B).
DEF: A *random variable* is a function X: S -> \R
DEF: E[X] = \sum P(s)X(s) // expected value of X ("average value")
s in S
DEF P(A|B) = P(A \cap B)/P(B)
Propositions:
- P(\emptyset) = 0 // by definition
- P(S) = 1
- P(A) + P(S \ A) = 1
- If A and B are disjoint events (that is, disjoint sets) then
P(A u B) = P(A) + P(B)
- ("sum bound")
P(A u B) <= P(A) + P(B)
- In general,
P(A u B) = Pr(A) + Pr(B) - P(A intersect B) // inclusion-exclusion principle
- Pr(A) = P(A|B1)P(B1) + P(A|B2)P(B2)
If B1,B2 disjoint sets whose union is S
- E(X+Y) = E(X) + E(Y) // expectation is linear.
----------------------------------------------------------------------
Eg 1: Dice.
The singular, the students assure me, is Die.
Like Mice and Mie. Or something like that.
----------------------------------------------------------------------
a You roll a fair die six times:
S = {1,2,3,4,5,6}
P(1)=P(2)=...=P(6)=1/6
"you roll an even number" is an event.
Event is A = {2,4,6}. P(A) = 3 * (1/6) = 1/2.
b Pair of dice.
S = {1,2,3,4,5,6} x {1,2,3,4,5,6}o
P((a,b)) = 1/36 for all (a,b) in S
Illustrate independence.
P(left die even and right die even) = P(left die even) P(right die even)
= (1/2)(1/2) =1/4
c Pair of dice, what's the chance of rolling an "8"?
Event E = {(2,6),(3,5),(4,4),(5,3),(6,2)}
P(E) = 5/36
Be careful: P(E) = |E|/|S| *if* we are assuming the *uniform*
distribution.
----------------------------------------------------------------------
Eg 2. Poker examples
----------------------------------------------------------------------
POKER DEFINITIONS:
Five cards dealt out. Order irrelevant.
What's the probability space?
Sample space has |S| = C(52,5)
Probability measure is uniform: P(a) = 1/|S|.
full house = 3 cards of one value, 2 cards of another value
two pairs = two cards of one value, two more cards of a second value,
the remaining card of a third value
A. How many poker hands are there?
Answer: C(52,5)=2,598,960 =|S|
B. What is the probability of being dealt a full house?
Answer: A full house can be identified by a pair of values,
like (J,8), where the first component of the pair is the value of what you
have three of and the second component is the value of which you have two.
So there are P(13,2) = 13*12 such pairs.
For each there are C(4,3)=4 ways to chose the first component
and C(4,2)=6 ways to choose the second
component. So all together there are
P(13,2) C(4,3)C(4,2) / C(52,5)
13*12*4*6=3,744 possible full houses.
The probability of being dealt one is therefore
P(13,2) C(4,3)C(4,2) / C(52,5)
= 3,744/2,598,960 \approx .001441
C. What's the probability of being dealt two pairs?
Answer: We can identify 2 pairs as in {J,8}.
Note that now the pair is now unordered.
There are C(13,2) such sets. For each there are C(4,2) ways to
choose the larger card and C(4,2) ways to choose the smaller card.
There are now 52-8=44 remaining cards one can choose as the fifth card.
So the total number of hands that are two pairs are:
C(13,2)*C(4,2)*C(4,2)*44 = 123,552.
The chance of being dealt two pairs is therefore
123,552/2,598,960 \approx .047539.
------------------------------------
Eg 3: Fair coin
------------------------------------
Flip a fair coin 100 times.
What is the probability space?
S = {0,1}^100
P(s) = 2^{-100} for all s in S.
What is the chance of getting exactly 50 out of the 100 coin flips
land heads?
C(100,50) 2^(-100) \approx 0.07959
(I said that Google would have a problem with C(100,50), but it
handles it without any problem at all).
------------------------------------
Eg 4: Biased coin
------------------------------------
Now, what if the coin is biased?
Say that the coin lands heads with probability .51, tails with probability .49.
each flip independent of the rest.
You flip an unfair coin 100 times. The coin lands heads
a fraction p=0.51 of the time:
S = {0,1}^100
P(x) = p^{#1(x)} (1-p)^{#0(x)} where #1(x) = the number of 1-bits
of 0-bits in the string x.
What's the Probability of 51 heads now?
C(100,5)(.51^5)(.49^50) \approx 7801
Makes sense -- 51 heads should now be the most likely number,
and things should fall off from there. Before, 50 was
the most likely number.
-----------------------------------------
Eg 5: Birthday phenomenon
-----------------------------------------
n=23 people gather in a room.
What' the chance that some two have the same birthday?
Assume nobody born 2/29, all other birthdays equiprobable.
S = [1..365]^23
P(same birthday) = 1 - (all birthday's different)
= 1 - (1-1/365)(1-364) ... (1-(n-1)/365)
= 1 - prod_{i=1}^{i=22} (1-1/i) = 0.507
More detailed analysis (perhaps prove later):
Let c(n,q) = probability of at least one collision in the
experiment of throwing q balls into n bins. Then
c(n,q) \approx 1-e^{-q^2/2n}
Solving for c(n,q)=1/2: q = \sqrt(ln 2) \sqrt(n)
\approx 1.1774 sqrt(n)
0.3 q(q-1)/n <= c(n,q) <= 0.5q^2/n
if q <= sqrt(2n)
---------------------------------------------------
Eg 5.1 An application: Cryptographic Hash Functions
(didn't do this in class)
---------------------------------------------------
SHA1: \bits^* -> \bits^160
About how long will it take to find a collision
if compute one new point every used?
roughly 2^80 msec,
10^37 years (1 year is about pi * 10^7 seconds)