Announcements             Syllabus             Schedule/Downloads


Schedule/Downloads (Spring 18)

The schedule is tentative and subject to change. (Last update: 05/07/18 )

Week Date Lecture 18 spring 15 spring 14 spring Video/Notes 14 spring Quiz/Exam/HW Subject

(Papoulis, 4th Ed.)

Topics  
1 2/19 1 mp4#1 asx#1 note#1   lecture#1  1.asx

note#1

quiz#01

quiz#01sol

Ch 1. The Meaning  of Probability

Syllabus

 
quiz#01 lecture#2  2.asx

note#2

Random Experiment, Outcome, Event, Set function,

 
2/21 2 mp4#2 asx#2 note#2   lecture#3  3.asx 

note#3

homework#01

Matlab homework#01

Ch 2. The Axioms of Probability

Sample Space, Event Space, Borel Field, Probability Measure, Countable Additivity  
2 2/26 3 mp4#3 asx#3 note#3 quiz#02 lecture#4  4.asx 

note#4

Conditional Probability, Total Probability Theorem,  Bayes' Theorem, Independent Events, Borel-Cantelli Lemma  
  lecture#5  5.asx 

note#5 

 

Ch 3. Repeated Trials

Combined Experiment, Independent combining, Repeated Trial, Monte-Hall Problem,  
quiz#03 lecture#6  6.asx 

note#6

2/28 4 mp4#4 asx#4 note#4   Supplementary note_pch04

lecture#7  7.asx

note#7

  Ch 4. The Concept of Random Variable Random Variable as a function, Cumulative Distribution Function (CDF) and its properties  
3 3/5 5 mp4#5 asx#5 note#5 3 types of random variables (discrete, continuous, mixed types), Histogram and, Probability Density Function (PDF), Properties of the Probability Density Function, Some common distributions,
3/7 6 mp4#6 asx#6 note#6 quiz#04

lecture#8  8.asx

note#8

  Some common distributions, Properties of a CDF,   
4 3/12 7 mp4#7 asx#7 note#7  

lecture#9  9.asx

note#9

  Conditional Distribution and Density Functions, Total Probability Theorem, Bayes's theorem, Gaussian approximation to a binomial r.v., Poisson approximation to a binomial r.v.  
3/14 8 mp4#8 asx#8 note#8 quiz#5

lecture#10  10.asx

note#10

Supplementary note_pch05 

  Ch 5. Functions of One Random Variable Function of a r.v., Direct method, Jacobi method,  
5 3/19 9 mp4#9 asx#9 note#9   

lecture#11  11.asx

note#11

  Inverse problem, Expected value, Alternate form of expectation,  
3/2110 mp4#10 asx#10 note#10 quiz#6

lecture#12  12.asx

note#12 

  Fundamental theorem of Expectation, Variance, Conditional mean, Total expectation theorem; Moments  
  3/22 1st midterm exam    exam01

exam01sol

  exam01

exam01solution

Alternate form of moments of non-negative random variable
6 3/26 11   mp4#11 asx#11 note#11   

lecture#13  13.asx

note#13  

   Inequalities: Markov, Bienayme, Chebyshev, Chernoff, Jensen; Complex-valued random variable, Characteristic function, Moment generating function, Central Limit Theorem: Gaussian approximation to Binomial  
3/2812 mp4#12 asx#12 note#12    lecture#14  14.asx

note#14  

homework#02

Matlab homework#02 

Ch 6. Two Random Variables

6-1 Two Random Variables and Their Joint CDF, Joint Probability Density Function, Bivariate Gaussianity,

  • Circular symmetry

  • Circularly symmetrical and Independent => Gaussian

  • Line masses

 
7 4/213 mp4#13 asx#13 note#13  quiz#7 lecture#15  15.asx

note#15  

Supplementary note_pch06

homework#03

6-2 A deterministic function of two random variables:

  • direct method,

  • linear (dep, indep),

  • nonlinear (max statistic, min statistic, etc.);

6-3 Two functions of two random variables

  • direct method: Example 6-21

  • Jacobi's method: derivation, cartesian coordinate -> polar coordinate

  • linear TF

  • auxiliary variable method, 

 
4/4 14 mp4#14 asx#14 note#14    lecture#16  16.asx

note#16  

homework#04

6-4 Joint Moments:

  • Expected Value and Correlation Coefficient (Correlation Coefficient, Covariance of Two R.V.'s)

  • covariance

  • correlation coefficient and equality condition,

  • uncorrelated, orthogonal,

  • variance of sum of two random variables,

6-5 Joint CF: independence and convolution; Normal random vector,

 
8 4/9 15 mp4#15 asx#15 note#15  quiz#8

lecture#17_1 lecture#17_2 

17_1.asx  17_2.asx

note#17  

 

6-6 Conditional Distribution and Density Functions

  • conditional density, total density, Bayes' theorem, independence

6-7 Conditional expectation:

  • conditional mean, conditional variance

  • joint Gaussian case

  • Definition of E{X|Y}

  • total expectation theorem

 
4/11 16 mp4#16 asx#16 note#16   

lecture#18  18.asx

note#18  

  Ch 7. Sequences of Random Variables 

7-1 Random Vector

  • joint pdf, cdf, CF, MGF

  • Transform: direct method, Jacobi's method

  • Independence, order statistics, Group Independence, conditional independence

  • Expectation: mean, covariance, correlation

    • complex random vector: complementary covariance, proper, improper

7-2 Conditional Density, CF, Multivariate Gaussian R.V.'s(N-variate Gaussian R.V.'s),

  • Chain rule,

  • total conditional pdf, conditional expectation, total expectation,

  • Gaussian random vector CF and pdf

  • Goodman's theorem for a proper-complex Gaussian random vector

            exam02

exam02solution

   
9 4/16 17 mp4#17 asx#17 note#17      lecture#19  19.asx note#19 

 

7-3 Mean Square Estimation

  • MMSE estimator = Conditional mean estimator

  • AMMSE estimator and conditional pdf of normal vector

  • LMMSE estimator and orthogonality principle

  • Z vector and orthonormal data transformation (whitening)

    • square-root of a covariance matrix

    • Cholesky factorization of a covariance matrix

 
 
 
4/18 18 mp4#18 asx#18 note#18       
  • linear independence

    • positive semi-definiteness of Rxx, Cxx

    • Equality of two random variables

    • linear independence and positive definiteness of Rxx

 

 

lecture#20_1 lecture#20_2

20_1.asx 20_2.asx

note#20

 

7-4 Stochastic convergence and Limit Theorems

  • a random sequence

  • Convergence of a sequence of real numbers, a sequence real functions

  • A random sequence converges to what?

  • Convergence of a sequence of real functions

  • Modes of convergence of a sequence of real random variables, SLLN, WLLN, CLT

    • Almost Sure Convergence, Convergence in Probability, Convergence in Quadratic Mean, Convergence in Distribution

 

  4/19 2nd midterm exam   exam02

exam02sol

       
10 4/23 19 mp4#19 asx#19 note#19   

lecture#21_1 lecture#21_2

21_1.asx 21_2.asx

note#21

 

lecture#22_1 lecture#22_2

 
 
  •  

    • a.s. =>p

    • in the pth mean => p

    • a.s. does not imply p

    • p does not imply a.s.

    • p => d

  • Convergence in distribution revisited

    • characteristic function

    • bounded continuous function

 
4/25 20 mp4#20 asx#20 note#20   

22_1.asx 22_2.asx

note#22

Supplementary note_pch07   

Matlab homework#03 
    • Law of large numbers: strong, weak

    • approximation to Binomial: Gaussian (CLT), Poisson

7-5 Random numbers: meaning and generation

  • Law of Large Numbers and Monte-Carlo integration

  • Generation of a pseudo-random number

  • Generation of random variable and vector

9-1. Definition

 
     

Part II Stochastic Processes

   
11 4/30 21 mp4#21 asx#21 note#21    lecture#23_1 lecture#23_2 lecture#23_3

23_1.asx 23_2.asx 23_3.asx

note#23

Supplementary note_ch6

  Ch. 9. General Concepts

9-1. Definition

  • DT and CT random processes

  • discrete and continuous random processes

  • Alternate definition of a random process

  • Full specification of a random process: Kolomogorov's extension theorem

    • Poisson point process

    • Poisson arrival process

  • n-th order distribution

  • second-order moments: real X, complex Z, real X and Y, complex Z and W

  • alpha-dependent random process

  • white noise

  • Equality and independence of two random processes

Ch 6. Random Processes - Temporal Characteristics

(Peyton Z. Peebles, Jr)

The Random Process Concept, Stationarity and Independence, Correlation Functions, Measurement of Correlation Functions, Gaussian Random Processes, Complex Random Processes

 

5/2 22 mp4#22 asx#22 note#22    lecture#24

24.asx

note#24

lecture#25

25.asx

note#25

 

 

9-1 (cont.)

  • Normal random process and existence theorem

  • stationary random processes

    • strict-sense stationarity, joint stationarity

    • n-th order stationarity

    • wide-sense stationarity, centering, covariance stationarity, joint wide-sense stationarity

      • second-order moments

    • asymptotic stationarity

    • stationary in an interval

    • process with stationary increment

5/2

Make-Up

(8pm @LG104)

23 mp4#23 asx#23 note#23    lecture#26_1 lecture#26_2

26_1.asx 26_2.asx

note#26 

 
  • periodic with period T

    • surely, almost surely

    • in the mean-square sense

      • periodic mean

      • doubly periodic autocorrelation function

  • cyclostationarity

    • periodic mean, periodic autocorrelation

9-2. System with Stochastic Inputs

  • deterministic vs. random mapping from a r.p. to a r.p.

12 5/9 24 mp4#24 asx#24 note#24  quiz#9    
  • Memoryless system

    • full characterization

    • A strict-sense stationary input leads to a strict-sense stationary output.

    • square law device, hardlimter, limiter

  • LTI system

    • Gaussian input leads to Gaussian output.

    • Fundamental theorem: mean, autocorrelation of output

    • WSS input leads to WSS output.

13 5/14 25 mp4#25 asx#25 note#25       
  •  

    • LCCDE

  • Vector-valued process and Multi-terminal (MIMO) Systems

    •  LTI system

    • Second-order moments

9-3. Power Spectrum

5/1626 mp4#26 asx#26 note#26       
  • Deterministic signals

    • Energy signal and Power signal

    • Energy spectral density (ESD) and necessary conditions to be satisfied

    • Necessary conditions to be satisfied by Power spectral density

  • Random processes

    • Instantaneous power, time-averaged power, ensemble-averaged power, average power, average cross power

    • Definitions of PSD and its properties

    • Definitions of xPSD and its properties

  • Wiener-Khinchine Theorem

    • Theorems for PSD and xPSD and special cases: WSS, jointly WSS

14 5/2127 mp4#27 asx#27 note#27       
  •  

    • Proof

  • Properties of PSD, xPSD revisited

  • LTI filtering of X(t) to have Y(t), S_XX(f), S_{XY}(f), S_{YY}(f)

  • Wireless Channel and Doppler Power spectrum

 

5/23 28 mp4#28 asx#28 note#28       
    • special case of WSSUS channel

    • scattering function and Doppler power spectrum

  • Line spectra

  • Vector processes

  • MS continuity, MS periodicity, and continuity and periodicity of autocorrelation function

15 5/28 29 mp4#29 asx#29 note#29       

9-4. DT Processes

9-5. Stochastic continuity, differentiation, integration

Ch. 10. Random Walks and Other Applications

10-1 Random Walk, The Wiener process

10-3. Modulation

10-4. Cyclostationary Processes

5/30 30 mp4#30 asx#30 note#30       

10-5. Bandlimited Processes and Sampling Theory

10-6. Deterministic signal in noise

10-7. Bispectra and System Identification

Ch. 11. Spectral Representation

11-1. Factorization and Innovations

11-2 Finite-Order Systems and State Variables

11-3 Fourier Series and KL expansion

11-4 Spectral representation of random processes

 

Ch. 12 Spectrum Estimation

12-1 Ergodicity

12-2 Spectrum Estimation

Ch. 15. Markov Chains

15-1. Introduction

15-2. Higher Transition Prob. and the Chapman-Kolmogorov Equation

15-3. Classification of States

15-4. Stationary Distributions and Limiting Probabilities

15-5. Transient States and Absorption Prob

15-6. Branching Processes 

Ch. 16. Markov Processes and Queueing Theory

16-1 Introduction

16-2. Markov Processes

16-3. Queueing Theory

16-4 Network and Queues 

16 6/7 Final exam

(covers upto Ch. 9)

  exam03

exam03sol

  exam03 

 

 
   

 

              
    28 mp4#1 asx#1 note#1  lecture#30_1 (73m)
lecture#30_2 (140m)

30_1.asx 30_2.asx

note Ch. 6

lecture#27

27.asx

note#27

lecture#28

28.asx

note#28

  Ch 6. Random Processes - Temporal Characteristics

(Peyton Z. Peebles, Jr)

The Random Process Concept, Stationarity and Independence, Ergodicity  
    29

mp4#1 asx#1 note#1

 
lecture#31_1
lecture#31_2
lecture#31_3

31_1.asx
31_2.asx
31_3.asx

note Ch. 7

lecture#29

29.asx

note#29 

  Ch 6. Random Processes - Temporal Characteristics

Ch 7. Random Processes - Spectral Characteristics

(Peyton Z. Peebles, Jr)

Correlation Functions, Measurement of Correlation Functions, Gaussian Random Processes, Poisson Random Processes, Power Density Spectrum  
   

30

mp4#1 asx#1 note#1  lecture#32_1
lecture#32_2
lecture#32_3
lecture#32_4

32_1.asx 32_2.asx 32_3.asx 32_4.asx

   

Ch 7. Random Processes - Spectral Characteristics

(Peyton Z. Peebles, Jr)

PSD & Its Properties, Relationship b/w PSD & Autocorrelation Function, xPSD & Its Properties, Relationship b/w xPSD & Cross-Correlation Function, Noise Definitions