The schedule is tentative and subject to change. (Last update: 05/07/18 )

Week Date Lecture 18 spring 15 spring 14 spring Video/Notes 14 spring Quiz/Exam/HW Subject

(Papoulis, 4th Ed.)

Topics
1 2/19 1 mp4#1 asx#1 note#1   lecture#1  1.asx quiz#01

Ch 1. The Meaning  of Probability

quiz#01 lecture#2  2.asx

Random Experiment, Outcome, Event, Set function,

2/21 2 mp4#2 asx#2 note#2   lecture#3  3.asx

note#3

homework#01

Ch 2. The Axioms of Probability

Sample Space, Event Space, Borel Field, Probability Measure, Countable Additivity
2 2/26 3 mp4#3 asx#3 note#3 quiz#02 lecture#4  4.asx

note#4

Conditional Probability, Total Probability Theorem,  Bayes' Theorem, Independent Events, Borel-Cantelli Lemma
lecture#5  5.asx

Ch 3. Repeated Trials

Combined Experiment, Independent combining, Repeated Trial, Monte-Hall Problem,
quiz#03 lecture#6  6.asx

note#6

2/28 4 mp4#4 asx#4 note#4   Supplementary note_pch04   Ch 4. The Concept of Random Variable Random Variable as a function, Cumulative Distribution Function (CDF) and its properties
3 3/5 5 mp4#5 asx#5 note#5 3 types of random variables (discrete, continuous, mixed types), Histogram and, Probability Density Function (PDF), Properties of the Probability Density Function, Some common distributions,
3/7 6 mp4#6 asx#6 note#6 quiz#04   Some common distributions, Properties of a CDF,
4 3/12 7 mp4#7 asx#7 note#7     Conditional Distribution and Density Functions, Total Probability Theorem, Bayes's theorem, Gaussian approximation to a binomial r.v., Poisson approximation to a binomial r.v.
3/14 8 mp4#8 asx#8 note#8 quiz#5   Ch 5. Functions of One Random Variable Function of a r.v., Direct method, Jacobi method,
5 3/19 9 mp4#9 asx#9 note#9      Inverse problem, Expected value, Alternate form of expectation,
3/2110 mp4#10 asx#10 note#10 quiz#6   Fundamental theorem of Expectation, Variance, Conditional mean, Total expectation theorem; Moments
3/22 1st midterm exam    exam01

exam01sol

exam01 Alternate form of moments of non-negative random variable
6 3/26 11   mp4#11 asx#11 note#11       Inequalities: Markov, Bienayme, Chebyshev, Chernoff, Jensen; Complex-valued random variable, Characteristic function, Moment generating function, Central Limit Theorem: Gaussian approximation to Binomial
3/2812 mp4#12 asx#12 note#12    lecture#14  14.asx homework#02 Ch 6. Two Random Variables

6-1 Two Random Variables and Their Joint CDF, Joint Probability Density Function, Bivariate Gaussianity,

• Circular symmetry

• Circularly symmetrical and Independent => Gaussian

• Line masses

7 4/213 mp4#13 asx#13 note#13  quiz#7 lecture#15  15.asx homework#03

6-2 A deterministic function of two random variables:

• direct method,

• linear (dep, indep),

• nonlinear (max statistic, min statistic, etc.);

6-3 Two functions of two random variables

• direct method: Example 6-21

• Jacobi's method: derivation, cartesian coordinate -> polar coordinate

• linear TF

• auxiliary variable method,

4/4 14 mp4#14 asx#14 note#14    lecture#16  16.asx homework#04

6-4 Joint Moments:

• Expected Value and Correlation Coefficient (Correlation Coefficient, Covariance of Two R.V.'s)

• covariance

• correlation coefficient and equality condition,

• uncorrelated, orthogonal,

• variance of sum of two random variables,

6-5 Joint CF: independence and convolution; Normal random vector,

8 4/9 15 mp4#15 asx#15 note#15  quiz#8

6-6 Conditional Distribution and Density Functions

• conditional density, total density, Bayes' theorem, independence

6-7 Conditional expectation:

• conditional mean, conditional variance

• joint Gaussian case

• Definition of E{X|Y}

• total expectation theorem

4/11 16 mp4#16 asx#16 note#16      Ch 7. Sequences of Random Variables

7-1 Random Vector

• joint pdf, cdf, CF, MGF

• Transform: direct method, Jacobi's method

• Independence, order statistics, Group Independence, conditional independence

• Expectation: mean, covariance, correlation

• complex random vector: complementary covariance, proper, improper

7-2 Conditional Density, CF, Multivariate Gaussian R.V.'s(N-variate Gaussian R.V.'s),

• Chain rule,

• total conditional pdf, conditional expectation, total expectation,

• Gaussian random vector CF and pdf

• Goodman's theorem for a proper-complex Gaussian random vector

exam02
9 4/16 17 mp4#17 asx#17 note#17      lecture#19  19.asx note#19

7-3 Mean Square Estimation

• MMSE estimator = Conditional mean estimator

• AMMSE estimator and conditional pdf of normal vector

• LMMSE estimator and orthogonality principle

• Z vector and orthonormal data transformation (whitening)

• square-root of a covariance matrix

• Cholesky factorization of a covariance matrix

4/18 18 mp4#18 asx#18 note#18
• linear independence

• positive semi-definiteness of Rxx, Cxx

• Equality of two random variables

• linear independence and positive definiteness of Rxx

7-4 Stochastic convergence and Limit Theorems

• a random sequence

• Convergence of a sequence of real numbers, a sequence real functions

• A random sequence converges to what?

• Convergence of a sequence of real functions

• Modes of convergence of a sequence of real random variables, SLLN, WLLN, CLT

• Almost Sure Convergence, Convergence in Probability, Convergence in Quadratic Mean, Convergence in Distribution

4/19 2nd midterm exam   exam02

exam02sol

10 4/23 19 mp4#19 asx#19 note#19

•

• a.s. =>p

• in the pth mean => p

• a.s. does not imply p

• p does not imply a.s.

• p => d

• Convergence in distribution revisited

• characteristic function

• bounded continuous function

4/25 20 mp4#20 asx#20 note#20    Matlab homework#03
• Law of large numbers: strong, weak

• approximation to Binomial: Gaussian (CLT), Poisson

7-5 Random numbers: meaning and generation

• Law of Large Numbers and Monte-Carlo integration

• Generation of a pseudo-random number

• Generation of random variable and vector

9-1. Definition

Part II Stochastic Processes

11 4/30 21 mp4#21 asx#21 note#21    lecture#23_1 lecture#23_2 lecture#23_3   Ch. 9. General Concepts

9-1. Definition

• DT and CT random processes

• discrete and continuous random processes

• Alternate definition of a random process

• Full specification of a random process: Kolomogorov's extension theorem

• Poisson point process

• Poisson arrival process

• n-th order distribution

• second-order moments: real X, complex Z, real X and Y, complex Z and W

• alpha-dependent random process

• white noise

• Equality and independence of two random processes

Ch 6. Random Processes - Temporal Characteristics

(Peyton Z. Peebles, Jr)

The Random Process Concept, Stationarity and Independence, Correlation Functions, Measurement of Correlation Functions, Gaussian Random Processes, Complex Random Processes

5/2 22 mp4#22 asx#22 note#22    lecture#24

note#24

note#25

9-1 (cont.)

• Normal random process and existence theorem

• stationary random processes

• strict-sense stationarity, joint stationarity

• n-th order stationarity

• wide-sense stationarity, centering, covariance stationarity, joint wide-sense stationarity

• second-order moments

• asymptotic stationarity

• stationary in an interval

• process with stationary increment

5/2

Make-Up

(8pm @LG104)

23 mp4#23 asx#23 note#23    lecture#26_1 lecture#26_2
• periodic with period T

• surely, almost surely

• in the mean-square sense

• periodic mean

• doubly periodic autocorrelation function

• cyclostationarity

• periodic mean, periodic autocorrelation

9-2. System with Stochastic Inputs

• deterministic vs. random mapping from a r.p. to a r.p.

12 5/9 24 mp4#24 asx#24 note#24  quiz#9
• Memoryless system

• full characterization

• A strict-sense stationary input leads to a strict-sense stationary output.

• square law device, hardlimter, limiter

• LTI system

• Gaussian input leads to Gaussian output.

• Fundamental theorem: mean, autocorrelation of output

• WSS input leads to WSS output.

13 5/14 25 mp4#25 asx#25 note#25
•

• LCCDE

• Vector-valued process and Multi-terminal (MIMO) Systems

•  LTI system

• Second-order moments

9-3. Power Spectrum

5/1626 mp4#26 asx#26 note#26
• Deterministic signals

• Energy signal and Power signal

• Energy spectral density (ESD) and necessary conditions to be satisfied

• Necessary conditions to be satisfied by Power spectral density

• Random processes

• Instantaneous power, time-averaged power, ensemble-averaged power, average power, average cross power

• Definitions of PSD and its properties

• Definitions of xPSD and its properties

• Wiener-Khinchine Theorem

• Theorems for PSD and xPSD and special cases: WSS, jointly WSS

14 5/2127 mp4#27 asx#27 note#27
•

• Proof

• Properties of PSD, xPSD revisited

• LTI filtering of X(t) to have Y(t), S_XX(f), S_{XY}(f), S_{YY}(f)

• Wireless Channel and Doppler Power spectrum

5/23 28 mp4#28 asx#28 note#28
• special case of WSSUS channel

• scattering function and Doppler power spectrum

• Line spectra

• Vector processes

• MS continuity, MS periodicity, and continuity and periodicity of autocorrelation function

15 5/28 29 mp4#29 asx#29 note#29

9-4. DT Processes

9-5. Stochastic continuity, differentiation, integration

Ch. 10. Random Walks and Other Applications

10-1 Random Walk, The Wiener process

10-3. Modulation

10-4. Cyclostationary Processes

5/30 30 mp4#30 asx#30 note#30

10-5. Bandlimited Processes and Sampling Theory

10-6. Deterministic signal in noise

10-7. Bispectra and System Identification

Ch. 11. Spectral Representation

11-1. Factorization and Innovations

11-2 Finite-Order Systems and State Variables

11-3 Fourier Series and KL expansion

11-4 Spectral representation of random processes

Ch. 12 Spectrum Estimation

12-1 Ergodicity

12-2 Spectrum Estimation

Ch. 15. Markov Chains

15-1. Introduction

15-2. Higher Transition Prob. and the Chapman-Kolmogorov Equation

15-3. Classification of States

15-4. Stationary Distributions and Limiting Probabilities

15-5. Transient States and Absorption Prob

15-6. Branching Processes

Ch. 16. Markov Processes and Queueing Theory

16-1 Introduction

16-2. Markov Processes

16-3. Queueing Theory

16-4 Network and Queues

16 6/7 Final exam

(covers upto Ch. 9)

exam03

exam03sol

exam03

28 mp4#1 asx#1 note#1  lecture#30_1 (73m)
lecture#30_2 (140m)
lecture#27

note#27

note#28

Ch 6. Random Processes - Temporal Characteristics

(Peyton Z. Peebles, Jr)

The Random Process Concept, Stationarity and Independence, Ergodicity
29   lecture#31_1
lecture#31_2
lecture#31_3

note Ch. 7

lecture#29   Ch 6. Random Processes - Temporal Characteristics

Ch 7. Random Processes - Spectral Characteristics

(Peyton Z. Peebles, Jr)

Correlation Functions, Measurement of Correlation Functions, Gaussian Random Processes, Poisson Random Processes, Power Density Spectrum

30

mp4#1 asx#1 note#1  lecture#32_1
lecture#32_2
lecture#32_3
lecture#32_4

Ch 7. Random Processes - Spectral Characteristics

(Peyton Z. Peebles, Jr)

PSD & Its Properties, Relationship b/w PSD & Autocorrelation Function, xPSD & Its Properties, Relationship b/w xPSD & Cross-Correlation Function, Noise Definitions