**
Announcements
Syllabus
Schedule/Downloads**

**
Schedule/Downloads ****(Spring 18)**

The schedule is tentative and subject to change. (Last update: 05/07/18 )

Week Date Lecture 18 spring 15 spring 14 spring Video/Notes 14 spring Quiz/Exam/HW Subject (Papoulis, 4th Ed.)

Topics 1 2/19 1 mp4#1 asx#1 note#1 lecture#1 1.asx quiz#01 Ch 1. The Meaning of Probability

quiz#01 lecture#2 2.asx Random Experiment, Outcome, Event, Set function,

2/21 2 mp4#2 asx#2 note#2 lecture#3 3.asx homework#01 Ch 2. The Axioms of Probability

Sample Space, Event Space, Borel Field, Probability Measure, Countable Additivity 2 2/26 3 mp4#3 asx#3 note#3 quiz#02 lecture#4 4.asx Conditional Probability, Total Probability Theorem, Bayes' Theorem, Independent Events, Borel-Cantelli Lemma lecture#5 5.asx Ch 3. Repeated Trials

Combined Experiment, Independent combining, Repeated Trial, Monte-Hall Problem, quiz#03 lecture#6 6.asx 2/28 4 mp4#4 asx#4 note#4 Supplementary note_pch04 Ch 4. The Concept of Random Variable Random Variable as a function, Cumulative Distribution Function (CDF) and its properties 3 3/5 5 mp4#5 asx#5 note#5 3 types of random variables (discrete, continuous, mixed types), Histogram and, Probability Density Function (PDF), Properties of the Probability Density Function, Some common distributions, 3/7 6 mp4#6 asx#6 note#6 quiz#04 Some common distributions, Properties of a CDF, 4 3/12 7 mp4#7 asx#7 note#7 Conditional Distribution and Density Functions, Total Probability Theorem, Bayes's theorem, Gaussian approximation to a binomial r.v., Poisson approximation to a binomial r.v. 3/14 8 mp4#8 asx#8 note#8 quiz#5 Ch 5. Functions of One Random Variable Function of a r.v., Direct method, Jacobi method, 5 3/19 9 mp4#9 asx#9 note#9 Inverse problem, Expected value, Alternate form of expectation, 3/21 10 mp4#10 asx#10 note#10 quiz#6 Fundamental theorem of Expectation, Variance, Conditional mean, Total expectation theorem; Moments 3/221st midterm exam exam01 exam01sol

exam01 Alternate form of moments of non-negative random variable 6 3/26 11 mp4#11 asx#11 note#11 Inequalities: Markov, Bienayme, Chebyshev, Chernoff, Jensen; Complex-valued random variable, Characteristic function, Moment generating function, Central Limit Theorem: Gaussian approximation to Binomial 3/28 12 mp4#12 asx#12 note#12 lecture#14 14.asx homework#02 Ch 6. Two Random Variables 6-1 Two Random Variables and Their Joint CDF, Joint Probability Density Function, Bivariate Gaussianity,

Circular symmetry

Circularly symmetrical and Independent => Gaussian

Line masses

7 4/2 13 mp4#13 asx#13 note#13 quiz#7 lecture#15 15.asx homework#03 6-2 A deterministic function of two random variables:

direct method,

linear (dep, indep),

nonlinear (max statistic, min statistic, etc.);

6-3 Two functions of two random variables

direct method: Example 6-21

Jacobi's method: derivation, cartesian coordinate -> polar coordinate

linear TF

auxiliary variable method,

4/4 14 mp4#14 asx#14 note#14 lecture#16 16.asx homework#04 6-4 Joint Moments:

Expected Value and Correlation Coefficient (Correlation Coefficient, Covariance of Two R.V.'s)

covariance

correlation coefficient and equality condition,

uncorrelated, orthogonal,

variance of sum of two random variables,

6-5 Joint CF: independence and convolution;

Normal random vector,8 4/9 15 mp4#15 asx#15 note#15 quiz#8 6-6 Conditional Distribution and Density Functions

conditional density, total density, Bayes' theorem, independence

6-7 Conditional expectation:

conditional mean, conditional variance

joint Gaussian case

Definition of E{X|Y}

total expectation theorem

4/11 16 mp4#16 asx#16 note#16 Ch 7. Sequences of Random Variables 7-1 Random Vector

joint pdf, cdf, CF, MGF

Transform: direct method, Jacobi's method

Independence, order statistics, Group Independence, conditional independence

Expectation: mean, covariance, correlation

complex random vector: complementary covariance, proper, improper

7-2 Conditional Density, CF, Multivariate Gaussian R.V.'s(N-variate Gaussian R.V.'s),

Chain rule,

total conditional pdf, conditional expectation, total expectation,

Gaussian random vector CF and pdf

Goodman's theorem for a proper-complex Gaussian random vector

exam02 9 4/16 17 mp4#17 asx#17 note#17 lecture#19 19.asx note#19

7-3 Mean Square Estimation

MMSE estimator = Conditional mean estimator

AMMSE estimator and conditional pdf of normal vector

LMMSE estimator and orthogonality principle

Z vector and orthonormal data transformation (whitening)

square-root of a covariance matrix

Cholesky factorization of a covariance matrix

4/18 18 mp4#18 asx#18 note#18

linear independence

positive semi-definiteness of Rxx, Cxx

Equality of two random variables

linear independence and positive definiteness of Rxx

7-4 Stochastic convergence and Limit Theorems

a random sequence

Convergence of a sequence of real numbers, a sequence real functions

A random sequence converges to what?

Convergence of a sequence of real functions

Modes of convergence of a sequence of real random variables, SLLN, WLLN, CLT

Almost Sure Convergence, Convergence in Probability, Convergence in Quadratic Mean, Convergence in Distribution

4/192nd midterm exam exam02 exam02sol

10 4/23 19 mp4#19 asx#19 note#19

a.s. =>p

in the pth mean => p

a.s. does not imply p

p does not imply a.s.

p => d

Convergence in distribution revisited

characteristic function

bounded continuous function

4/25 20 mp4#20 asx#20 note#20 Matlab homework#03

Law of large numbers: strong, weak

approximation to Binomial: Gaussian (CLT), Poisson

7-5 Random numbers: meaning and generation

Law of Large Numbers and Monte-Carlo integration

Generation of a pseudo-random number

Generation of random variable and vector

9-1. Definition

Part II Stochastic Processes

11 4/30 21 mp4#21 asx#21 note#21 lecture#23_1 lecture#23_2 lecture#23_3 Ch. 9. General Concepts 9-1. Definition

DT and CT random processes

discrete and continuous random processes

Alternate definition of a random process

Full specification of a random process: Kolomogorov's extension theorem

Poisson point process

Poisson arrival process

n-th order distribution

second-order moments: real X, complex Z, real X and Y, complex Z and W

alpha-dependent random process

white noise

Equality and independence of two random processes

Ch 6. Random Processes - Temporal Characteristics (Peyton Z. Peebles, Jr)

The Random Process Concept, Stationarity and Independence, Correlation Functions, Measurement of Correlation Functions, Gaussian Random Processes, Complex Random Processes

5/2 22 mp4#22 asx#22 note#22 lecture#24

9-1 (cont.)

Normal random process and existence theorem

stationary random processes

strict-sense stationarity, joint stationarity

n-th order stationarity

wide-sense stationarity, centering, covariance stationarity, joint wide-sense stationarity

second-order moments

~~asymptotic stationarity~~

~~stationary in an interval~~

~~process with stationary increment~~5/2

Make-Up

(8pm @LG104)23mp4#23 asx#23 note#23 lecture#26_1 lecture#26_2

periodic with period T

surely, almost surely

in the mean-square sense

periodic mean

doubly periodic autocorrelation function

cyclostationarity

periodic mean, periodic autocorrelation

9-2. System with Stochastic Inputs

deterministic vs. random mapping from a r.p. to a r.p.

12 5/9 24 mp4#24 asx#24 note#24 quiz#9

Memoryless system

full characterization

A strict-sense stationary input leads to a strict-sense stationary output.

square law device, hardlimter, limiter

LTI system

Gaussian input leads to Gaussian output.

Fundamental theorem: mean, autocorrelation of output

WSS input leads to WSS output.

13 5/14 25 mp4#25 asx#25 note#25

LCCDE

Vector-valued process and Multi-terminal (MIMO) Systems

LTI system

Second-order moments

9-3. Power Spectrum

5/16 26 mp4#26 asx#26 note#26

Deterministic signals

Energy signal and Power signal

Energy spectral density (ESD) and necessary conditions to be satisfied

Necessary conditions to be satisfied by Power spectral density

Random processes

Instantaneous power, time-averaged power, ensemble-averaged power, average power, average cross power

Definitions of PSD and its properties

Definitions of xPSD and its properties

Wiener-Khinchine Theorem

Theorems for PSD and xPSD and special cases: WSS, jointly WSS

14 5/21 27 mp4#27 asx#27 note#27

Proof

Properties of PSD, xPSD revisited

LTI filtering of X(t) to have Y(t), S_XX(f), S_{XY}(f), S_{YY}(f)

Wireless Channel and Doppler Power spectrum

5/23 28 mp4#28 asx#28 note#28

special case of WSSUS channel

scattering function and Doppler power spectrum

Line spectra

Vector processes

MS continuity, MS periodicity, and continuity and periodicity of autocorrelation function

15 5/28 29 mp4#29 asx#29 note#29 9-4. DT Processes

9-5. Stochastic continuity, differentiation, integration

Ch. 10. Random Walks and Other Applications 10-1 Random Walk, The Wiener process

10-3. Modulation

10-4. Cyclostationary Processes

5/30 30 mp4#30 asx#30 note#30 10-5. Bandlimited Processes and Sampling Theory

10-6. Deterministic signal in noise

10-7. Bispectra and System Identification

Ch. 11. Spectral Representation 11-1. Factorization and Innovations

11-2 Finite-Order Systems and State Variables

11-3 Fourier Series and KL expansion

11-4 Spectral representation of random processes

Ch. 12 Spectrum Estimation 12-1 Ergodicity

12-2 Spectrum Estimation

Ch. 15. Markov Chains 15-1. Introduction

15-2. Higher Transition Prob. and the Chapman-Kolmogorov Equation

15-3. Classification of States

15-4. Stationary Distributions and Limiting Probabilities

15-5. Transient States and Absorption Prob

15-6. Branching Processes

Ch. 16. Markov Processes and Queueing Theory 16-1 Introduction

16-2. Markov Processes

16-3. Queueing Theory

16-4 Network and Queues

16 6/7Final exam (covers upto Ch. 9)

exam03 exam03sol

exam03

28 mp4#1 asx#1 note#1 lecture#30_1 (73m)

lecture#30_2 (140m)lecture#27 Ch 6. Random Processes - Temporal Characteristics (Peyton Z. Peebles, Jr)

The Random Process Concept, Stationarity and Independence, Ergodicity 29 lecture#31_1

lecture#31_2

lecture#31_3lecture#29 Ch 6. Random Processes - Temporal Characteristics Ch 7. Random Processes - Spectral Characteristics

(Peyton Z. Peebles, Jr)

Correlation Functions, Measurement of Correlation Functions, Gaussian Random Processes, Poisson Random Processes, Power Density Spectrum 30

mp4#1 asx#1 note#1 lecture#32_1

lecture#32_2

lecture#32_3

lecture#32_4Ch 7. Random Processes - Spectral Characteristics

(Peyton Z. Peebles, Jr)

PSD & Its Properties, Relationship b/w PSD & Autocorrelation Function, xPSD & Its Properties, Relationship b/w xPSD & Cross-Correlation Function, Noise Definitions