Announcements             Syllabus             Schedule/Downloads


Schedule/Downloads (Spring 2013)

 http://www.postech.ac.kr/lab/ee/cisl/orange-ball.gif The schedule is tentative and subject to change. (Last update: 12/21/16)

Week

2015

Date

Lecture

Topics

Key Words

Reading Assignments*

Handouts

 (강의노트)

/Homeworks

Exam 2016 Fall Exam 2015 Spring

한국어 강의

 asx (mp4)

/판서

/HW, Exam 2013

Quiz/ Exam 2012

Video/Board Writing/HW 2010

Quiz/ Exam 2006

Homework/ Exam   2004

1

3/5

1

Introduction

zero-error data compression, source coding theorem, channel coding theorem, rate-distortion theory

 

Handout 1

   

한국어 강의 1-1, 1-2

(1-1, 1-2) 판서 1

 

Video 1, 2, 3 Note 1, 2, 3, 4

Homework 1 (Solutions)

3/6

2 Make-up

21:00-22:15

@LG106

Zero-Error Data Compression

Problem Formulation

Encoder, Decoder, Source Code, Expected Length, Uniquely Decodable (UD) Code

Y 2.5, PDC 2.1, 2.2

Handout 2

   

한국어 강의 2-1

(2-1)

판서 1

 

Video Note 1, 2, 3, 4, 5, 6, 7, 8, 9

Homework 2 (Solutions)

3/7

3

한국어 강의 3-1

(3-1)

판서 1

2

3/12

4

Inequalities for UD Codes

Kraft's Inequality, Fundamental Inequality, Entropy Bound, Some Properties of Entropy

Y 3.1, PDC 2.3, 2.4

Handout 3

   

한국어 강의 4-1

(4-1)

판서 1

Quiz#1

Video Note 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13

Quiz#1 Quiz#1 solutions

 

3/14

5

Prefix-Free (PF) Codes

Existence Theorem for PF Codes

Y 3.2, PDC 2.4

Handout 4

   

한국어 강의 5-1

(5-1)

판서 1, 2, 3, 4

 

 

Video Note 1, 2, 3, 4, 5, 6, 7, 8

Homework 3 (Solutions)

Quiz #1

3/15

Quiz #1 17:00-18:00 @LG106

Quiz#1

Quiz#1sol

3

3/19

6

Huffman Codes

Huffman Procedure, Huffman Code, Optimality of Huffman Codes, Asymptotic Optimality of Huffman Codes

C 5.6, 5.8, PDC 2.5

Handout 5

   

한국어 강의 6-1

(6-1)

 

판서 1

Quiz#2

Video Note 1, 2, 3, 4, 5, 6, 7, 8, 9

Quiz#2 Quiz#2 solutions

Homework 4 (Solutions)

 

Video Note 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

Source Coding

Problem Formulation, Pre-requisites

Formulation of block source-coding, Chebyshev Inequality, Convergence of a sequence

S 7.4, 7.5, PDC 2.6

Handout 6

   

Quiz#3

Video Note 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13

Homework 5 (Solutions)

3/20

7 Make-up

21:00-22:15

@LG106

Source Coding Theorem

Convergence in Probability, Weak Law of Large Numbers, Weak Asymptotic Equipartition Property  (AEP), Source Coding Theorem

C 3, Y 4.1-4.4, Y 2.8, 4.3, PDC 2.7

한국어 강의 7-1

(7-1)

판서 1

 

Video  Note 1, 2, 3, 4, 5, 6, 7, 8, 9

Exam#1

Exam#1 solutions

 

3/21

8

한국어 강의 8-1

(8-1)

판서 1

 

Quiz #2

4

3/26

No Class

3/28

9

한국어 강의 9-1, 9-2

(9-1, 9-2)

판서 1

Shannon's Information Measures

Entropy, Mutual Information, Relative Entropy

Entropy, Joint Entropy, Conditional Entropy, Mutual Information

C 2.1-2.6, 2.8,

 

Handout 7

   

 

Video Note 1, 2, 3, 4, 5, 6, 7,

Homework 6 (Solutions)

 

12exam01.pdf

12exam01sol.pdf

10exam01.pdf

10exam01sol.pdf

Quiz#3 Quiz#3 solutions

Midterm Exam 1 (Solutions)

3/29

Quiz #2 17:00-18:00 @LG106

5

4/2

10

한국어 강의 10-1, 10-2

(10-1, 10-2)

판서 1

 

Video Note  1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13

4/3

11 Make-up
21:00-22:15
@LG106

Chain Rules, Conditional Mutual Information, Chain Rules, Kullback Leibler Distance, Information Divergence, Fano's Inequality, Markov Chain, Data Processing Theorem

Handout 8

Exam 01

sol

 

한국어 강의 11-1 (11-1)

판서 1

Quiz#4

Video Note 1, 2, 3, 4, 5, 6, 7, 8, 9

4/4

12

 

한국어 강의 12-1, 12-2, 12-3

(12-1, 12-2, 12-3)

판서 1

 

Video Note 1, 2,3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13

4/4

1st Midterm 19:30-24:00 @LG104 (Covers Lec. 1 - Lec. 9)

Exam 01

Exam 01

6

4/9

No Class

Channel Coding for DMC

Definitions and Problem Formulation

Channel Coding Problem,  Discrete Memoryless Channel (DMC), Code Rate, Error Rate

C 8.5

Handout 9

   

한국어 강의 13-1

(13-1)

 

판서 1

Quiz#5

Video Note 1, 2, 3, 4, 5, 6, 7

Homework 7 (Solutions)

4/10

13 Make-up
21:00-22:15
@LG106

4/11

14

Sketch of Achievability of max I(X;Y)

(M,n) code, Channel Encoder, Channel Decoder, Sketch of achievability of max I(X;Y)

Handout 9b

   

한국어 강의 14-1

(14-1)

 

판서 1

 

Video Note 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12

 

7

4/16

15

Joint AEP

Random Coding, Joint AEP, Maximal Probability of Error, Markov inequalities

C 8.1-8.4, 8.6

 

Handout 10a

   

한국어 강의 15-1

(15-1)

 

판서 1

 

Video Note 1, 2, 3, 4, 5, 6, 7, 8, 9

Quiz#4 Quiz#4 solutions

Homework 8 (Solutions)

4/18

16

한국어 강의 16-1

(16-1)

 

판서 1 

Quiz#6

Video Note 1, 2, 3, 4, 5

4/19

Quiz #3 17:00-18:00 @LG106

             

8

4/23

17

Noisy Channel Coding Theorem and Its Weak Converse Typical Set Decoding, Lower Bound on Maximal Probability, Approximate Necessary Conditions to Achieve Channel Capacity, C 8.7, 8.9, 8.10

Handout 11a

   

한국어 강의 17-1

(17-1)

 

판서 1

4/24

18 Make-up
21:00-22:15
@LG106

Fano's Inequality and Converse of Channel Coding Theorm

 

Handout 11b

   

한국어 강의 18-1

(18-1)

 

판서 1  

Quiz#7

Video Note 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11

Quiz #3

4/25

19

Mutual Information of a DMC, Convex Set, Convex Functions, Channel Capacity of a DMC

 

Handout 11c

   

한국어 강의 19-1

(19-1)

 

판서 1

 

Video Note 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12

9

4/30

20

 

한국어 강의 20-1

(20-1)

 

판서 1

 

Quiz #4

Video  Note 1, 2, 3, 4, 5, 6

5/1

21 Make-up

21:00-22:15
@LG106

Feedback Capacity, Joint Source Channel Coding Theorem

Discrete Memoryless Channel with Feedback, Feedback Capacity, Joint Source/Channel Coding

C 8.12, 8.13

Handout 11d

   

한국어 강의 21-1

(21-1)

 

판서 1 

Quiz#8

Video Note 1, 2, 3, 4, 5

 

5/2 No Class

 

5/3

Quiz #4 17:00-18:00 @LG106

10

5/6

22 Make-up

21:00-22:15
@LG106

Channel Coding for Gaussian Noise Channel

Shannon's Information Measures for Continuous Random Variables

Differential Entropy, Mutual Information, Relative Entropy, AEP

C 9.1-9.6

Handout 12

   

한국어 강의 22-1

(22-1)

 

판서 1

 

Video Note 1, 2, 3, 4, 5, 6, 7, 8

Homework 9 (Solutions)

5/7

No Class  

5/8

23 Make-up

21:00-22:15
@LG106

한국어 강의 23-1

(23-1)

 

판서 1

Quiz #5

 

5/9

24

Capacity of Gaussian CMC

Gaussian CMC, Capacity of Gaussian CMC

C 10.1, 10.2

Handout 13

Exam 02

sol

 

한국어 강의 24-1

(24-1)

 

판서 1 

 

Quiz#9

Video Note 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

   

5/10

2nd Midterm 19:30-24:00 @LG106 (Covers Lec. 1 - Lec. 21)

Exam 02

Exam 02

Exam#2

Exam#2 solutions

Homework 10 (Solutions)

11

5/14

No Class

NCCT for Gaussian CMC and Its Converse 

 

 

 

Video Note 1, 2, 3, 4, 5, 6, 7, 8, 9

5/15

25 Make-up

21:00-22:15

@LG106

 
 

한국어 강의 25-1

(25-1)

 

판서 1 

12exam02.pdf

12exam02sol.pdf

10exam02.pdf

10exam02sol.pdf

5/16

26

Sphere Packing Argument, Parallel Gaussian Channels and Water-Filling

C 10.3-10.6

Handout 14c

   

한국어 강의 26-1

(26-1)

 

판서 1  

Quiz#10

Video Note 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11

Quiz#5 Quiz#5 solutions

Midterm Exam 2 (Solutions)

5/17

27 Make-up

09:00-10:15

@LG106

Continuous-Time Band-Limited White/Colored Gaussian Noise Channel, Gaussian Channels with Feedback

한국어 강의 27-1

(27-1)

 

판서 1

 

Video  Note  1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11

 

Quiz #6

12

5/21

28

Rate Distortion Theory

Problem Formulation

Vector Quantization, Operational Rate-Distortion Function

C 13.1, 13.2

PDC 3, PDCut Lecs.#6-7

Handout 15

   

한국어 강의 28-1

(28-1)

 

판서 1 

Quiz#11

Video Note 1, 2, 3, 4, 5, 6, 7, 8, 9

Homework 11 (Solutions)

5/22

29

Make-up

21:00-22:15

@LG106

Rate Distortion Theorem

Informational Rate-Distortion Function, Rate-Distortion Theorem, Sphere Covering Argument as Sketch of Direct Part of Rate Distortion Theorem

C 13.2-13.5

Handout 16c

   

한국어 강의 29-1

(29-1)

 

판서 1

 

Video Note 1, 2, 3, 4, 5, 6, 7, 8

 

5/23

No Class

5/24

Quiz #5 17:00-18:00 @LG106

13

5/28

30

Converse of Rate-Distortion Theorem, Distortion Typical Sequences and Set, Distortion Typical Set Encoding, Indicator Function, Direct Part of Rate-Distortion Theorem

한국어 강의 30-1

(30-1)

 

판서 1 

 

Video  Note 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

 

5/29

31 Make-up

21:00-22:15

@LG106

한국어 강의 31-1

(31-1)

 

판서 1

5/30

32

AWGN channel and Shannon Bound on BER

AWGN Channel Revisited

Gaussian Codebook, DT Complex AWGN channel, Outage, CT bandpass AWGN channel

 

Handout 16e

   

한국어 강의 32-1

(32-1)

 

판서 1 

Quiz#12

Video Note 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

 

14

6/4

33

Shannon Bound on BER

Capacity as a function of Eb/N0, Shannon Limit, Bandwidth Efficiency Plane, Joint Source-Channel Coding with Distortion, Shannon bound on Pb vs. Eb/N0

 

Handout 16f

   

한국어 강의 33-1

(33-1)

 

판서 1

2016 오류정정부호 단기강좌 중 해당 부분 한국어 강의

pdf

video

 

Video Note 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11

Homework 12

6/4

34

Make-up

21:00-22:15

@LG106

 

 

한국어 강의 34-1

(34-1)

 

판서 1

 

Video Note 1, 2, 3, 4, 5, 6, 7, 8, 9, 10

   

 

Miscellaneous

Entropy Rate, Universal Source Coding, Error Exponent, Occam's Razor, etc.

 

Handout 17c

           

 

6/25

Final Exam 19:30-24:00 @LG106

 

 

 

Exam 03 Exam 03

Note_pdf 1

 

One note  1

Final Exam

(Solutions)

Final Exam (Solutions)

Final Exam (Solutions)

Final Exam (Solutions)

Exam 03

* References

C2: T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed.,  John Wiley and Sons, Inc., 2006

C: T. M. Cover and J. A. Thomas, Elements of Information Theory. John Wiley and Sons, Inc., 1991.

G: R. G. Gallager, Information Theory and Reliable Communication. John Wiley and Sons, Inc., 1968

N: A. El Gamal and Y.-H. Kim, Network Information Theory. Cambridge Univ. Press, 2011.

PDC: R. G. Gallager, Principles of Digital Communication. Cambridge Univ. Press, 2008.

S: H. Stark and J. W. Woods, Probability, Random Processes, and Estimation Theory for Engineers, 2nd Ed., Prentice-Hall, Inc., 1994.

Y: R. W. Yeung, A First Course in Information Theory. Kluwer Academic/Plenum Publishers, 2002.

Recommended Papers

·         Fifty years of Shannon theory

Verdu, S.;
 

·         Quantization

Gray, R.M.; Neuhoff, D.L.;
 

·         The art of signaling: fifty years of coding theory

Calderbank, A.R.;
 

·         Claude E. Shannon: a retrospective on his life, work, and impact

Gallager, R.G.;