INFORMATION THEORY

ECE 729

Course Description

Definition of measures of information and their properties, capacity of discrete and continuous channels with noise, source and channel coding theorems, fundamentals of channel coding, noiseless source coding, and source coding with a fidelity criterion. Knowledge of basic probability analysis [such asE C E 331,STAT/MATH 431, orSTAT 311] required.

Prerequisties

Graduate/professional standing

Satisfies

This course does not satisfy any prerequisites.

Credits

Not Reported

Offered

Not Reported

Grade Point Average

3.42

2.96% from Historical

Completion Rate

100%

1.99% from Historical

A Rate

61.54%

55.36% from Historical

Class Size

-15.58% from Historical

Cumulative Grade Distribution

Instructors (2025 Fall)

Sorted by ratings from Rate My Professors

Similar Courses

COMMUNICATION SYSTEMS II

ECE 437

Statistical analysis of information transmission systems. Probability of error, design of receivers for digital transmission through additive white Gaussian noise channels and bandlimited channels. Spread spectrum communication systems. Channel capacity, source and error control coding.

COMMUNICATION SYSTEMS I

ECE 436

Amplitude, frequency, pulse, and pulse-code modulation. Narrow-band noise representation and signal-to-noise ratios for various modulation schemes. Pulse shaping, timing recovery, carrier synchronization, and equalization. Sampling, quantization and coding.

PROBABILITY AND INFORMATION THEORY IN MACHINE LEARNING

COMPSCI/ECE 561

Probabilistic tools for machine learning and analysis of real-world datasets. Introductory topics include classification, regression, probability theory, decision theory and quantifying information with entropy, relative entropy and mutual information. Additional topics include naive Bayes, probabilistic graphical models, discriminant analysis, logistic regression, expectation maximization, source coding and variational inference.

PROBABILITY AND RANDOM PROCESSES

ECE 730

Review of basic probability. Advanced probability concepts. Random vectors; linear filtering of random processes; stationarity; power spectral densities; estimation; convergence; Markov chains; Poisson process; Wiener process. Knowledge of basic probability analysis [such asE C E 331,STAT/MATH 431, orSTAT 311] strongly encouraged.

SIGNALS, INFORMATION, AND COMPUTATION

ECE 203

Introduction to the signals, information, and computational techniques in electrical engineering.