SIGNALS,

SYSTEMS,

and INFERENCE

Class Notes for

6.011: Introduction to

Communication, Control and

Signal Processing

Spring 2010

Alan V. Oppenheim and George C. Verghese

c Alan V. Oppenheim and George C. Verghese 2010

c
Alan
V. Oppenheim and George C. Verghese, 2010

Contents
1 Introduction
2 Signals and Systems
2.1

2.2

2.3

2.4

21

Signals, Systems, Models, Properties . . . . . . . . . . . . . . . . . .

21

2.1.1 System/Model Properties . . . . . . . . . . . . . . . . . . . .

22

Linear, Time-Invariant Systems . . . . . . . . . . . . . . . . . . . . .

24

2.2.1 Impulse-Response Representation of LTI Systems . . . . . . .

24

2.2.2 Eigenfunction and Transform Representation of LTI Systems

26

2.2.3 Fourier Transforms . . . . . . . . . . . . . . . . . . . . . . . .

29

Deterministic Signals and their Fourier Transforms . . . . . . . . . .

30

2.3.1 Signal Classes and their Fourier Transforms . . . . . . . . . .

30

2.3.2 Parsevals Identity, Energy Spectral Density, Deterministic

Autocorrelation . . . . . . . . . . . . . . . . . . . . . . . . . .

32

The Bilateral Laplace and Z-Transforms . . . . . . . . . . . . . . . .

35

The Bilateral Z-Transform . . . . . . . . . . . . . . . . . . .

35

2.4.2 The Inverse Z-Transform . . . . . . . . . . . . . . . . . . . .

38

2.4.3 The Bilateral Laplace Transform . . . . . . . . . . . . . . . .

39

Discrete-Time Processing of Continuous-Time Signals . . . . . . . .

40

2.5.1 Basic Structure for DT Processing of CT Signals . . . . . . .

40

2.5.2 DT Filtering, and Overall CT Response . . . . . . . . . . . .

42

2.5.3 Non-Ideal D/C converters . . . . . . . . . . . . . . . . . . . .

45

2.4.1

2.5

3 Transform Representation of Signals and LTI Systems

47

3.1

Fourier Transform Magnitude and Phase . . . . . . . . . . . . . . . .

47

3.2

Group Delay and The Eect of Nonlinear Phase . . . . . . . . . . .

50

3.3

All-Pass and Minimum-Phase Systems . . . . . . . . . . . . . . . . .

57

3.3.1 All-Pass Systems . . . . . . . . . . . . . . . . . . . . . . . . .

58

3.3.2 Minimum-Phase Systems . . . . . . . . . . . . . . . . . . . .

60

Spectral Factorization . . . . . . . . . . . . . . . . . . . . . . . . . .

63

3.4

c
Alan
V. Oppenheim and George C. Verghese, 2010

4 State-Space Models

65

4.1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

65

4.2

Input-output and internal descriptions . . . . . . . . . . . . . . . . .

66

4.2.1 An RLC circuit . . . . . . . . . . . . . . . . . . . . . . . . . .

66

4.2.2 A delay-adder-gain system . . . . . . . . . . . . . . . . . . . .

68

State-Space Models . . . . . . . . . . . . . . . . . . . . . . . . . . . .

70

4.3.1 DT State-Space Models . . . . . . . . . . . . . . . . . . . . .

70

4.3.2 CT State-Space Models . . . . . . . . . . . . . . . . . . . . .

71

4.3.3 Characteristics of State-Space Models . . . . . . . . . . . . .

72

4.4 Equilibria and Linearization of

Nonlinear State-Space Models . . . . . . . . . . . . . . . . . . . . . .

73

4.4.1 Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . .

74

4.4.2 Linearization . . . . . . . . . . . . . . . . . . . . . . . . . . .

75

State-Space Models from InputOutput Models . . . . . . . . . . . .

80

4.5.1 Determining a state-space model from an impulse response

or transfer function . . . . . . . . . . . . . . . . . . . . . . . .

80

4.5.2 Determining a state-space model from an inputoutput dif

ference equation . . . . . . . . . . . . . . . . . . . . . . . . .

83

4.3

4.5

5 Properties of LTI State-Space Models

85

5.1

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

85

5.2

The Zero-Input Response and Modal Representation . . . . . . . . .

85

5.2.1 Modal representation of the ZIR . . . . . . . . . . . . . . . .

87

5.2.2 Asymptotic stability . . . . . . . . . . . . . . . . . . . . . . .

89

Coordinate Transformations . . . . . . . . . . . . . . . . . . . . . . .

89

5.3.1 Transformation to Modal Coordinates . . . . . . . . . . . . .

90

5.4 The Complete Response . . . . . . . . . . . . . . . . . . . . . . . . .

91

5.5 Transfer Function, Hidden Modes,

Reachability, Observability . . . . . . . . . . . . . . . . . . . . . . .

92

5.3

6 State Observers and State Feedback

101

6.1

Plant and Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

6.2

State Estimation by Real-Time Simulation . . . . . . . . . . . . . . . 102

6.3

The State Observer . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

c
Alan
V. Oppenheim and George C. Verghese, 2010

6.4

State Feedback Control . . . . . . . . . . . . . . . . . . . . . . . . . 108

6.4.1

6.5

Proof of Eigenvalue Placement Results . . . . . . . . . . . . . 116

Observer-Based Feedback Control . . . . . . . . . . . . . . . . . . . . 117

7 Probabilistic Models
7.1

121

The Basic Probability Model . . . . . . . . . . . . . . . . . . . . . . 121

7.2 Conditional Probability, Bayes Rule, and Independence . . . . . . . 122

7.3 Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

7.4 Cumulative Distribution, Probability Density, and Probability Mass

Function For Random Variables . . . . . . . . . . . . . . . . . . . . . 125

7.5 Jointly Distributed Random Variables . . . . . . . . . . . . . . . . . 127

7.6 Expectations, Moments and Variance . . . . . . . . . . . . . . . . . . 129

7.7 Correlation and Covariance for Bivariate Random Variables . . . . . 132

7.8 A Vector-Space Picture for Correlation Properties of Random Variables137

139

8.1

Estimation of a Continuous Random Variable . . . . . . . . . . . . . 140

8.2

From Estimates to an Estimator . . . . . . . . . . . . . . . . . . . . 145

8.2.1

8.3

Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

Linear Minimum Mean Square Error Estimation . . . . . . . . . . . 150

9 Random Processes

161

9.1

Denition and examples of a random process . . . . . . . . . . . . . 161

9.2

Strict-Sense Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . 166

9.3

Wide-Sense Stationarity . . . . . . . . . . . . . . . . . . . . . . . . . 167

9.3.1

Some Properties of WSS Correlation and Covariance Functions168

9.4

Summary of Denitions and Notation . . . . . . . . . . . . . . . . . 169

9.5

Further Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

9.6

Ergodicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

9.7

Linear Estimation of Random Processes . . . . . . . . . . . . . . . . 173

9.8

9.7.1

Linear Prediction . . . . . . . . . . . . . . . . . . . . . . . . . 174

9.7.2

Linear FIR Filtering . . . . . . . . . . . . . . . . . . . . . . . 175

The Eect of LTI Systems on WSS Processes . . . . . . . . . . . . . 176

c
Alan
V. Oppenheim and George C. Verghese, 2010

10 Power Spectral Density

183

10.1 Expected Instantaneous Power and Power Spectral Density . . . . . 183

10.2 Einstein-Wiener-Khinchin Theorem on Expected Time-Averaged Power185

10.2.2 Invoking Ergodicity . . . . . . . . . . . . . . . . . . . . . . . 187

10.2.3 Modeling Filters and Whitening Filters . . . . . . . . . . . . 188

10.3 Sampling of Bandlimited Random Processes . . . . . . . . . . . . . . 190

11 Wiener Filtering
11.1 Noncausal DT Wiener Filter

195

. . . . . . . . . . . . . . . . . . . . . . 196

11.2 Noncausal CT Wiener Filter . . . . . . . . . . . . . . . . . . . . . . . 203

11.2.1 Orthogonality Property . . . . . . . . . . . . . . . . . . . . . 205

11.3 Causal Wiener Filtering . . . . . . . . . . . . . . . . . . . . . . . . . 205

11.3.1 Dealing with Nonzero Means . . . . . . . . . . . . . . . . . . 209

12 Pulse Amplitude Modulation (PAM), Quadrature Amplitude Mod

ulation (QAM)
211

12.1 Pulse Amplitude Modulation . . . . . . . . . . . . . . . . . . . . . . 211

12.1.1 The Transmitted Signal . . . . . . . . . . . . . . . . . . . . . 211

12.1.2 The Received Signal . . . . . . . . . . . . . . . . . . . . . . . 213

12.1.3 Frequency-Domain Characterizations . . . . . . . . . . . . . . 213

12.1.4 Inter-Symbol Interference at the Receiver . . . . . . . . . . . 215

12.2 Nyquist Pulses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217

12.3 Carrier Transmission . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

12.3.1 FSK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

12.3.2 PSK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220

12.3.3 QAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

13 Hypothesis Testing

227

13.1 Binary Pulse Amplitude Modulation in Noise . . . . . . . . . . . . . 227

13.2 Binary Hypothesis Testing . . . . . . . . . . . . . . . . . . . . . . . . 229

13.2.1 Deciding with Minimum Probability of Error: The MAP Rule 230

13.2.2 Understanding Pe : False Alarm, Miss and Detection . . . . . 231

c
Alan
V. Oppenheim and George C. Verghese, 2010

13.2.3 The Likelihood Ratio Test . . . . . . . . . . . . . . . . . . . . 233

13.2.4 Other Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . 233

13.2.5 Neyman-Pearson Detection and Receiver Operating Charac

teristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

13.3 Minimum Risk Decisions . . . . . . . . . . . . . . . . . . . . . . . . . 238

13.4 Hypothesis Testing in Coded Digital Communication . . . . . . . . . 240

13.4.1 Optimal a priori Decision . . . . . . . . . . . . . . . . . . . . 241

13.4.2 The Transmission Model . . . . . . . . . . . . . . . . . . . . . 242

13.4.3 Optimal a posteriori Decision . . . . . . . . . . . . . . . . . . 243

14 Signal Detection

247

14.1 Signal Detection as Hypothesis Testing . . . . . . . . . . . . . . . . . 247

14.2 Optimal Detection in White Gaussian Noise . . . . . . . . . . . . . . 247

14.2.1 Matched Filtering . . . . . . . . . . . . . . . . . . . . . . . . 250

14.2.2 Signal Classication . . . . . . . . . . . . . . . . . . . . . . . 251

14.3 A General Detector Structure . . . . . . . . . . . . . . . . . . . . . . 251

14.3.1 Pulse Detection in White Noise . . . . . . . . . . . . . . . . . 252

14.3.2 Maximizing SNR . . . . . . . . . . . . . . . . . . . . . . . . . 255

14.3.3 Continuous-Time Matched Filters . . . . . . . . . . . . . . . 256

14.3.4 Pulse Detection in Colored Noise . . . . . . . . . . . . . . . . 259

c
Alan
V. Oppenheim and George C. Verghese, 2010

c
Alan
V. Oppenheim and George C. Verghese, 2010

MIT OpenCourseWare
http://ocw.mit.edu

6.011 Introduction to Communication, Control, and Signal Processing

For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.