Course: EE 306 - Signals and Systems II

Weekly outline

General
- Announcements Forum
- YouTube Playlist:
  
  Playlist Link: EE 306 - Spring 2021-2022
7 March - 13 March
- EE 306 - Signals and Systems II - Lecture 1: Review of Probability Fundamentals File
  Lecture Content:
  
  0:00 Intro
  8:14 Random Experiment
  10:53 Sample Space
  13:58 Events
  17:06 Fundamentals of Set Theory
  20:36 Properties of Set Operations
  30:13 Probability Axioms
- EE 306 - Signals and Systems II - Lecture 2: Conditional Probability, Bayes Rule and Independence File
  Lecture Content:
  
  0:00 Conditional Probability
  10:30 Total Probability Theorem
  18:20 Bayes' Rule
  26:08 Independence of Events
  29:12 Counting Methods
- EE 306 - Signals and Systems II - Lecture 3: Review of Random Variables File
  Lecture Content:
  
  0:00 Recap of Previous Lecture
  1:13 Random Variables
  4:38 Expectation, Variance and Moments
  8:14 Important Distributions (Discrete Random Variables)
  11:39 Important Distributions (Discrete Random Variables) Example 1: Expectation of Geometric Random Variable
  16:04 Important Distributions (Discrete Random Variables) Example 2: Binomial and Poisson Random Variables
  22:11 Important Distributions (Continuous Random Variables)
  36:03 Additional Notes
  41:40 Example: Football Game Tickets
- Lecture 1-2-3 Notes Folder
14 March - 20 March
- EE 306 - Signals and Systems II - Lecture 4: Functions of Random Variables and Transform Methods File
  Lecture Content:
  
  0:00 Recap of Previous Lecture
  1:12 Functions of Random Variables
  7:12 Functions of Random Variables Example 1: Squared Random Variable
  14:46 Functions of Random Variables Example 2: Computer Generation of Random Variables
  26:22 Transform Methods
  26:52 Characteristic Functions (Continuous Random Variables)
  31:17 Probability Generating Functions (Discrete Random Variables)
  37:17 Transform Methods Example 1: Characteristic Function of Gaussian Random Variable
- EE 306 - Signals and Systems II - Lecture 5: Probability Bounds and Recursion File
  Lecture Content:
  
  0:00 Probability Bounds
  2:20 Prove Markov Bound
  4:18 Probability Bounds Example 1: Comparison of Markov Bound to Exact Probability of a Binomial Variable
  12:21 Probability Bounds Example 2: Application of Chebyshev Bound to Bernoulli Trials
  22:44 Recursion
  24:30 Recursion Example 1: Recursion to Derive Probability
  36:58 Recursion Example 2: Recursion to Derive Expectation
- EE 306 - Signals and Systems II - Lecture 6: Pairs of Random Variables File
  Lecture Content:
  
  0:00 Recap of Previous Lecture
  0:41 Pairs of Random Variables
  4:41 Useful CDF Relations
  9:47 Example 1: Pair of Discrete Random Variables
  22:30 Example 2: Pair of Continuous Random Variables
  35:52 Jointly Gaussian Random Variables
  38:42 Conditional PMF/PDF and Expectation
- Lecture 4-5-6 Notes Folder
21 March - 27 March
- EE 306 - Signals and Systems II - Lecture 7: Covariance, Correlation Coefficent, Functions of Pairs of Random Variables File
  Lecture Content:
  
  0:00 Covariance and Correlation Coefficient
  5:48 Example 1: Range of Correlation Coefficient (Proof)
  11:07 Example 2: Expected Value of Function of Two Random Variables
  14:42 Functions of Pairs of Random Variables
  26:40 Example 3: PDF of Sum of Two Independent Random Variables
- EE 306 - Signals and Systems II - Lecture 8: Vectors of Random Variables, Sum of Random Variables File
  Lecture Content:
  
  0:00 Vectors of Random Variables
  9:33 Example 1: Three Jointly Gaussian Random Variables
  16:47 Example 2: Deterministic Matrix and Random Vector Multiplication
  25:27 Sums of Random Variables
  33:59 Example 3: Law of Large Numbers
  37:43 Example 4: Central Limit Theorem
- EE 306 - Signals and Systems II - Lecture 9: Parameter Estimation, Linear MMSE Estimator File
  Lecture Content:
  
  0:00 Recap of Previous Lecture
  1:38 Estimation
  5:15 Parameter Estimation
  11:03 Example 1: Sample Mean Estimator
  21:16 Example 2: Maximum Estimator
  32:06 Minimum MSE Linear Estimator (Random Variable Estimation)
- Lecture 7-8-9 Notes Folder
28 March - 3 April
- EE 306 - Signals and Systems II - Lecture 10: Linear MMSE Estimator for Vectors, Generalized MMSE Estimator File
  Lecture Content:
  
  0:00 Recap of Previous Lecture
  0:48 MMSE Linear Estimator (continued)
  4:31 Gradient and Hessian
  14:11 MMSE Generalized Estimator
  22:06 Example 1: Estimators of Two Joint Random Variables
  32:21 Linear Least Squares (LLS)
  39:38 Example 2: Ordinary Least Squares (OLS)
- EE 306 - Signals and Systems II - Lecture 11: Linear Programming and Duality in Optimization File
  Lecture Content:
  
  0:00 Recap of Previous Lecture
  3:21 Linear Programming
  6:11 Example 1: 2D Optimization on a Pentagonal Region
  14:16 Convex-Concave Functions
  22:37 Example 2: Convexity/Concavity of Quadratic and Logarithmic Functions
  25:07 Duality in Optimization
  36:53 Example 3: Minimization of Vector Squared Norm with Equality Constraint
  41:02 Example 4: Minimization of Linear Function with Equality Constraint
- EE 306 - Signals and Systems II - Lecture 12: Lagrange Dual Problem, Descent Methods File
  Lecture Content:
  
  0:00 Recap of Previous Lecture
  1:32 Notes on Convexity and Duality
  4:42 Lagrange Dual Problem
  11:22 Example 1: Dual Problem of Linear Programming Problem
  15:08 Karush-Kuhn-Tucker (KKT) Conditions
  24:27 Example 2: Water Filling on Communication Channels
  39:25 Descent Methods
  44:18 Gradient Descent Algorithm
- Lecture 10-11-12 Notes Folder
4 April - 10 April
- EE 306 - Signals and Systems II - Lecture 13: Discrete Stochastic Processes and Introduction to Markov Chains File
  Lecture Content:
  
  0:00 Introduction to Module 2
  0:58 Discrete Stochastic Processes Examples
  10:28 Definition of a Discrete Stochastic Process
  15:16 Introduction to Markov Chains
  17:32 Example 1: Two State Markov Chain (Rain/no rain)
  28:17 Example 2: Two State Markov Chain (Passing Calculus I)
  37:06 Transition Probabilities
  42:52 Example 3: Random Walk with Infinite Number of States
  48:58 Example 4: Random Walk with Finite Number of States (Spider and Insect)
- EE 306 - Signals and Systems II - Lecture 14: n-step Transition Probabilities File
  Lecture Content:
  
  0:00 n-step Transition Probabilities
  12:54 Example 1: 4-State Markov Chain with 2 Steady State Outcomes (Spider Web)
  27:11 Example 2: 3-State Markov Chain with 2 Absorbing States (Passing Calculus II)
  30:48 Example 3: Finding a Markov Chain Model (Raining)
  37:57 Example 4: 2 Umbrella Problem
- EE 306 - Signals and Systems II - Lecture 15: 2 Umbrella Problem and Random Walk File
  Lecture Content:
  
  0:00 Example 1: Solution of the “2 Umbrella Problem” from Previous Lecture
  15:47 Example 2: Solution of the “Passing EE202 Problem”
  30:57 Example 3: First passage probability in a Random Walk
  43:48 Example 4: Expected first passage time for a Random Walk in a Finite State Space
- Lecture 13-14-15 Notes Folder
11 April - 17 April
- EE 306 - Signals and Systems II - Lecture 16: Classification of States File
  Lecture Content:
  
  0:00 Classification of States
  0:35 Accessibility
  5:40 Communicating States
  14:08 Recurrent and Transient States
  24:20 The number of visits to a Transient State
  26:50 Further Discussion on Recurrent States
  33:18 Recurrent Classes
- EE 306 - Signals and Systems II - Lecture 17: Several Examples for Classification of States File
  Lecture Content:
  
  0:00 Periodic States
  13:22 Example 1: Periodic Markov Chain with 3x3 Transition Matrix
  17:35 Example 2: Aperiodic Markov Chain with 2x2 Transition Matrix
  26:36 Example 3: MC with 2 absorbing states (“Passing EE202” example from previous lectures)
  30:30 Discussion of examples 2 and 3: ergodic and non-ergodic processes
  36:00 Relation Between Convergence and Periodicity
  42:00 Example 4: Markov Chain with Multiple Recurrent Classes
  51:25 Example 5: Markov Chain with a Single Recurrent Class
- EE 306 - Signals and Systems II - Lecture 18: Steady State Probabilities and Long Term Averages File
  Lecture Content:
  
  0:00 Steady State Probabilities
  2:07 Example 1: Steady State for 3-state Markov Chain (“2 LED”s)
  17:00 Theorem: Global Balance Equations for Stationary Probabilities
  26:08 Long Term Frequency of Occurence
  32:28 Example 2: Long Term Averages in the 2 Umbrella Problem
  40:26 Example 3: Long Term Averages in the Rain/No Rain Problem
- Lecture 16-17-18 Notes Folder
18 April - 24 April
- EE 306 - Signals and Systems II - Lecture 19 – Part 1: Birth-Death Chains, Mean First Passage and Recurrence Times File
  Lecture Content:
  
  0:00 Birth-Death Chains
  3:40 Example 1: Random Walk with Reflecting Barriers
  8:27 Example 2: Geo/Geo/1 Queue
  20:40 Mean First Passage and Recurrence Times
  28:00 Example 3: Expected Number of Rainy Days (Rain/No Rain)
  33:40 Example 4: Mean First Passage and Recurrence Times for MC with 1 Recurrent Class and 2 Transient States
- EE 306 - Signals and Systems II - Lecture 19 – Part 2: Example for Mean First Passage and Recurrence Times File
  Lecture Content:
  
  0:00 Example 4: Mean First Passage and Recurrence Times for MC with 1 Recurrent Class and 2 Transient States (continued)
- EE 306 - Signals and Systems II - Lecture 20: Exponential Random Variable File
  Lecture Content:
  
  0:00 Exponential Random Variable
  4:28 Memorylessness Property of Exponential Distribution
  8:21 Example 1: Waiting Time with Exponential Distribution (Bus Inter-Arrival Times)
  13:39 Racing Exponentials
  18:17 Example 2: Service Times at Bank with Exponential Distributions
- EE 306 - Signals and Systems II - Lecture 21: Counting Processes File
  Lecture Content:
  
  0:00 Counting Processes
  2:02 Sample Path of a Counting Process
  4:40 Equivalent Characterization of Counting Process by Arrival Times
  10:12 Counting Process Properties: Property 1
  14:20 Counting Process Properties: Property 2
  15:30 Counting Process Properties: Property 3
  19:50 Counting Process Properties: Property 4
- Lecture 19-20-21 Notes Folder
25 April - 1 May
- EE 306 - Signals and Systems II - Lecture 22: Poisson Process, Its Properties and Moment Generating Functions File
  Lecture Content:
  
  0:00 Definition 1 of Poisson Process
  2:15 Example 1: Arrival Times of 3 Buses
  4:43 Residual Time
  8:17 Fresh Start and Independent Increments Properties
  11:35 Stationary Increments Property
  14:50 Example 2: Probability of Arrival in an Interval of 𝛿
  22:24 Little-o Notation for Linear Decay
  27:09 Definition 2 of Poisson Process (Based on Little-o Notation)
  33:43 Moment Generating Functions and Poisson Distribution
- EE 306 - Signals and Systems II - Lecture 23: Poisson Processes Continued File
  Lecture Content:
  
  0:00 Recap of Previous Lecture
  2:08 Definition 3 of Poisson Process (Based on Poisson Distribution)
  3:57 Equivalence of Definitions 2 and 3 of Poisson Process
  9:06 Equivalence of Definitions 1 and 3 of Poisson Process
  13:30 Example 1: Reception of Emails Modelled as a Poisson Process
  20:36 Waiting Times in a Poisson Process
  32:06 Example 2: Entering Building A
  38:20 Time Reversed Poisson Process
  41:13 Example 3: Number of People in the Bus before Boarding
- EE 306 - Signals and Systems II - Lecture 24: Splitting and Merging Poisson Processes File
  Lecture Content:
  
  0:00 Recap of Previous Lecture
  4:05 Example 1: 3 Buses Departing from METU
  11:52 Splitting Poisson Processes (Proof by Using Baby Bernoulli)
  19:48 Splitting Poisson Processes (Proof by Using Definition 1 of Poisson Process)
  30:30 Splitting Poisson Processes (Proof of Two Split Processes Being Indpendent)
  41:30 Merging Independent Poisson Processes (Proof by Racing Exponentials)
- Lecture 22-23-24 Notes Folder
Holiday
9 May - 15 May
- EE 306 - Signals and Systems II - Lecture 25: A Long Example Related to Poisson Process (Bus Departure Example) File
  Lecture Content:
  
  0:00 Summary of Poisson Process
  9:11 Example 1: Bus Departure from METU (long example)
  12:03 Example 1, Part a: Expected Waiting Time after Bus Leaves
  13:29 Example 1, Part b: Expected Waiting Time at Random Arrival
  15:05 Example 1, Part c: 5 Bus Departure Probability
  17:50 Example 1, Part d: 3 Buses out of 5 Going to Specific Location
  20:25 Example 1, Part e: Expected Value of People Getting on Bus
  31:50 Example 1, Part f: Expected Value of People on Board at Random Time
  37:44 Example 1, Part g: Probability of 6 Bus Departures in First 10 Minutes
- EE 306 - Signals and Systems II - Lecture 26: Random Telegraph Signal and Shot Noise File
  Lecture Content:
  
  0:00 Example 1: Random Telegraph Signal
  18:00 Example 2: Shot Noise
- Lecture 25-26 Notes Folder
- Complete Lecture Notes for Module II File
16 May - 22 May
- Complete Lecture Notes for Module III (Lectures 27-41) File
- EE 306 - Signals and Systems II - Lecture 27: Deterministic and Stochastic Modelling File
  Lecture Content:
  
  0:00 Example 1: Deterministic Modelling (Projectile Motion of a Cannonball)
  11:35 Example 2: Deterministic Modelling (Reconstruction of a Bandlimited Signal from the Samples)
  16:10 Stochastic Modelling
  19:06 Example 3: Random Slope Signal
  24:55 Description of Random Processes (1st Order p.d.f Description)
  28:28 Example 4: Uniformly Distributed Random Slope Signal
  40:40 Description of Random Processes (2nd Order Joint p.d.f Description)
  42:20 Example 5: Uniformly Distributed Random Slope Signal at Two Time Instants
- EE 306 - Signals and Systems II - Lecture 28: Stochastic Modelling Continued File
  Lecture Content:
  
  0:00 Example 1: Uniformly Distributed Random Slope Signal at Two Time Instants (continued from previous lecture)
  12:47 Comments on Nth Order Joint p.d.f. Description
  23:18 Example 2: Calculation of 1st Order p.d.f from 2nd Order p.d.f.
  35:23 Partial Description of Random Processes with Moment Descriptions
  40:07 Example 3: Mean and Autocorrelation of Random Slope Signal (1st Order Description)
- EE 306 - Signals and Systems II - Lecture 29: p.d.f Description of Random Phase Cosine Signal File
  Lecture Content:
  
  0:00 Recap of Previous Lecture
  4:44 Example 1: Mean and Autocorrelation of Random Slope Signal (2nd Order Description)
  10:00 Example 2: Random Phase Cosine Signal
  16:40 Example 2: Random Phase Cosine Signal (1st Order p.d.f. Description)
  42:30 Example 2: Random Phase Cosine Signal (2nd Order p.d.f. Description)
- Lecture Notes - Module III (All) Folder
23 May - 29 May
- EE 306 - Signals and Systems II - Lecture 30: Stationarity File
  Lecture Content:
  
  0:00 Example 1: Moment Description of Random Phase Cosine (Mean Function)
  5:57 Example 1: Moment Description of Random Phase Cosine (Autocorrelation Function)
  11:44 Example 1: Moment Description of Random Phase Cosine (Covariance Function)
  12:34 Notes on Moment Descriptions
  23:55 Stationary Processes
  29:00 1st Order Stationarity
  33:10 Example 2: 1st Order Stationarity of Random Phase Cosine and Random Slope Signals
  38:13 2nd Order Stationarity
  52:04 Nth Order Stationarity
- EE 306 - Signals and Systems II - Lecture 31: Stationarity File
  Lecture Content:
  
  0:00 Nth Order Stationarity Continued
  0:59 Notes on Stationarity
  6:02 Example 1: Stationarity of Random Phase Cosine Signal
  11:00 Strict Sense Stationarity
  12:30 Brief Summary of Stationarity
  21:25 Stationarity in Moment Descriptions (Mean Function)
  25:34 Stationarity in Moment Descriptions (Autocorrelation Function)
  30:00 Example 2: Stationarity in Autocorrelation for Random Phase Cosine Signal
  35:13 Stationarity in Covariance Function
  38:15 Wide Sense Stationarity
  39:52 Comments on Stationarity
- EE 306 - Signals and Systems II - Lecture 32: Gaussian Processes File
  Lecture Content:
  
  0:00 Recap of Previous Lecture
  0:59 Example 1: Wide Sense Stationarity of Some Processes
  23:50 Gaussian Processes
  32:29 Notes on Gaussian Processes
30 May - 5 June
- EE 306 - Signals and Systems II - Lecture 33 - A Gaussian Processes Example and LTI Processing of WSS Random Processes File
  Lecture Content:
  
  0:00 Example 1: Gaussian Process with ‘sinc’ Autocorrelation
  4:41 Example 1: Gaussian Process with ‘sinc’ Autocorrelation (Sampling Rate for Independent Samples)
  17:23 Example 1: Gaussian Process with ‘sinc’ Autocorrelation (Joint Density of Independent Samples)
  24:38 LTI Processing of WSS Random Processes
  26:20 Jointly WSS Processes
  29:14 Steps of Showing Output is WSS for WSS Input
  31:03 Steps of Showing Output is WSS for WSS Input (Mean Function)
  36:13 Steps of Showing Output is WSS for WSS Input (Autocorrelation Function)
- EE 306 - Signals and Systems II - Lecture 34 - Moment Characterization of a Process for WSS Input and White Noise File
  Lecture Content:
  
  0:00 Example 1: Moment Characterization of Output Process for WSS Input
  16:50 Comments on the Previous Example for Input as Gaussian Process
  29:01 Comments on the Previous Example for Input with Impulsive Autocorrelation
  38:45 White Noise
- EE 306 - Signals and Systems II - Lecture 35 - Properties of Autocorrelation Function of WSS Processes File
  Lecture Content:
  
  0:00 Properties of Autocorrelation Function of WSS Processes
  1:55 Properties of Autocorrelation Function of WSS Processes (Hermitian Symmetry)
  4:50 Properties of Autocorrelation Function of WSS Processes (Positive Semi-definiteness)
  14:43 Properties of Autocorrelation Function of WSS Processes (Non-negativity of Zero-shift Term)
  18:20 Properties of Autocorrelation Function of WSS Processes (Zero-shift Term is Absolutely Largest)
  31:45 Example 1: Autocorrelation Function Candidates
  35:57 Comments on Necessity and Sufficieny of Autocorrelation Properties
  41:07 Example 2: Autocorrelation and Crosscorrelation of a Discerete Process Passed Through an LTI System
6 June - 12 June
- EE 306 - Signals and Systems II - Lecture 36 - Power Spectral Density File
  Lecture Content:
  
  0:00 Power Spectral Density Brief Introduction
  2:43 Power Spectral Density Definition
  9:25 Power Spectral Density Properties (Area under the PSD Function)
  14:46 Power Spectral Density Properties (Real-valued Function)
  18:29 Power Spectral Density Properties (Nonnegativity)
  28:30 Power Spectral Density Conclusion (Why it is Called ‘Density’?)
  33:37 Example 1: Power Spectral Density of Random Phase Cosine
  44:23 Example 2: Power Spectral Density of Sum of Complex Exponentials with Random Coefficients
- EE 306 - Signals and Systems II - Lecture 37 - Power Spectral Density Continued & Hilbert Transform File
  Lecture Content:
  
  0:00 Recap of Previous Lecture
  1:45 Notes on Power Spectral Density (Processing of White Noise with an LTI System)
  9:57 Notes on Power Spectral Density (Flatness of White Noise in Frequency Domain)
  12:50 Notes on Power Spectral Density (Sufficient Condition on Power Spectral Density for Valid Autocorrelation Functions)
  21:20 Notes on Power Spectral Density (Classification of Processes by Power Spectral Density Based on Passband Properties)
  23:35 Representation of Bandpass Signals/Processes (Hilbert Transform)
  35:07 Note on Stability of Hilbert Transform
  37:53 Example 1: Hilbert Transform of a Sinusoid
- EE 306 - Signals and Systems II - Lecture 38 - Bandpass Signal Representation File
  Lecture Content:
  
  0:00 Single Sideband Modulations as a Hilbert Transform Application
  10:30 Bandpass Signal Representation
  14:27 Bandpass Signal Representation (Lowpass Equivalent of a Bandpass Signal-Finding Analytic/Pre-envelope Signal)
  19:04 Bandpass Signal Representation (Lowpass Equivalent of a Bandpass Signal-Shifting Analytic/Pre-envelope Signal to DC)
  22:50 Notes on Lowpass Equivalent Signal (Complex Valued Signal)
  25:04 Notes on Lowpass Equivalent Signal (Location of the Center Frequency)
  26:36 Obtaining Bandpass Signal from Lowpass Equivalent
  38:20 Example 1: Lowpass Equivalent of a Sinusoid (Center Frequency as the Sinusoid Frequency)
  43:41 Example 1: Lowpass Equivalent of a Sinusoid (Center Frequency Different from the Sinusoid Frequency)
  50:28 Generation of Bandpass Signals
  55:15 Generation of I/Q Signals
13 June - 19 June
- EE 306 - Signals and Systems II - Lecture 39 - Representation of Bandpass Processes File
  Lecture Content:
  
  0:00 Recap of Previous Lecture
  2:54 Representation of Bandpass Systems
  12:28 Representation of Bandpass WSS Processes
  14:34 Obtaining Zero-Mean Information from Power Spectral Density
  20:18 Steps to Represent a Bandpass Process with an Equivalent Lowpass Process
  23:08 Step 1: Analytic Process Generation
  36:06 Step 2: Lowpass Equivalent Process Generation
  43:50 Notes on Lowpass Equavialent of a Bandpass Process
- EE 306 - Signals and Systems II - Lecture 40 - In-phase and Quadrature (I/Q) Components of a Bandpass Process File
  Lecture Content:
  
  0:00 Finding I/Q Components of a Bandpass Process
  6:42 Statistics of I/Q Components
  17:17 Statistics of I/Q Components (Relations Between Correlations of the Bandpass Process and Its Hilbert Transform)
  26:25 Statistics of I/Q Components (Final Trigonometric Calculations)
- EE 306 - Signals and Systems II - Lecture 41 - Spectral Density of I/Q Components of a Bandpass Process File
  Lecture Content:
  
  0:00 Recap of Previous Lecture
  13:20 Power Spectral Density of I/Q Components of a Bandpass Process
  23:00 Cross Power Spectral Density of I/Q Components of a Bandpass Process
  30:06 Example 1: Cross and Power Spectral Densities of I/Q Componets of a Process with Triangular Spectrum
  42:35 Notes on Spectral Density of I/Q Components of a Bandpass Process
  53:35 Some Results Based on Practical Assumptions