M.

Tech in Digital Signal Processing

COURSE STRUCTURE

July 2012

Department of Avionics Indian Institute of Space Science and Technology, Thiruvananthapuram, 695547

Preamble: Signal processing is an important research area of electrical engineering and applied mathematics that deals with operations on or analysis of signals, in either discrete or continuous time, to perform useful operations on those signals. In the term Signal Processing, signal refers to sound, images, time-varying measurement values and sensor data, for example biological data such as electrocardiograms, control system signals, telecommunication transmission signals such as radio signals, and many others. These Signals are analog or digital electrical representations of time-varying or spatial-varying physical quantities. There is significant impact of signal processing research in space research as well as in the current state of the art technology such as communication, biomedical etc. Many universities offer graduate and under graduate program in signal processing, however, there focus is not directly deals with space requirement or space related research problem. In IIST we propose to offer systematic course in Signal processing that can give desired exposure to ISRO scientist as well as to the other eligible students who will enrol for this course. The curriculum designed to enhance three important skills i.e., Theoretical research, Programming or Realization of theory using software and hardware, and finding applications to apply knowledge gained during the course. The proposed curriculum in beginning (first two semesters) will give necessary knowledge and exposure in signal processing theory and lab and later (3rd and 4th semester) these will be used to solve important research problem. The proposed Mtech program in Signal Processing from the Department of Avionics, Indian Institute of Space Science and Technology aims at bridging the gap between requirement of young research human resource and the Indian space research programme. The structure of the proposed course is aims at providing graduate students both a solid theoretical understanding and introduction to a comprehensive research skill in the development of novel signal processing techniques and uses these techniques to solve important research problems.

Course plan for proposed M.Tech in Signal Processing Ist Semester

Code Course title 1 2 3 4 5 Total Advanced Signal Analysis and Processing* Mathematical Methods for Signal Processing DSP in digital communication* Pattern Recognition and Machine Learning Image and Video Processing * Lecture hours 3 3 3 3 3 15 Practical hours 0 0 3 0 3 6 Practical Credits 0 0 1 0 1 2 Total credits 3 3 4 3 4 17

IInd Semester
Code Course title 1 2 Statistical Signal Processing Digital signal processors for real time applications * Elective I Elective II Seminar Lecture hours 3 3 Practical hours 0 3 Practical Credits 0 1 Total credits 3 4

3 3 0 1 4 4 3 0 1 4 5 0 0 0 3 Total 12 3 3 18 *: Courses with either laboratory component or numerical or computational component

IIIrd Semester
Code Course title M.Tech Thesis research work (pilot results, problem definition, work progress, thesis evaluation) Elective III Total Lecture hours 0 Practical hours 0 Practical Credits 0 Total credits 12

0 0

0 0

0 0

3 15

IVth Semester
Code Course title M.Tech Thesis (internal and external evaluation) seminar Total Lecture hours 0 0 0 Practical hours 0 0 0 Practical Credits 0 0 0 Total credits 18 2 20

Total credits : 17+ 18 + 15 + 20 = 70

Core courses: 1. Advanced Signal Analysis and Processing 2. Mathematical Methods for Signal Processing 3. DSP in digital communication* 4. Digital signal processors for real time applications * 5. Video and Image Processing * 6. Pattern Recognition and Machine Learning 7. Statistical Signal Processing

*: Courses with lab components included Electives (These can be either self study courses or regular lecture courses. Decided by the factors like number of students needing it, the expertise and understanding of individual student, etc.): 1. 2. 3. 4. Speech signal processing and coding Information theory and coding Soft computing and its application in Signal Processing Computer vision

5. Multimedia processing

Proposed Syllabus

1. Advanced Signal Analysis and Processing Continuous-time and discrete-time signals and systems; Spectral analysis:CTFT and DTFT, DFT and FFT; Sampling, Quantization, Decimation and Interpolation; Ztransform; Digital filters: FIR and IIR filters, Digital-filter realisations and design; Probabilistic method in digtal signal processing; DSP processors; Application of DSP: Speech signal processing. Discrete and Continuous time signals and systems, LTI systems, Convolution, Difference equations. Frequency domain representation: Fourier transform and its properties. TimeFrequency representations; Nonstationary Processes; Discrete Wavelet Transforms; Discrete Time-Frequency Transforms; Random discrete signals. Sampling and reconstruction: Change of sampling rate. Normed vector spaces, basis, linear independence, orthogonality. Linear systems of equations. Over- and Underdetermined systems. Row- and Column spaces, Null spaces. Least square and minimum norm solutions. Inverse and pseudo inverse, Symmetry transformations. Eigenvectors and eigenvalues. Hilbert transforms, band pass representations and complex envelope. Base band pulse transmission, matched filtering, ISI, equalization. Coherent and noncoherent detection. Discrete random processes, Linear prediction, Digital Wiener filtering, Least mean squares adaptive filter, Orthogonalized adaptive filter, Least squares adaptive filters, Other adaptive filtering techniques, Blind adaptive filtering Texts/References: 1. A.V. Oppenheim and R.W. Schafer: Discrete- Time Signal Processing; PHI, 1997. 2. John G. Proakis Dimitris G. Manolakis: Digital Signal Processing: Principles, Algorithms and Applications; PHI, 1997 3. S. K. Mitra: Digital Signal Processing; TMH, 1998. 4. S. J. Orfanidis: Introduction to Digital Signal Processing; Prentice-Hall, 1996. 5. L.R. Rabiner and R.W. Schafer: Digital Processing of Speech Signal; Prentice-Hall, 1978. 6. Burrus et al: Computer-Based Exercises for Signal Processing using MATLAB; Prentice- Hall, 1994. 7. Computational Signal Processing with Wavelets, Teolis, Anthony 1996, 352 p. 128 illus. A product of Birkhuser Boston, springer 8. Adaptive Filter Theory, S. Haykin, Prentice-Hall, 4-th edition, 2001. 9. Fundamentals of Adaptive Filtering, Ali H. Sayed, John Wiley, 2003. 10. Statistical and Adaptive Signal Processing: Spectral Estimation, Signal Modeling, Adaptive Filtering and Array Processing, D. Manolakis, V. Ingle, S. Kogan, McGraw Hill, 1999. 11. Signal Processing: The Modern Approach, J. Candy, McGraw Hill, 1988. 12. Digital Spectral Analysis, S. Marple, Prentice-Hall, 1987. 13. Adaptive Signal Processing, B. Widrow, S. Stearns, Prentice-Hall, 1985. 2. Mathematical Methods for Signal Processing Linear Algebra: Basic analysis and topology. Vector spaces, linear operators and matrices. Decomposition theorems and eigen-analysis. Quadratic forms. PerronFrobenius theorems.

Probability: Spaces and random variables. Distributions. Transformations and moment analysis. Stochastic processes and covariance analysis. Estimation theory. It will also cover modern techniques of digital signal processing and spectral estimation of discrete-time or discrete-space sequences derived by the sampling of continuous-time or continuous-space signals. The class covers the mathematical foundation needed to understand the various signal processing techniques as well as the techniques themselves. Topics include the discrete Fourier transform, the discrete Hilbert transform, the singular-value decomposition, the wavelet transform, classical spectral estimates (periodogram and correlogram), autoregressive, autoregressivemoving average spectral estimates, and Burg maximum entropy method. Generalized inverses, regularization of ill-posed problems. Eigen and singular value decompositions, generalized problems. Interpolation and approximation by least squares and minimax error criteria. Optimization techniques for linear and nonlinear problems. Applications in various areas of signal processing.

Texts/References: 1. K. Hoffman and R. Kunze: Introduction to Linear Algebra; Prentice-Hall, 1996, 2/e. 2. R. Horn and C. Johnson: Matrix Analysis; Cambridge, C.U.P.,1991 3. A. Papoulis: Probability, Random Variables and Stochastic Processes; McGrawHill, 1991, 3/e. 4. H. Stark and J.W. Woods: Probability, Random Variables and Estimation Theory for Engineers; Prentice Hall, 1994. 5. Gene H. Golub and Charles F. van Loan, Matrix Computations, John Hopkins University Press, 1996. 6. Prof. J. Reillys online course notes on Matrix Computations http://www.ece.mcmaster.ca/~reilly/html/gradcourse/graduate_courses.html 7. Todd K. Moon and Wynn C. Stirling, Mathematical methods and algorithms for signal processing, Prentice Hall, 2000. 8. Roger A. Horn and Charles R. Johnson, Matrix analysis, Cambridge University Press, 1990. 3. DSP in digital communication* This course provides attendees with a solid and intuitive understanding of the latest digital communication technologies for both wireless and wireline systems. The course covers both the Digital Signal Processing (DSP) techniques and the physical layer modulation schemes which are facilitating the vast improvements in spectral efficiency being implemented in 3rd and 4th generation wireless systems. Current and future wireless standards shall be used as case studies to illustrate these techniques. DSP theory review, Single carrier modulation, Pulse shaping and matched filtering Intersymbol interference, Propagation channels, Equalisation, Spread spectrum techniques, OFDM, Synchronisation, Multi-antenna techniques, Modelling digital modulation and demodulation; explaining the modulation scheme and evaluating the performances for BPSK, QPSK, 16-PSK, 16-QAM, 64-QAM. Noise in electronic communication systems. Baseband coding; convolutional coding and decoding (Viterbi). System performance; BER, BLER with and without the addition of baseband coding. Orthogonal Frequency Divisional Multiplexing (OFDM). Propagation and receiver Noise Figure.

Text book 1. communication system by b carlon (TMH) 2. Digital communication by Bernard shlar , pearson education 3. Principle of communication by Taub and schilling (TMH)

4. Digital signal processor for real time applications Computational characteristics of DSP algorithms and applications; their influence on defining a generic instruction-set architecture for DSPs. Architectural requirement of DSPs: high throughput, low cost, low power, small code size, embedded applications. Techniques for enhancing computational throughput: parallelism and pipelining. Data-path of DSPs: multiple on-chip memories and buses, dedicated address generator units, specialized processing units (hardware multiplier, ALU, shifter) and on-chip peripherals for communication and control. Control-unit of DSPs: pipelined instruction execution, specialized hardware for zerooverhead looping, interrupts. Architecture of Texas Instruments fixed-point and floating-point DSPs: brief description of TMS320 C5x /C54x/C3x DSPs; Programmers model. Architecture of Analog Devices fixed-point and floating-point DSPs: brief description of ADSP 218x / 2106x DSPs; Programmers model. Advanced DSPs: TIs TMS 320C6x, ADIs Tiger-SHARC, Lucent Technologies DSP 16000 VLIW processors. Applications: a few case studies of application of DSPs in communication and multimedia. Introduction to FPGA, RTOS, OS, Basics of Embedded systems Texts/References: 1. P. Pirsch: Architectures for Digital Signal Processing; John Wiley, 1999. 2. R. J. Higgins: Digital Signal Processing in VLSI; Prentice-Hall, 1990. 3. Texas Instruments TMSC5x, C54x and C6x Users Manuals. 4. Analog Devices ADSP 2100-family and 2106x-family Users Manuals. 5. K. Parhi: VLSI Digital Signal Processing Systems; John Wiley, 1999. 6. K. Parhi and T. Nishitani: Digital Signal Processing for Multimedia Systems; Marcel Dekker, 1999. 7. IEEE Signal Processing Magazine: Oct 88, Jan 89, July 97, Jan 98, March 98 and March 2000. 5. Video and Image Processing Human visual system and image perception; monochrome and colour vision models; image acquisition and display: video I/O devices; standard video formats; image digitization, display and storage; 2-D signals and systems; image transforms- 2D DFT, DCT, KLT, Harr transform and discrete wavelet transform; image enhancement: histogram processing, spatial-filtering, frequency-domain filtering; image restoration: linear degradation model, inverse filtering, Wiener filtering; image compression: lossy and lossless compression, entropy coding, transform coding, subband coding, image compression standards, video compression- motion compensation, video compression standards; image analysis: edge and line detection, segmentation, feature

extraction, classification; image texture analysis; morphological image processing: binary morphology- erosion, dilation, opening and closing operations, applications; basic gray-scale morphology operations; colour image processing: colour models and colour image processing Experiments are based on MATLAB implementation of algorithms covered in the course. Fundamentals of digital video processing. Coverage includes spatio-temporal sampling, motion analysis, parametric motion models, motion-compensated filtering, and video processing operations including noise reduction, restoration, superresolution, deinterlacing and video sampling structure conversion, and compression (frame-based and object-based methods). A number of advanced topics will be covered, including video segmentation and layered video representations, watermarking, video streaming, compressed-domain video processing, and digital TV. Texts/References: 1. A. K. Jain, Fundamentals of Digital Image processing, Pearson Education, 1989. 2. R. C. Gonzalez and R. E. Woods: Digital Image Processing, Pearson Education, 2001 3. R. C. Gonzalez , R. E. Woods and S. L. Eddins: Digital Image Processing using MATLAB, Pearson Education, 2004. 4. G. A. Baxes: Digital Image Processing; John Wiley, 1994 5. R.J. Schalkoff: Digital Image Processing and Computer Vision; John Wiley, 1989. 6. Sid Ahmed: Image Processing; McGraw -Hill, 1994. 7. S.J. Solari: Digital Video and Audio Compression; McGraw-Hill, 1996. 8. Video Processing and Communications" by Yao Wang, Joern Ostermann, and YaQin Zhang, Prentice Hall, 2002, ISBN 0-13-017547-1. 9. "Digital Video Processing" by M. Tekalp (Prentice Hall, 1995, ISBN 0-13-1900757) 10. "The Art of Digital Video", J. Watkinson, 3rd edition, Focal Press, 2000. 11. "Video Demystified", K. Jack, 3rd edition, Llh Technology Publishing, 2001. 12 "Motion Analysis and Image Sequence Processing", edited by M.I. Sezan and R.L. Lagendijk, Kluwer Academic Publishers, 1993. 13. "Image and Video Compression Standards: Algorithms and Architectures", V. Bhaskaran and K. Konstantinides, Kluwer Academic Publishers, 2nd edition, 1997.

6. SPEECH SIGNAL PROCESSING AND CODING Introduction: speech production and perception, information sources in speech, linguistic aspect of speech, acoustic and articulatory phonetics, nature of speech, models for speech analysis and perception; Short-term processing: need, approach, time, frequency and time-frequency analysis; Short-term Fourier transform (STFT): overview of Fourier representation, non-stationary signals, development of STFT, transform and filter-bank views of STFT; Cesptrum analysis: Basis and development, delta, delta-delta and mel-cepstrum, homomorphic signal processing, real and complex cepstrum; Linear Prediction (LP) analysis: Basis and development, Levinson-Durbins method, normalized error, LP spectrum, LP cepstrum, LP residual; Sinusoidal analysis: Basis and development, phase unwrapping, sinusoidal analysis and synthesis of speech; Speech coding: Need and parameters, classification,

waveform coders, speech-specific coders, GSM, CDMA and other mobile coders; Applications: Some applications like pitch extraction, spectral analysis and coding standard. Texts/References: 1. L.R. Rabiner and R.W. Schafer, Digital Processing of Speech Signals Pearson Education, Delhi, India, 2004 2. J. R. Deller, Jr., J. H. L. Hansen and J. G. Proakis Discrete-Time Processing of Speech Signals, Wiley-IEEE Press, NY, USA, 1999. 3. D. OShaughnessy, Speech Communications: Human and Machine, Second Edition,University Press, 2005. 4. T. F. Quatieri, Discrete time processing of speech signals, Pearson Education, 2005. 5. L. R. Rabiner, B. H. Jhuang and B. Yegnanarayana, Fundamentals of speech recognition, Pearson Education, 2009.

7. Information theory and coding Communication sytems and principles of information theory; measure of information; coding for discrete; discrete memoryless channels and channels capacity; noisy channels; coding theorem; techniques for coding and decoding memoryless channel with discrete time; waveform channels; source coding with a fidelity criterion. This course covers information theory and coding within the context of modern digital communications applications. We begin with a directed review of probability and digital modulation schemes. We then introduce information theory and employ it to study bounds on source/channel coding and communication channel performance. Source coding is considered because it provides a straightforward example of the utility entropy, an information theory measure. Channel coding is considered because channel capacity, another information theory measure, provides a theoretical bound which is the goal of channel coding. We then proceed with an in depth treatment of block and convolutional channel coding, with both soft and hard decoding. Bit-error-rate performance is studied relative to channel capacity. Advanced topics such as Reed-Solomon codes, space time codes, concatenated codes, turbo coding and LDPC codes are introduced. Bandwidth efficient trellis coded modulation is also overviewed. Reference 1. Information Theory and Coding by Norman Abramson 2. Digital Communications, 5-th edition, by John Proakis & Masoud Salehi, McGrawHill, 2008.

8. Statistical Signal Processing Power Spectrum Estimation-Parametric and Maximum Entropy Methods, Wiener, Kalman Filtering, Levinson-Durban Algorithms Least Square Method, Adaptive Filtering, Nonstationary Signal Analysis, Wigner-Ville Distribution, Wavelet Analysis. Power Spectrum Estimation, model order selection, Prony, Pisarenko, MUSIC, ESPRIT algorithms, least square estimation, cholesky, LDU-OR, SV

decomposition. Transversal & reasnic least square lattice filters, Signal Analysis with Higher order Spectra, Array processing, Beam foming, Time-delay estimation. Estimation Theory, Maximum Likelihood estimation (MLE): exact and approximate methods (EM, alternating max, etc), Cramer-Rao lower bound (CRLB), Minimum variance unbiased estimation, best linear unbiased estimation, Bayesian inference & Least Squares Estimation , Basic ideas, adaptive techniques, Recursive LS, etc, Kalman filtering (sequential Bayes), Finite state Hidden Markov Models: forwardbackward algorithm, Viterbi (ML state estimation), parameter estimation (f-b + EM), Monte Carlo methods: importance sampling, MCMC, particle filtering, applications in numerical integration (MMSE estimation or error probability computation) and in numerical optimization (e.g. annealing) Detection Theory: Likelihood Ratio testing, Bayes detectors, Minimax detectors, Multiple hypothesis tests Neyman-Pearson detectors (matched filter, estimatorcorrelator etc), Wald sequential test, Generalized likelihood ratio tests (GLRTs), Wald and Rao scoring tests, Applications

Texts/References: 1. Statistical Signal Processing (Paperback) by Louis Scharf 2. S.M. Kay's Fundamentals of Statistical Signal Processing: Estimation Theory (Vol 1), Detection Theory (Vol 2) 3. Kailath, Sayed and Hassibi, Linear Estimation 4. V. Poor, An Introduction to Signal Detection and Estimation 5. H.Van Trees, Detection, Estimation, and Modulation Theory 6. J.S. Liu, Monte Carlo Strategies in Scientific Computing. Springer-Verlag, 2001. 7. B.D. Ripley, Stochastic Simulation. Wiley, 1987. 9. Pattern Recognition and machine learning Bayes decision theory, Supervised and unsupervised classification: parametric and non-parametric schemes. Nearest neighbour classification. Decision trees for classification. Clustering schemes, regression, and time series analysis. Discriminant analysis, model performance , dimensionality reduction, data normalization, Basic Learning techniques (kernel methods etc.), online and offline data analysis. Knowledge-based classification. Soft computing paradigms for classification and clustering. Applications to document analysis and recognition. Texts/References: 1. Duda and Hart Pattern classification John willey and sons 2. TM Mitchel Machine learning McGraw Hills 1997 3. Christopher M. Bishop, "Pattern Recognition and Machine Learning", Springer, 2006. 10. Soft computing and its application in Signal Processing Soft Computing: Introduction, requirement, different tools and techniques, usefulness and applications. Fuzzy sets and Fuzzy logic: Introduction, Fuzzy sets versus crisp sets, operations on fuzzy sets, Extension principle, Fuzzy relations and relation equations, Fuzzy numbers, Linguistic variables, Fuzzy logic, Linguistic hedges, Applications, fuzzy controllers, fuzzy pattern recognition, fuzzy image processing, fuzzy database.

Artificial Neural Network: Introduction, basic models, Hebb's learning, Adaline, Perceptron, Multilayer feed forward network, Back propagation, Different issues regarding convergence of Multilayer Perceptron, Competitive learning, SelfOrganizing Feature Maps, Adaptive Resonance Theory, Associative Memories, Applications. Evolutionary and Stochastic techniques: Genetic Algorithm (GA), different operators of GA, analysis of selection operations, Hypothesis of building blocks, Schema theorem and convergence of Genetic Algorithm, Simulated annealing and Stochastic models, Boltzmann Machine, Applications. Rough Set: Introduction, Imprecise Categories Approximations and Rough Sets, Reduction of Knowledge, Decision Tables, and Applications. Hybrid Systems: Neural-Network-Based Fuzzy Systems, Fuzzy Logic-Based Neural Networks, Genetic Algorithm for Neural Network Design and Learning, Fuzzy Logic and Genetic Algorithm for Optimization, Applications. Applications of soft computing to signal processing Texts/References: 1. Neural Fuzzy Systems, Chin-Teng Lin & C. S. George Lee, Prentice Hall PTR. 2. Fuzzy Sets and Fuzzy Logic, Klir & Yuan, PHI, 1997. 3. Neural Networks, S. Haykin, Pearson Education, 2ed, 2001. 4. Genetic Algorithms in Search and Optimization, and Machine Learning, D. E. Goldberg, Addison-Wesley, 1989. 5. Neural Networks, Fuzzy logic, and Genetic Algorithms, S. Rajasekaran & G. A. V. Pai, PHI. 6. Neuro-Fuzzy and Soft Computing, Jang, Sun, & Mizutani, PHI. 7. Learning and Soft Computing, V. Kecman, MIT Press, 2001. 8. Rough Sets, Z. Pawlak, Kluwer Academic Publisher, 1991. 9. Intelligent Hybrid Systems, D. Ruan, Kluwer Academic Publisher, 1997. 11. Computer vision Image Formation Models, Monocular imaging system, Orthographic & Perspective Projection , Camera model and Camera calibration , Binocular imaging systems Image Processing and Feature Extraction , Image representations (continuous and discrete), Edge detection, Motion Estimation , Regularization theory, Optical computation , Stereo Vision , Motion estimation , Structure from motion Shape Representation and Segmentation , Deformable curves and surfaces , Snakes and active contours , Level set representations , Fourier and wavelet descriptors Medial representations , Multiresolution analysis Object recognition , Hough transforms and other simple object recognition methods , Shape correspondence and shape matching , Principal component analysis, Shape priors for recognition Texts/References: 1. Computer Vision - A modern approach, by D. Forsyth and J. Ponce, Prentice Hall 2. Introductory Techniques for 3D Computer Vision, by E. Trucco and A. Verri, Publisher: Prentice Hall. 3. Robot Vision, by B. K. P. Horn, McGraw-Hill.

12. Multimedia Processing The course focuses on international standards related to image/video/audio formulated by ISO/IEC/ITU. Short-term Fourier Transform & Continuous Wavelet Transform, CWT and its discretization, Discrete Wavelet Transforms, 2 - D Wavelet Transforms, Coding Techniques in 2 - D Wavelet Transforms, Emphasis will be on the family of MPEG 1/2/4 (Moving Picture Experts Group), H.26x (x = 1, 2, 3), JPEG/JPEGLS/JPEG2000 (Joint Photographic Experts Group), JBIG 1/2 (Joint Binary Image Group), H.264/MPEG-4 Part 10 AVC (Advanced Video Coding) and the emerging H.265 standard (HEVC). Other standards such as WMV-9 (Windows Media Video) of Microsoft and AVS China (Advanced Video Standard) and their similarities/differences with H.264 will also be presented, also audio coding AC3, AAC, AAC + SBR, G.72xseries, MPEG-1, 2, 4 audio. Course will be supplemented with ftp/web sites, software, standards documents, test sequences etc. Industry worldwide is very active in developing products (software/hardware) based on these standards with emphasis on the latest standard H.264. Some of these are digital TV, HDTV, HD-DVD, set-top-box, handheld devices with multimedia capabilities, digital cameras, camcorders, electronic games, IPTV, video streaming, iPods etc. Motion Estimation : Matching Criteria, Generalised Matching, Generalised Deformation Model in Motion Estimation, Synchronization of Media Multimedia Content Representation and Retrieval, Video Content Representation, Content-based Video : Motion Representation, Content-based Video : Low to High-level Representation, Content Retrieval Schemes Text and Reference: O. Marques, Practical image and video processing using MATLAB, Hoboken, NJ: Wiley, 2011.