Scribd Logo
Upload

Welcome to Scribd!

  • ITC Tutorial 4

    Uploaded by

    Anon Poet
    2 views13 pages
    This document contains information about an upcoming tutorial on information theory and coding to be given by Dr. Atri Mukhopadhyay and Dr. Sumit K. Chattopadhyay, assistant professors in the EEE Department at BITS Pilani Hyderabad Campus. The tutorial will cover topics such as entropy, mutual information, and channel capacity calculations. It provides examples of calculating entropy for single and joint random variables, deriving the mutual information between two random variables, and finding the maximum channel capacity.

    Original Description:

    Original Title

    ITC_Tutorial_4

    Copyright

    © © All Rights Reserved

    Available Formats

    PPTX, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    This document contains information about an upcoming tutorial on information theory and coding to be given by Dr. Atri Mukhopadhyay and Dr. Sumit K. Chattopadhyay, assistant professors in the EEE Department at BITS Pilani Hyderabad Campus. The tutorial will cover topics such as entropy, mutual information, and channel capacity calculations. It provides examples of calculating entropy for single and joint random variables, deriving the mutual information between two random variables, and finding the maximum channel capacity.

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pptx, pdf, or txt
    2 views13 pages

    ITC Tutorial 4

    Uploaded by

    Anon Poet
    This document contains information about an upcoming tutorial on information theory and coding to be given by Dr. Atri Mukhopadhyay and Dr. Sumit K. Chattopadhyay, assistant professors in the EEE Department at BITS Pilani Hyderabad Campus. The tutorial will cover topics such as entropy, mutual information, and channel capacity calculations. It provides examples of calculating entropy for single and joint random variables, deriving the mutual information between two random variables, and finding the maximum channel capacity.

    Copyright:

    © All Rights Reserved
    Download as PPTX, PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pptx, pdf, or txt
    of 13

    Information Theory and

    Coding - Tutorial

    Dr. Atri Mukhopadhyay, Dr. Sumit K. Chattopadhyay


    Assistant Professor
    EEE Department
    BITS Pilani Hyderabad Campus

    ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


    Q1

    06/29/2023 2

    ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


    Solution

    06/29/2023 3

    ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


    Solution

    06/29/2023 4

    ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


    Solution

    06/29/2023 5

    ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


    Q2

    Y
    X

    06/29/2023 6

    ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


    1
    𝑝1 𝑝 1+𝑝 (1 − 𝑝 1)

    Input Probabilities Output Probabilities

    1− 𝑝 1 (1 − 𝑝)(1 − 𝑝 1)

    So, we have P(X), P(Y|X) and P(Y)


    Use

    06/29/2023 7

    ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


    𝑑 ( 𝐼 ( 𝑋 ; 𝑌 )) 1 − 𝑝1
    =( 1− 𝑝 ) 𝑙𝑜𝑔 − 𝑝𝑙𝑜𝑔𝑝
    𝑑 𝑝1 𝑝1 +𝑝 (1 −𝑝 1)

    Solve for

    06/29/2023 8

    ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


    06/29/2023 9

    ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


    06/29/2023 10

    ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


    Q3

    06/29/2023 11

    ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


    𝐻 ( 1 1
    3 3
    1 1 1
    3 3 3 ) ( )
    2 1
    + ( 𝑝 1 − 𝑝3 ) , , − ( 𝑝 1 − 𝑝 3 ) − ( 𝑝 1 +𝑝 3 ) 𝐻 , − ( 1 −𝑝 1 − 𝑝 3 ) 𝑙𝑜𝑔 3
    3 3

    06/29/2023 12

    ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION


    06/29/2023 13

    ELECTRICAL ELECTRONICS COMMUNICATION INSTRUMENTATION

    You might also like