Scribd Logo
Upload

Welcome to Scribd!

  • ITC Quiz 1 (Possible Answer)

    Uploaded by

    sohaibhaider
    38 views2 pages
    This document describes Shannon-Fano encoding using a sample message set with 15 possible messages and their probabilities. It shows the binary, Shannon-Fano codes obtained, and how the codes can be uniquely decoded. It then calculates the average code length, entropy, efficiency and redundancy of the codes. Shannon-Fano encoding is only 100% efficient when the message probabilities are powers of 2.

    Original Description:

    ITC Course Solution

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    This document describes Shannon-Fano encoding using a sample message set with 15 possible messages and their probabilities. It shows the binary, Shannon-Fano codes obtained, and how the codes can be uniquely decoded. It then calculates the average code length, entropy, efficiency and redundancy of the codes. Shannon-Fano encoding is only 100% efficient when the message probabilities are powers of 2.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    38 views2 pages

    ITC Quiz 1 (Possible Answer)

    Uploaded by

    sohaibhaider
    This document describes Shannon-Fano encoding using a sample message set with 15 possible messages and their probabilities. It shows the binary, Shannon-Fano codes obtained, and how the codes can be uniquely decoded. It then calculates the average code length, entropy, efficiency and redundancy of the codes. Shannon-Fano encoding is only 100% efficient when the message probabilities are powers of 2.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    Download as pdf or txt
    of 2

    ShannonFano

    Encoding Example: 3

    Message







    Probability 0.35 0.15 0.12 0.12 0.08 0.04 0.04 0.04
    0.02
    0.01
    0.01
    0.005
    0.005
    0.005
    0.005
    Binary 0000 0001 0010 0011 0100 0101 0110 0111 1000 1001
    1010
    1011
    1100
    1101
    1110
    ShanFano 00 01 100 101 1100 1101 11100 11101 111100 111101 1111100 1111101 1111110 11111110 11111111
    OR
    ShanFano 00 01 100 101 1100 1101 11100 11101 111100 111101 1111100 1111101 11111100 11111101 1111111
    Code Length
    X1, X2, X3, X4, X5, X6, X7, X8, X9, X10, X11, X12, X13, X14, X15

    X1, X2
    X3, X4, X5, X6, X7, X8, X9, X10, X11, X12, X13, X14, X15
    2
    X1
    X2
    X3, X4
    X5, X6, X7, X8, X9, X10, X11, X12, X13, X14, X15
    3


    X3
    X4
    X5, X6
    X7, X8, X9, X10, X11, X12, X13, X14, X15
    4

    X5
    X6
    X7, X8
    X9, X10, X11, X12, X13, X14, X15
    5

    X7
    X8
    X9, X10
    X11, X12, X13, X14, X15
    6

    X9
    X10
    X11, X12
    X13, X14, X15
    7

    X11
    X12
    X13
    X14, X15
    8


    X14
    X15

    OR

    X13, X14
    X15
    8

    X13
    X14


    x1,x2,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12,x13,x14,x15
    x1,x2

    x3,x4,x5,x6,x7,x8,x9,x10,x11,x12,x13,x14,x15

    x1

    x2

    00

    x3,x4

    01

    x5,x6,x7,x8,x9,x10,x11,x12,x13,x14,x15
    11

    10

    x3

    x5,x6

    x4

    100

    x7,x8,x9,x10,x11,x12,x13,x14,x15

    110

    101

    x5

    111

    x6

    1100

    x7,x8

    1101

    x9,x10,x11,x12,x13,x14,x15

    1110

    x7

    11100

    1111

    x8

    x9,x10

    11101

    x11,x12,x13,x14,x15

    11110

    x9

    111100

    11111

    x10

    x11,x12

    111101

    x13,x14,x15

    111110

    x11

    1111100

    111111

    x12

    1111101

    x13

    x14,x15

    1111110

    1111111

    x14

    11111110

    x15

    11111111

    OR
    x1,x2,x3,x4,x5,x6,x7,x8,x9,x10,x11,x12,x13,x14,x15
    x1,x2

    x3,x4,x5,x6,x7,x8,x9,x10,x11,x12,x13,x14,x15

    x1

    00

    x2

    x3,x4

    01

    x5,x6,x7,x8,x9,x10,x11,x12,x13,x14,x15
    11

    10

    x3

    100

    x5,x6

    x4

    x7,x8,x9,x10,x11,x12,x13,x14,x15

    110

    101

    x5

    1100

    111

    x6

    x7,x8

    1101

    x9,x10,x11,x12,x13,x14,x15

    1110

    x7

    11100

    1111

    x8

    x9,x10

    11101

    x11,x12,x13,x14,x15

    11110

    x9

    111100

    11111

    x10

    x11,x12

    111101

    x13,x14,x15

    111110

    x11

    1111100

    111111

    x12

    x13,x14

    1111101

    x13

    11111100

    1111111

    x14

    11111101



    MSCS/CCN/F13/107

    x15

    1111110

    Page4of5

    Determine whether the codes are Uniquely Decipherable.


    2

    2 2

    2 2

    2 2

    2 2

    2 2

    3 2

    2 2


    b Obtain ShannonFano Codes.
    Message





    Probability 0.35 0.15 0.12 0.12 0.08 0.04
    ShanFano
    Code

    00

    01

    00

    01

    0.04

    0.04

    0.02

    0.01

    0.01


    0.005


    0.005

    0.005

    Determine the Average Code Length, Entropy, Efficiency and Redundancy of these codes.
    2

    0.35

    0.15

    2 3

    0.02

    0.01

    0.12

    0.01

    2 7

    0.08

    0.04

    0.005

    2 8

    2 5

    0.005

    0.04
    3

    0.35 log 0.35

    0.15 log 0.15

    2 0.12 log 0.12

    0.08 log 0.08

    3 0.04 log 0.04

    0.02 log 0.02

    2 0.01 log 0.01

    4 0.005 log 0.005

    2.922
    2.922
    3
    1

    0.974

    0.974

    97.4%

    0.026

    2.6%

    The ShannonFano encoding is 100% efficient when the probability of occurrence of each
    message symbol

    is of form:
    2
    2

    0.25


    0.35

    Hence proved that the following cannot be 100% efficient.

    MSCS/CCN/F13/107

    0.005

    100 101 1100 1101 11100 11101 111100 111101 1111100 1111101 1111110 11111110 11111111
    OR
    100 101 1100 1101 11100 11101 111100 111101 1111100 1111101 11111100 11111101 1111111

    Page5of5

    You might also like