Multimedia Watermarking Techniques

Frank Hartung and Martin Kutter ECE 738 In Young Chung

Outline
1.

2.
3. 4.

Terminology Requirement Basic Watermarking Principles Watermarking techniques for

1. Text Document 2. Image 3. Video 4. Audio 5.other media

5.

Conclusion

Terminology

Watermarking

techniques that allow secret communication, usually by embedding or hiding the secret information

Data hiding and data embedding Fingerprinting and Labeling Embedded signatures Visible watermarks

applications where the existence of the embedded data are publicly known, but there is no need to protect it

special application of watermarking related to copyright protection

stand for watermark in early publication

visible pattern, like logos, inserted into images or video

Watermarking Requirements
1)

2)

3)

4)

As much information as possible Capacity Only be accessible by authorized parties by means of cryptographic key security Resist against hostile attacks Robusteness Invisibility Imperceptibility
Robustness Imperceptibility

Capacity

Supplemental W. Requirements
1)

2)

Watermark Recovery may or may not allowed to use the original data. Real time watermarking requirements e.g., video fingerprinting (We will see this later)

Design Issues

Watermark Security and Keys Robustness Imperceptability Capacity Watermark Detection

Robustness

Capacity

Imperceptibility

Basic Watermarking Principles

Three issues in the design of a watermarking system

1) Design of the Watermark signal

W f0 (I , K )

or W f 0 ( I , K , X )

2) Design of the embedding method

3) Design of the corresponding extraction method

Y f1 ( X ,W )

I g( X ,Y , K )

or

I g (Y , K )

Where, W: watermark signal, I: watermark information. K: key, X: host data, Y: watermarked signal

Basic Watermarking Principles

The public or secret key is used to enforce security Many watermarking schemes use spread-spectrum methods they add a PN signal with low amplitude to the host data. Correlator is used for watermark detection

Watermarking Techniques 1. Text Document Watermarking

Text Document Watermarking

Two methods to hide information 1) in the semantics; in the meaning and ordering or the words 2) in the format * In the layout and the appearance

- Example of word shifting coding -

Text Document Watermarking; Three coding methods

1)

Line shift coding *Assumption; lines are uniformly spaced doesnt need original for watermark extraction word-shift coding

2)

*Assumption; space between words are usually variable needs original for Watermark extraction
3)

feature coding; slightly modifies the features Goal: making watermark removal more expensive than obtaining the right to copy from the copy right owner

Watermarking Techniques 2. Image Watermarking

Why is Image watermarking so important? 1) There is a large demand and productions 2) Most of watermarking research
much more than video or audio watermark

Common ideas for Image watermarking

A lot of watermarking methods are very similar and differ only in parts Three topics 1) watermark signal design 2) embedding 3) recovery (Detection) Watermark signal design The watermark signal; typically a pseudorandom signal with low amplitude e.g., Gaussian, uniform, or bipolar pdf * The watermark signal: often designed in spatial domain, sometimes in DCT or block-wise DCT

I.

Common ideas of Image watermarking

ii.

Signal embedding (where?) * embed watermark signal mostly to the luminance channel alone * sometimes in color channels * in the spatial domain * or in the DCT,DFT and DWT (full-image DCT or block wise DCT domain) advantage in terms of visibility and security

Common ideas of Image watermarking

Argue? about embedding domain Low, medium or high freq.?

For maximum robustness embed watermark signal adaptively where the host data populate (typically the low frequency)

Common ideas of Image watermarking

iii.

Recovery (Detection) * Usually done by correlation method; a correlation receiver or a matched filter

Acknowledgements

The Following slides organized by Author name The author of this paper put stress on embedding domain/methods So, we will
mainly deal with embedding domain/method of numerous watermarking methods

Tirkel

Publication: Electrical Water Mark in 1993 Proposal m-sequence PN code embedding in LSB plane
* To gain full access to the LSB plane without much distortion compress original image to 7 bits through histogram manipulation

the decoding process use the unique and optimal autocorrelation of m-sequences

Matsui and Tanaka

Publication: Video Steganography: How to secretly Embed a Signature in a picture Proposal Predictive code schemes using key table * exploit correlation between adjacent pixels by coding the prediction error instead of coding the individual gray scale value

i X i X i 1

And embed a watermark in forms of a binary string

Smith

New approach Digital watermarking and digital modulation (especially, direct sequence spread spectrum modulation) share similar concepts

More in depth analysis of 2-D amplitude modulation was given by Hernandez

Bender
I.

Proposal 2 Methods Patchwork 1. randomly selected pairs of pixels (ai , bi ) Are used to hide 1 bit by increasing the ais by one and decreasing the bis by one 2. The expected value of the sum of N pixel pairs

ContII.

Texture Block Coding 1. Watermark is embedded by copying one image texture block to another area in the image with similar texture 2. Recovery- autocorrelation * Remarkable point- high robustness to any kind of distortion, since both image area distorted in a similar way autocorrelation still works

Pitas and Kaskalis

Proposal Signature casting on digital images - based on same basic idea as the patchwork

1. The watermark S {sm,n } is the same size as the original image (here, # of ones = # of zero)
2. The original image is divided into two sets A and B of equal size

I : Original Image , xmn : Luminance value

Contn3. The watermark is superimposed by changing the elements of the subset A by positive factor k

A' {xmn k , xmn A}

*k is positive integer 4. The watermarked image is given by the union of A and B

Langelaar

Proposal Block base spatial watermarking - Improved version of previous method 1. 2. 3. 4. 5. The image is tiled into square blocks (8x8) Each block is selected pseudorandomly To embed 1, k x P added to the block To embed 0, k x P subtracted to the block Each selected block has a PN pattern P * k: scaling factor, P : PN pattern

Bruyndonckx

Proposal Watermarking with the use of pixel classification Purpose- increase the performance of the block base spatial watermarking methods 1. Select blocks (PN) & classify the block based on three types of contrast between zones; hard, progressive and noise contrast 2. Each zone subdivided into two categories A and B based on gird defined by the coder 3. Each pixel is assigned to one of four zone/category combinations e.g., 1/A,2/B

Cont4. A bit b is embedded by modifying the zone/category means to satisfy the following constraints

* * m1 ... m A 2 B: the modified zone/category mean values

S: the watermark embedding strength

5. the modification of the mean values is done by applying equal luminance variations for all pixels belonging to the same zone 7. To increase robustness the authors suggest to perform redundant bit embedding and use error correcting codes 6. Good robustness to JPEG compression is reported

Kutter

Proposal Improved spread-spectrum watermarking in the spatial domain * Exclusively works with the blue image component, in the RGB color space maximize the watermarking strength and minimize visual artifacts * preprocess the image prior to watermark decoding increased robustness, applicable to any spread-spectrum spatial domain watermarking

Cont1. A single bit b is embedded at a pseudorandomly selected location (I,j) by either adding or subtracting (Amplitude modulation)

Where Bi, j describes the blue value at location (I,j) Li , j :the Luminance at the same location : the embedding strength

Cont2. To recover an embedded bit, an estimate of original value is computed

c : the size of the cross-shaped neighborhood 3. The bit value is determined by looking at the sign of the difference

*To increase robustness, (Author suggest) each signature bit is embedded several times and to extract, the sum of all differences i , j is used

i , j Bi , j B i , j

Macq

Proposal Watermarking adapted to the HVS using masking and modulation 1. The watermark in spatial domain is low-pass filtered, frequency modulated, masked and then added to the host image 2. A secret key is used to determine the modulation freq. 3. Masking uses an extension of the masking phenomena for monochromatic signals, called gratings

Recovery- demodulation followed by a correlation function

Voyatzis and Pitas

Proposal Watermarking by inserting logo like patterns using torus automorphism * A 2-D torus automorphism: a kind of spatial transformation * It is defined by

* Iterated action of A on a point 0 form a dynamical system expressed by

Cont* This system mixes the point in chaotic way. * Under certain circumstances, the automorphisms may have periodicity

1-iteration

2-iteration

10-iteration

T-iteration

Cont

How/where to embed?

1. Watermark is mixed using the automophism AN

2. then, overlaid on a selected block in the original image e.g., LSB

Recovery- extracting the mixed watermark then, reconstruct the watermark using the automorphism AT N

* Where, T is the automorphism period.

Raymond and Wolfgang

Proposal Watermarking technique to verify image authenticity based on an approach similar to the m-sequence approach

1. A random sequence generated is mapped from {0,1} to {-1,1}, arranged into a suitable block and added to image

Recovery overlays the watermarked block with the watermark block and compute inner product and compares the result with ideal value

Cont-

The test statistics

is defined as

If the watermark is unchanged , =0 When is greater than a defined tolerance, the block fails the watermark test

Chen and Wornell

Proposal Quantized Index Watermark (QIM) *Please recall even-odd embedding and predictive coding scheme using table *Not based on spread-spectrum modulation but,quantization modulation *Based on a set of N-dimensional quantizers. *Quantizers satisfy a distortion constraint - each reconstruction values from one quantizer are far away from the others 1. The message to be transmitted is used as in index for quantizer section 2. Selected quantizer is used to embed information 3. Spatial or DCT domain used

Koch

Proposal Efficient watermarking in DCT domain for JPEG (first introduction) 1. The image is divided into square blocks of size 8x8 for which the DCT is computed

2. From a pseudoramdomly selected blocks, a pair of midfrequency coefficients are selected from 12 predetermined pairs
3. To embed a bit- the coefficient are modified such that the difference between them (a pair of coefficient) is positive or negative * Good robustness JPEG (Q=50%)

Swanson

Proposal DCT domain watermarking technique, Based on frequency masking of DCT blocks 1. Input image is split up in to square matrix and DCT is computed 2. Frequency mask is computed based on the knowledge that a masking grating raises the visual threshold for signal gratings around the mask freq. 3. The resulting perceptual mask is scaled and multiplied by PN(DCT)-sequence 4. This watermark is added to the corresponding DCT block

Podilchuk

Proposal Perceptual watermarking using the just noticeable difference (JND) to determine an image-dependent watermark modulation mask

wu ,v ; the computed JND based on visual models

JNDu ,v

I u ,v ; the transform coefficients of the original image

; the watermark values

Recovery- based on the correlation * Robust to JPEG compression, cropping, scaling and additive noise

Boland and Cox

Proposal Frequency-domain watermarking (First) perceptual adaptive methods which is based on modulation 1. Generate the watermark with statistical distribution ; e.g., N(0,1) 2. The watermark is inserted into the image

Cont* ; determine the strength * The watermark embedded 1000 strongest DCT coefficients

Detection- given by the normalized correlation coefficient * Boland propose a similar techniques DCT, DWT, Walsh-Hadamard, FFT

Summary

Several different image watermarking methods Most watermarking methods are based on the same basic principle - small, pseudorandom changes are applied to selected region; spatial or transform domain Recovery correlation-like similarity measures. Usually, the number of modified coefficient is much larger than the number of bits to be encoded Embedding domain have a influence on the robustness spatial less robust to noise like attack / E.g.- JPEG - more robust to cropping, translating. freq. less robust to cropping which destroy the embedding water mark (DCT,DFT,DWT)

Watermarking Techniques 3. Video Watermarking

Common idea for video watermarking

Video sequences consists of a series of consecutive and equally time-spaced still images in general, very similar with image watermarking so, image watermark method is applicable to video directly Important differences * available signal space; for image; very limited for video; much larger signal space (# of pixels)

*video watermark imposes real or near real-time watermarking system complexity issue is much more important

Cont

The structure of video as a sequence of images give rise to particular attacks frame averaging, frame dropping and frame swapping (Only in video) Two competing requirements *A good watermarking scheme 1. may recover the full watermark from a short part of the sequence 2. distribute watermark information over several consecutive frame to have robustness against frame dropping depends on application Compressed/ uncompressed video

Hartung and Girod

Proposal Watermarking of compressed video for fingerprint application * Used spread spectrum approach and added an additive watermark into video * The watermark is generated using a PN signal with the same dimension as the video signal *Each information bit is redundantly embedded into many pixels 1. For each compressed video fame, 8x8 DCT transformed watermark signal is added to DCT coefficient of the video

Cont*This method done for I,P, and B frames * Rate control- by comparing each encoded watermarked DCT coefficient versus the corresponding encoded unwatermarked coefficient

(because video steam use variable length coding, the watermarked signal may or may not need more bits encoding than the unwatermarked one)
If more bits required, the coefficient is not used for embedding

Recovery correlation using the same PN sequence used for generation of watermark

Jordan

Proposal Watermarking of compressed video that embed information in motion vector of motion -compensated prediction schemes. 1. Motion vectors slightly modified in pseudorandom way

Watermark embedding/detection available as long as the video is in compressed format After decompression? the watermark still be recovered by recompressing and detecting Artifacts? Because the blocks pointed to by the original and the modified vector are very similar no visible artifacts

Hsu and Wu

Proposal Watermarking for compressed video using middle-freq DCT * Extension of their image watermarking method 1. modifies middle freq DCT coefficients in relation to spatially (I-frames) or temporally ( for P- and B- blocks) neighboring block * Prior to embedding, the watermark signal is spatially scrambled to make it robust to cropping

Drawback- for watermark extraction, original video and watermark should be known

Langelaar

Proposal 2 methods 1) data hiding method 1. adds the label directly in the MPEG bit stream by replacing variable length codes (VLC) of DCT coefficient * In MPEG-2 code tables there are pairs of code which represent the same run and levels that deviate only by one from each other We can choose one as bit 1 the other as 0

Drawback- the label can be easily removed by decompression and recompression

Cont2) watermarking 1. for each bit to be embedded, a set of n 8x8-block is pseudorandomly taken from the video frame (here, n typically 16~64) 2. Seudorandomly divide into two subsets of equal size. 3. For each subset, the energy of the high-freq. DCT coefficient is measured 4. In order to embed the bit, the energy of the high-freq coefficient in one or the other subset is reduced by removing high freq. coefficients

Cont-

Block diagram of watermark embedding into DCT coefficients of compressed video

Recovery - 1. select the same set of blocks 2. divide it into the same subsets 3. Compare the energy of the high-freq. coefficient in each of the subsets

* Here, We can use the secret key for the selection of blocks

Cont

Drawbacks 1. robustness is limited 2. Re-encoding increases the error rate of the embedded signal much

Swanson

Proposal Multiscale watermarking method working on uncompressed video *Same scheme as Image watermarking 1. video sequence is segmented into scenes 2. temporal wavelet transform is applied to each scenes, yielding temporal low-pass and high-pass frames 3. the watermark is embedded into each of the temporal components (here, low-freq.) 4.Inverse transform the watermarked components to get the watermarked video

Cont

Interesting properties The watermark has some components that change over time, since they are embedded in low frequency coefficient this allow robustness against frame averaging, frame dropping

Drawback- very high complexity

Linnartz

Proposal Embed information in the GOP structure of the MPEG-2 compressed video

*There is a maximum distance between two successive I-frames

*The frame type signaled in the frame header and can be switched randomly from frame to frame.

Cont* A typical GOP in display order (N=12)

* If I-frame is fixed we have 2048 variations

*However, most available video codecs never use most of the admissible GOP structure purposely use irregular GOP structure, that are very unlikely, to embed information

Drawback- use this method only during compression - decompress and recompression will remove this information completely Merit - Low complexity

Darmstaedter

Proposal Embed a spatial-domain low-pass spread spectrum watermark into 8x8 pixel blocks of video sequences 1. The blocks are classified according to their activity *Low activity are not watermarked 2. Low-pass pseudorandom pattern is added to each block * Each block conveys one bit watermark information

Cont* The block repeated over several blocks and several frames for robustness

Recovery-done in spatial domain after decompression using correlation concept with threshold

Deguillaume

Proposal Spread spectrum watermark into 3-D block of video using 3-D DFT * Embed a spread spectrum watermark into 3-D blocks of video by employing a 3-D DFT and add to the transform coefficients

Busch

Proposal Apply a still-image watermarking method working on DCT blocks to video sequences 1. The watermarks are embedded into the luminance component of umcompressed video and retrieved after decompression *In order to improve the invisibility of the watermarks, especially at edge, blocks are selected depend on activity (high-activity block selected)

Cont

Recovery- 0~50% error rate are reported depend on the sequence the author propose to embed the watermark into several consecutive frames (over 50frame a few percent error rate reported)

Kalker

Proposal Video Watermark for video broadcasting monitoring application *Called JAWS (just another watermarking system) * For low complexity, both watermarking and detection are performed in the spatial domain watermarking before compression detection after decompression

Cont* The watermark size is 128x128 and repeated (tiled) to fill whole video frame * To avoid visible artifacts, the watermark is, on a pixel-bypixel basis, scaled with scale factor * Scale factor is derived from an activity measure * Activity measure is computed using a Laplacian high-pass filter

Detection- correlation detector is used - in case of presence of spatial shift, a search over all possible shift is performed (128x128 position)

Summary

The proposed methods span a wide complexity range from low complexity to considerable complexity In general, the more complex methods provide higher robustness Most methods operate on uncompressed video; a few methods embed watermark directly into compressed video e.g., DCT embedding or the motion vector embedding, GOP structure embedding

Watermarking Techniques 3. Audio Watermarking

Common ideas for audio watermarking

Compared to image and video, audio signal has much less samples per time interval amount of information which can be embedded is much lower HAS (Human Audible System) is much more sensitive than the HVS

Boney

Propose A spread spectrum approach * PN sequence is filtered in several stage to exploit masking effects of the HAS * The watermark is low-pass filtered by using full audio compression/decompression scheme to guarantee that it survive audio compression

Tilki and Beex

Propose interactive television application where they embed information into the audio components of a television signal 1. The information to be embedded is partitioned in blocks of 35bits 2. Each information bit is modulated using a sinusoidal carrier of a specific frequency with low amplitude and added to audio signal if the sinusoidal carrier for a specific bit is present=> 1 otherwise => 0 * The frequencies of the sinusoidal carrier are above 2.4 kHz To reduce interference from the audio signal , the audio signal is attenuated at frequencies above 2.4 kHz

Bender

Propose phase coding ; use the phase information as a data space 1. For encoding, a Fourier Transform is applied and the phase value of each freq. component are lined up as matrix 2. Binary information can be added into this matrix by modifying the phase component. * HAS is not very sensitive to the phase distortion of the sound

Watermarking Techniques 4. other Multimedia DATA

Ohbuchi

Propose embed visible and invisible watermarks into 3-D polygonal models * This model comprise primitives like points, lines, ploygons, and ployhedrons * Modify geometry or topology for watermarking

2 methods 1) pseudorandomly selects sets of four adjacent triangles * embed information by displacing the vertices of the four triangles up to 1% of the shortest edge of the rectangular bounding box of the entire 3-D model

Cont

2) pseudorandomly selects tetrahedron from the mesh and embeds information in the volume ratio of consecutive tetrahedron by modification of the vertices

Hartung

Proposal A spread-spectrum method for watermarking of MPEG-4 FAPs *MPEG-4 features model-based animation 3-D head modules using FAP (Facial Animation Parameter) *There are FAP like rotate head, open mouth or raise right cornerlip

1. In order to embed information, the parameters first have to be estimated from the sequence
2. The watermarks are embedded into the animation parameter
*Adaptive amplitude attenuation prevent visible distortion of animated head level.

Cont* Interesting point is that the watermark is not embedded in pixels but in the semantics( the way the head and face move)

Conclusion

We reviewed the most important aspects design requirements, system issues, and techniques for digital watermarking We have elaborated on numerous watermarking techniques for still images, video, audio, text, and other multimedia data. Majority of techniques are similar and based on modulation with a PN signal. Hypothesis test using correlation is used in the watermark recovery Although working systems are available, research has to continue

Thanks for your attention

- Questions? -