Bitstream-Corrupted JPEG Images are Restorable

: Two-stage Compensation and Alignment Framework

Authors: Wenyang Liu, Yi Wang, Kim-Hui Yap, Lap-Pui Chau

Affiliation: Nanyang Technological University, Singapore, and The Hong Kong Polytechnic University, Hong Kong
Conference: CVPR 2023

Presenter: 201911186 신지수

1. Problem Statement
열화된 JPEG 이미지 -
>
Traditional JPEG image restoration methods:
Focus on restoring degraded images, such as those with noise, downscaled,
blurred, or masked, back to high-quality original images.

<- 비트스트림이 손상된 JPEG 이미지

In the JPEG decoding process, bit errors can cause the decoding process
to stop, and the bits that follow cannot be decoded.

Bit errors in JPEG images can occur at any time

due to physical damage to storage media, transmission errors, software bugs, and other factors.
NAND 플래시 메모리

- 현대 휴대용 장치에서 데이터를 저장하는 데 널리 사용됨

- ECC (Error Correction Code) 의 오류 수정 능력 <<< 비트 에러율이면 , 복구된 데이터에 예상하지 못한 비트 에러를
발생시킴
- 최근 기술 트렌드에 의해 더 짧은 수정 , Raw Bit Error Rate 는 빠르게 증가중임

File Carving

- 신뢰할 수 없는 NAND 플래시메모리로부터 파일을 복구할 수 있는 필수적인 메모리 포렌식 기법

- 기존의 JPEG file carving 기법은 JPEG 파일 시스템의 메타데이터가 없는 경우를 다루며 , JPEG 파일 자체가 손상된
경우를 고려한 방법은 거의 없다 .

Android 5.0 의 full-disk encyption (FDE)

- 사용자의 프라이버시 보호를 위해 도입

- 안드로이드 장치가 암호화되면 모든 user-created data 가 자동으로 암호화되고 , 디스크에서 액세스하기 전에 자동으로 복호화된다 .
- 비트 에러들은 직접적으로 안드로이드 장치의 암호화된 데이터에 반영되며 , 이는 복호화된 파일의 비트 에러들을 더 심각하게 만든다 .
최근 , 딥러닝의 강력한 feature representation 능력으로 이미지 복구 문제에서 큰 성과를 보여줌

하지만 사진과 같이 예측할 수 없는 색 왜곡 , 비트 오류로 인한 Block Shifting 문제로 적합하지 않다

이 논문의 실험을 통해 다음과 같은 결과를 낼 수 있음
2. Solution
문제 상황의 해결을 위해 , 두 가지 단계를 거쳐 이미지를
복구한다 .

1. Robust Decoder
2. Two-stage Compensation and Alignment Framework
2.1. Robust Decoder

- Error-Resilient Mechanism
- Handling Decoding Failures:
- detects two main types of failures during the Huffman decoding process:
- 1. Encountering an invalid Huffman codeword that is not found in the standard JPEG tables.
- 2. Overflow of decoded coefficients in a block, exceeding the permissible count of 64.
- Self-Synchronization:
- 초기 오류가 디코딩 전체 과정에 영향을 끼치지 않게 함

오류가 포함된 비트스트림에서도 JPEG 이미지를 완전하고 정확하게 디코딩하여 이미지 복원 프로세스의 첫 단계로서 중요한 기능을
수행한다
2.2. Two-Stage Compensation
2.2.1. Self-Compensation and Alignment

- Segment Detection and Normalization: Divide

the image into segments and normalize the
brightness of each segment to share the same
DC shift.
- Block Alignment: Resolve block movement
issues by aligning each row's blocks with the
top row.

- Detect segments centered on DC error

propagation, adjusting brightness through
normalization.
- Use alignment algorithms to optimally shift
each row to its correct position.

More natural image restoration with corrected colors and positions.

Serve as a fine-tuning stage in the restoration process.
2.2.2 Guided-Compensation and Alignment

- Thumbnail Integration:
Use thumbnails, which are less affected by bit errors, to guide the restoration process.
- Pix2pix Network:
Combine the self-compensated image with thumbnails to provide initial guidance.
- Laplacian Pyramid Fusion Network:
Integrate high-frequency and low-frequency details at various resolutions to produce the final output.

• More natural image restoration with corrected colors and positions.

• Serve as a fine-tuning stage in the restoration process.
3. Experiment
Visual comparison with bicubic and SOTA super-resolution methods for low-resolution thumbnails at scale factor 6.

Comparison with SCA, SCA+EPDN, SCA+GCA>>

Visual comparison of the GCA results on varying BERs.
The bicubic upsampled thumbnail and the ground truth are given for comparison. .

.
4. Summary and Future Directions
Experimental Results and Efficacy
- Demonstrated superior restoration performance across benchmarks
involving various resolutions and bit error rates.
- Notably, it restores high-resolution 2K images with high precision
(achieving up to 38.92 dB PSNR), which is a significant improvement
of 5.52 dB over existing methods.

Future Research Directions

- Further explore the bitstream corruption issue and develop methods
capable of handling a broader range of decoding failure scenarios.
- Expand the generalizability of the technology to apply to various
forms of digital images and video files.
Significance and Contribution of the Study
- This research addresses the novel challenge of restoring JPEG
images with bit errors in encrypted bitstreams.
- Unlike traditional pixel-domain-based restoration methods, it
resolves unpredictable color shifts and block movements
occurring in the bitstream domain.
Thank You