Signal Processing: Image Communication 63 (2018) 149–160

Contents lists available at ScienceDirect

Signal Processing: Image Communication

journal homepage: www.elsevier.com/locate/image

A novel contrast enhancement forensics based on convolutional neural

networks✩
Jee-Young Sun, Seung-Wook Kim, Sang-Won Lee, Sung-Jea Ko *
Department of Electrical Engineering, Korea University, Anam-ro 145, Seongbuk-gu, Seoul, 02481, Republic of Korea

a r t i c l e i n f o a b s t r a c t

Keywords: Contrast enhancement (CE), one of the most popular digital image retouching technologies, is frequently utilized
Digital image forensics for malicious purposes. As a consequence, verifying the authenticity of digital images in CE forensics has recently
Contrast enhancement drawn significant attention. Current CE forensic methods can be performed using relatively simple handcrafted
Convolutional neural networks features based on first-and second-order statistics, but these methods have encountered difficulties in detecting
Deep learning
modern counter-forensic attacks. In this paper, we present a novel CE forensic method based on convolutional
Gray level co-occurrence matrix
neural network (CNN). To the best of our knowledge, this is the first work that applies CNN to CE forensics.
Unlike the conventional CNN in other research fields that generally accepts the original image as its input, in the
proposed method, we feed the CNN with the gray-level co-occurrence matrix (GLCM) which contains traceable
features for CE forensics, and is always of the same size, even for input images of different resolutions. By
learning the hierarchical feature representations and optimizing the classification results, the proposed CNN can
extract a variety of appropriate features to detect the manipulation. The performance of the proposed method
is compared to that of three conventional forensic methods. The comparative evaluation is conducted within a
dataset consisting of unaltered images, contrast-enhanced images, and counter-forensically attacked images. The
experimental results indicate that the proposed method outperforms conventional forensic methods in terms of
forgery-detection accuracy, especially in dealing with counter-forensic attacks.

1. Introduction into the processing history of an image [5]. Furthermore, since CE

As image and video editing techniques rapidly develop, image
manipulation has become an easy process that can be exploited for
malicious purposes, such as copyright infringement, and spreading false
information in the news media or litigation. In recent years, various
digital forensic methods have been proposed to verify the authenticity of
multimedia data. Digital image forensics identifies traceable statistical
artifacts left behind after an image alteration and distinguishes forgeries
from unaltered images. In general, image manipulation leaves unique
fingerprints on images; thus, most digital image forensic methods focus
on detecting different types of image manipulations, which are broadly
divided into two categories: (1) content-preserving operations including
resampling [1], compression [2], median filtering [3,4], and contrast
enhancement (CE) [5–7]; and (2) content-changing operations, such as
splicing and copy-move manipulation [8–10]. Although the content-
preserving operations may not pertain to malicious image tampering,
detecting these alteration is still forensically significant. Especially, the
detection of the globally applied CE manipulation can provide insight
is frequently employed to disguise the evidence of image tampering,
detecting such a manipulation can provide useful prior information
in the identification of content-changing operations. Thus, this paper
focus on the development of a forensic method of detecting the CE
manipulation.
Several CE forensic approaches [5–7] have been proposed in recent
years. Earlier CE forensic methods [5,6] utilized the manipulation
traces that can be observed in the 1D grayscale histogram of images.
These simple 1D histogram-based evidences, however, are easy to be
concealed after a further processing, which may render current CE
forensic methods unreliable. Techniques introduced for this purpose are
referred to as counter-forensics, or anti-forensics. Some anti-forensic
techniques [11,12] fool the 1D histogram-based CE forensic meth-
ods [5,6] by removing the visual clues in the 1D histogram through
adding a pixel-wise random noise to the image. To address such anti-
forensic methods, De Rosa et al. [7] proposed a CE forensic algorithm
to classify the unaltered, contrast-enhanced, and counter-forensically

✩
This paper has supplementary downloadable material available at http://doi.org/10.6084/m9.figshare.5160982.v1, provided by the authors. This includes .mat
format dataset and .py format source codes for training and testing the proposed CNN for contrast enhancement forensics.
* Corresponding author.
E-mail addresses: jysun@dali.korea.ac.kr (J.-Y. Sun), swkim@dali.korea.ac.kr (S.-W. Kim), swlee@dali.korea.ac.kr (S.-W. Lee), sjko@korea.ac.kr (S.-J. Ko).

https://doi.org/10.1016/j.image.2018.02.001
Received 27 September 2017; Received in revised form 11 January 2018; Accepted 1 February 2018
Available online 12 February 2018
0923-5965/© 2018 Elsevier B.V. All rights reserved.
J.-Y. Sun et al.
attacked images employing a 3-class support vector machine (SVM)
training scheme. Nevertheless, a recently developed optimization-based
anti-forensic attack [13] performs CE without a significant distortion
of the first-and second-order statistics, which effectively deceives the
conventional CE forensic methods. The aforementioned conventional
CE forensic methods [5–7] utilize handcrafted features based on the
observed visual clues and have separate feature extraction and classi-
fication stages that cannot be simultaneously optimized in an iterative
scheme. As a result, most conventional methods exhibit unsatisfactory
performance when detecting images that are manipulated by counter-
forensic attacks, even though they can detect commonly contrast-
enhanced images fairly well.
To cope with not only common CE manipulations but also counter-
forensic attacks, we propose a novel CE forensic method based on a
deep learning framework. Convolutional neural network (CNN) [14]
is a deep multi-layer neural network that is motivated by how human
brains process visual information. It learns feature representations by
using convolution kernels and automatically fulfills the classification.
Then, the classification result is utilized to guide the feature extraction
through a back-propagation algorithm. CNNs have shown impressive
performance in modern artificial intelligence (AI) tasks that include
object detection and segmentation [15], and recently, the CNN has been
applied to a number of approaches [16–22] regarding digital image
forensics. These approaches focus on the detection of specific image
manipulation techniques such as median filtering [16,17], Gaussian
blurring [17], JPEG compression [18–20], and some content-changing
operations [21,22]. In these methods, an input image itself or a specific
form of input is fed into the CNN structure, and the appropriate features
are extracted through an iterative training scheme. In the proposed
method of the present study, the GLCM is utilized as the input to the
CNN structure.
From the perspective of CE forensics, input images of different
resolutions need to be resized to have a fixed input resolution of the
CNN structure. The image resizing process, however, damages the pixel
intensity information, and this may be harmful for CE forensics in the
extraction of traceable features. Image cropping does not change the
pixel intensity, but the one-by-one checking of the cropped patches of
the input image by the CNN could be a cumbersome process. Moreover,
the features that are trained with the input image itself are more related
to the image content rather than the manipulation evidence. Therefore,
in the proposed method, instead of resizing or cropping the input image,
the GLCM of the input image is computed and fed into the CNN. The
GLCM is obtained by accumulating the occurrence of the pixel intensity
pairs between each pixel and its neighboring pixels. Besides, the GLCM
can always be obtained with the same size, even if the input images have
different resolutions. A detailed explanation on the GLCM is provided
in Section 3.1.
In the proposed CNN-based CE forensic method, instead of construct-
ing better-handcrafted features, a wide range of global and local features
that can distinguish between the GLCMs of unaltered images and
those of the forgeries are automatically learned during optimizing the
classification results. To the best of our knowledge, this work is the first
CE forensic attempt that employs a CNN and uses the GLCM as the input
to the CNN. The trained features from the GLCM are more powerful than
the previous handcrafted features and are even more accurate than the
features that are trained using the image itself. To verify the effective-
ness of the GLCM, we compared the forgery detection performance of the
deep learning-based CE forensic models, which are trained using three
different types of inputs (i.e., resized input image, randomly cropped
input image, and the GLCM of an input image). Then, the performance
of the proposed method was compared with that of three state-of-
the-art CE forensic methods. This comparative evaluation is conducted
within the dataset consisting of unaltered images, three different types
of images processed by common CE technologies, and images attacked
by using three state-of-the-art counter-forensic approaches [11–13].
The experimental results show that the proposed method handles not
150
Signal Processing: Image Communication 63 (2018) 149–160
only the common CE manipulations but also the recent counter-forensic
attacks. Moreover, the proposed method outperforms the conventional
ones in terms of forgery detection accuracy. The remainder of this paper
is organized as follows. Section 2 briefly explains the previous works on
CE forensics and deep-learning based digital image forensics. Section 3
describes the proposed CNN-based CE forensic method. Experimental
settings and results are provided in Sections 4 and 5, respectively, while
Section 6 concludes the paper.
2. Related works
2.1. Contrast enhancement forensic approaches
Several approaches [5–7] have been proposed to identify whether
a digital image has been processed by CE or not. Earlier CE forensic
methods [5,6] observed the differences in the 1D grayscale histogram of
the unaltered image and that of the manipulated one. Stamm and Liu [5]
maintained that the 1D histogram of a pure image has a smooth envelope
in general, whereas that of a contrast-enhanced image has a ragged
envelope that contains peaks and valleys. In this method, the Fourier
transform of the 1D histogram is performed and the high frequency
energy F is calculated. The target image is identified as a contrast-
enhanced one when F is larger than a certain threshold value. Cao
et al. [6] introduced a slightly different observation on the 1D histogram
of the contrast-enhanced image. Since the histogram peaks can occur
by other image processing techniques such as JPEG compression, they
considered only the zero-height gap in the 1D histogram as the trace
of CE. In this method, the number of gap bins N in the 1D histogram
is counted; then if N is larger than a certain threshold, the image
is classified as a contrast-enhanced one. These simple 1D histogram-
based features, however, are easy to be removed by processing with the
counter-forensic techniques.
To address this, De Rosa et al. [7] tried to classify the images into the
unaltered, contrast-enhanced, and counter-forensically attacked images
by introducing a SVM training-based CE forensic algorithm that uses
GLCM. In this method, a 3-class SVM is utilized for the classification,
and the feature space for SVM training is a 1D histogram calculated by
accumulating the variance value of each row of the GLCM. Although
this work is the first attempt to apply second-order statistics to CE
forensics, it is difficult to say that the 2D information was well-utilized
because the GLCM was only used to generate another simple 1D feature.
The aforementioned CE forensic methods [5–7] are based on manually
selected features and have separate stages for feature extraction and
classification, thereby yielding unsatisfactory performance in detection
of the images manipulated of counter-forensic attacks.
2.2. Deep-learning based digital image forensics
Although deep learning based CE forensics has not been studied, a
number of attempts wherein the deep learning models were applied
have been made to address the other manipulations in digital image
forensics. One of the first works that employed CNNs for image forensics
is [16]. In this paper, the authors developed a CNN-based median
filtered image detector. To better expose the manipulation traces, a
pre-processing step was added in this case to obtain the median filter
residual, which is defined as the difference between the input image
and its median filtered output. By feeding the residual into the CNN,
the detector could extract the forgery-related features and achieved a
performance that is superior to the one that is trained by the direct use
of an image as the input. In [17], instead of the preprocessing step, Bayer
and Stamm proposed a constrained convolutional layer that enables the
CNN to focus on the local relationships among the pixels; the layer is
constrained to suppress the image content so that the content-dependent
features cannot be learned. This method can deal not only with median
filtering, but also with other filter-based alterations such as Gaussian
J.-Y. Sun et al.
blurring, the addition of additive white Gaussian noise (AWGN), and
the bilinear interpolation.
Wang et al. [18] proposed a CNN-based double JPEG compression
detector that exploits a 1D feature vector, a histogram consisting of 99
bins, as the CNN input. They first generated nine histograms of the
DCT coefficients that had been arranged in a zigzag order at the first
nine alternating current (AC) sub-bands. For each sub-band, only the
coefficient values in the range of {−5, −4, −3, −2, −1, 0, 1, 2, 3, 4, 5} are
considered to be accumulated to a histogram. So the concatenation of
the nine histograms with 11 bins resulted in the 1D histogram with 99
bins. In [19], a multi-domain CNN is introduced to distinguish between
uncompressed, single-compressed, and double-compressed images. The
multi-domain CNN consists of the spatial-domain CNN that receives
the input as a three-channel color image and the frequency-domain
CNN that accepts the 1D histogram of the DCT coefficients as the
input. Two feature vectors that are obtained from the two CNNs are
combined, thereby making the final decision on the JPEG compression.
When the CNN models are trained with the DCT coefficients, these two
methods use 1D convolutional CNN whose input is not a 2D image. Barni
et al. [20] proposed a 2D convolutional CNN-based JPEG compression
detector that can accept the DCT images as the CNN input. They also
performed an experiment to compare the effects of three different inputs
(i.e., mean subtracted image, noise residual image, and DCT feature
image) for both aligned and non-aligned double JPEG compressed
images.
Some approaches that deal with content-changing alterations have
been introduced as well. Bondi et al. [21] presented a CNN for the detec-
tion of the splicing region by performing a camera model identification.
In this work, the forgery is defined as the image that is created by
pasting one or more patches onto a pristine image, and it is assumed
that the patches and the image have been taken from different camera
models. The input image is first divided into several non-overlapping
patches, and each patch is fed into the CNN to be classified into its
corresponding camera model. Then, a clustering algorithm is employed
to estimate the splicing region by using the patch-wise classified labels
and the confidence values that are obtained from the CNN. Finally, a
CNN model which performs a two-stage classification for the detection
of counter-forensic source anonymization attacks is proposed in [22].
At the first stage, the CNN model determines whether the image is
authentic or counter-forensically modified. Then, the model fulfills
a multi-class classification to identify the specific attack among the
seam carving, fingerprint copying, and adaptive photo response non-
uniformity (PRNU) denoising.
3. Materials and methods
In this section, we first comment on the GLCM, including the problem
of using the original image as the CNN input and how the GLCM can
deal with such a problem. After describing the acquisition of the GLCM,
the next subsection further explains the architecture of the proposed
network and the loss function.
3.1. Gray level co-occurrence matrix
Images of different resolutions should be resized to a specific size to
be fed into the CNN because it only accepts a fixed-sized input. Image
resizing and cropping are the common techniques to resize an image.
However, when the image resolution is changed by using an image
resizing technique, the pixel intensity may change slightly, which can
damage manipulation traces for CE forensics. For example, an unaltered
image in Fig. 1(a) and its contrast-enhanced image in Fig. 1(b) are
resized by the same scale using bicubic interpolation, resulting in two
outputs as shown in Fig. 1(c) and (d), respectively. Fig. 1(e) and (f)
illustrate the randomly cropped images of Fig. 1(a) and (b), and their
corresponding 1D histograms are depicted in Fig. 2. The histogram of
the resized and contrast-enhanced image in Fig. 2(d) no longer has
151
Signal Processing: Image Communication 63 (2018) 149–160
as many peaks and zeros as that of the original-sized and contrast-
enhanced image in Fig. 2(b). In Table 1, the two features employed
in the conventional CE forensic methods [5,6] (i.e., the high frequency
energy metric F and the number of zero-gap bins N ) are provided to
see how the resizing process affects the traceable features. In the case of
the original-sized images, the differences between the unaltered image
and the manipulated one are large enough for both features, and this
allows to distinguish the forgery from the pure images. However, the
conspicuous differences disappear after resizing, which indicates that
the image resizing negatively affects the manipulation traces for CE
forensics.
Although F and N of the cropped images are still reliable to identify
the authenticity as shown in Table 1, image cropping could be a
cumbersome process in a real world scenario, because the input images
are cropped into several patches and the cropped patches should be
repeatedly fed into the CNN one-by-one to detect the manipulation.
Furthermore, features that are extracted from original image are more
related to image content rather than the trace of manipulation. Thus,
instead of the images, we utilize GLCMs that are more forensically
informative as the CNN input in the proposed method.
GLCM represents the 2D distribution of the pixel intensity pairs
consisting of the intensity values of a pixel and its neighboring pixels.
Given a grayscale image I of size 𝑀 × 𝑁 and an offset (𝛥𝑥, 𝛥𝑦) , GLCM is
obtained by accumulating the occurrences of a gray-value pair (𝑖, 𝑗) of
two pixels that are separated by the offset. Ignoring the computation of
image boundaries, GLCM is defined as:
∑ 𝑀 ∑
∑ 𝑁
𝐺𝐿𝐶𝑀 (𝑖, 𝑗) = I (𝐼 (𝑝, 𝑞) = 𝑖 ∧ 𝐼 (𝑝 + 𝛥𝑥, 𝑞 + 𝛥𝑦) = 𝑗) , (1)
(𝛥𝑥,𝛥𝑦)∈𝑶 𝑝=1 𝑞=1
where I (⋅) is an indicator function that returns 1 if the condition
in the parentheses is satisfied, and 0 otherwise. GLCM offers the
following three advantages: (1) GLCM is obtained by using pixel
intensities of an input image, without any changes to the intensity
values. (2) GLCM represents the relationships between the intensities
of each pixel and other pixels in its local neighborhood, which can
be a useful information for CE forensics. (3) GLCM can always be
obtained in the same size, even for the input images of different
resolutions. We first compute the GLCMs with eight different offsets,
𝑂 = {(0, 1), (0, −1), (1, 0), (−1, 0), (1, 1), (1, −1), (−1, 1), (−1, −1)}, which
represent the relations between the center pixel and the pixels in its
8-directional neighborhood, and then element-wise sum of the eight
GLCMs results in one accumulated GLCM. In our method, the quanti-
zation level of GLCM is set to 256, whereby the pixel intensity value
of the center pixel 𝑖 and that of its neighboring pixel 𝑗 in (1) are in
the range of [0, 255], and therefore, the size of the GLCM is fixed to
256 × 256. Then, the accumulated 1-channel GLCM is fed into the CNN
instead of the input image. To verify the effectiveness of using GLCM, we
compared the performance of deep learning-based CE forensic models
that are trained using three different types of inputs: the resized image,
the cropped image, and the GLCM of an image. The corresponding
experimental results will be discussed in Section 5.1.
Fig. 3 shows the GLCMs for the unaltered, contrast-enhanced, and
counter-forensic-attack images. The GLCMs for the contrast-enhanced
images in Fig. 3(b)–(d) have conspicuous traces, including the transi-
tions from/to peaks and zeros, constructing grid-like patterns. On the
other hand, there is no noticeable difference in appearance between
the GLCM of the pure image in Fig. 3(a) and the GLCMs of the
images after anti-forensic techniques which are shown in Fig. 3(e)–(g).
Thus, conventional handcrafted features which are selected by human
observation are not sufficient to distinguish unaltered images from anti-
forensic-attack images. In the proposed CNN-based CE forensic method,
the CNN first fulfills classification with the use of initial features,
and the classification loss is back-propagated to update the weights
of convolution kernels. Then, the updated convolution kernels extract
new filter responses (i.e., local or global features of the GLCM for CE
forensics, and these features are utilized to perform the classification
J.-Y. Sun et al. Signal Processing: Image Communication 63 (2018) 149–160

Fig. 1. Original-sized images, corresponding resized images, and corresponding randomly cropped images. (a) Unaltered image, (b) contrast-enhanced image after
gamma correction is applied, (c) resized image of (a), (d) resized image of (b), (e) cropped image of (a), and (f) cropped image of (b).

Fig. 2. 1D histograms of the images in Fig. 1. (a) Histogram of Fig. 1(a), (b) histogram of Fig. 1(b), (c) histogram of Fig. 1(c), (d) histogram of Fig. 1(d), (e) histogram
of Fig. 1(e), and (f) histogram of Fig. 1(f).

Fig. 3. Gray level co-occurrence matrix of (a) unaltered image, contrast-enhanced images after (b) histogram stretching (HS), (c) gamma correction (GC), and
(d) S-curve mapping (SM), and images manipulated by the following counter-forensic attacks: (e) Gaussian dithering (GD) [11], (f) internal bit depth increasing
(IBD) [12], and (g) total variation optimization (TVO) [13].

Table 1
Comparison of CE forensic features before and after image resizing and cropping.
Feature Original-sized Resized Cropped
Unaltered Enhanced Unaltered Enhanced Unaltered Enhanced
𝐹a 0.39 365.11 0.41 0.23 0.26 108.15
𝑁a 1 22 1 1 1 22
a
Features for CE forensics; F = high frequency energy and N = number of gap bins.

152
J.-Y. Sun et al.
at the next iteration. By repeating these classification and updating of
the kernel parameters, the proposed CNN can learn further features that
are hard to be visible to human eyes from the GLCMs. The proposed
CE forensic method, therefore, achieves reliable performance to detect
both the commonly contrast-enhanced images and those processed by
counter-forensic attacks.
Notably, most of the information is distributed along the diagonal
line of the GLCM, as shown in Fig. 3, because the intensity values
of adjacent pixels tend to be similar. Inspired by this observation, we
additionally trained the proposed CNN using the partial GLCM that
is cropped along the diagonal direction, and the performance will be
described in Section 5.2.
3.2. CNN architectures for CE forensics
The proposed CNN architecture is presented in Fig. 4. The grayscale
GLCM image with a size of 256 × 256 is fed into the network, and then
low-level features are extracted using convolutional kernels. The first
convolutional layer convolves the GLCM with 50 kernels of size 7 × 7.
The size of the output is 25 × 256 × 50, which means that the number of
feature maps is 50 and the resolution of each feature map is 256 × 256.
Following every convolution layer, rectified linear unit (ReLU) is used
as the element-wise non-linear activation function in our work. After
ReLU activation, a max pooling operation is performed with a window
size of 2 × 2, and a stride of 2, leading to the same number of feature
maps with a decreased spatial resolution, 128 × 128 ×50. The feature
maps become inputs for the next convolutional layer, and the same
operations (i.e., convolution, activation, and max pooling) are repeated.
Then, an output with a size of 32 ×32 × 50 passes through the two fully
connected layers with the length of 100 and it is fed into the last layer
of the CNN, where the softmax function is used for classification. At
the end of every fully connected layer, Batch normalization (BN) [23]
and ReLU activation are applied. Since BN can act as a regularizer, the
dropout layer and other regularization terms are not used in this work.
The softmax function, which returns the class probability, is defined as
follows:
( ) 𝑒𝑧𝑗
𝜎 𝑧𝑗 = ∑𝐾 for 𝑗 = 1, … , 𝐾, (2)
𝑧𝑘
𝑘=1 𝑒
where K is the number of classes, z is the vector consisting of K
output values
( ) of the last layer, and the softmax probability of the 𝑗th
class, 𝜎 𝑧𝑗 , is in the range of [0, 1]. The proposed CNN is trained
for both two-class and three-class classification. The two-class (𝐾 =
2) classification detects manipulated images from unaltered images,
whereas the three-class (𝐾 = 3) classification categorizes images into
three groups: unaltered images, common contrast-enhanced images, and
those attacked by counter-forensic techniques.
Adam optimizer is utilized as the optimization solver with the
hyper-parameters recommended in [24]. The objective loss function is
formulated by using a cross-entropy loss of the softmax class probability
defined as:
∑
𝐾
( )
𝐿=− 𝑦𝑗 log 𝜎 𝑧𝑗 , (3)
𝑗=1
where 𝑦𝑗 ∈ {0, 1} is the ground truth label of the 𝑗th class.
Fig. 5 shows the visualization of 50 convolution kernels in the
first convolutional layer of the aforementioned CNN trained using the
GLCM as the input. These filters look more structured rather than being
random; some filters have strong horizontal/vertical edges and cross-
like patterns, while others have relatively flat patterns. Note that the
GLCMs of contrast-enhanced images have some grid-like patterns, while
that of the unaltered image does not have any, as shown in Fig. 3.
Thus, the GLCMs of unaltered images show high and continuous filter
responses after the convolutions with relatively flat filters, while the
convolution kernels with strong edges generate high responses when
they are convolved with the grid-like regions in the GLCMs of contrast-
enhanced images. Examples and analysis of the convolutional kernels
and their corresponding feature maps will be discussed in Section 5.1.
153
Signal Processing: Image Communication 63 (2018) 149–160
4. Experimental setup
Two experiments were conducted to evaluate the performance of
the proposed CNN-based CE forensic method. First, a performance
comparison was conducted among three deep neural network models for
CE forensics that utilize the following different types of inputs: resized
images, cropped images, and the GLCM of the images. In the second
experiment, the forgery detection performance of the proposed method
is compared with that of three state-of-the-art CE forensic methods. In
this section, training, validation, and test dataset are first explained, and
then the experimental cases and comparison targets are introduced. The
implementation details including the parameter settings and training
strategies are described in the last subsection.
4.1. Dataset
To evaluate the CE detection performance of the proposed method,
we generated a dataset by using 5000 randomly chosen images from
MS COCO dataset [25], which is frequently used for object detection
and segmentation purposes. After converting the images to the YUV
color space, the Y -channels of the images were assigned as the unaltered
image data. Then, each of the 5000 Y -channel images was manipulated
by three popular CE techniques, histogram stretching (HS), gamma
correction (GC), and S-curve mapping (SM), resulting in a total of 15,000
contrast-enhanced images. In HS, the input pixel values were linearly
mapped, such that 1% of the total pixels were saturated at an intensity
value of 0, and another 1% at an intensity value of 255. In the case of GC,
we split the 5000 images into four groups of 1250 images and applied
the GC with different gamma values (𝛾 = 0.5, 0.8, 1.2, 1.5) for each
group. Also, three recently-proposed counter-forensic attacks, Gaussian
dithering (GD) [11], internal bit-depth increasing (IBD) [12], and total
variation optimization (TVO) [13] were applied to the Y -channel images
to generate the counter-forensic-attack images. For the entire seven
types of images, the GLCMs were computed and used as inputs for the
proposed network. In total, we obtained 35,000 GLCM data acquired
from 5000 unaltered, 15,000 enhanced, and 15,000 attacked images.
For each type of GLCM data, we randomly selected 80% for training
and validation and assigned the remaining 20% as the test dataset.
4.2. Experimental cases and comparison targets
Regarding the first experiment, the three different types of CNN
inputs, the resized images, cropped images, and the GLCM of the images,
were fed into the same network architecture, and their corresponding
CE forensic models were trained by means of transfer learning, which
performs fine-tuning using a pre-trained model. We utilized Inception-
ResNet-v2 (IRN-v2) [26], one of the most popular base networks in the
research field of object classification and detection, as the pre-trained
model, and the performance evaluation was conducted under the first
and second experimental cases, as shown in Table 2. In Cases 1 and 2,
we measured the performance of the binary classification between the
unaltered and manipulated images. However, the forgery of each case is
of a different type; the CNN models tried to detect the contrast-enhanced
and counter-forensic-attack images in the first and second experimental
cases, respectively. In Case 1, four types of GLCM that had been obtained
by the unaltered images and the images that were processed by the HS,
GC, and SM were used. In Table 2, the GLCMs of the unaltered images
were assigned with ‘‘U’’, and those of the contrast-enhanced images was
labeled as ‘‘E’’. Similarly, Case 2 was also performed using four types of
data, but in this case, the GLCMs of the unaltered images and those of the
images attacked by three anti-forensic techniques, which were assigned
with the label ‘‘A’’, were used to evaluate the detection performance.
In the next experiment, the forgery detection performance of the
proposed method was compared with that of the three state-of-the-art
CE forensic methods. In contrast to the first experiment, two additional
experimental cases, which are indicated by Cases 3 and 4 in Table 2,
J.-Y. Sun et al. Signal Processing: Image Communication 63 (2018) 149–160

Fig. 4. CNN architecture of the proposed method.

Fig. 5. Visualization of the convolution kernels in the first convolutional layer.

Table 2
Experimental cases with different data distributions.
Type Unaltered HSa GCa SMa GDb IBDb TVOb
# of Train+Val 4000 4000 4000 4000 4000 4000 4000
data Test 1000 1000 1000 1000 1000 1000 1000
Case 1 Uc Ec E E – – –
Case 2 2-class U – – – Ac A A
Case 3 U Fc F F F F F
Case 4 3-class U E E E A A A
a
Common CE techniques; HS=histogram stretching, GC=gamma correction, and SM=s-curve mapping.
b
State-of-the-art counter-forensic methods; GD=Gaussian dithering [11], IBD=internal bit depth increasing
[12], and TVO=total variation optimization [13].
c
Types of image used for each case; U=unaltered images, E=contrast-enhanced images,
A=anti-forensic-attack images, and F=forgery images (E ∩ A).

were added. In the third case, all seven types of GLCM were divided
into the following two classes: unaltered images and images that are
assigned with ‘‘F’’. In this case, the GLCMs of the common contrast-
enhanced images and those of the images that had passed through
the counter-forensic techniques were considered together as forgeries.
While the two-class classification was performed for the three previously
mentioned cases, Case 4 evaluated the performance of the three-class
classification. In this case, all of the GLCM data were classified into the
following three classes: unaltered images, common contrast-enhanced
images, and those generated by anti-forensic attacks.
The comparison targets for the second experiments [5–7] are im-
plemented in MATLAB. The two handcrafted features, high frequency
energy F and the number of gap bin N, are calculated according
to the equations provided in [5] and [6], respectively. Then, each
method identifies the authenticity of input images by using a specific
threshold. In [6], the threshold value for N is predetermined as 0,
which means the image is verified as a contrast-enhanced one when
there are one or more gap bins. Since the threshold value for F has not
been specified in [5], it was explored heuristically among 20 different
values from 0 to 2 with equal intervals. After measuring the accuracy
of classifying the unaltered and contrast-enhanced images varying the
threshold, the best performing threshold of 1.4 was finally selected
for further experiments. The images processed by the counter-forensic
attacks were not considered in the determination of threshold, because
the conventional method [5] is introduced to discriminate unaltered
images from common contrast-enhanced images only. The CE forensic
methods in [5] and [6] always classify the images into two categories,
the unaltered image versus the forgery. Thus, these two methods were
evaluated for the first three experimental cases only. On the other
hand, the SVM training-based CE forensic method [7] can perform a
three-class classification as well as a binary classification, and thus
the performance of this method was assessed in all four experimental
cases. The handcrafted feature proposed in [7] was extracted by using
the source code which is downloadable on the authors’ website [27].

154
J.-Y. Sun et al.
After the feature extraction, SVM classifier was implemented by using
MATLAB built-in functions. The SVM classifier was trained and tested
using the same dataset used for the proposed method.
4.3. Implementation details
This section discusses the implementation details including hyper-
parameter settings and training strategies. For each type of input in the
first experiment, two models that are trained using different ways of
transfer learning, were utilized. One model (IRN-v2-LL) was obtained
by replacing the last layer of the IRN-v2 into a new fully connected
layer that is connected to two output nodes for the binary classification,
and the parameters in the last layer were trained only; the other pre-
trained parameters of IRN-v2-LL were not changed. Fig. 6 illustrates
the network architecture of the IRN-v2-LL. The colored blocks in Fig. 6
are the pre-defined modules of the INR-v2, and a detailed description
of these modules is explained in [26]. The network architecture of the
other model (IRN-v2-FT) is the same as shown in Fig. 6, but all of the
parameters in this model were fine-tuned from the IRN-v2. We used the
default hyper-parameter setting of the IRN-v2 in the transfer learning,
except for the preprocessing function that performs random cropping.
For both models, the training session was ceased after 5000 iteration
steps.
Regarding training of our proposed CNN structure which is shown
in Fig. 4, the weights in the network were initialized with Xavier
initialization [28], and there was no pre-training stage. We trained
the model using Adam optimizer [24] with 𝛽1 = 0.9, 𝛽2 = 0.999,
and the initial learning rate of 0.001, which are the default settings
recommended in [24]. Also, the mini-batch training was carried out with
different batch sizes depending on the experimental cases. In Cases 1 and
2 that use four types of GLCM as the dataset, we trained the model with
the batch size of 40 containing 10 GLCM inputs for each type. When
training the CNN in the remaining two experimental cases, 70 GLCM
inputs were grouped into one batch since all seven types of GLCM were
used in these cases. To determine the learning rate schedule, we trained
the model and monitored the convergence of the classification loss on
the validation data, by using the k-fold cross validation with 𝑘 = 10.
As mentioned above, there were seven types of GLCM input, each of
which contained 5000 GLCMs. For each type, 4000 GLCM inputs that
were used for the cross validation were equally divided into 10 folds,
and subsequent training and validation of the model were performed
10 times. At each iteration, a single fold was held-out for validation,
while the remaining 9 folds were used for training. We trained the
model with an initial learning rate of 0.001 for the first 5 epochs and
decreased the learning rate by a factor of 10 for the next 10 epochs. After
setting the learning rate schedule, the data used for cross validation was
randomly split into a ratio of 3:1 for training and validation. By training
the proposed network with the learning rate schedule mentioned above,
the model with the minimum classification loss on the validation set
was selected as the final model. The training data was not shuffled
between epochs in our work. The proposed method was implemented on
Tensorflow [29], an open source deep learning library recently released
by Google, and the proposed network was trained on a single NVIDIA
Titan X Pascal GPU.
5. Experimental results
For the quantitative analysis of the proposed CNN-based CE forensic
method, a comparison of the receiver operating characteristics (ROC)
curves and the area under the ROC curve (AUC) was performed over
all of the experimental cases. The ROC curve plots the true positive
rate (TPR) versus the false positive rate (FPR), where the TPRs and the
FPRs are obtained by varying the detection threshold. In this work, the
forged and unaltered image were considered as positive and negative,
respectively, because the main purpose of a CE forensic method is the
detection of the manipulated images. In addition, we compute the TPR,
155
Signal Processing: Image Communication 63 (2018) 149–160
Table 3
Comparison of detection performance (Case 1: Unaltered images vs. Contrast-
enhanced images).
Method TNR TPR ACC
Unaltered HS GC SM
Stamm and Liu [5] 0.896 0.969 1 1 0.966
Cao et al. [6] 0.558 1 0.977 0.899 0.859
De Rosa et al. [7] 0.955 0.878 0.974 0.966 0.943
Cropped GLCM 0.998 0.986 1 1 0.996
Proposed 0.996 0.988 0.998 0.999 0.995
the true negative rate (TNR), and the total accuracy (ACC) for each CE
forensic method in the second experiment using each method’s decision
criterion. The TPR, TNR, and ACC are defined as follows:
𝑇𝑃
TPR = , (4)
𝑇𝑃 + 𝐹𝑁
𝑇𝑁
TNR = , (5)
𝑇𝑁 + 𝐹𝑃
𝑇𝑃 + 𝑇𝑁
ACC = , (6)
𝑇𝑃 + 𝐹𝑁 + 𝑇𝑁 + 𝐹𝑃
where TP, TN, FP, and FN represent the numbers of the true positive,
true negative, false positive, and false negative samples, respectively. A
proficient forgery detector should exhibit a high ACC, and this is possible
when both the TPR and the TNR are high. To calculate the previously
mentioned metrics, the finally predicted class of the proposed CNN-
based CE forensic method is determined as the one with the maximum-
value softmax probability. The resultant TPR, TNR, and ACC values are
provided in Tables 3–6.
5.1. Performance analysis depending on the CNN inputs
Prior to the performance comparison, we provide a discussion on the
convolutional kernels that are trained using three different types of input
and their corresponding feature maps. Figs. 7–9 illustrate the filters
of the first convolutional layer and their corresponding feature maps
obtained after the CNN models were trained with resized input images,
cropped input images, and the GLCM of input images, respectively.
To compare the manner in which the appearance of the feature maps
change according to the image content and the manipulation, the filter
responses of four different images, two unaltered images and their
contrast-enhanced versions, are provided. The intra-class similarity and
the inter-class similarity of the feature maps should be high and low,
respectively, to achieve an enhanced forgery detection performance.
That is, in Figs. 7–9, the feature maps for the unaltered images in the
columns (b) and (d) should be similar to each other, and the appearances
of those for the forgeries in the columns (c) and (e) should also be
similar.
The feature maps shown in Figs. 7 and 8 are image-related; they
seem to be blurred, brightness-changed, or edge images. Thus, the filter
responses of the columns (b) and (c) are similar, and the appearance of
those in the column (d) are similar to those in the column (e), which
implies that these feature maps depend on the image content rather
than the manipulation. On the other hand, the feature maps in Fig. 9
obtained from the CNN that had been trained using the GLCM as the
input show different patterns. The first filter in Fig. 9(a) is relatively
flat, which exhibits high responses when it is convolved with the flat
regions in the GLCM. Thus, the filter responses from the GLCM of the
unaltered images in Fig. 9(b) and (d) are continuous, whereas those from
the GLCM of the contrast-enhanced images in Fig. 9(c) and (e) showed
uneven responses. Also, the feature maps generated using the second
and third convolutional kernels in Fig. 9(a) exhibited high responses
on the grid-like regions in the GLCM of the contrast-enhanced images.
As shown in Fig. 9, the CNN that is trained using the GLCM generates
feature maps with higher intra-class and lower inter-class similarities
compared with the other input types. This implies that the GLCM is an
effective CNN input for the detection of the CE manipulation.
J.-Y. Sun et al.
Fig. 6. Network architecture of IRN-v2-LL and IRN-v2-FT.
Fig. 7. Convolutional kernels (a) of the CNN trained using resized input images and their corresponding feature maps generated from (b) unaltered image 1, (c)
contrast-enhanced image 1, (d) unaltered image 2, and (e) contrast-enhanced image 2.
Fig. 10 shows the ROC curves and their corresponding AUC values
for the models learned through the transfer learning from IRN-v2.
Compared with the ROC curves from IRN-v2-LL in Fig. 10(a) and (b),
those from IRN-v2-FT in Fig. 10(c) and (d) provides better performance
in terms of AUC for the CE forensic model using the GLCM input. As
shown in Fig. 10(a) and (c), the models trained using the GLCM input
outperforms those with other input types in Case 1. However, as shown
in Fig. 10(b) and (d), the GLCM does not show much superiority in
Case 2 that evaluates the ability to distinguish the images that are
manipulated after counter-forensic attacks from the unaltered images;
in fact, even the models trained using the GLCM in Case 2 yields unsat-
isfactory performance because the traceable features in GLCM has faded
out during passing through the deep layers in the IRN-v2 architecture.
A more detailed analysis will be discussed in Section 5.2.2 with the
experimental result of our proposed CNN for the same experimental
case. Although the advantage of using GLCM as the CNN input has
rarely shown in Case 2, the ROC curves in Fig. 10(a) and (c) imply
that extracting the manipulation clues from the GLCM would be more
effective than tracing it in the resized or cropped images.
5.2. Performance comparison with the state-of-the-art CE forensic methods
To verify the practical effectiveness of the CNN-based CE forensic
approach, a comparative evaluation of the proposed method and the
156
Signal Processing: Image Communication 63 (2018) 149–160
three state-of-the-art CE forensic algorithms was performed under the
four experimental cases mentioned in Section 4.2. In this experiment,
the partial GLCM with a size of 64 × 256 was acquired by cropping the
GLCM along the diagonal direction, and it was utilized as another CNN
input, as mentioned in Section 3.1. The performance of the CE forensic
model that was trained using the cropped GLCM was also compared
with the others in all of the experimental cases. The ROC curves are
depicted in Fig. 11. Since the three-class classification was done in Case
4, two sets of the ROC curves were provided, as shown in Fig. 11(d)
and (e), which were drawn in consideration of the contrast-enhanced
and counter-forensic-attack images as positive samples, respectively. A
detailed explanation on the experimental result for each case will be
discussed in the following subsections.
5.2.1. Case 1: Unaltered images vs. Contrast-enhanced images
In the first case, the test dataset consisted of four types of GLCM input
obtained from the unaltered images and the contrast-enhanced images
that are processed by the HS, GC, and SM. The ROC curves for Case 1 are
shown in Fig. 11(a). Both the ROC curves depicting the performance of
the proposed CNN models trained using the original-sized and partial
GLCM show significantly high performance. Table 3 compares the
detection performance in terms of TPR, TNR, and ACC for each type
of image and the highest ACC value and the second-highest one are
J.-Y. Sun et al. Signal Processing: Image Communication 63 (2018) 149–160

Fig. 8. Convolutional kernels (a) of the CNN trained using cropped input images and their corresponding feature maps generated from (b) unaltered image 1,
(c) contrast-enhanced image 1, (d) unaltered image 2, and (e) contrast-enhanced image 2.

Fig. 9. Convolutional kernels (a) of the CNN trained using GLCM of input images and their corresponding feature maps generated from (b) unaltered image 1,
(c) contrast-enhanced image 1, (d) unaltered image 2, and (e) contrast-enhanced image 2.

written in bold and underlined, respectively. Although the proposed
method took the first place showing the highest mean accuracy, other
methods yielded acceptable performance as well, because the commonly
contrast-enhanced images have sufficient traces of CE manipulation.
5.2.2. Case 2: Unaltered images vs. Anti-forensic-attack images
In contrast to the results of Case 1, Table 4 and the ROC curves
in Fig. 11(b) show that all conventional methods are fooled in Case 2
since the images attacked by anti-forensic techniques no longer have the
striking evidence. As shown in Table 4, the forensic method in [5] shows
very low TPR values for images attacked by GD [11] and IBD [12], which
means most attacked images are classified as unaltered ones. Similarly,
the conventional method [6] suffers from difficulties in detecting the
TVO-attacked images. Besides, in almost every type of image, the CE
forensic methods in [6] and [7] have the TPR and the TNR values of
about 0.5, indicating that these algorithms randomly fulfill the classifi-
cation regardless of the input image. Meanwhile, the proposed algorithm
outperforms the conventional methods in terms of the ACC having high
TPR and TNR. Note that both the proposed method and the conventional
method introduced by De Rosa et al. [7] utilize the GLCM in the first
stage of the CE detection. Nevertheless, the two methods exhibited
significantly different performance. In the conventional method [7],
the 1D histogram of variance value calculated from every row of the
GLCM is considered as the only feature space for the SVM classifier. By

157
J.-Y. Sun et al. Signal Processing: Image Communication 63 (2018) 149–160

Fig. 10. ROC curves and their corresponding AUC values for the models learned by transfer learning from IRN-v2. (a) IRN-v2-LL in case 1, (b) IRN-v2-LL in case 2,
(c) IRN-v2-FT in case 1, and (d) IRN-v2-FT in case 2.

Table 4 impressive performance in the research field of object recognition and

Comparison of detection performance (Case 2: Unaltered images vs. Anti- segmentation. From the perspective of CE forensics, however, it is more
forensic-attack images). important to find the manipulation clues that are hidden in the local
Method TNR TPR ACC region of the GLCM than to observe the traces with the global contextual
Unaltered GD IBD TVO information. Also, a blurring effect arises from numerous mathematical
Stamm and Liu [5] 0.896 0.013 0.135 0.681 0.432 operations in the IRN-v2 such as convolution and pooling layers, and this
Cao et al. [6] 0.558 0.467 0.463 0.035 0.465 may negatively affect the local information, resulting in a performance
De Rosa et al. [7] 0.506 0.520 0.509 0.586 0.530 degradation in the classification of the images manipulated by using
Cropped GLCM 0.777 0.975 0.871 0.997 0.905
counter-forensic attacks from the unaltered images.
Proposed (GLCM) 0.801 0.981 0.912 0.998 0.923

5.2.3. Case 3: Unaltered images vs. Manipulated images

contrast, in the proposed method, the effective features of the GLCM
input are extracted by the convolutional kernels, and the parameters
for the feature extraction and classification are learned simultaneously
by the iterative end-to-end training. This integrated process enables
the proposed method to extract not only the visually distinguishing
features of the GLCM but also the features which are hard to detect
based on the human perception. The experimental results verified that
the proposed CNN-based CE forensic model exhibits a high performance
in the detection of the images processed by anti-forensic attacks, and
thus the deep features trained by the proposed CNN are more powerful
than the handcrafted features in [7].
As shown in Fig. 11(b), the red ROC curve and the AUC value of
the proposed method are higher than the black curve and its AUC
that correspond to the CE forensic model trained with the cropped
GLCM. This indicates that the features of the whole-sized GLCM are
more appropriate than those of the diagonally cropped GLCM, thereby
yielding a better performance. Although it seems that less or no in-
formation exists outside of the diagonal region, traceable CE forensic
features do exist, and the proposed CNN using the whole-sized GLCM
can effectively extract them through the iterative training scheme. In
addition, the CE forensic model trained with the proposed CNN structure
provides a greatly enhanced performance than the models fine-tuned
from the IRN-v2 in Fig. 10(b) and (d). In general, the deeper layers of
CNN structure have wider receptive fields, thereby allowing the CNN
to capture further global contextual information. Indeed, the contextual
information that can be obtained from the DNN structures has shown an
In the third experiment, the contrast-enhanced and anti-forensic-
attack images were considered together as manipulated images. The
TPR, TNR, and ACC in classifying the unaltered images and the forgeries
are given in Table 5 and the ROC curves for Case 3 are depicted in
Fig. 11(c). The threshold-based methods [5,6] provided exactly the
same TPR and TNR as Cases 1 and 2. The other two methods, the
SVM training-based [7] and the proposed CNN-based one, however, had
different TPR and TNR because CE forensic models were re-trained in
these methods by using the new dataset that is different from Cases
1 and 2. Also, all of the ROC curves in Fig. 11(c) are different from
those in Fig. 11(a) and (b) because the positive dataset in Case 3
contains both the enhanced and attacked images. In Fig. 11(c), the red
and black ROC curves that correspond to the proposed method show
higher performance than the conventional methods yielding higher AUC
values. In Table 5, for the conventional methods, the TPR values of
detecting the forgeries obtained by GD [11], IBD [12], and TVO [13] are
lower than those of detecting other types of images. This implies that
the conventional methods are not robust against the counter-forensic
attacks. However, the proposed method exhibits high performance in
the detection of the attacked image as well, showing the highest ACC
value written in bold.
5.2.4. Case 4: Unaltered images vs. Contrast-enhanced images vs. Anti-
forensic attack images
The last experimental case was to classify the test images into three
classes, unaltered images, contrast-enhanced images, and anti-forensic-
attack images. For each method, we computed the confusion matrices

158
J.-Y. Sun et al. Signal Processing: Image Communication 63 (2018) 149–160

Fig. 11. ROC curves and their corresponding AUC values for the proposed CNN-based CE forensic methods and state-of-the-art CE forensic methods. (a) Case 1:
unaltered vs. contrast-enhanced images, (b) Case 2: unaltered vs. anti-forensic-attack images, (c) Case 3: unaltered vs. manipulated images, and Case 4: unaltered
vs. (d) contrast-enhanced vs. (e) anti-forensic-attack images.

Table 5
Comparison of detection performance (Case 3: Unaltered images vs. Manipulated images).
Method TNR TPR ACC
Unaltered HS GC SM GD IBD TVO
Stamm and Liu [5] 0.896 0.969 1 1 0.013 0.135 0.681 0.671
Cao et al. [6] 0.558 1 0.977 0.899 0.467 0.463 0.035 0.628
De Rosa et al. [7] 0.672 0.888 0.939 0.963 0.339 0.350 0.423 0.653
Cropped GLCM 0.740 0.994 1 1 0.973 0.895 0.998 0.943
Proposed (GLCM) 0.810 0.989 0.999 0.999 0.984 0.905 0.998 0.955

Table 6
Comparison of detection performance (Case 4: Unaltered images vs. Contrast-enhanced images vs. Anti-forensic-
attack images).
Method Class TNR TPR (Enhanced) TPR (Attacked) ACC
Unaltered HS GC SM GD IBD TVO
Unaltered 0.478 0.075 0.033 0.031 0.459 0.447 0.410
De Rosa et al. [7] Enhanced 0.017 0.806 0.927 0.916 0.012 0.024 0.078 0.671
Attacked 0.505 0.119 0.040 0.053 0.529 0.529 0.512
Unaltered 0.753 0.008 0 0 0.024 0.126 0.006
Cropped GLCM Enhanced 0.003 0.988 0.999 0.999 0 0 0 0.940
Attacked 0.244 0.004 0.001 0.001 0.976 0.874 0.994
Unaltered 0.821 0.011 0.001 0 0.026 0.097 0.005
Proposed Enhanced 0.001 0.987 0.997 0.999 0 0 0.001 0.954
Attacked 0.178 0.002 0.002 0.001 0.974 0.903 0.994

on the test dataset, as reported in Table 6. In the confusion matrices, elements written in bold indicate the proportion of correctly classified
each column corresponds to the real label of the test images, while each images. For example, the values in ‘‘TPR (Enhanced)’’ column represent
row indicates the predicted label of the test images. Thus, the diagonal the proportions of the test data predicted as the contrast-enhanced

159
J.-Y. Sun et al.
images. In this column, all of the methods have high rates in the second
rows and very low rates in the other two rows, indicating that the
entire methods detected common contrast-enhanced images fairly well.
However, for the classification of the unaltered and attacked images,
the conventional and proposed method show a significant difference in
their performance. In Table 6, the conventional method [7] has low
TNR and TPR values for detecting the unaltered and counter-forensic-
attack images, respectively. More specifically, the conventional method
showed the proportions of correctly classified labels that are close to
0.5 for ‘‘TNR’’ and ‘‘TPR (Attacked)’’ columns; the other half are falsely
classified as each other, implying that the conventional method had dif-
ficulties in distinguishing between unaltered images and anti-forensic-
attack images. On the other hand, the proposed method provides high
accuracy values in both ‘‘TNR’’ and ‘‘TPR (Attacked)’’ columns. This can
be seen from the ROC curves in Fig. 11(d) and (e) as well. As the ROC
curve can be defined for binary classification, we provide two sets of
ROC curves which are drawn by using two different positive datasets:
the enhanced images in Fig. 11(d) and the attacked images in Fig. 11(e).
The performance of our proposed method is significantly high for both
ROC curves in Fig. 11(d) and (e). The conventional method, however,
shows high performance only for Fig. 11(d) and yields unsatisfactory
performance in detection of anti-forensic attacks in Fig. 11(e). In other
words, the proposed method achieved better performance on the entire
types of test images, especially in discrimination between the images
manipulated by counter-forensic techniques and the unaltered images.
6. Conclusion
In this paper, we proposed a novel CNN-based CE forensic method.
In our method, the GLCM of the input image is fed into the CNN. Deep
features are then extracted from the GLCM by using convolution kernels,
and the CE forensic classification is carried out based on these features.
Via back-propagation algorithm, the classification results help guide
for updating the convolution kernel parameters, thereby generating a
new feature space for the classification at the next iteration. As such,
the proposed CNN simultaneously optimizes the feature extraction and
classification in an iterative training scheme. The forgery detection
performance of the proposed method was compared to that of three
state-of-the-art CE forensic methods and the test dataset consisted of
the unaltered, common contrast-enhanced, and counter-forensic-attack
images. The proposed method outperformed the conventional ones in
terms of the forgery detection accuracy, especially in the identification
of the state-of-the-art counter-forensic attacks that were based on the
1D histogram and the 2D GLCM. Furthermore, the proposed CNN
can be fine-tuned by using an extended dataset including the GLCM
of the images processed by any other CE manipulation techniques.
Therefore, the proposed network and deep-learning based CE forensics
framework are applicable to identify potential anti-forensic attacks
arise in the future. In conclusion, the proposed CNN-based CE forensic
method achieved a significant improvement in performance regarding
CE forgery detection, demonstrating a robust CE forensic method against
the state-of-the-art anti-forensic techniques.
Acknowledgements
This work was supported by Institute for Information & commu-
nications Technology Promotion (IITP) grant funded by the Korea
government (MSIP) (2014-0-00077, Development of global multi-target
tracking and event prediction techniques based on real-time large-scale
video analysis).
References
[1] A.C. Popescu, H. Farid, Exposing digital forgeries by detecting traces of resampling,
IEEE Trans. Signal Process. 53 (2005) 758–767.
[2] W. Luo, J. Huang, G. Qiu, JPEG error analysis and its applications to digital image
forensics, IEEE Trans. Inf. Forens. Security 5 (2010) 480–491.
160
Signal Processing: Image Communication 63 (2018) 149–160
[3] X. Kang, M.C. Stamm, A. Peng, K.J.R. Liu, Robust median filtering forensics using
an autoregressive model, IEEE Trans. Inf. Forens. Security 8 (2013) 1456–1468.
[4] H. Yuan, Blind forensics of median filtering in digital images, IEEE Trans. Inf. Forens.
Security 6 (2011) 1335–1345.
[5] M.C. Stamm, K.J.R. Liu, Forensic detection of image manipulation using statistical
intrinsic fingerprints, IEEE Trans. Inf. Forens. Security 5 (2010) 492–506.
[6] G. Cao, Y. Zhao, R. Ni, X. Li, Contrast enhancement-based forensics in digital images,
IEEE Trans. Inf. Forens. Security 9 (2014) 515–525.
[7] A.D. Rosa, M. Fontani, M. Massai, A. Piva, M. Barni, Second-order statistics analysis
to cope with contrast enhancement counter-forensics, IEEE Signal Process. Lett. 22
(2015) 1132–1136.
[8] X. Zhao, S. Wang, S. Li, J. Li, Passive image-splicing detection by a 2D noncausal
Markov model, IEEE Trans. Circuits Syst. Video Technol. 25 (2015) 185–189.
[9] I. Amerini, L. Ballan, R. Caldelli, A.D. Bimbo, G. Serra, A SIFT-based forensic method
for copy-move attack detection and transformation recover, IEEE Trans. Inf. Forens.
Security 6 (2011) 1099–1110.
[10] J. Li, X. Li, B. Yang, X. Sun, Segmentation-based image copy-move forgery detection
scheme, IEEE Trans. Inf. Forens. Security 10 (2015) 507–518.
[11] G. Cao, Y. Zhao, R. Ni, H. Tian, Anti-forensics of contrast enhancement in digital
images, in: Proceedings of the 12th ACM workshop Multimedia and Security, Roma,
Italy, Sep. 9–10, 2010.
[12] C.W. Kwok, O.C. Au, S.H. Chui, Alternative anti-forensics method for contrast
enhancement, in: International Workshop Digital Watermarking, Atlantic City, NY,
Oct. 23–26, 2011.
[13] H. Ravi, A.V. Subramanyam, S. Emmanuel, ACE-an effective anti-forensic contrast
enhancement technique, IEEE Signal Process. Lett. 23 (2016) 212–216.
[14] A. Krizhavsky, I. Sutskever, G.E. Hinton, ImageNet classification with deep convo-
lutional neural networks, in: Proceedings of the 25th International Conference on
Neural Information Processing Systems, Lake Tahoe, Nevada, Dec. 3–6, 2012.
[15] Y. Guo, Y. Liu, A. Oerlemans, S. Lao, S. Wu, M.S. Lew, Deep learning for visual
understanding: A review, Neurocomputing 187 (2016) 27–48.
[16] J. Chen, X. Kang, Y. Liu, Z.J. Wang, Median filtering forensics based on convolutional
neural networks, IEEE Signal Process. Lett. 22 (2015) 1849–1853.
[17] B. Bayar, M.C. Stamm, A deep learning approach to universal image manipulation
detection using a new convolutional layer, in: Proceedings of the 4th ACM Workshop
on Information Hiding and Multimedia Security, Vigo, Galicia, Spain, Jun. 20–22,
2016.
[18] Q. Wang, R. Zhang, Double JPEG compression forensics based on a convolutional
neural network, EURASIP J. Info. Security 23 (2016).
[19] I. Amerini T. Uricchio, L. Ballan, R. Caldelli, Localization of JPEG double compres-
sion through multi-domain convolutional neural networks. in: IEEE Conference on
Computer Vision and Pattern Recognition Workshops, Honolulu, Hawaii, Jul. 21–26,
2017.
[20] M. Barni, L. Bondi, N. Bonettini, P. Bestagini, A. Costanzo, M. Maggini, B. Tondi, S.
Tubaro, et al., Aligned and non-aligned double JPEG detection using convolutional
neural networks, J. Vis. Commun. Image Represent. 49 (2017) 153–163.
[21] L. Bondi, S. Lameri, D. Güera, P. Bestagini, E.J. Delp, S. Tubaro, et al., Tampering
detection and localization through clustering of camera-based CNN features, in:
IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu,
Hawaii, Jul. 21–26, 2017.
[22] V.U. Sameer, R. Naskar, N. Musthyala, K. Kokkalla, Deep learning based counter-
forensic image classification for camera model identification, in: International
Workshop on Digital Watermarking, Magdeburg, Germany, Aug. 23–25, 2017.
[23] S. Ioffe, C. Szegedy, Batch normalization: Accelerating deep network training by re-
ducing internal covariate shift, in: Proceedings of the 32th International Conference
on Machine Learning, Lille, France, Jul. 6–11, 2015.
[24] D.P. Kingma, J. Ba, Adam: A method for stochastic optimization, in: International
Conference for Learning Representations, San Diego, CA, May 7–9, 2015.
[25] T.Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, C.L.
Zitnick, et al., Microsoft COCO: Common objects in context, in: Proceedings of the
13th European Conference on Computer Vision, Zurich, Switzerland, Sep. 6–12,
2014.
[26] C. Szegedy, S. Ioffe, V. Vanhoucke, A.A. Alemi, Inception-v4, Inception-ResNet and
the impact of residual connection on learning, in: Proceedings of the 31st AAAI
Conference on Artificial Intelligence, San Francisco, CA, Feb. 4–9, 2017.
[27] A.D. Rosa, M. Fontani, M. Massai, A. Piva, M. Barni, Demo second order CE detection.
[Source code]. Available: https://lesc.dinfo.unifi.it/it/node/187.
[28] X. Glorot, Y. Bengio, Understanding the difficulty of training deep feedforward
neural networks, in: Proceedings of the 13th International Conference on Artificial
Intelligence and Statistics, Sardinia, Italy, May 13–15, 2010.
[29] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G.S. Corrado, A.
Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M.
Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Leveberg, D. Mane, R. Monga,
S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K.
Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viegas, O. Vinyals, P. Warder,
M. Wattenberg, M. Wicke, Y. Yu, X. Zheng, et al., TensorFlow: Large-scale machine
learning on heterogeneous systems. 2015, Software available from tensorflow.org.
[Online]. Available: http://tensorflow.org/.