Knowledge-Based Systems 194 (2020) 105554

Contents lists available at ScienceDirect

Knowledge-Based Systems
journal homepage: www.elsevier.com/locate/knosys

Visual object tracking with adaptive structural convolutional network✩

∗
Di Yuan a ,1 , Xin Li a ,1 , Zhenyu He a,b , , Qiao Liu a , Shuwei Lu a
a
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, China
b
Peng Cheng Laboratory, Shenzhen 518055, China

article info a b s t r a c t

Article history: Convolutional Neural Networks (CNN) have been demonstrated to achieve state-of-the-art performance
Received 2 October 2019 in visual object tracking task. However, existing CNN-based trackers usually use holistic target samples
Received in revised form 20 January 2020 to train their networks. Once the target undergoes complicated situations (e.g., occlusion, background
Accepted 22 January 2020
clutter, and deformation), the tracking performance degrades badly. In this paper, we propose an
Available online 24 January 2020
adaptive structural convolutional filter model to enhance the robustness of deep regression trackers
Keywords: (named: ASCT). Specifically, we first design a mask set to generate local filters to capture local
Visual tracking structures of the target. Meanwhile, we adopt an adaptive weighting fusion strategy for these local
Convolution neural network filters to adapt to the changes in the target appearance, which can enhance the robustness of the
Structural filters tracker effectively. Besides, we develop an end-to-end trainable network comprising feature extraction,
Adaptive weighting decision making, and model updating modules for effective training. Extensive experimental results
on large benchmark datasets demonstrate the effectiveness of the proposed ASCT tracker performs
favorably against the state-of-the-art trackers.
© 2020 Elsevier B.V. All rights reserved.

1. Introduction improves the calculation efficiency. However, due to the bound-

Visual object tracking plays a crucial role in the computer vi-
sion community that finds numerous applications such as
human–computer interaction, motion analysis, action recogni-
tion, video surveillance, and autonomous driving, to name only
a few. Given the initial information in the first frame, the track-
ing task aims to locate the target in the subsequent frames
accurately. Although many algorithms have been proposed and
have made much progress in recent years, tracking remains a
challenging problem due to the realistic and complicated tracking
environment.
Recently, discriminative correlation filters (DCFs) have been
introduced into the target tracking task and shown a pretty
tracking speed and accuracy [1–8]. Existing DCFs-based methods
perform candidate generation through cyclic shifts from a search
image patch. According to the Convolution Theorem, the convo-
lution calculation in the time domain can be transformed into
element-wise multiplication in the Fourier domain, which greatly
✩ No author associated with this paper has disclosed any potential or
pertinent conflicts which may be perceived to have impending conflict with
this work. For full disclosure statements refer to https://doi.org/10.1016/j.knosys.
2020.105554.
∗ Corresponding author at: School of Computer Science and Technology,
Harbin Institute of Technology, Shenzhen, China.
E-mail address: zhenyuhe@hit.edu.cn (Z. He).
1 Contribution equally.
ary effects caused by the Fourier transform and the synthetic
samples generated by the cyclic shift, the tracking performance
of these DCFs-based trackers is far from the actual demand [9–
14]. With the emergence of the large-scale dataset, CNN-based
trackers have shown their great capacity in tracking task which
greatly improves the tracking performance [15–22]. In general,
CNN-based methods pre-train their networks on a large-scale
dataset (e.g., ImageNet [23]) and finetune the networks with the
information of the first frame in the tracking sequence. This
effectively avoids the boundary effects and also increases the
tracking result efficiency. Meanwhile, due to the efficiency of
DCFs-based trackers, some CNN-based trackers also attempt to
embed correlation filters as a layer into their convolutional neural
networks to improve the tracking efficiency [24–26]. In [25], Chen
and Tao proposed a convolutional regression network for visual
tracking and solve the regression problem by optimizing a one-
channel-output convolution layer. Song et al. [26] proposed to
reformulate DCFs as a one-layer convolutional neural network
in tracking task. However, the convolutional filters trained with
holistic samples cannot accurately model the local structures of
the tracking target, which are crucial for robust and accurate
tracking performance.
On the other hand, the importance of each local structure is
changing along with the variations of the target. It is crucial to
adjust the weights of the local parts to focus on the more discrim-
inative parts constantly. Ma et al. [28] utilized features from three
convolutional layers to exploit both semantic information and

https://doi.org/10.1016/j.knosys.2020.105554
0950-7051/© 2020 Elsevier B.V. All rights reserved.
2 D. Yuan, X. Li, Z. He et al. / Knowledge-Based Systems 194 (2020) 105554
Fig. 1. Comparisons of the proposed tracker (ASCT) and other representative trackers (DeepSRDCF [15], C-COT [16], CREST [26] and SiamFC [27]) on several
challenging sequences. These trackers perform differently as various features, network structures, and scale estimation strategies are used. The proposed ASCT tracker
performs favorably against these tracking sequences.
spatial details for visual tracking. The weights of these correlation
filters are fixed during the whole tracking process. In [26], Song
et al. developed a spatiotemporal residual network for target
tracking, but in their tracking network the base layer, the spatial
residual layer, and the temporal residual has the same weight.
Several other tracking methods [3,29] generate attention maps
to adjust the weights of different areas, however, the attention
without exact boundaries may corrode the local structures.
To address the above-mentioned issues, we propose an adap-
tive structural convolutional filter network for visual target track-
ing. The local filters are generated by adding binary masks to
the original filter whose size is the same as the target size.
Specifically, each local filter is performed by setting the mask
value of the interesting areas as 1 and others as 0. Considering
the variations of the target, we design an adaptive weighting
mechanism to focus on the more discriminative local filters. For
each local filter, we combine the peak sidelobe ratio and the
Laplacian distribution to adaptively determine the corresponding
weight. And then, we integrate the local filter layer and the
original filter layer to make the tracker have better robustness
and tracking performance. We find that the integrated adaptive
structural convolutional filters networks usually predict the ac-
curate scale and get compact bounding boxes as shown in Fig. 1.
The main contributions of this paper are as follows:
• We propose an adaptive structural convolutional filter net-
work for visual target tracking. The local filter layer can
effectively capture the structural patterns of the target and
can easily integrate with the original filter layer, which can
generate more accurate and robust tracking results.
• We develop an adaptive weighting strategy to improve the
stability of the target tracking framework, which can be used
to determine the corresponding weight by applying the peak
sidelobe ratio and the Laplacian distribution on the local
filter.
• We conduct extensive experiments on the OTB-2015, VOT-
2016, UAV-123 and TC-128 datasets. These experimental re-
sults demonstrate that our ASCT tracker performs favorably
against the state-of-the-art trackers.
The rest of this paper is structured as follows. We first in-
troduce some related works in Section 2. Next, we propose the
adaptive structural convolutional filter network for visual object
tracking, including the introduction of the basic tracker, the struc-
tural convolutional filters model, the adaptive weighting strategy
and the whole tracking process in Section 3. Subsequently, we
introduce the implementation details and the evaluation crite-
rion, evaluate and discuss our approach on some comprehensive
benchmark datasets in Section 4. Finally, we briefly present the
conclusion of our work in Section 5.
2. Related works
In this section, we introduce the tracking methods closely
related to our work in the proper context. A comprehensive
review of the tracking methods is beyond the scope of this paper,
and some survey papers can be found in [30–32].
DCFs-based trackers. DCFs-based trackers have been widely
used in recent years. As a pioneering work, Bolme et al. [9]
proposed a MOSSE filter for visual tracking. Although the MOSSE
tracker with a fast-tracking speed which nearly 670 frames per
second, the tracking performance is not satisfactory.
Subsequently, several variants of the MOSSE tracker have been
proposed to improve the tracking performance [1,4,10–13,33,34].
Henriques et al. [1,10] propose to adopt kernel methods and
HOG features into correlation filters framework for accurately
target tracking. In [13], Danelljan et al. used color features to
replace grayscale features for target representation, which can
improve the tracking performance. For fast and robust tracking,
Zhang et al. [33] exploited the dense spatio-temporal context
information in their STC tracker. In the correlation filters frame-
work, generating training samples by using a cyclic shift will
bring boundary effects, which can affect tracking performance
to a certain extent. To handle these boundary effects, Galoogahi
et al. [4,11] proposes took benefit of intrinsic computational re-
dundancy in the Fourier domain and gives a BACF tracker, which
can efficiently modulate the varies of a target in the foreground
and background. Some spatially regularized DCFs [2,15] track-
ers adopt large space support to learn their correlation filters,
 D. Yuan, X. Li, Z. He et al. / Knowledge-Based Systems 194 (2020) 105554
Fig. 2. The framework of the proposed algorithm. We generate structural responses with local convolution filters. Then, we fuse the structural responses with the
base response from a holistic filter to generate the final response map. The weights of the local filters are adaptively updated to focus on more discriminative parts.
which can effectively reduce the boundary effect. Although these
trackers have an acceptable tracking speed, there is still have a
problem by using the holistic target samples to train their model
in complex tracking scenarios. To address these drawbacks, we
propose an adaptive structural convolutional network to improve
the tracking performance, which can adaptively weight the local
target information to strengthen the representational ability of
the model.
Part-based trackers. These trackers described above are sus-
ceptible to noise interference and reduce its stability. To make
the tracker more robust, several patch-based correlation filter
trackers have been proposed [35–42]. Liu et al. [35] proposed a
patch-based tracking method with multi-CFs models. The com-
bination of multiple parts can effectively deal with the effects of
noise. Li et al. [43] introduced a reliable part-based tracking al-
gorithm, which attempts to evaluate and utilize the reliable parts
to tracking the object through a whole tracking process. In [44],
a deformable part-based correlation filter tracking approach has
been proposed to deal with long-term tracking tasks that depend
on coupled interactions between a global filter and several part
filters. Deformable parts models show great potential in tracking
by principally addressing nonrigid object deformations and self
occlusions [45], but a potentially large number of degrees of
freedom have to be estimated for object localization and sim-
plifications of the constellation topology are often assumed to
make the inference tractable. Lukezic et al. [46] presented a new
formulation of the constellation model with correlation filters
that treat the geometric and visual constraints within a single
convex cost function and derived a highly efficient optimization
for maximum a posteriori inference of a fully connected con-
stellation. For different tracking targets, Rasmussen et al. [47]
proposed a framework by combining and sharing information
among several state estimation processes operating on the same
underlying visual targets. These part-based tracking algorithms
all improve the robustness of the tracker to varying degrees.
Nevertheless, the integration strategy of these trackers often uses
a fixed weight to determine the weight of each patch, which
cannot show the advantages of the part-based tracker well. In this
paper, we develop an adaptive weighting strategy to mitigate this
defect, which can adaptively apply the peak sidelobe ratio and the
Laplacian distribution to determining the corresponding weight
of each local filter.
CNN-based trackers. In recent years, deep learning-based
tracking methods are gaining popularity. Convolutional neural
network (CNN) [19,23,48–52] is the most fashionable deep learn-
ing model in visual tracking, due to the formidable capability
on feature extraction and representation. As demonstrated em-
pirically in [53–55], features play the most important role in
the tracking method. Since hand-crafted features show natural
defects in the tracking process, Huang et al. [56] proposed an
adaptive approach to select cheap features and deep features
3
and formulate this adaptive tracking task as a decision-making
process. In [29], Choi et al. presented a context-aware deep
feature compression framework to achieve a real-time tracking
performance. The CNT [57] tracker uses a two-layer feed-forward
convolutional network to generates an effective representation of
the tracking target which achieves high-speed performance even
on a CPU operator. Li et al. [50] introduced a DeepTrack method,
which employed a CNN architecture to evaluates the similarity
of the target and these candidate samples. The representation
ability of features extracted from a single CNN layer can be im-
proved by extracted from multi-layers. Inspired by this discovery,
Ma et al. [28] proposed a hierarchical convolutional features
framework for visual object tracking. Gao et al. [58] proposed
an end-to-end network model to learn reinforced attentional
representation for accurate target tracking. Qi et al. [17] proposed
to use an adaptive Hedge method to hedge several CNN trackers
into a strong one. In [59] Gao et al. designed a lightweight
Siamese network that can capture more global and local con-
textual information at multiple scales. In [60], a TFCR tracking
framework have been proposed to balance the disequilibrium of
positive and negative samples. All of these trackers train their
networks with the holistic target, and when the target is occluded
or disturbed by noise, the tracking results of there trackers will be
greatly reduced. Different from the aforementioned trackers, our
ASCT tracker uses an adaptive structural convolutional network
to adaptively enhance target representation, which can improve
the tracking performance effectively.
3. Adaptive structural convolutional networks
In this section, we introduce the adaptive structural convo-
lutional network for visual tracking task. Firstly, we give a brief
introduction to the baseline tracker. Then, we present the struc-
tural convolutional filter network. Furthermore, we describe the
adaptive weighting strategy to adjust the weight of different local
filters effectively. Finally, we show the tracking pipeline of the
proposed tracker, which is shown in Fig. 2.
3.1. Base framework
CREST [26] reformulates DCFs as a one-layer convolutional
neural network and integrates feature extraction, response map
generation, and model update into neural networks for end-
to-end training. This method brings end-to-end training to the
convolutional neural network and achieves effective training. We
use the base layer of the CREST network as our base framework.
DCFs-based tracking method usually learns a discriminant
classifier to predict the target center by searching for the maxi-
mum response value [1,4]. Formally, learning the correlation filter
is to solve the following minimization problem:
W = arg min ∥W ∗ X − Y ∥2 + λ∥W ∥2 , (1)
W
4 D. Yuan, X. Li, Z. He et al. / Knowledge-Based Systems 194 (2020) 105554
Fig. 3. Example of the implementation of the local part convolution layer.
where W denotes the correlation filter, X denotes the input
samples, Y is the corresponding Gaussian label, and λ denotes the
regularization parameter.
The CREST tracker reformulates the learning process of DCFs
as the loss minimization of the convolutional neural network and
a weight decay term with an ℓ 2 regularization.
L(W ) = LW (X ) + λ∥W ∥ , 2
2
where LW (X ) = ∥F (X ) − Y ∥ . F (X ) is the network output and Y is
the ground-truth label. It is the base layer in the CREST network.
Our ASCT tracker is the improvement of the base layer of the
CREST network without residual learning. More details about the
CREST tracker can be found in [26].
3.2. Structural convolutional filters
Although the CREST tracker achieves good performance, its
convolutional filters trained with holistic samples are less effec-
tive for capturing the discriminative local structures of the target.
The local structures play a significant role in tracking perfor-
mance, especially during complicated tracking scenes (e.g., back-
ground clutter, occlusion, deformation). In order to enhance the
robustness and accuracy of the deep regression tracking frame-
work, we propose an adaptive structural convolutional filter net-
work for visual tracking.
Intuitively, the target can be divided into several local parts,
and the response outputs of each local filter are calculated sepa-
rately. If the occlusion or interference occurs in some local areas,
the target position can still be located accurately through other
unobstructed or undisturbed parts. Therefore, based on defining
DCFs as a convolutional layer, we decompose the convolutional
filter into M local parts, and then use a convolutional with a
part filter set to calculate the corresponding response map for
each local regions. The part filters are combined with the base
convolution layer to improve the robustness of the tracker.
Specifically, the tracker needs to predict the position of the
target in subsequent frames based on the given target position
in the first frame. After extracting the features, the tracker feeds
the features into the base layer and the local filters layer. These
two layers calculate the responses correspondingly. Then the final
response is computed as:
W t = φ ∗ Bt + ϕ ∗ V t ,
where φ and ϕ denotes the weight factors of the base layer and
the part-fused layer, W t denotes the final response output at tth
frame, Bt denotes the base layer response output at tth frame,
V t denotes the fused response output of the M local filters at tth
frame.
M
∑
Vt = γmt ∗ Pmt , (4)
m=1
where γmt denote the parameter of mth local filter at tth frame,
(2) Pmt
denotes the response output of mth local filter at tth frame.
For each local filter, the size of the convolution kernel is the
same as the target size. We use binary masks to set the interesting
part as 1 and the other parts as 0 to divide the target into M
local parts by performing M mask operations. Fig. 3 shows an
example of dividing the target into M local parts by the binary
masks. During the training and updating process, we only update
the weights of the local part with a nonzero mask value. During
updating the size of the response output of each local filter is
equivalent to the base layer which can bring convenience to the
fusion of different response outputs.
3.3. Adaptive weighting strategy
In the tracking process, there will be inconsonant changes in
each local part of the target appearance in different scenes, such
as occlusion deformation, etc. If the algorithm uses a fixed weight
to directly add the response outputs of the M local filters into
the integrated output, the reliability of the local filter may be
inconsistent with the corresponding weight, thereby reducing the
tracking performance. The response value should be suppressed
if the local part is occluded, and vice versa. Therefore, we propose
an adaptive weighting strategy to achieve adaption.
For DCFs-based trackers, the peak sidelobe ratio can be used
to quantify the sharpness of the correlation peak and it can be
calculated as:
gmax − µs1
psr = , (5)
σs1
where gmax denotes the peak value of the response map, mus1 and
σs1 denote the mean value and standard deviation of the response
map respectively.
Fig. 4 is an illustration of the peak sidelobe ratio corresponding
to a moving target in different image frames. As can be seen
from Fig. 4, target in the upper left sub-figure is in the center
of the bounding box, and the corresponding response map is
(3) single-peak and continuous smooth in the lower left sub-figure.
The peak-side lobe ratio is 7.0 correspondingly. Meanwhile, in
the upper right sub-figure, the target deviates from the center of
the bounding box, and the corresponding response map is multi-
peak and ambiguous. At this time, the peak sidelobe ratio is 5.5.
 D. Yuan, X. Li, Z. He et al. / Knowledge-Based Systems 194 (2020) 105554 5

Fig. 4. An illustration of the peak sidelobe ratio in different image frames of a moving target.

Therefore, the larger the peak sidelobe ratio is, the closer the
target center located by our tracker is to the center of the real
target bounding box.
If the target is not occluded, the response map is usually
unimodal. However, when the target is partially occluded, the
response map will appear multi-peak. At this time, the peak
sidelobe ratio will not effectively enhance the response of the
real target region. Since the target’s motion distance between two
frames follows the Laplacian distribution [61]. We can take this
discovery to compensate for the lack of the peak sidelobe ratio.
The Laplacian distribution is computed as:

1 |x−µ|
f (x, µ, b) = e(− b
)
, (6)
2b
where µ is the position parameter, and b is the scale parameter. Fig. 5. Laplace distribution under different parameters. As can be seen from the
area under the curve, the target center position is mainly changed in a small
As is apparent from Fig. 5, µ determines the center position of the range.
target in the current frame, b determines the extent to which the
target may move, and x determines the distance the target moves
between two frames. Regardless of how the values of µ and b 3.4. Tracking via ASCT
change, the center position of the target always moves within a
small range. The tracking process of our ASCT algorithm includes four
Based on the peak sidelobe ratio and Laplacian distribution, stages: model initialization, online detection, scale estimation,
and model update.
the weight γ of the tth frame can be calculated as:
Model initialization. Given the first frame with the ground-
γmt = psrmt ∗ αmt , truth in tracking tasks, we extract training samples centered on
the target location. We use VGG-16 [62] for feature extraction
1
⎧
⎨ α, abs(dist(max(gmt ) − max(g t −1 ))) ≥ ,
⎪
(7)
and fixed network parameters. Under the zero-mean Gaussian
αmt = 5 distribution, all parameters of the base layer and the local part
⎩ 1, abs(dist(max(g t ) − max(g t −1 ))) < 1 ,
⎪ layers are randomly initialized.
m
5 Online detection. When a new image frame arrives, the search
t area is extracted based on the target center position of the pre-
where psrmdenote the peak sidelobe ratio of mth local filter in
t vious frame. Enter the search area into the network to generate
tth frame, abs(dist(max(gm ) − max(g t −1 ))) denote the distance
a response map. The position in the response map that has the
between the maximum value of the local filter response value of largest response value is the center of the target at the current
current frame and the maximum value of the response value of frame. Specifically for our ASCT network, the search area is sent
previous frame, where the Euclidean distance is used. to the M + 1 convolutional layers, and each response output is
6 D. Yuan, X. Li, Z. He et al. / Knowledge-Based Systems 194 (2020) 105554
adaptively added for the final response map. The peak point of
the final response map is the location of the target center.
Scale estimation. Based on the determined target center lo-
cation, search windows of different sizes are extracted and sent
to our ASCT network to obtain the corresponding response maps.
The width wt and height ht of the target at tth frame is updated
as (wt , ht ) = β (wt∗ , h∗t ) + (1 −β )(wt −1 , ht −1 ), where wt∗ , h∗t are the
width and height of the sample which has the maximum response
value in current frame. The factor β enables the smooth change
of the target size. In general, β takes a small value to avoid the
target drift.
Model update. Throughout the tracking process, the algorithm
continuously generates training data for online updates. For each
frame, after predicting the target position, the search window
samples can be directly entered into the network of our ASCT
network for an online update as the training set of the algorithm.
4. Experiments
We evaluate the proposed ASCT tracker with other state-of-
the-art trackers including SRDCF [2], BACF [4], DeepSRDCF [15],
HDT [17], C–COT [16], MDNet_N [19], CREST_base, CREST [26],
SiamAN, SiamRN, SiamFC [27], ACFN [3], TRACA [29], MetaCREST
[63], SRDCFdecon [64], CNT [57], Staple [65] and EBT [66] on
four widely used benchmarks OTB-2015, TC-128, UAV-123 and
VOT-2016 [67–70].
4.1. Implementation details and evaluation criterion
Implementation Details: The tracking task gives the target
ground-truth in the first image frame. We can get the training
samples with labels from the given frame in the areas 5 times
of target both width and height. The local part number M is set
to 4. The feature extraction network adopts VGG-16 [62] with the
first two pooling layers. The ASCT tracker extracts the feature map
from the conv4-3 layer and reduces the feature map channel from
512 channels to 64 channels by principal component analysis. The
regression response map is generated using a two-dimensional
Gaussian function with a peak value of 1.0. The scale estimate
parameter β is set to 0.6 and the fusion local tracker parameter
α is set to 0.75. Except for the discussion part, the weight factors
φ and ϕ are set to 1, which means the base layer and the part-
fused layer has the same importance in target representation.
The default setting is set for the base layer of the CREST [26]
network, we do not use the residual learning. Our experiments
are performed on a PC with an i7 4.2 GHz CPU, 32 GB RAM and
an Nvidia GTX Titan X GPU with MatConvNet toolbox.
Evaluation criterion: We evaluate the proposed ASCT tracker on
OTB-2015, TC-128, UAV-123 and VOT-2016 benchmarks. For the
OTB-2015, UAV-123 and TC-128 benchmarks, we use one-pass
evaluation (OPE) as the evaluation indexes. OPE has two parts:
precision plots and success plots. The precision plots show the
accurate percentage of predicted position and ground-truth under
different thresholds. While, the success plots measured by an
average overlap, which accounts for both size and position. For
the VOT-2016 benchmark, we use the expected average overlap
(EAO), robustness and accuracy as the evaluation indexes. Ac-
curacy is an indicator of the performance of a direct response
algorithm. EAO is an estimator used to estimate the average
overlap that a tracker expects to achieve over a large number
of short-term sequences with the same visual attributes as a
given dataset. Robustness is an algorithmic stability indicator
that becomes more unstable as the value increases. The EAO
combines the raw values of per-frame accuracies and failures in
a principled manner and has a clear practical interpretation. The
EAO measures the expected no-reset overlap of a tracker run on
a short-term sequence.
4.2. Experiment on OTB-2015
In this section, to validate the effectiveness of the proposed
tracker, we make comparisons of our ASCT tracker with some
state-of-the-art trackers including DeepSRDCF [15], HDT [17],
CREST_base, CREST [26] ACFN [3], TRACA [29], BACF [4], SRDCF
[2], MetaCREST [63], SRDCFdecon [64], CNT [57] and SiamFC [27]
on OTB-2015 [67] dataset with 100 different video sequences.
Fig. 6 shows the precision and the success plots of OPE re-
sults of our ASCT tracker and the 12 state-of-the-art trackers
on the OTB-2015 dataset. Our tracker (0.860/0.644) is the best
in the precision plots and the success plots, which obtained
great improvements on the baseline–CREST_base tracker. Com-
pared to the convolutional residual learning tracker CREST [26]
with the precision score (0.838) and success score (0.623), the
proposed ASCT tracker has obtained improvements approximate
2.63% and 3.37%. While, compared to the offline meta-learning-
based MetaCREST tracker [63] (0.857/0.637), the proposed ASCT
tracker is better than it both in precision score and success score.
This result demonstrates that adaptive structural convolutional
networks for visual tracking are very effective and promising in
practice.
To further verify the effectiveness of the proposed algorithm,
we analyze and compare the performance of the tracker on vari-
ous attributes as shown in Table 1. As we can show that the pro-
posed tracker has the top3 performance in almost of all attributes.
For the attributes of fast motion (FM), illumination variation
(IV), out-of-plane rotation (OPR), and scale variation (SV), the
proposed ASCT tracker achieved the best performance, which can
be attributed to the adaptive structural convolutional networks.
For other attributes such as background clutter (BC), deformation
(DEF), in-plane rotation (IPR), and occlusion (OCC), our tracker is
very closed to the MetaCREST tracker [63] which benefit from
incorporating the actual tracking scenarios into meta-learning.
For the remaining attributes low resolution (LR), out of view
(OOV), motion blur (MB), our ASCT tracker is also very close to the
best tracker. All of those results show that the adaptive structural
convolutional networks can improve the tracking performance.
4.3. Experiment on TC-128
We use TC-128 [68] dataset to evaluate the tracking per-
formance of the proposed ASCT tracker with 10 state-of-the-
art tracking methods using the source tracking results includ-
ing ECO [24], BACF [4], SRDCF [2], DeepSRDCF [15], Staple [65],
MEEM [71], SiamFC [27], MCPF [72], SRDCFdecon [64], CNT [57]
and HDT [17].
The comparison results are shown in Table 2. Among the
compared trackers, the BACF, SRDCF, DeepSRDCF, Staple, MEEM,
SiamFC, MCPF, SRDCFdecon, HDT and CNT tracking methods
achieve the precision and AUC scores of (66.0%, 49.6%), (69.6%,
51.6%), (74.0%, 54.1%), (66.8%, 50.9%), (70.8%, 50.0%),
(69.4%, 50.5%), (76.9%, 55.2%), (72.9%, 54.3%), (68.6%, 48.0%) and
(44.9%, 33.5%) respectively. In contrast, our ASCT tracker performs
well in both metrics (77.1%, 55.8%). The ASCT method obtains
a performance gain of 11.1% and 6.2% on the precision and
AUC scores against the BACF method and outperforms the CNN
based correlation filter trackers (DeepSRDCF, SRDCFdecon, etc.)
by a significant margin. The ASCT method performs almost like
the best ECO tracker and significantly outperforms other track-
ers (Staple, MEEM and SiamFC). Compared to the HDT tracker
with hierarchical CNN features representation, our ASCT tracker
achieves a performance gain of 8.5% and 7.8% in terms of precision
and AUC scores. Overall, the proposed ASCT tracker performs
favorably against the state-of-the-art trackers in terms of both
in precision and AUC metrics.
 D. Yuan, X. Li, Z. He et al. / Knowledge-Based Systems 194 (2020) 105554 7

Fig. 6. The precision plots and success plots of OPE on OTB-2015 over 100 standard benchmark video sequences. The legend contains the average distance precision
score at 20 pixels and the area-under-curve (AUC) score for each tracker. To mark it clear, we just plot the top ten trackers.

Table 1
AUC scores of the BACF, ACFN, SRDCFdecon, SRDCF, DeepSRDCF, HDT, CNT, SiamFC, TRACA, MetaCREST, CREST_base, CREST and the proposed ASCT on OTB-2015 for
different attributes. The first, second and third best scores are highlighted in red, blue and green colors, respectively.
Trackers FM BC MB DEF IV IPR LR OCC OPR OV SV
ASCT(Ours) 64.9 63.9 64.5 61.8 66.9 61.9 59.8 60.0 63.6 56.3 62.0
CREST_base [26] 52.8 57.4 52.7 51.7 60.9 56.9 52.4 54.3 57.8 53.5 52.7
CREST [26] 62.7 61.8 65.5 56.9 64.4 61.7 47.3 59.2 61.5 56.6 57.2
TRACA [29] 58.1 60.1 59.8 56.1 62.2 58.5 50.2 57.6 59.3 56.6 55.8
MetaCREST [63] 62.7 67.4 65.4 62.2 63.5 63.5 47.2 61.2 62.7 56.0 58.2
BACF [4] 59.8 64.1 58.7 59.9 63.1 58.2 51.7 57.4 58.4 51.6 57.0
ACFN [3] 56.3 53.8 56.4 53.5 56.7 54.4 51.5 53.9 54.3 50.0 54.9
SRDCFdecon [64] 60.6 64.1 63.9 55.3 64.6 57.3 51.7 58.9 59.1 51.0 60.7
SRDCF [2] 59.7 58.3 59.4 54.4 61.3 54.4 51.4 55.9 55.0 46.0 56.1
DeepSRDCF [15] 62.8 62.7 64.2 56.6 64.6 58.9 56.1 60.1 60.7 55.3 60.5
SiamFC [27] 56.8 52.3 55.0 50.6 56.8 55.7 61.8 54.3 58.8 50.6 55.2
HDT [17] 56.8 57.8 57.5 54.3 53.5 55.5 40.1 52.8 53.3 47.2 48.6
CNT [57] 30.6 49.0 32.6 39.8 46.2 41.3 40.6 43.4 43.6 47.5 41.0

Table 2
Precision and AUC scores of the ECO, BACF, SRDCF, DeepSRDCF, Staple, MEEM, SiamFC, MCPF, SRDCFdecon, HDT, CNT and the proposed ASCT on TC-128 dataset. The
first, second and third best scores are highlighted in red, blue and green colors, respectively.
Trackers ASCT ECO BACF SRDCF DeepSRDCF Staple MEEM SiamFC MCPF SRDCFdecon HDT CNT
Ours [24] [4] [2] [15] [65] [71] [27] [72] [64] [17] [57]
Precision scores 77.1 80.0 66.0 69.6 74.0 66.8 70.8 69.4 76.9 72.9 68.6 44.9
AUC scores 55.8 60.5 49.6 51.6 54.1 50.9 50.0 50.5 55.2 54.3 48.0 33.5

Table 3
Precision and AUC scores of the ECO, CFNet, SRDCF, MUSTer, SAMF, MEEM, SiamFC, DSST, KCF, ASLA, BACF, CNT and the proposed ASCT on UAV-123 dataset. The
first, second and third best scores are highlighted in red, blue and green colors, respectively.
Trackers ASCT ECO CFNet SRDCF MUSTer SAMF MEEM SiamFC DSST KCF ASLA BACF CNT
Ours [24] [6] [2] [73] [74] [71] [27] [12] [1] [75] [4] [57]
Precision scores 71.6 74.1 65.1 67.6 59.1 59.2 62.7 72.6 58.6 52.3 57.1 65.4 52.4
AUC scores 50.6 52.5 43.6 46.4 39.1 39.6 39.2 49.8 35.6 33.1 40.7 45.7 36.9

4.4. Experiment on UAV-123
We evaluate our ASCT tracker on UAV-123 [69] dataset, and
compare it with 11 state-of-the-art tracking methods using the
source tracking codes or results including ECO [24], CFNet [6],
SRDCF [2], MUSTer [73], SAMF [74], MEEM [71], SiamFC [27],
DSST [12], KCF [1], BACF [4], CNT [57] and ASLA [75].
The comparison results are shown in Table 3. Among the
compared trackers, our ASCT tracker performs second or third
best in both metrics (71.6%, 50.6%) and it is very close to the
best tracker (74.1%, 52.5%). Compared to DCFs-based trackers (e.g.,
BACF, DSST), the proposed ASCT method obtains performance
gain significant improvement on both precision scores and AUC
scores. Compared to these deep learning-based trackers (e.g.,
CFNet, CNT), our ASCT tracker also obtains performance gain
some improvement on both precision scores and AUC scores. In
general, out ASCT tracker performs favorably against the state-of-
the-art trackers both in precision scores and AUC scores.
In Fig. 7, we further analyze the tracking performance under
different challenge attributes (e.g. full occlusion, background clut-
ter and scale variation) annotated in the UAV-123 [69] dataset.
We show the AUC score under six challenge attributes. The re-
sults indicate that our ASCT tracker is more effective than the
ECO tracker and the SiamFC tracker in handling out-of-view and
8 D. Yuan, X. Li, Z. He et al. / Knowledge-Based Systems 194 (2020) 105554

Fig. 7. The success plots over six tracking challenges, including out-of-view, full occlusion, background clutter, similar object, illumination variation, and scale variation
on UAV-123 dataset. To mark it clear, we just plot the top ten trackers.

Table 4 overlap (EAO) score (0.33) and the best robustness value (0.89).
Performance comparison for 10 state-of-the-art algorithms on the VOT-2016 Our ASCT and SiamRN achieved the best accuracy value (0.55).
dataset. The evaluation metrics include expected average overlap (EAO), accuracy
and robustness value. The first, second and third best scores are highlighted in
Meanwhile, the performance of our ASCT tracker is similar to
red, blue and green colors, respectively. those of Staple and EBT on EAO score and robustness value.
Trackers EAO Accuracy Robustness Besides, these trackers perform better than CREST, DeepSRDCF,
ASCT(Ours) 0.29 0.55 1.16 SRDCF, SiamRN, SiamAN, and MDNet_N trackers. According to
Staple [65] 0.30 0.54 1.42 the analysis of VOT-2016 benchmark report and the definition of
EBT [66] 0.29 0.46 1.05 the state-of-the-art bound, our ASCT tracker performs favorably
C-COT [16] 0.33 0.53 0.89 against the state-of-the-art tracking methods.
DeepSRDCF [15] 0.28 0.52 1.23
SRDCF [2] 0.25 0.53 1.50
MDNet_N [19] 0.26 0.54 0.91 4.6. Qualitative comparison
SiamAN [27] 0.24 0.53 1.36
SiamRN [27] 0.28 0.55 1.37 Our proposed tracker (ASCT) significantly improves the track-
CREST [26] 0.28 0.51 1.08
ing performance compared to other representative trackers in-
clude BACF [4], DeepSRDCF [15], CREST [26], SiamFC [27], TRACA
[29] and MetaCREST [63] in the visual object tracking task.
full occlusion. This is because the adaptive weighted integrated Fig. 8 shows a qualitative comparison of these seven trackers
layer can capture changes in appearance, effectively updating on some challenging tracking sequences. The tracking results
the model. Our ASCT performed slightly worse than ECO and of BACF [4] tracker display that in the scene of illumination
SiamFC when similar targets appeared in the tracking scene. This variation, fast motion, and background cluster, it is easily inter-
is mainly because our tracker will focus on some local infor- fered and reducing the tracking performance. This may because
mation. When the target undergoes illumination variation and it adopts the HOG feature, which cannot model the target ap-
scales variation, the ECO tracker’s powerful representation capa- pearance very well in complex tracking scenes. In comparison,
bilities make its tracking performance exceed our ASCT tracker DeepSRDCF [15] tracker investigates the impact of CNN-based
and SiamFC tracker. features on the DCFs-based framework. It performs well on oc-
clusion and background cluster (e.g., freeman1). However, direct
4.5. Experiment on VOT-2016 fusion will limit the processing potential of the model on the
illumination variation (e.g., matrix, skating1) and fast motion (e.g.,
For more thorough evaluations, we use VOT-2016 [70] bench- motorRolling, skiing). Because of the use of attentional correla-
mark to validate the performance of the proposed ASCT tracker tion filter network, TRACA [29] tracker achieved good tracking
against 9 state-of-the-art trackers including DeepSRDCF [15], results in illumination variation (e.g., matrix) and fast motion
C–COT [16], MDNet_N [19], CREST [26], SRDCF [2], SiamAN, (e.g., skiing). In other sequences, its tracking performance still
SiamRN [27], Staple [65] and EBT [66]. The tracking performance has much space for improvement. Form these tracking results,
is measured both in terms of expected average overlap, accuracy we can see that when tracking target are affected by complex
and robustness as in [70]. scenes such as fast motion, background cluster, motion blur, and
The comparison results are shown in Table 4. Among the ten etc, these compared trackers may quickly lose the object target
compared trackers, C–COT [16] obtains the best expected average (e.g., motorRolling, matrix, skating1), but our ASCT tracker can
 D. Yuan, X. Li, Z. He et al. / Knowledge-Based Systems 194 (2020) 105554 9

Fig. 8. Qualitative comparison of our ASCT tracker and other representative trackers (BACF [4], DeepSRDCF [15], CREST [26], SiamFC [27], TRACA [29] and
MetaCREST [63]) on some visual object tracking sequences (bolt2, matrix, car4, motorRolling, carScale, ironman, freeman1, freeman4, human3, human6, human8,
human9, skating1, skating2-2, skiing and walking) with fast motion, scale variation, illumination variation, background clutter, deformation and other challenges.

Table 5 representation. Therefore, it is extremely important to select the

Precision scores and AUC scores of our ASCT tracker under different weight weight factors of φ and ϕ . Due to the weight factors mainly
factor φ & ϕ on OTB-2015 dataset. The first, second and third best scores are
highlighted in red, blue and green colors, respectively.
reflects the importance of different layers, we assume that the
Weight factors OTB-2015 dataset
sum of φ and ϕ is 1. We successively choose φ as [1.0, 0.9, 0.8,
0.7, 0.6, 0.5, 0.4, 0.3, 0.2, 0.1, 0.0], and ϕ as [0.0, 0.1, 0.2, 0.3, 0.4,
φ ϕ Precision scores AUC scores
0.5, 0.6, 0.7, 0.8, 0.9, 1.0] correspondingly.
1.0 0.0 83.4 62.1
Table 5 shows the comparison results on OTB-2015 [67]
0.9 0.1 82.4 61.9
0.8 0.2 82.3 61.5 dataset. When the factor φ is 1.0, which means we just using
0.7 0.3 83.9 62.7 the base layer to tracking the moving target. Otherwise, when
0.6 0.4 82.4 61.5 the factor ϕ is 1.0, which means we just using the local layer
0.5 0.5 86.0 64.4 to tracking the moving target. From Table 5 we can see that
0.4 0.6 83.1 62.2
a common tracking performance can be obtained by using the
0.3 0.7 82.3 61.7
0.2 0.8 82.7 61.5 base layer independently, while the tracking performance by only
0.1 0.9 82.4 60.5 using the local layer is relatively poor. The fusion of these two
0.0 1.0 76.4 55.9 parts with different weights will also bring different tracking
performance. When the weights of these two parts are the same,
a relatively ideal tracking performance can be obtained. This also
shows that although the target representation of local layers can
still be able to locate it faultlessly. Unlike the contrast trackers,
work for the tracker, it still needs a reasonable weight allocation.
benefit from the adaptive structural convolutional network, our
Meanwhile, our ASCT tracker performs well against the base layer
ASCT tracker can handle fast motion, scale variation, illumination
tracker (by 2.4%/2.3%) and the local layer tracker (by 9.6%/8.5%)
variation, deformation, and other challenges very well.
schemes in terms of average precision and success scores. Overall,
these results show that the adaptive structural convolutional
4.7. Discussion network helps improve CNN based tracking methods to obtain
a competitive tracking performance.
In this section, we mainly discuss the weight factors of the Meanwhile, we explored the impact of the adaptive weight-
base layer and the local layer that are essential to our tracking ing strategy on the proposed ASCT tracker. Table 6 shows the
performance. Since our tracker combines the advantages of both performance comparison of the ASCT tracker with/without the
the base layer target representation and the local layer target adaptive weighting strategy on the OTB-2015 dataset [67]. We
10 D. Yuan, X. Li, Z. He et al. / Knowledge-Based Systems 194 (2020) 105554

Table 6
Precision scores and AUC scores of our ASCT tracker with the adaptive weighting strategy (ASCT_waws) and without the adaptive weighting strategy (ASCT_woaws)
on the OTB-2015 dataset.
Indexes Trackers FM BC MB DEF IV IPR LR OCC OPR OV SV Overall
ASCT_waws 64.9 63.9 64.5 61.8 66.9 61.9 59.8 60.0 63.6 56.3 62.0 64.4
AUC scores
ASCT_woaws 63.1 59.5 63.1 57.1 64.5 59.9 59.4 60.5 61.3 58.1 60.9 62.5
ASCT_waws 82.5 86.8 79.6 85.6 87.8 86.2 94.6 79.9 87.2 69.6 83.2 86.0
Prec. scores
ASCT_woaws 81.3 79.4 77.4 78.5 84.8 82.6 94.2 81.2 83.7 74.2 81.8 83.7

can see that the tracker with an adaptive weighting strategy
(ASCT_waws) is significantly better than the tracker without an
adaptive weighting strategy (ASCT_woaws), which fully demon-
strates the effectiveness of our adaptive weighting.
5. Conclusions
In this paper, we propose an adaptive structural convolutional
filter network for visual tracking. We generate a set of local filters
to capture the local structures of the target. The response outputs
of these filters are combined with the response output of the base
layer to enhance the final response output of our ASCT tracker.
Besides, we propose an adaptive weighting strategy to adjust
the weight of different local filters based on the peak sidelobe
ratio and Laplacian distribution. The adaptive response output
is weighted for each local filter to improve the stability of the
proposed algorithm. Extensive experimental results on the OTB-
2015, VOT-2016, UAV-123 and TC-128 benchmarks validate the
effectiveness and stability of the proposed ASCT tracker.
Acknowledgments
This study was supported by the National Natural
Science Foundation of China (Grant No. 61672183), by the Natural
Science Foundation of Guangdong Province (Grant No.
2015A030313544), by the Shenzhen Research Council (Grant
No. JCYJ2017041310455226946, JCYJ20170815113552036,
JCYJ20160226201453085), partially by the projects ‘‘PCL Future
Greater-Bay Area Network Facilities for Large-scale Experiments
and Applications (PCL2018KP001)’’ and ‘‘The Verficiation Plat-
form of Multi-tier Coverage Communication Network for Oceans
(PCL2018KP002)’’, and by Shenzhen Medical Biometrics Percep-
tion and Analysis Engineering Laboratory. Di Yuan is supported
by a scholarship from China Scholarship Council (CSC).
References
[1] J.F. Henriques, R. Caseiro, P. Martins, J. Batista, High-speed tracking with
kernelized correlation filters, IEEE Trans. Pattern Anal. Mach. Intell. 37 (3)
(2014) 583–596.
[2] M. Danelljan, G. Hager, F.S. Khan, M. Felsberg, Learning spatially regular-
ized correlation filters for visual tracking, in: International Conference on
Computer Vision, 2015, pp. 4310–4318.
[3] J. Choi, H.J. Chang, S. Yun, T. Fischer, Y. Demiris, Y.C. Jin, Attentional
correlation filter network for adaptive visual tracking, in: Computer Vision
and Pattern Recognition, 2017, pp. 4828–4837.
[4] H.K. Galoogahi, A. Fagg, S. Lucey, Learning background-aware correlation
filters for visual tracking, in: International Conference on Computer Vision,
2017, pp. 1135–1143.
[5] X. Li, Q. Liu, Z. He, H. Wang, C. Zhang, W.S. Chen, A multi-view model for
visual tracking via correlation filters, Knowl.-Based Syst. 113 (2016) 88–99.
[6] J. Valmadre, L. Bertinetto, J. Henriques, A. Vedaldi, P.H.S. Torr, End-to-end
representation learning for correlation filter based tracking, in: Computer
Vision and Pattern Recognition, 2017, pp. 2085–2813.
[7] K. Zhang, X. Li, H. Song, Q. Liu, L. Wei, Visual tracking using spatio-
temporally nonlocally regularized correlation filter, Pattern Recognit. 83
(2018) 185–195.
[8] D. Yuan, X. Lu, D. Li, Y. Liang, X. Zhang, Particle filter re-detection for
visual tracking via correlation filters, Multimedia Tools Appl. 78 (11) (2019)
14277–14301.
[9] D.S. Bolme, J.R. Beveridge, B.A. Draper, Y.M. Lui, Visual object tracking using
adaptive correlation filters, in: Computer Vision and Pattern Recognition,
2010, pp. 2544–2550.
[10] J.F. Henriques, C. Rui, P. Martins, J. Batista, Exploiting the circulant
structure of tracking-by-detection with kernels, in: European Conference
on Computer Vision, 2012, pp. 702–715.
[11] H.K. Galoogahi, T. Sim, S. Lucey, Correlation filters with limited boundaries,
in: Computer Vision and Pattern Recognition, 2015, pp. 4630–4638.
[12] M. Danelljan, G. Hager, F.S. Khan, M. Felsberg, Discriminative scale space
tracking, IEEE Trans. Pattern Anal. Mach. Intell. 39 (8) (2017) 1561–1575.
[13] M. Danelljan, F.S. Khan, M. Felsberg, J.V.D. Weijer, Adaptive color attributes
for real-time visual tracking, in: Computer Vision and Pattern Recognition,
2014, pp. 1090–1097.
[14] N. Fan, J. Li, Z. He, C. Zhang, X. Li, Region-filtering correlation tracking,
Knowl.-Based Syst. 172 (2019) 95–103.
[15] M. Danelljan, G. Hager, F.S. Khan, M. Felsberg, Convolutional features for
correlation filter based visual tracking, in: International Conference on
Computer Vision Workshops, 2015, pp. 621–629.
[16] M. Danelljan, A. Robinson, F.S. Khan, M. Felsberg, Beyond correlation filters:
Learning continuous convolution operators for visual tracking, in: European
Conference on Computer Vision, 2016, pp. 472–488.
[17] Y. Qi, S. Zhang, L. Qin, H. Yao, Q. Huang, J. Lim, M.H. Yang, Hedged
deep tracking, in: Computer Vision and Pattern Recognition, 2016, pp.
4303–4311.
[18] X. Li, Q. Liu, N. Fan, Z. He, H. Wang, Hierarchical spatial-aware siamese
network for thermal infrared object tracking, Knowl.-Based Syst. 166
(2019) 71–81.
[19] H. Nam, B. Han, Learning multi-domain convolutional neural networks for
visual tracking, in: Computer Vision and Pattern Recognition, 2016, pp.
4293–4302.
[20] X. Li, C. Ma, B. Wu, Z. He, M.-H. Yang, Target-aware deep tracking, in:
Computer Vision and Pattern Recognition, 2019, pp. 1369–1378.
[21] K. Zhang, Q. Liu, W. Yi, M.H. Yang, Robust visual tracking via convolutional
networks without training, IEEE Trans. Image Process. 25 (4) (2015)
1779–1792.
[22] B. Liu, Q. Liu, Z. Zhu, T. Zhang, Y. Yang, Msst-resnet: Deep multi-scale
spatiotemporal features for robust visual object tracking, Knowl.-Based
Syst. 164 (2019) 235–252.
[23] A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deep
convolutional neural networks, in: International Conference on Neural
Information Processing Systems, 2012, pp. 1097–1105.
[24] M. Danelljan, G. Bhat, F.S. Khan, M. Felsberg, Eco: Efficient convolution
operators for tracking, in: Computer Vision and Pattern Recognition, 2017,
pp. 6638–6646.
[25] K. Chen, W. Tao, Convolutional regression for visual tracking, IEEE Trans.
Image Process. 27 (7) (2018) 3611–3620.
[26] Y. Song, C. Ma, L. Gong, J. Zhang, R.W. Lau, M.-H. Yang, Crest: Convolutional
residual learning for visual tracking, in: International Conference on
Computer Vision, 2017, pp. 2574–2583.
[27] L. Bertinetto, J. Valmadre, J.F. Henriques, A. Vedaldi, P.H.S. Torr,
Fully-convolutional siamese networks for object tracking, in: European
Conference on Computer Vision Workshop, 2016, pp. 850–865.
[28] C. Ma, J.B. Huang, X. Yang, M.H. Yang, Hierarchical convolutional features
for visual tracking, in: International Conference on Computer Vision, 2015,
pp. 3074–3082.
[29] J. Choi, H.J. Chang, T. Fischer, S. Yun, K. Lee, J. Jeong, Y. Demiris, Y.C. Jin,
Context-aware deep feature compression for high-speed visual tracking,
in: Computer Vision and Pattern Recognition, 2018, pp. 479–488.
[30] Q. Liu, Z. He, X. Li, Y. Zheng, Ptb-tir: A thermal infrared pedestrian tracking
benchmark, IEEE Trans. Multimed. (2019) http://dx.doi.org/10.1109/TMM.
2019.2932615.
[31] P. Li, D. Wang, L. Wang, H. Lu, Deep visual tracking: Review and
experimental comparison, Pattern Recognit. 76 (2018) 323–338.
[32] H. Lu, P. Li, D. Wang, Visual object tracking: A survey, Pattern Recognit.
Artif. Intell. 31 (1) (2018) 61–76.
[33] K. Zhang, L. Zhang, Q. Liu, D. Zhang, M.-H. Yang, Fast visual tracking
via dense spatio-temporal context learning, in: European Conference on
Computer Vision, 2014, pp. 127–141.
[34] K. Zhang, J. Fan, Q. Liu, J. Yang, W. Lian, Parallel attentive correlation
tracking, IEEE Trans. Image Process. 28 (1) (2019) 479–491.
[35] T. Liu, G. Wang, Q. Yang, Real-time part-based visual tracking via adaptive
correlation filters, in: Computer Vision and Pattern Recognition, 2015, pp.
4902–4912.
 D. Yuan, X. Li, Z. He et al. / Knowledge-Based Systems 194 (2020) 105554
[36] Q. Peng, Y. Cheung, X. You, Y. Tang, A hybrid of local and global saliencies
for detecting image salient region and appearance, IEEE Trans. Syst. Man
Cybern.: Syst. 47 (1) (2017) 86–97.
[37] S. Liu, T. Zhang, X. Cao, C. Xu, Structural correlation filter for robust
visual tracking, in: Computer Vision and Pattern Recognition, 2016, pp.
4312–4320.
[38] D. Yuan, X. Zhang, J. Liu, D. Li, A multiple feature fused model for visual
object tracking via correlation filters, Multimedia Tools Appl. 78 (19)
(2019) 27271–27290.
[39] T. Liu, X. Cao, J. Jiang, Visual object tracking with partition loss schemes,
IEEE Trans. Intell. Transp. Syst. 18 (3) (2017) 633–642.
[40] Z. He, S. Yi, Y.M. Cheung, X. You, Y.Y. Tang, Robust object tracking via key
patch sparse representation, IEEE Trans. Cybern. 47 (2) (2017) 354–364.
[41] S. Xin, N.M. Cheung, H. Yao, Y. Guo, Non-rigid object tracking via
deformable patches using shape-preserved kcf and level sets, in: IEEE
International Conference on Computer Vision, 2017, pp. 5496–5504.
[42] S. Zhang, H. Zhou, J. Feng, X. Li, Robust visual tracking using structurally
random projection and weighted least squares, IEEE Trans. Circuits Syst.
Video Technol. 25 (11) (2015) 1749–1760.
[43] Y. Li, J. Zhu, S.C.H. Hoi, Reliable patch trackers: Robust visual tracking by
exploiting reliable patches, in: Computer Vision and Pattern Recognition,
2015, pp. 353–361.
[44] O. Akin, E. Erdem, A. Erdem, K. Mikolajczyk, Deformable part-based
tracking by coupled global and local correlation filters, J. Vis. Commun.
Image Represent. 38 (2016) 763–774.
[45] R. Yao, Q. Shi, C. Shen, Y. Zhang, A.V.D. Hengel, Part-based visual tracking
with online latent structural learning, in: Computer Vision and Pattern
Recognition, 2013, pp. 2363–2370.
[46] A. Lukezic, L. Cehovin, M. Kristan, Deformable parts correlation filters for
robust visual tracking, IEEE Trans. Cybern. 48 (6) (2018) 1849–1861.
[47] C. Rasmussen, G.D. Hager, Joint probabilistic techniques for tracking multi-
part objects, in: Computer Vision and Pattern Recognition, 1998, pp.
16–21.
[48] C. Tian, Y. Xu, Z. Li, W. Zuo, L. Fei, H. Liu, Attention-guided cnn for image
denoising, Neural Netw. (2020) http://dx.doi.org/10.1016/j.neunet.2019.12.
024.
[49] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition,
in: Computer Vision and Pattern Recognition, 2016, pp. 770–778.
[50] H. Li, Y. Li, F. Porikli, Deeptrack: Learning discriminative feature represen-
tations by convolutional neural networks for visual tracking, in: British
Machine Vision Conference, 2014, pp. 1–12.
[51] Q. Liu, X. Lu, Z. He, C. Zhang, W.S. Chen, Deep convolutional neural
networks for thermal infrared object tracking, Knowl.-Based Syst. (2017)
189–198.
[52] C. Tian, Y. Xu, W. Zuo, Image denoising using deep cnn with batch
renormalization, Neural Netw. 121 (2019) 461–473.
[53] N. Wang, J. Shi, D.Y. Yeung, J. Jia, Understanding and diagnosing visual
tracking systems, in: International Conference on Computer Vision, 2015,
pp. 3101–3109.
[54] K. Zhang, Q. Liu, Y. Jian, M.H. Yang, Visual tracking via boolean map
representations, Pattern Recognit. 81 (2018) 147–160.
[55] S. Zhang, X. Lan, H. Yao, H. Zhou, D. Tao, X. Li, A biologically inspired
appearance model for robust visual tracking, IEEE Trans. Neural Netw.
Learn. Syst. 25 (11) (2015) 1749–1760.
[56] C. Huang, S. Lucey, D. Ramanan, Learning policies for adaptive tracking
with deep feature cascades, in: International Conference on Computer
Vision, 2017, pp. 105–114.
11
[57] K. Zhang, Q. Liu, Y. Wu, M.H. Yang, Robust visual tracking via convolutional
networks without training, IEEE Trans. Image Process. 25 (4) (2016)
1779–1792.
[58] P. Gao, Q. Zhang, F. Wang, L. Xiao, H. Fujita, Y. Zhang, Learning reinforced
attentional representation for end-to-end visual tracking, Inform. Sci. 517
(2020) 52–67.
[59] P. Gao, R. Yuan, F. Wang, L. Xiao, H. Fujita, Y. Zhang, Siamese attentional
keypoint network for high performance visual tracking, Knowl.-Based Syst.
(2019) http://dx.doi.org/10.1016/j.knosys.2019.105448.
[60] D. Yuan, N. Fan, Z. He, Learning target-focusing convolutional regression
model for visual object tracking, Knowl.-Based Syst. (2020) http://dx.doi.
org/10.1016/j.knosys.2020.105526.
[61] D. Held, S. Thrun, S. Savarese, Learning to track at 100 fps with deep
regression networks, in: European Conference on Computer Vision, 2016,
pp. 749–765.
[62] K. Simonyan, A. Zisserman, Very deep convolutional networks for
large-scale image recognition, in: International Conference on Learning
Representations, 2015, pp. 1–14.
[63] E. Park, A.C. Berg, Meta-tracker: Fast and robust online adaptation for
visual object trackers, in: European Conference on Computer Vision, 2018,
pp. 1–17.
[64] M. Danelljan, G. Hager, F.S. Khan, M. Felsberg, Adaptive decontamination
of the training set: A unified formulation for discriminative visual tracking,
in: Computer Vision and Pattern Recognition, 2016, pp. 1430–1438.
[65] L. Bertinetto, J. Valmadre, S. Golodetz, O. Miksik, P. Torr, Staple: Comple-
mentary learners for real-time tracking, in: Computer Vision and Pattern
Recognition, 2016, pp. 1401–1409.
[66] G. Zhu, F. Porikli, H. Li, Beyond local search: Tracking objects every-
where with instance-specific proposals, in: Computer Vision and Pattern
Recognition, 2016, pp. 943–951.
[67] Y. Wu, J. Lim, M.H. Yang, Object tracking benchmark, IEEE Trans. Pattern
Anal. Mach. Intell. 37 (9) (2015) 1834–1848.
[68] P. Liang, E. Blasch, H. Ling, Encoding color information for visual tracking:
Algorithms and benchmark., IEEE Trans. Image Process. 24 (12) (2015)
5630–5644.
[69] M. Mueller, N. Smith, B. Ghanem, A benchmark and simulator for uav
tracking, in: European Conference on Computer Vision, 2016, pp. 445–461.
[70] M. Kristan, A. Leonardis, J. Matas, et al., The visual object tracking vot2016
challenge results, in: European Conference on Computer Vision, 2016, pp.
191–217.
[71] J. Zhang, S. Ma, S. Sclaroff, Meem: Robust tracking via multiple experts
using entropy minimization, in: European Conference on Computer Vision,
2014, pp. 188–203.
[72] T. Zhang, C. Xu, M.H. Yang, Multi-task correlation particle filter for robust
object tracking, in: Computer Vision and Pattern Recognition, 2017, pp.
4819–4827.
[73] Z. Hong, C. Zhe, C. Wang, M. Xue, D. Prokhorov, D. Tao, Multi-store tracker
(muster): a cognitive psychology inspired approach to object tracking, in:
Computer Vision and Pattern Recognition, 2015, pp. 749–758.
[74] Y. Li, J. Zhu, A scale adaptive kernel correlation filter tracker with feature
integration, in: European Conference on Computer Vision Workshops,
2014, pp. 254–265.
[75] X. Jia, H. Lu, M.H. Yang, Visual tracking via adaptive structural local sparse
appearance model, in: Computer Vision and Pattern Recognition, 2012, pp.
1822–1829.