Professional Documents
Culture Documents
Neural Information Processing: 25th International Conference, ICONIP 2018, Siem Reap, Cambodia, December 13-16, 2018, Proceedings, Part VI Long Cheng
Neural Information Processing: 25th International Conference, ICONIP 2018, Siem Reap, Cambodia, December 13-16, 2018, Proceedings, Part VI Long Cheng
Neural Information Processing: 25th International Conference, ICONIP 2018, Siem Reap, Cambodia, December 13-16, 2018, Proceedings, Part VI Long Cheng
https://textbookfull.com/product/neural-information-
processing-25th-international-conference-iconip-2018-siem-reap-
cambodia-december-13-16-2018-proceedings-part-vi-long-cheng/
https://textbookfull.com/product/neural-information-
processing-25th-international-conference-iconip-2018-siem-reap-
cambodia-december-13-16-2018-proceedings-part-ii-long-cheng/
https://textbookfull.com/product/neural-information-
processing-25th-international-conference-iconip-2018-siem-reap-
cambodia-december-13-16-2018-proceedings-part-iii-long-cheng/
https://textbookfull.com/product/neural-information-
processing-25th-international-conference-iconip-2018-siem-reap-
cambodia-december-13-16-2018-proceedings-part-v-long-cheng/
Neural Information Processing: 25th International
Conference, ICONIP 2018, Siem Reap, Cambodia, December
13–16, 2018, Proceedings, Part VII Long Cheng
https://textbookfull.com/product/neural-information-
processing-25th-international-conference-iconip-2018-siem-reap-
cambodia-december-13-16-2018-proceedings-part-vii-long-cheng/
https://textbookfull.com/product/neural-information-
processing-27th-international-conference-iconip-2020-bangkok-
thailand-november-18-22-2020-proceedings-part-iv-haiqin-yang/
https://textbookfull.com/product/neural-information-
processing-27th-international-conference-iconip-2020-bangkok-
thailand-november-18-22-2020-proceedings-part-v-haiqin-yang/
https://textbookfull.com/product/neural-information-
processing-27th-international-conference-iconip-2020-bangkok-
thailand-november-23-27-2020-proceedings-part-i-haiqin-yang/
https://textbookfull.com/product/neural-information-
processing-27th-international-conference-iconip-2020-bangkok-
thailand-november-23-27-2020-proceedings-part-ii-haiqin-yang/
Long Cheng
Andrew Chi Sing Leung
Seiichi Ozawa (Eds.)
LNCS 11306
Neural
Information Processing
25th International Conference, ICONIP 2018
Siem Reap, Cambodia, December 13–16, 2018
Proceedings, Part VI
123
Lecture Notes in Computer Science 11306
Commenced Publication in 1973
Founding and Former Series Editors:
Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen
Editorial Board
David Hutchison
Lancaster University, Lancaster, UK
Takeo Kanade
Carnegie Mellon University, Pittsburgh, PA, USA
Josef Kittler
University of Surrey, Guildford, UK
Jon M. Kleinberg
Cornell University, Ithaca, NY, USA
Friedemann Mattern
ETH Zurich, Zurich, Switzerland
John C. Mitchell
Stanford University, Stanford, CA, USA
Moni Naor
Weizmann Institute of Science, Rehovot, Israel
C. Pandu Rangan
Indian Institute of Technology Madras, Chennai, India
Bernhard Steffen
TU Dortmund University, Dortmund, Germany
Demetri Terzopoulos
University of California, Los Angeles, CA, USA
Doug Tygar
University of California, Berkeley, CA, USA
Gerhard Weikum
Max Planck Institute for Informatics, Saarbrücken, Germany
More information about this series at http://www.springer.com/series/7407
Long Cheng Andrew Chi Sing Leung
•
Neural
Information Processing
25th International Conference, ICONIP 2018
Siem Reap, Cambodia, December 13–16, 2018
Proceedings, Part VI
123
Editors
Long Cheng Seiichi Ozawa
The Chinese Academy of Sciences Kobe University
Beijing, China Kobe, Japan
Andrew Chi Sing Leung
City University of Hong Kong
Kowloon, Hong Kong SAR, China
This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface
speakers, authors, reviewers, volunteers, and participants for their contribution and
support in making ICONIP 2018 a successful event.
General Chair
Jun Wang City University of Hong Kong,
Hong Kong SAR, China
Advisory Chairs
Akira Hirose University of Tokyo, Tokyo, Japan
Soo-Young Lee Korea Advanced Institute of Science and Technology,
South Korea
Derong Liu Institute of Automation, Chinese Academy of Sciences,
China
Nikhil R. Pal Indian Statistics Institute, India
Program Chairs
Long Cheng Institute of Automation, Chinese Academy of Sciences,
China
Andrew C. S. Leung City University of Hong Kong, Hong Kong SAR,
China
Seiichi Ozawa Kobe University, Japan
Tutorial Chairs
Hiroaki Gomi NTT Communication Science Laboratories, Japan
Takashi Morie Kyushu Institute of Technology, Japan
Kay Chen Tan City University of Hong Kong, Hong Kong SAR,
China
Dongbin Zhao Institute of Automation, Chinese Academy of Sciences,
China
VIII ICONIP 2018 Organization
Publicity Chairs
Zeng-Guang Hou Institute of Automation, Chinese Academy of Sciences,
China
Tingwen Huang Texas A&M University at Qatar, Qatar
Chia-Feng Juang National Chung-Hsing University, Taiwan
Tomohiro Shibata Kyushu Institute of Technology, Japan
Publication Chairs
Xinyi Le Shanghai Jiao Tong University, China
Sitian Qin Harbin Institute of Technology Weihai, China
Zheng Yan University Technology Sydney, Australia
Shaofu Yang Southeast University, China
Registration Chairs
Shenshen Gu Shanghai University, China
Qingshan Liu Southeast University, China
Ka Chun Wong City University of Hong Kong,
Hong Kong SAR, China
Conference Secretariat
Ying Qu Dalian University of Technology, China
Program Committee
Hussein Abbass University of New South Wales at Canberra, Australia
Choon Ki Ahn Korea University, South Korea
Igor Aizenberg Texas A&M University at Texarkana, USA
Shotaro Akaho National Institute of Advanced Industrial Science
and Technology, Japan
Abdulrazak Alhababi UNIMAS, Malaysia
Cecilio Angulo Universitat Politècnica de Catalunya, Spain
Sabri Arik Istanbul University, Turkey
Mubasher Baig National University of Computer and Emerging
Sciences Lahore, India
Sang-Woo Ban Dongguk University, South Korea
Tao Ban National Institute of Information and Communications
Technology, Japan
Boris Bačić Auckland University of Technology, New Zealand
Xu Bin Northwestern Polytechnical University, China
David Bong Universiti Malaysia Sarawak, Malaysia
Salim Bouzerdoum University of Wollongong, Australia
Ivo Bukovsky Czech Technical University, Czech Republic
ICONIP 2018 Organization IX
Time-Series Analysis
Social Systems
1 Introduction
Time series prediction impacts on humans’ daily life, e.g. weather forecasting,
wind speed forecasting for wind power systems [1], and financial trend forecast-
ing [2]. In 2015, Scardapane et al. proposed an algorithm called Online Sequential
Extreme Learning Machine (OS-ELM) with Kernel that enables implicit feature
mappings by utilising kernel method [3]. The problem of OS-ELM–the unstable
prediction results–was solved by kernel method. Its unstable feature mapping in
OS-ELM was replaced by the kernel matrix. This can solve the problem of predic-
tion deterministic in model as it is known that Extreme Learning Machine (ELM)
randomly chooses its parameters, thus its generalisation ability cannot be guar-
anteed. However, the algorithm requires an extensive computational resource in
the learning process, especially when the large-scale data sets are considered.
c Springer Nature Switzerland AG 2018
L. Cheng et al. (Eds.): ICONIP 2018, LNCS 11306, pp. 3–13, 2018.
https://doi.org/10.1007/978-3-030-04224-0_1
4 Z. Liu et al.
Thus, Scardapane et. al. further employed Kernel Adaptive Filters (KAF) to
reduce the size of hidden neurons in the process of their algorithm–Approximate
Linear Dependency (ALD) and Fixed-Budget (FB). It was found that ALD can
reduce computational complexity while enhance the overall performance. Fur-
thermore, the majority of data sets are non-stationary that means the charac-
teristics of data streams may change–underlying distribution changes over time.
This directly causes concept drift problem. Although there are many methods
that deal with concept drift [4–6], however, these methods are only focus on clas-
sification problem. Hence, this paper pays more attention to the concept drift
problem in time series prediction.
Besides, the hidden nodes of Kernel OS-ELM (KOS-ELM) include all infor-
mation of training data, which keep the old information and cannot be over-
written with time progress. Reservoir Computing (RC) is a framework for
computation–an extension of neural networks–that is employed in time series
prediction. It is a fixed dynamical system that absorbs the information from
their recent input history. As time passes, the old information in the reservoir
is dissipated and overwritten. There are two major types of RC–Liquid-State
Machine (LSM) and Echo State Network (ESN). Again, due to the random
weight selection, this causes the unstable of forecasting performance. Therefore,
this paper defines infinitely large reservoir called recursive kernels. These kernels
can be readily analysed in terms of dynamical stability. To deal with an infinitely
large hidden states, kernel trick is applied. Thus, we end up with a kernel func-
tion that is the inner product of the hidden states of infinite networks.
In order to solve the limitations of KOS-ELM and generate the fixed dynam-
ical reservoir in its learning part, we apply recurrent multi-step algorithm with
recursive kernel to release the restriction of prediction horizon. Furthermore, the
KAF is applied to reduce the extensive computational resources. Moreover, Drift
Detector Mechanism (DDM) is also considered in order to enable the model to
have a good generalisation for concept drift in time series prediction and improve
the performance of prediction. Finally, the modified meta-cognitive strategy–a
new strategy for regression prediction–is applied to decide when the coming data
in the learning part need to be updated, retrained or discarded.
2 Methodology
This section explains the theory of recursive kernel method and modifies DDM
in the prediction model of KOS-ELM. At the end of the section, we present how
a new meta-cognitive strategy is applied in our proposed time series prediction
model.
Time series data is a series of data points listed in time order. Assuming that
the data X = [X1 , . . . , XN ] is given where N is the number of time series data.
Then, the data is transformed into a matrix form in order to achieve the goal of
Handling Concept Drift in Time-Series Data: Meta-RRKOS-ELM 5
multi-step prediction. This technique was previously applied in [7,8]. This gives
the following training input data and target data,
⎡ ⎤ ⎡ ⎤
X1,1 · · · X1,D X1,D+p
⎢ ⎥ ⎢ ⎥
S = ⎣ ... . . . ... ⎦ ; Yp = ⎣ ..
. ⎦, (1)
XL,1 . . . XL,D XL,D+p
where D is the size of time window, L is the number of training data, p is the
p-th step in the prediction (p = 1, 2, . . . , P ).
Although KOS-ELM can perform well in time series prediction [3], its size of
prediction horizon is restricted to utilise in the real-world application. The recur-
rent multi-step algorithm is a feedback network which backward from the output
of current step to the input of the next step. In order to extend the prediction
horizon, we employ recurrent multi-step algorithm in the prediction model, more
details can be found at [8]. Based on the theory of recurrent multi-step algorithm,
the training input data at the (p + 1)-th becomes as follow,
⎡ ⎤
X1,1+p · · · X1,D y1−1 · · · y1−p
⎢ ⎥
⎢ .. .. .. .. . . .. ⎥
⎢
Sp+1 = ⎢ . . . ⎥
. . . ⎥, (2)
⎣ −1 −p
⎦
XL,1+p . . . XL,D yL · · · yL
where y −p represents the predicting values of training data in the p-th step and
(D + 1)≥ P .
Assuming that there is a kernel function k(x, x ) with data points x and x . The
kernel function can be defined by k(x, x ) = φ(x)φ(x ), where φ is a feature map.
A norm can be defined using inner product of inputs and the kernel function has
the following property:
φ(x)φ(x ) = k(x, x ). (5)
If k has φ associated with it for which (5) is valid, this means that this same
feature map can be applied for recurrent kernels. The recurrent kernel with input
data x and x can be written as follow:
rt,t (x, x ) = k(kt−1 (x, x ) + x(t)x (t ), kt−1 (x, x) + x(t)x(t), kt −1 (x , x ) + x (t )x (t )). (6)
A recurrent formula that can be applied for any kernel function with the form
specified in (5) [11]. This recurrent formula also requires us to compute the
recurrent kernels kt−1 (x, x) and kt −1 (x , x ). Generally, a kernel function is only
a function of the inner product of its arguments, and not their quadratic norms.
In this case the corresponding recurrent kernel can be simplified to the following
form:
rt,t (x, x ) = k(rt−1,t −1 (x, x ) + x(t)x (t )), (7)
In RKOS-ELM with DDM, Radial Basis Function (RBF), which is a popular
2
kernel function, is considered, k(x, x ) = exp(− ||x−x
2σ 2
||
), where σ is a kernel
Handling Concept Drift in Time-Series Data: Meta-RRKOS-ELM 7
parameter. Although ALD plays a significant role in filtering input data and
reduce the number of hidden nodes, the computing process of kernel matrix
only pays attention to the kernel matrix between the current input data and
the current memory. In order to deal with the infinitely large hidden states of
RKOS-ELM with DDM, we apply recursive kernel–which is the inner product
of the hidden states of infinite networks–to replace RBF kernel method in the
learning part. Thus, the old information of kernel matrix can be dissipated and
overwritten by the recursive kernel over the time passes. Therefore, the reservoir
of predicting model that is made by the recursive kernel has the more efficient,
completed, and variety of information than by the common kernel method. In this
paper, we utilise the recursive RBF kernel method to construct a kernel matrix.
The algorithm is called “Recurrent Recursive RBF Kernel Online Sequential
Extreme Learning Machine with DDM” (RRKOS-ELM-DDM).
According to (6), the recursive RBF can be represented as:
βl−1 − zl rl−1 el
βl = , (10)
rl−1 el
where λ is the slope that controls the rate of self-adaptation and is set close to
1. If the coming sample data does not have concept drift problem or has the
Another random document with
no related content on Scribd:
A. D. 1897 (Zanzibar).
Abolition of slavery.
AFRICA: A. D. 1899.
Railway development.
"Article I.
From the coming into force of the present Convention, the
import duty on spirituous liquors, as that duty is regulated
by the General Act of Brussels, shall be raised throughout the
zone where there does not exist the system of total
prohibition provided by Article XCI, of the said General Act,
to the rate of 70 fr. the hectolitre at 50 degrees centigrade,
for a period of six years. It may, exceptionally, be at the
rate of 60 fr. only the hectolitre at 50 degrees centigrade in
the Colony of Togo and in that of Dahomey. The import duty
shall be augmented proportionally for each degree above 50
degrees centigrade; It may be diminished proportionally for
each degree below 50 degrees centigrade. At the end of the
above-mentioned period of six years, the import duty shall be
submitted to revision, taking as a basis the results produced
by the preceding rate. The Powers retain the right of
maintaining and increasing the duty beyond the minimum fixed
by the present Article in the regions where they now possess
that right.
Article II.
In accordance with Article XCIII of the General Act of
Brussels, distilled drinks made in the regions mentioned in
Article XCII of the said General Act, and intended for
consumption, shall pay an excise duty. This excise duty, the
collection of which the Powers undertake to insure as far as
possible, shall not be lower than the minimum import duty
fixed by Article I. of the present Convention.
Article III.
It is understood that the Powers who signed the General Act of
Brussels, or who have acceded to it, and who are not
represented at the present Conference, preserve the right of
acceding to the present Convention."
AFRICA: A. D. 1899.
Progress of the Telegraph line from the Cape to Cairo.
AFRICA: A. D. 1899-1900.
Summary of the partition of the Continent.
"Seven European nations, as before remarked, now control
territories in Africa, two of them having areas equal in each
case to about the entire land area of the United States, while
a few small territories remain as independent States.
Beginning at the northeast, Egypt and Tripoli are nominally at
least tributaries of Turkey, though the Egyptian Government,
which was given large latitude by that of Turkey, has of late
years formed such relations with Great Britain that, in
financial matters at least, her guidance is recognized; next
west, Algeria, French; then Morocco on the extreme northwest,
an independent Government and an absolute despotism; next on
the south, Spain's territory of Rio de Oro; then the Senegal
territories, belonging to the French, and connecting through
the desert of Sahara with her Algeria; then a group of small
divisions controlled by England, along the Gulf of Guinea;
then Liberia, the black Republic; Togoland, controlled by the
Germans; Dahomey, a French protectorate; the Niger territory,
one-third the size of the United States, controlled by
England; Kamerun, controlled by Germany; French Kongo; then
the Kongo Free State, under the auspices of the King of
Belgium, and occupying the very heart of equatorial Africa;
then Portuguese Angola; next, German South west Africa; and
finally in the march down the Atlantic side, Cape Colony,
British.
{5}
Following up the eastern side comes the British colony of
Natal; then just inland from this the two Boer Republics, the
Orange Free State and the South African Republic, both of
which are entirely in the interior, without ocean frontage;
next, Portuguese Africa, and west of this the great territory
known as 'Rhodesia'; then German Africa, which extends almost
to the equator; north of these, British East Africa, fronting
on the Indian Ocean, and merging northwardly with the Egyptian
Sudan, which was recently recovered from the Mahdi by the joint
operation of British and Egyptian troops, and the British flag
placed side by side with that of Egypt; next north, upon the
coast, Italian territory and a small tract opposite the
entrance to the Red Sea controlled by England; and a few
hundred miles west of the entrance to the Red Sea, the
independent Kingdom of Abyssinia. This division of African
territory, nearly all of it made within the memory of the
present generation, forms the present political map of Africa.
With England and France controlling an area equal in each case
to that of the United States; Germany, a territory one-third
the size of the United States; Portugal, with an area somewhat
less; the Kongo Free State in the great equatorial basin, but
having a frontage upon the Atlantic with an area nearly
one-third that of the United States; Italy and Spain, each
with a comparatively small area of territory; Egypt, with
relations quite as much British as Turkish; Tripoli, Turkish,
and the five independent States of Morocco, Liberia,
Abyssinia, and the two Boer Republics—nothing remains
unclaimed, even in the desert wastes, while in the high
altitudes and subtropical climate of southeast Africa
civilization and progress are making rapid advancement."
United States Bureau of Statistics,
Monthly Summary, August, 1899.
POP.
TOTAL FOREIGN PER SQ.
AREA. POPULATION. POPULATION MILE IMPORTS.
EXPORTS.
French Africa. 3,028,000 28,155,000 922,000 9.3
$70,116,000 $69,354,000
British Africa. 2,761,000 35,160,000 455,000 12.8
131,398,000 131,835,000
Turkish Africa. 1,760,000 21,300,000 113,000 12.2
54,091,000 62,548,000
German Africa. 944,000 11,270,000 4,000 12.0
4,993,000 2,349,000
Belgian Africa. 900,000 30,000,000 2,000 33.3
4,522,000 3,309,000
Portuguese Africa. 790,000 8,059,000 3,000 10.2
11,863,000 6,730,000
Spanish Africa. 243,000 36,000 … 0.5
… …
Italian Africa. 188,000 850,000 … 4.5
… …
Independent States.
AFRIKANDER CONGRESS.
See (in this volume)
SOUTH AFRICA (Cape Colony): A. D. 1900 (DECEMBER).
AFRIKANDERS:
Joining the invading Boers.
AFRIKANDERS:
Opposition to the annexation of the Boer Republics.
{6}
ALABAMA: A. D. 1899.
Dispensary Laws.
ALASKA: A. D. 1898-1899.
Discovery of the Cape Nome gold mining region.
"The news of a rich strike at Nome worked its way up the Yukon
River during the winter, and as soon as the ice broke in June
a large crowd came down from Rampart City, followed by a
larger crowd from Dawson. The 'Yukoners,' as these people were
called, were already disgusted with the hardships,
disappointments, and Canadian misgovernment which they had met
with on the upper river. … Those to whom enough faith had been
given to go over to Cape Nome were disgusted and angered to
find that pretty much the whole district was already staked,
and that the claims taken were two or three times as large as
those commonly allowed on the upper river. Another grievance
was the great abuse of the power of attorney, by means of
which an immense number of claims had been taken up, so that
in many cases (according to common report) single individuals
held or controlled from 50 to 100 claims apiece. …
"A year ago [that is, in the winter of 1898-1899] a few Eskimo
huts and one or two sod houses of white men were the only
human habitations along 60 miles of the present Nome coast.
Last June [1899] a dozen or score of tents contained the whole
population. By October a town of 5,000 inhabitants fronting
the ocean was crowded for a mile or more along the beach.
Hundreds of galvanized-iron and wooden buildings were
irregularly scattered along two or three thoroughfares,
running parallel with the coast line. There is every
description of building, from the dens of the poor
prospectors, built of driftwood, canvas, and sod, to the large
companies' warehouses, stores, and the army barracks—a city,
as it were, sprung up in the night, built under the most
adverse circumstances on the barren seacoast, a coast without
harbor, all the supplies being landed through the surf. … The
country contributes nothing toward the support of the
population except a few fish and a limited supply of
driftwood.
{8}
ALASKA: A. D. 1900.
Civil Government.
ALASKA: A. D. 1900.
Exploration of Seward peninsula.
"III.
The line of demarcation between the possessions of the High
Contracting Parties upon the Coasts of the Continent and the
Islands of America to the North-West, shall be drawn in the
following manner: Commencing from the southernmost point of
the Island called Prince of Wales Island, which point lies in
the parallel of 54 degrees 40 minutes, North Latitude, and
between the 131st and 133d Degree of West Longitude (Meridian
of Greenwich), the said line shall ascend to the North along
the Channel called Portland Channel, as far as the Point of
the Continent where it strikes the 56th Degree of North
Latitude; from this last mentioned Point the line of
demarcation shall follow the summit of the mountains situated
parallel to the coast, as far as the point of intersection of
the 141st Degree of West Longitude (of the same meridian),
and, finally, from the said point of intersection, the said
Meridian Line of the 141st Degree, in its prolongation as far
as the Frozen Ocean, shall form the limit between the Russian
and British Possessions on the Continent of America to the
North West.
"IV.
With reference to the line of demarcation laid down in the
preceding Article, it is understood:
1st.
That the Island called Prince of Wales Island shall belong
wholly to Russia.
2d.
That wherever the summit of the mountains which extend in a
direction parallel to the Coast, from the 56th Degree of North
Latitude to the point of intersection of the 141st Degree of
West Longitude, shall prove to be at the distance of more than
ten marine leagues from the Ocean, the limit between the
British Possessions and the line of Coast which is to belong
to Russia, as above mentioned, shall be formed by a line
parallel to the windings of the Coast, and which shall never
exceed the distance of ten marine leagues therefrom."
Map of Alaska.
H. Townsend,
The Alaskan Boundary Question
(Fortnightly Review, September, 1899).
ALEXANDRIA:
Discovery of the Serapeion.
ALEXANDRIA:
Patriarchate re-established.
AMATONGALAND:
Annexed, with Zululand, to Natal.
AMERICA:
The Projected Intercontinental Railway.
AMERICA, Central.
AMERICAN ABORIGINES.