Download pdf Recent Advances In Information And Communication Technology 2019 Proceedings Of The 15Th International Conference On Computing And Information Technology Ic2It 2019 Pongsarun Boonyopakorn ebook full chapter

You might also like

Download as pdf or txt
Download as pdf or txt
You are on page 1of 54

Recent Advances in Information and

Communication Technology 2019:


Proceedings of the 15th International
Conference on Computing and
Information Technology (IC2IT 2019)
Pongsarun Boonyopakorn
Visit to download the full and correct content document:
https://textbookfull.com/product/recent-advances-in-information-and-communication-t
echnology-2019-proceedings-of-the-15th-international-conference-on-computing-and-
information-technology-ic2it-2019-pongsarun-boonyopakorn/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Advances in Information Communication Technology and


Computing Proceedings of AICTC 2019 Vishal Goar

https://textbookfull.com/product/advances-in-information-
communication-technology-and-computing-proceedings-of-
aictc-2019-vishal-goar/

Information Systems Architecture and Technology


Proceedings of 40th Anniversary International
Conference on Information Systems Architecture and
Technology ISAT 2019 Part I Leszek Borzemski
https://textbookfull.com/product/information-systems-
architecture-and-technology-proceedings-of-40th-anniversary-
international-conference-on-information-systems-architecture-and-
technology-isat-2019-part-i-leszek-borzemski/

Information Systems Architecture and Technology


Proceedings of 40th Anniversary International
Conference on Information Systems Architecture and
Technology ISAT 2019 Part III Zofia Wilimowska
https://textbookfull.com/product/information-systems-
architecture-and-technology-proceedings-of-40th-anniversary-
international-conference-on-information-systems-architecture-and-
technology-isat-2019-part-iii-zofia-wilimowska/

Information Systems Architecture and Technology


Proceedings of 40th Anniversary International
Conference on Information Systems Architecture and
Technology ISAT 2019 Part II Jerzy ■wi■tek
https://textbookfull.com/product/information-systems-
architecture-and-technology-proceedings-of-40th-anniversary-
international-conference-on-information-systems-architecture-and-
Information Communication and Computing Technology 4th
International Conference ICICCT 2019 New Delhi India
May 11 2019 Revised Selected Papers Abdullah Bin Gani

https://textbookfull.com/product/information-communication-and-
computing-technology-4th-international-conference-
icicct-2019-new-delhi-india-may-11-2019-revised-selected-papers-
abdullah-bin-gani/

Information Technology and Systems Proceedings of ICITS


2019 Álvaro Rocha

https://textbookfull.com/product/information-technology-and-
systems-proceedings-of-icits-2019-alvaro-rocha/

Information and Communication Technology for


Competitive Strategies Proceedings of Third
International Conference on ICTCS 2017 Simon Fong

https://textbookfull.com/product/information-and-communication-
technology-for-competitive-strategies-proceedings-of-third-
international-conference-on-ictcs-2017-simon-fong/

Recent Advances in Information Technology 1st Edition


Waldemar Wójcik

https://textbookfull.com/product/recent-advances-in-information-
technology-1st-edition-waldemar-wojcik/

Advances in Information and Communication Technology


and Systems Mykhailo Ilchenko

https://textbookfull.com/product/advances-in-information-and-
communication-technology-and-systems-mykhailo-ilchenko/
Advances in Intelligent Systems and Computing 936

Pongsarun Boonyopakorn
Phayung Meesad
Sunantha Sodsee
Herwig Unger Editors

Recent Advances
in Information and
Communication
Technology 2019
Proceedings of the 15th International
Conference on Computing and
Information Technology (IC2IT 2019)
Advances in Intelligent Systems and Computing

Volume 936

Series Editor
Janusz Kacprzyk, Systems Research Institute, Polish Academy of Sciences,
Warsaw, Poland

Advisory Editors
Nikhil R. Pal, Indian Statistical Institute, Kolkata, India
Rafael Bello Perez, Faculty of Mathematics, Physics and Computing,
Universidad Central de Las Villas, Santa Clara, Cuba
Emilio S. Corchado, University of Salamanca, Salamanca, Spain
Hani Hagras, Electronic Engineering, University of Essex, Colchester, UK
László T. Kóczy, Department of Automation, Széchenyi István University,
Gyor, Hungary
Vladik Kreinovich, Department of Computer Science, University of Texas
at El Paso, El Paso, TX, USA
Chin-Teng Lin, Department of Electrical Engineering, National Chiao
Tung University, Hsinchu, Taiwan
Jie Lu, Faculty of Engineering and Information Technology,
University of Technology Sydney, Sydney, NSW, Australia
Patricia Melin, Graduate Program of Computer Science, Tijuana Institute
of Technology, Tijuana, Mexico
Nadia Nedjah, Department of Electronics Engineering, University of Rio de Janeiro,
Rio de Janeiro, Brazil
Ngoc Thanh Nguyen, Faculty of Computer Science and Management,
Wrocław University of Technology, Wrocław, Poland
Jun Wang, Department of Mechanical and Automation Engineering,
The Chinese University of Hong Kong, Shatin, Hong Kong
The series “Advances in Intelligent Systems and Computing” contains publications
on theory, applications, and design methods of Intelligent Systems and Intelligent
Computing. Virtually all disciplines such as engineering, natural sciences, computer
and information science, ICT, economics, business, e-commerce, environment,
healthcare, life science are covered. The list of topics spans all the areas of modern
intelligent systems and computing such as: computational intelligence, soft comput-
ing including neural networks, fuzzy systems, evolutionary computing and the fusion
of these paradigms, social intelligence, ambient intelligence, computational neuro-
science, artificial life, virtual worlds and society, cognitive science and systems,
Perception and Vision, DNA and immune based systems, self-organizing and
adaptive systems, e-Learning and teaching, human-centered and human-centric
computing, recommender systems, intelligent control, robotics and mechatronics
including human-machine teaming, knowledge-based paradigms, learning para-
digms, machine ethics, intelligent data analysis, knowledge management, intelligent
agents, intelligent decision making and support, intelligent network security, trust
management, interactive entertainment, Web intelligence and multimedia.
The publications within “Advances in Intelligent Systems and Computing” are
primarily proceedings of important conferences, symposia and congresses. They
cover significant recent developments in the field, both of a foundational and
applicable character. An important characteristic feature of the series is the short
publication time and world-wide distribution. This permits a rapid and broad
dissemination of research results.

** Indexing: The books of this series are submitted to ISI Proceedings,


EI-Compendex, DBLP, SCOPUS, Google Scholar and Springerlink **

More information about this series at http://www.springer.com/series/11156


Pongsarun Boonyopakorn•

Phayung Meesad Sunantha Sodsee


• •

Herwig Unger
Editors

Recent Advances
in Information
and Communication
Technology 2019
Proceedings of the 15th International
Conference on Computing and Information
Technology (IC2IT 2019)

123
Editors
Pongsarun Boonyopakorn Phayung Meesad
Faculty of Information Technology Faculty of Information Technology
King Mongkut’s University of Technology King Mongkut’s University of Technology
North Bangkok North Bangkok
Bangkok, Thailand Bangkok, Thailand

Sunantha Sodsee Herwig Unger


Faculty of Information Technology LG Kommunikationsnetze
King Mongkut’s University of Technology FernUniversität in Hagen
North Bangkok Hagen, Germany
Bangkok, Thailand

ISSN 2194-5357 ISSN 2194-5365 (electronic)


Advances in Intelligent Systems and Computing
ISBN 978-3-030-19860-2 ISBN 978-3-030-19861-9 (eBook)
https://doi.org/10.1007/978-3-030-19861-9
© Springer Nature Switzerland AG 2020
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar
methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt from
the relevant protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authors or the editors give a warranty, expressed or implied, with respect to the material contained
herein or for any errors or omissions that may have been made. The publisher remains neutral with regard
to jurisdictional claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface

Within the last few years, computer scientists and engineers as well as users had to
face rough and fast changes in computer interfaces and of the abilities of machines
and services offered. And this is only the beginning: the research results obtained
let all of us expect more significant progress and challenges in our life within the
next decade. A major contribution to these developments was brought about by
achievements in the area of artificial intelligence. Machine learning, natural lan-
guage processing, speech recognition, image and video processing are only the
major research and engineering directions, which made possible autonomous
driving, language assistants, automatic translation and answering systems as well as
other innovative applications such as more human-oriented interfaces. Those
changes also reflect economic changes in the world, which are increasingly dom-
inated by the needs of an enhanced globalisation and worldwide cooperation (in-
cluding its competitive aspects) and by emerging global problems.
All those developments were mostly carried out by private companies, domi-
nated by a few great players in the Internet business. Universities very often just
follow such fast developments instead of being the motors of innovation and, thus,
may have to justify their existence within the next few years. Consequently, uni-
versities have to rethink their approach to teaching and research and return to be
really innovative in the area of basic research. This includes to abolish the perverse
and inefficient ‘publish or perish’ approach as well as the omnipresent practice of
evaluating ‘all and everything’ and the overwhelming paperwork and to return to
the roots of science, viz. the competition for the best strategic ideas within a dispute
far ahead of what is presently needed, but what will be the challenge in future.
This also concerns scientific conferences, which became more and more ineffi-
cient market places to present, advertise and ‘sell’ one’s latest results and to put
money in the pockets of many commercial organisers, many of which are just like
‘day flies’ and disappear after some very few events, only.
Differing from those events, the International Conference on Computing and
Information Technology (IC2IT) celebrates its 15th anniversary in 2019—in the
year of its major sponsor’s 60th birthday, the King Mongkut’s University of
Technology North Bangkok—by still obeying the classic, less stagy rules of

v
vi Preface

science. To ensure high quality, 59 submissions from 19 countries were thoroughly


reviewed by at least two, usually even three members of the International
Programme Committee (IPC). Through a positive vote of at least two IPC members,
32 submissions have been accepted for the presentation at the conference and
inclusion in the conference proceedings, which are published in the well-established
and worldwide-distributed series on Advances in Intelligent and Soft Computing
edited by Janusz Kacprzyk for seven years now.
In contrast to other conferences, in this volume the above addressed, typical
areas of artificial intelligence such as data mining with its theory and applications,
image and natural language processing as well as the needed computational envi-
ronments are not presented in an isolated manner neglecting the complexity of
contemporary computer systems. Reacting to user needs in real time mostly
requires strong computing facilities and background knowledge, which usually
would overload a single, in particular mobile device. Appropriate achievements in
networking, mobile and cloud computing as well as computer organisation are
incorporated in powerful, innovative solutions. Finally, software development and
security methodologies must generate holistic, optimised solutions improving the
users’ comfort when applying new artefacts.
Naturally, to unite these different experiences and approaches in one conference
and one book requires a great preparation effort. Therefore, the volume’s editors
have to thank the authors as well as the members of the programme committee and
all participating university partners in both Thailand and overseas for their out-
standing support and academic cooperation. As usual, a special ‘thank you’ is given
to all staff members of the Faculty of Information Technology at King Mongkut’s
University of Technology North Bangkok, who have carried out many technical
and organisational tasks.
For many years, the signees conducted and significantly influenced the organi-
sation of the IC2IT conference. They think that it is now time to say ‘goodbye’ and
put the organisation of IC2IT 2020 in the hands of younger colleagues, which
already participated in the organisation of the 2019 event. They are eager to realise
their ideas and bring the needed, new impulses to our event.
Finally, we are convinced that all authors, presenters and participants will make
IC2IT 2019 a successful event. So we wish all of them a pleasant time in Bangkok
and hope that the conference will find the right way for the next generation of
scientists: fruitful discussions of competitive ideas, which sustainable new systems
may incorporate.

July 2019 Phayung Meesad


Herwig Unger
Organisation

Program Committee

M. Ahmed Waikato, New Zealand


M. Aiello Uni Stuttgart, Germany
S. Auwatanamongkol NIDA, Thailand
T. Bernard Li-Parad/Syscom CReSTIC
S. Boonkrong SUT, Thailand
S. Butcharoen TOT, Thailand
M. Caspar DLR, Germany
T. Chintakovid KMUTNB, Thailand
K. Chochiang PSU, Thailand
K. Dittakan PSU, Thailand
T. Eggendorfer HS Weingarten, Germany
M. Hagan Okstate U, USA
W. Halang FernUni, Germany
C. Haruechaiyasak NECTEC, Thailand
W. Jitsakul KMUTNB, Thailand
S. Jungjit Thaksin U, Thailand
M. Komkhao RMUTP, Thailand
P. Kropf UniNE, Switzerland
M. Kubek FernUni, Germany
K. Kyamakya AAU, Austria
U. Lechner UniBW, Germany
H. Lefmann TU Chemnitz, Germany
Z. Li FernUni, Germany
K. Nimkerdphol RMUTT, Thailand
K. Pasupa KMITL, Thailand
M. Phongpaibul TU, Thailand
J. Polpinij MSU, Thailand
N. Porrawatpreyakorn KMUTNB, Thailand

vii
viii Organisation

M. Sodanil KMUTNB, Thailand


S. Sodsee KMUTNB, Thailand
T. Srikacha TOT, Thailand
T. Sucontphunt NIDA, Thailand
C. Thaenchaikun PSU, Thailand
J. Thaenthong PSU, Thailand
D. Thammasiri NPRU, Thailand
K. Thongglin PSU, Thailand
J. Thongkam MSU, Thailand
N. Tongtep PSU, Thailand
D. H. Tran HNUE, Vietnam
D. Tutsch Uni Wuppertal, Germany
M. Weiser Okstate U, USA
K. Woraratpanya KMITL, Thailand

Organising Partners

In cooperation with
King Mongkut’s University of Technology North Bangkok (KMUTNB)
FernUniversitaet in Hagen, Germany (FernUni)
Chemnitz University, Germany (CUT)
Oklahoma State University, USA (OSU)
Edith Cowan University, Western Australia (ECU)
Hanoi National University of Education, Vietnam (HNUE)
Gesellschaft für Informatik (GI)
Mahasarakham University (MSU)
Ubon Ratchathani University (UBU)
Kanchanaburi Rajabhat University (KRU)
Nakhon Pathom Rajabhat University (NPRU)
Phetchaburi Rajabhat University (PBRU)
Rajamangala University of Technology Krungthep (RMUTK)
Rajamangala University of Technology Thanyaburi (RMUTT)
Prince of Songkla University, Phuket Campus (PSU)
National Institute of Development Administration (NIDA)
Sisaket Rajabhat University (SSKRU)
Council of IT Deans of Thailand (CITT)
IEEE CIS Thailand
Contents

Data Mining
Dual Increment Shapelets: A Scalable Shapelet Discovery
for Time Series Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Nattakit Vichit and Chotirat Ann Ratanamahatana
Analyzing and Visualizing Anomalies and Events in Time Series
of Network Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Qinpei Zhao, Yinjia Zhang, Yang Shi, and Jiangfeng Li
The Grid-Based Spatial ARIMA Model: An Innovation
for Short-Term Predictions of Ocean Current Patterns
with Big HF Radar Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Ratchanont Pongto, Nopparat Wiwattanaphon, Peerapon Lekpong,
Siam Lawawirojwong, Siwapon Srisonphan, Kerk F. Kee,
and Kulsawasd Jitkajornwanich
Parameter-Free Outlier Scoring Algorithm Using the Acute Angle
Order Difference Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Pollaton Pumruckthum, Somjai Boonsiri, and Krung Sinapiromsaran
Improved Weighted Least Square Radiometric Calibration Based
Noise and Outlier Rejection by Adjacent Comparagraph
and Brightness Transfer Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Chanchai Techawatcharapaikul, Pradit Mittrapiyanurak,
and Werapon Chiracharit

Application of Data Mining, Language, and Text Processing


Fuzzy TF-IDF Weighting in Synonym for Diabetes Question
and Answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Ketsara Phetkrachang and Nichnan Kittiphattanabawon

ix
x Contents

Feature Comparison for Automatic Bug Report Classification . . . . . . . . 69


Bancha Luaphol, Boonchoo Srikudkao, Tontrakant Kachai,
Natthakit Srikanjanapert, Jantima Polpinij, and Poramin Bheganan
A Novel Three Phase Approach for Single Sample Ear Recognition . . . 79
Nitin Kumar
Comparison of Thai Sentence Sentiment Tagging Methods
Using Thai Sentiment Resource . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
Kanlaya Thong-iad and Ponrudee Netisopakul
Collecting Child Psychiatry Documents of Clinical Trials
from PubMed by the SVM Text Classification Method
with the MATF Weighting Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
Jantima Polpinij, Tontrakant Kachai, Kanyarat Nasomboon,
and Poramin Bheganan

Image Processing
Instance-Based Learning for Blood Vessel Segmentation
in Retinal Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
Worapan Kusakunniran, Sarattha Kanchanapreechakorn,
and Kittikhun Thongkanchorn
Floor Projection Type Serious Game System for Lower Limb
Rehabilitation Using Image Processing . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Kazuo Hemmi, Yuki Kondo, Takuro Tobina, and Takeshi Nishimura
Image Processing Technique for Gender Determination
from Medical Microscope Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Wichan Thumthong, Hathaichanok Chompoopuen, and Pita Jaupunphol
Accelerate the Detection Frame Rate of YOLO
Object Detection Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
Wattanapong Kurdthongmee
Analyze Facial Expression Recognition Based on Curvelet
Transform via Extreme Learning Machine . . . . . . . . . . . . . . . . . . . . . . 148
Sarutte Atsawaruangsuk, Tatpong Katanyukul, and Pattarawit Polpinit
Ensemble Model for Segmentation of Lateral Ventricles from 3D
Magnetic Resonance Imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
Akadej Udomchaiporn, Khitichai Lertrungwichean, Pokpakorn Klinkasen,
and Chawanwut Nuchprasert
Deep Convolutional Neural Network with Edge Feature
for Image Denoising . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
Supakorn Chupraphawan and Chotirat Ann Ratanamahatana
Contents xi

The Combination of Different Cell Sizes of HOG with KELM


for Vehicle Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
Natthariya Laopracha

Network, Cloud and Management


Dynamic Data Management for an Associative P2P Memory . . . . . . . . 193
Supaporn Simcharoen
Extremely Fast Neural Computation Using Tally
Numeral Arithmetic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
Kosuke Imamura
Traceable CP-ABE for Outsourced Big Data in Cloud Storage . . . . . . . 213
Praveen Kumar Premkamal, Syam Kumar Pasupuleti,
and P. J. A. Alphonse
A Novel Solution for Virtual Server on the Data Consistency
Maintenance in Cloud Storage Systems . . . . . . . . . . . . . . . . . . . . . . . . . 227
Van Thang Doan, Vo Quang Hoang Khang, Ha Huy Cuong Nguyen,
Cong Phap Huynh, and Phayung Meesad
Data Integration Patterns in the Context of Enterprise
Data Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
Roland Petrasch
Evolutionary Dynamics of Service Provider Legacy Network
Migration to Software Defined IPv6 Network . . . . . . . . . . . . . . . . . . . . 245
Babu R. Dawadi, Danda B. Rawat, and Shashidhar R. Joshi
Hiding Patient Injury Information in Medical Images
with QR Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
Akkarat Boonyapalanant, Mahasak Ketcham,
and Manussawee Piyaneeranart
Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
Data Mining
Dual Increment Shapelets: A Scalable Shapelet
Discovery for Time Series Classification

Nattakit Vichit and Chotirat Ann Ratanamahatana(&)

Department of Computer Engineering, Faculty of Engineering,


Chulalongkorn University, 254 Phayathai Road, Pathumwan,
Bangkok 10330, Thailand
6070178221@student.chula.ac.th,
chotirat.r@chula.ac.th

Abstract. As time series data become more complex and users expect more
sophisticated information, numerous algorithms have been proposed to solve
these challenges. Among those algorithms to classify time series data, shapelet –
a discriminative subsequence of time series data – is considered a practical
approach due to its accurate and insightful classification. However, previously
proposed shapelet algorithms still suffer from exceedingly high computational
complexity, as a result, limiting its scalability to larger datasets. Therefore, in
this work, we propose a novel algorithm that speeds up shapelet discovery
process. Our algorithm so called “Dual Increment Shapelets (DIS)” is a com-
bination of two-layered incremental neural network and filtering process based
on subsequence characteristics. Empirical experiments on forty datasets evi-
dently demonstrate that our proposed work could achieve large speedup while
maintaining its accuracy. Unlike the previous algorithm that mainly emphasizes
speedup of the search algorithm, DIS essentially reduces the number of shapelet
candidates based on subsequence characteristics. As a result, our DIS algorithm
could achieve more than three orders of magnitude speedup, comparing with the
baseline algorithms, while preserving the accuracy of the state-of-the-art
algorithm.

Keywords: Time series  Data mining  Classification  Shapelet

1 Introduction

A time series is a sequence of data point which represents data in a time interval. Apart
from classical time series data such as stock market data, other types of data have
recently been shown to work effectively and efficiently for various data mining tasks
once transformed into time series data, including shape classification, movement
tracking, medical diagnosis (ECG/EKG, EEG, etc.), motif discovery, anomaly detec-
tion, classification, clustering, etc.
In working with time series data, many challenges that affect the performance of the
time series mining tasks include occlusion, distortion of warping, uniform scaling,
uncertainty, and wandering baselines [1]. Therefore, the algorithms must be designed to
overcome the complexity of a time series with such invariances.

© Springer Nature Switzerland AG 2020


P. Boonyopakorn et al. (Eds.): IC2IT 2019, AISC 936, pp. 3–14, 2020.
https://doi.org/10.1007/978-3-030-19861-9_1
4 N. Vichit and C. A. Ratanamahatana

A large amount of studies have been conducted to improve time series classification
tasks. A well known approach is based on distance, such as Euclidean Distance (ED) or
Dynamic Time Warping (DTW) combined with 1-NN [2]. However, these algorithms
have been shown to be sensitive to missing data that might lead to misclassification. To
solve this problem, many researchers opted to classify time series data by discovering
and using local features rather than the whole time series sequence, e.g., interval-based
classifiers, dictionary-based classifiers, and shapelet-based classifiers.
Above all three aforementioned algorithms, the shapelet-based classifiers have been
recently shown to provide the most accurate and interpretable results with the speediest
testing. Due to its flexibility, the shapelet-based classifiers are applicable for both local
features and whole time series. This flexible quality allows the extensive improvement
of the classification accuracy. Comparing with the lazy algorithm such as DWT based
1-NN classifier, the eager algorithm like shapelet-based classifiers could save more
time to classify the target classes. Moreover, shapelet-based classifiers provide inter-
pretable results, which facilitate the domain experts in the fields to effectively interpret
the results. For example, the shapelets that are generated by the shapelet-based clas-
sifiers could evidently provide prominent features of ‘Urtica Dioica’ and ‘Verbena
Urticifolia’ [3]. Based on these shapelet-based classifier results, the botanists could
identify the types of the leaves by having only a quick glance of the shapelets.
Another application of shapelet-based classifiers are classification of video clips of
actions; a hand’s centroid position in each frame is tracked. Figure 1 compares time
series sequences of a 150-frame video clip between the actions of a person grasping a
gun from a holster, pointing it to a target, then putting it back to the holster versus
another person resting his hand on the side, only pointing his finger to a target, then
putting his hand back to the resting position. As seen in the figure, the 100th–130th time
interval clearly shows the difference between the person holding a gun and the one
without. Even though the two time series sequences demonstrate the discrepancies
between the two actions in the 50th–90th time interval (different heights of the hand
pointing a finger vs. pointing a gun), the shapelet-based classifiers see them as irrel-
evant invariances and successfully differentiate the two actions using only the essential
intervals. It can be observed that the overly captured invariances could mislead the
results. Thus, shapelet-based classifiers are considered more appropriate and effective.

Fig. 1. A comparison between time series sequences of the actions of a person holding and
pointing a gun (Gun time series) and a person pointing a finger (No gun time series).
Dual Increment Shapelets: A Scalable Shapelet Discovery for Time Series Classification 5

In spite of its benefits, shapelet-based classifiers still suffer from a major drawback;
they are too slow that they become infeasible with large datasets. Even in state-of-the-
art approach, the algorithm still requires to generate a large number of shapelet can-
didates. A group of shapelet candidates could be as large as Oðn2 m4 Þ when n is the
number of time series in the dataset, and m is the length of each time series. For
instance, if a dataset contains only 70 instances, where each instance is 500 data points
long, the shapelet candidates could accumulate to more than 1012 candidates, which
become infeasible for the classifier to give prediction results within reasonable amount
of time unless adequate number of candidates are pruned out.
To resolve the issues of infeasibility and practicality of the existing shapelet-based
classifiers, we design a new algorithm, which optimizes the shapelet candidate selec-
tion process, while preserving the accuracy of the results and no-false-dismissal
property. More specifically, our proposed work can effectively prune out shapelet
candidates using a so-called “Dual Increment Shapelets (DIS)” that is a combination of
two-layered incremental neural network and filtering process based on subsequence
characteristics.
The rest of the paper is organized as follows. Section 2 gives fundamental back-
ground on shapelets and reviews on its related work. Section 3 presents our proposed
algorithm. Subsequently, the empirical evidence, interpretation of the results, and
discussions are provided in Sect. 4. Finally, the conclusion, limitations, and sugges-
tions for future studies are discussed in Sect. 5.

2 Related Work
2.1 Shapelet
Shapelet is introduced by [4] as a decision tree algorithm to search for the best shapelet
candidates. Shapelet candidates are generated through testing on every possible sub-
sequence length. All shapelet candidates will then be evaluated on their qualities using
an information gain measure, which returns the best split point. For example, as shown
in Fig. 2, a group of data in the first half (represented by circles) has better quality than
a group of data in the second half (represented by a mixture of rectangles and a circle).

Fig. 2. A split point is determined to provide the best quality within each group of the data [4]

Since the traditional process of finding shapelets is very time-consuming, Fast


shapelet [3] has been introduced to reduce the running time through dimensionality
reduction (SAX) [5] and random projection [6] to group similar shapelets. Regardless
of an order of magnitude speedup, the accuracy has to be significantly sacrificed.
Shapelet Transform is then proposed by [5]. Its shapelet finding process is sepa-
rated from the classifier, which allows us to use any classifiers and shapelet quality
6 N. Vichit and C. A. Ratanamahatana

measures. Its running time could be improved by opting in faster shapelet quality
evaluation such as Mood’s Median and F-stat [7]. By using F-stat quality evaluation,
more than two orders of magnitude speedup can be achieved because it removes the
need of checking every node in the subtrees. To improve the accuracy, inferior decision
tree classifiers could be replaced with better algorithms such as Rotation Forest [8] or
Support Vector Machine [9].
Learning Shapelet are introduced by [10]. Instead of looping and evaluating the
accuracy of shapelet candidates one by one, learning shapelet uses a gradient descent to
learn near-optimal shapelets.
To compare among these baseline algorithms, both accuracy and speed are desir-
able. In terms of accuracy, Shapelet Transform is the most accurate as referred to the as
state of the art, followed by Learning Shapelet and Fast Shapelet, respectively. On the
contrary, as expected, in terms of speed, Shapelet Transform is the slowest, while Fast
Shapelet is the fastest.

2.2 Shapelet Characteristic Filtering


As shapelet candidates vary in their shapes and characteristics, we can use these
attributes to filter out useless candidates. In fact, key points of shapelets can be selected
using this criteria [11]; the points that most reduce the distance to original time series
are selected to generate shapelet candidates.
As seen in Fig. 3, selected points (KP3-KP4 and KP5-KP6-KP7) are used to
generate shapelet candidates. With this strategy, we do not only reduce a lot of data
points and shapelet candidates, but also maintain the best-quality candidates.

Fig. 3. Examples of shapelet candidates generated by key points. Candidate S1 is generated


from key point 3 (KP3) to key point 4 (KP4), and candidate (S2) is generated from key point 5
(KP5) to key point 7 (KP7)

2.3 Self-organizing Incremental Neural Network


Self-organizing incremental neural network is introduce by [12]. It uses DTW com-
bined with a neural network to cluster the data in one pass. The amount of an output
group is varied by the size of the dataset. Incremental shapelet [13] acts as a shapelet
candidate selector. First, it generates shapelets by using the skip length parameter in
shapelet length. Then, it employs self-organizing incremental neural network method to
Dual Increment Shapelets: A Scalable Shapelet Discovery for Time Series Classification 7

select shapelet candidate. However, this skip length parameter can significantly reduce
classification accuracy [14]. Another downside is that the incremental shapelet also
limits the same shapelet candidate length to be grouped together.

3 Our Proposed Work: Dual Increment Shapelets

Our algorithm is divided into 3 steps as follows:

3.1 Shapelet Candidate Selection


Not all shapelet candidates have the same quality. A Candidate with large variation is
considered having higher quality than a candidate with small variation (e.g., straight
line). We adopt a Local Farthest Deviation Points (LFDP) algorithm [15] to select only
candidates with high variation. For each time series sequence, LFDP algorithm first
marks the start and end of the set of selected points and then set the distance to infinity
(lines 2–4). LFDP algorithm continues selecting a point until its distance to the original
time series falls below the threshold (lines 5–8). The selected points are all appended
together to the list of important points (line 9). These reference points are used to mark
the starting and ending points of a shapelet candidate, as shown in Fig. 3. As a result,
we could filter out a large number of shapelet candidates, while maintaining shapelet
candidate’s quality.

3.2 Incremental Neural Network


An Incremental Neural Network is our preferable choice of clustering algorithm as it is
a one-pass algorithm, which is suitable for large datasets. Moreover, it has an ability to
self-adjust the number of groups by candidate’s variation. We adopt this idea from
work in [11]. However, instead of one single layer, we have modified the algorithm to
become a two-layered network, i.e., Candidates Averaging layer, and Class Purity
layer.
8 N. Vichit and C. A. Ratanamahatana

Candidates Averaging Layer


A large number of candidates are similar in shape and size. Similar candidates would
result in similar shapelet quality. If we cluster similar shapelets together, we would be
able to estimate the quality of shapelets groupwise rather than individually. We use the
length and shapelet average within the same group to calculate shapelet quality.
The first layer will average candidates of the same length from the same original
time series sequence, whose distance between the candidates was below the threshold.
As shown in Algorithm 2, the inputs are candidates that are grouped by their length.
For all candidates with the same length (line 4), we insert them one by one into our
incremental neural network (line 5). After inserting all data with the same length, we
then average of all candidates in each node and append it to the list of filtered can-
didates (line 6). The algorithm continues until all lengthwise candidate grouping are
completed (line 2). The outputs are the averaged filtered candidates, based on their
lengths.

Class Purity Layer


The measurement criteria for the best shapelet relies on the power of class discrimi-
nation [16]. To measure shapelet quality, we adopt the idea from [3] and [16] in
measuring the quality of the group using class purity of a shapelets group. During the
shapelet grouping process, we insert the class metadata into the groups to help in group
purity calculation. We used this approximated group quality to filter out some low-
quality candidate groups instead of selecting each candidate one by one. Regardless of
the length, this second layer could effectively group similar shapelets together.
Dual Increment Shapelets: A Scalable Shapelet Discovery for Time Series Classification 9

3.3 Dual Increment Shapelets


With those strategies above, we could actually reduce the number of candidates to be
evaluated by a few orders of magnitude. The remaining groups have good candidate
potentials. All of the remaining candidates are then used to find the distance to all time
series in the dataset. The underlying algorithm is very similar to the shapelet transform.
Lastly, the results of the transform step are classified by SVM algorithm with a purpose
to find the best shapelet candidates. With L1 regularization [17], weights are consol-
idated with some candidates that contribute the best accuracy. As a result, an acceptable
number of shapelets are acquired.

The dual increment shapelets algorithm starts by first selecting important points
obtained from the LFDP algorithm. Then, those selected points are used to generate
shapelet candidates (lines 1–2). The generated candidates were transferred to the first
layer of the incremental neural network to average candidates that share the same
length (line 3). After that, these candidates were transferred to the second layer of the
incremental neural network which hosted a group of similar shapelet candidates (line
4). The quality of every group in layer 2 is measured by the class purity (lines 6–7).
The selected groups above the threshold are kept in the selected candidate list (line 8).
Then, they are classified by SVM with L1 regularization to obtain the best shapelets
(line 9).

4 Experiments and Results

To examine the accuracy and effectiveness of our proposed DIS, the datasets from the
UCR repository [18] are tested across all four shapelet-based algorithms: our proposed
DIS, Shapelet Transform (ST), Learning Shapelet (LS), and Fast Shapelet (FS). In this
paper, the three algorithms (ST, LS, and FS) will be referred to as the state-of-the-art
baselines. Using Java WEKA framework [19], we acquire the codes from [10] to
reimplement and recreate the results of the baselines. However, due to the exceedingly
large time complexity, some baseline algorithms are unable to complete the
10 N. Vichit and C. A. Ratanamahatana

classification in some large datasets within 24-h period. Specifically, only 40 datasets
from the total of 76 datasets could be run by Shapelet Transform algorithm. Therefore,
even though our proposed DIS algorithm could be successfully trained within the time
limit, for fair comparison, only these 40 datasets are chosen for the experiments.

4.1 Running Time Comparison to the Baselines


To evaluate the performance, every algorithm is repeated 5 times, and the average
running time is reported. Table 1 reports the average running time for all shapelet-
based classifiers.

Table 1. Average running time in millisecond for shapelet-based classifiers


Datasets FS LS ST DIS
Adiac 101177 99454633 10692757 14834
ArrowHead 10815 228300 4841654 1596
Beef 75734 1915378 3123344 9364
BeetleFly 22934 179908 5866064 9474
BirdChicken 17178 159999 6281603 8060
Car 119352 3926827 50858285 52557
CBF 3301 49966 213188 516
ChlorineConcentration 191170 1820407 52458469 12248
Coffee 6244 76629 302256 1013
DiatomSizeReduction 6882 309793 93089 831
DistalPhalanxOutlineAgeGroup 10385 410264 15641115 2609
DistalPhalanxOutlineCorrect 23501 287386 10530021 3377
DistalPhalanxTW 13456 1933645 5874907 2045
ECGFiveDays 2267 15433 159845 224
FaceAll 190672 37933408 78029678 26308
FaceFour 24148 500485 7792586 1994
FacesUCR 54225 12490998 19879931 5122
GunPoint 2578 45133 895632 679
ItalyPowerDemand 1166 4777 2730 295
Lightning7 87039 4702008 54935741 19362
MedicalImages 46871 7818573 28067449 6739
MiddlePhalanxOutlineAgeGroup 10184 412895 8862617 1526
MiddlePhalanxTW 14331 1917648 14721367 3744
MoteStrain 1738 6538 10036 175
OliveOil 47847 1623811 3082948 8836
PhalangesOutlinesCorrect 88116 968687 99125785 16083
Plane 14181 1658518 10914489 2583
ProximalPhalanxOutlineAgeGroup 9819 410964 8685065 1355
ProximalPhalanxOutlineCorrect 18283 269331 7089362 2377
(continued)
Dual Increment Shapelets: A Scalable Shapelet Discovery for Time Series Classification 11

Table 1. (continued)
Datasets FS LS ST DIS
ProximalPhalanxTW 10817 1938259 9705892 1767
ShapeletSim 31187 210767 542623 7698
SonyAIBORobotSurface1 1504 4862 8410 220
SonyAIBORobotSurface2 1561 6323 12257 281
SwedishLeaf 104532 36721071 86573650 16996
Symbols 24673 1648839 15207771 4569
SyntheticControl 10487 934441 2413403 3405
ToeSegmentation1 11377 109016 8114483 3792
ToeSegmentation2 14778 197764 17057920 6911
Trace 45370 1624364 87329377 15214
TwoLeadECG 1468 7206 4381 175

Table 1 shows that our proposed DIS algorithm evidently and significantly out-
performs all other algorithms in terms of the running time. Especially in multiclass
problems, DIS has an extra ability to prune more candidates than the binary class
problems, being as much as 6,400 times faster in some datasets. These results indicate
that the characteristics of the datasets (length, number of instances, number of classes,
etc.) could be influential factors affecting the speed of DIS. In the other words, the
characteristics of the dataset directly affect the pruning power of the shapelet candi-
dates, e.g., ItalyPowerDemand vs. FaceAll.
The types of algorithms also influence the running time. The results from Table 1
revealed that our DIS outperformed all the baselines since it is a one-pass candidate
filtering algorithm. To be more specific, comparing to DIS, Shapelet Transform is the
slowest one (2460.567 times slower), followed by Learning Shapelet (525.664 times
slower), and Fast Shapelet (6.180 times slower), respectively, as shown in Table 2. It
can be explained that Brute Force algorithm like Shapelet Transform requires very
large running time to run. The Gradient descent algorithm underlying the Learning
Shapelet is faster than the Brute Force algorithm, and the dimensionality reduced
algorithm could greatly help reduce the running time, making Fast Shapelet the fastest
one among all the baselines.

Table 2. A comparison of the results of average speedup of DIS and the baselines
Fast shapelet Learning shapelet Shapelet transform
6.180 525.664 2460.567

4.2 Accuracy Comparison to the Baselines


In terms of accuracy, our proposed DIS and the baselines were compared in order to
compare the accuracy of DIS against all the baselines. It should be noted that the
Shapelet Transform employed in this session was parameterized using the same
parameters reported in [20]. The results are provided in Table 3.
12 N. Vichit and C. A. Ratanamahatana

Table 3. Classification accuracies for shapelet-based classifiers


Dataset FS LS ST DIS
Adiac 0.550 0.519 0.130 0.729
ArrowHead 0.577 0.823 0.720 0.777
Beef 0.567 0.800 0.567 0.767
BeetleFly 0.650 0.750 0.800 0.800
BirdChicken 0.900 0.800 0.750 0.900
Car 0.733 0.800 0.633 0.883
CBF 0.919 0.990 0.956 0.941
ChlorineConcentration 0.591 0.591 0.616 0.648
Coffee 0.964 1.000 1.000 0.964
DiatomSizeReduction 0.879 0.967 0.765 0.941
DistalPhalanxOutlineAgeGroup 0.640 0.719 0.691 0.698
DistalPhalanxOutlineCorrect 0.728 0.786 0.670 0.775
DistalPhalanxTW 0.655 0.626 0.647 0.647
ECGFiveDays 0.995 1.000 1.000 1.000
FaceAll 0.620 0.775 0.653 0.778
FaceFour 0.920 0.966 0.750 0.943
FacesUCR 0.738 0.944 0.671 0.878
GunPoint 0.940 1.000 0.953 0.993
ItalyPowerDemand 0.906 0.963 0.943 0.958
Lightning7 0.630 0.808 0.425 0.740
MedicalImages 0.605 0.686 0.471 0.689
MiddlePhalanxOutlineAgeGroup 0.539 0.584 0.532 0.584
MiddlePhalanxTW 0.461 0.506 0.526 0.494
MoteStrain 0.798 0.858 0.839 0.874
OliveOil 0.633 0.700 0.767 0.867
PhalangesOutlinesCorrect 0.724 0.748 0.685 0.833
Plane 0.990 1.000 0.924 1.000
ProximalPhalanxOutlineAgeGroup 0.776 0.815 0.737 0.834
ProximalPhalanxOutlineCorrect 0.838 0.849 0.715 0.887
ProximalPhalanxTW 0.727 0.810 0.654 0.790
ShapeletSim 1.000 0.978 1.000 0.972
SonyAIBORobotSurface1 0.686 0.827 0.947 0.429
SonyAIBORobotSurface2 0.790 0.890 0.876 0.824
SwedishLeaf 0.789 0.917 0.755 0.928
Symbols 0.937 0.930 0.823 0.930
SyntheticControl 0.937 0.997 0.957 0.990
ToeSegmentation1 0.943 0.930 0.934 0.956
ToeSegmentation2 0.692 0.923 0.892 0.800
Trace 1.000 1.000 0.980 1.000
TwoLeadECG 0.946 0.997 0.970 0.991
Dual Increment Shapelets: A Scalable Shapelet Discovery for Time Series Classification 13

As the tradeoff for speed, our accuracies are expected to be lowered. However, DIS
does surprisingly well, being a winner among all baselines in as many as 18 datasets.
The Gradient descent algorithm in Learning Shapelet could yield good results as
expected, but the Fast Shapelet is the least accurate despite its fastest speed, due to the
dimensionality reduced data.
Table 4 reports the performance of our proposed DIS algorithm in terms of
accuracy, comparing to the three baselines. Our DIS generally outperforms Fast Sha-
pelet and Shapelet Transform, but is slightly less accurate than the Learning Shapelet.
One possible explanation involves their underlying behavior of the algorithms. The
Gradient descent algorithm could effectively search for the qualified shapelets. The DIS
might prune some qualified shapelet candidates out, leading to the possible deviated
results.

Table 4. A comparison of the results of accuracy of DIS and each baseline algorithm
Result Fast shapelet Learning shapelet Shapelet transform
Win 33 15 30
Lose 4 20 7
Tie 3 5 3

5 Conclusions and Future Work

In this work, a novel algorithm so-called “Dual Increment Shapelets (DIS)” is intro-
duced. We evaluate its performance in terms of speed and accuracy against the three
baselines: Fast Shapelet, Learning Shapelet, and Shapelet Transform. The results reveal
that our DIS is the fastest algorithm comparing with all the baselines. With the sig-
nificantly improved speed, the accuracy of DIS has only been slightly sacrificed. The
key to its success is good candidate filtering ability, which is a result of LFDP can-
didate generation method combined with two-layered Incremental Neural Network.
With these capacities, our DIS offers a promising algorithm to handle large datasets
while maintaining satisfactorily high accuracy. However, as a future work, we could
improve its ability to pin-point qualified shapelets, as a result, speeding up the training
time.

References
1. Keogh, E., Wei, L., Xi, X., Vlachos, M., Lee, S.-H., Protopapas, P.: Supporting exact
indexing of arbitrarily rotated shapes and periodic time series under Euclidean and warping
distance measures. VLDB J. 18, 611–630 (2009)
2. Keogh, E., Ratanamahatana, C.A.: Exact indexing of dynamic time warping. Knowl. Inf.
Syst. 7, 358–386 (2005)
14 N. Vichit and C. A. Ratanamahatana

3. Rakthanmanon, T., Keogh, E.: Fast shapelets: a scalable algorithm for discovering time
series shapelets. In: Proceedings of the 2013 SIAM International Conference on Data
Mining, pp. 668–676. SIAM (2013)
4. Ye, L., Keogh, E.: Time series shapelets: a new primitive for data mining. In: Proceedings of
the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data
Mining, pp. 947–956. ACM (2009)
5. Lin, J., Keogh, E., Wei, L., Lonardi, S.: Experiencing SAX: a novel symbolic representation
of time series. Data Min. Knowl. Discov. 15, 107–144 (2007)
6. Bingham, E., Mannila, H.: Random projection in dimensionality reduction: applications to
image and text data. In: Proceedings of the Seventh ACM SIGKDD International
Conference on Knowledge Discovery and Data Mining, pp. 245–250. ACM (2001)
7. Lines, J., Bagnall, A.: Alternative quality measures for time series shapelets. In: International
Conference on Intelligent Data Engineering and Automated Learning, pp. 475–483. Springer
(2012)
8. Rodriguez, J.J., Kuncheva, L.I., Alonso, C.J.: Rotation forest: a new classifier ensemble
method. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1619–1630 (2006)
9. Rodríguez, J.J., Alonso, C.J.: Support vector machines of interval-based features for time
series classification. In: International Conference on Innovative Techniques and Applications
of Artificial Intelligence, pp. 244–257. Springer (2004)
10. Grabocka, J., Schilling, N., Wistuba, M., Schmidt-Thieme, L.: Learning time-series
shapelets. In: Proceedings of the 20th ACM SIGKDD International Conference on
Knowledge Discovery and Data Mining, pp. 392–401. ACM (2014)
11. Ji, C., Liu, S., Yang, C., Pan, L., Wu, L., Meng, X.: A shapelet selection algorithm for time
series classification: new directions. Procedia Comput. Sci. 129, 461–467 (2018)
12. Okada, S., Hasegawa, O.: Motion recognition based on dynamic-time warping method with
self-organizing incremental neural network. In: 2008 19th International Conference on
Pattern Recognition, ICPR 2008, pp. 1–4. IEEE (2008)
13. Yang, Y., Deng, Q., Shen, F., Zhao, J., Luo, C.: A shapelet learning method for time series
classification. In: 2016 IEEE 28th International Conference on Tools with Artificial
Intelligence (ICTAI), pp. 423–430. IEEE (2016)
14. Bostrom, A., Bagnall, A., Lines, J.: Evaluating Improvements to the Shapelet Transform
(2016). www-bcf.usc.edu
15. Ji, C., Liu, S., Yang, C., Wu, L., Pan, L., Meng, X.: A piecewise linear representation
method based on importance data points for time series data. In: 2016 IEEE 20th
International Conference on Computer Supported Cooperative Work in Design (CSCWD),
pp. 111–116. IEEE (2016)
16. Bostrom, A., Bagnall, A.: Binary shapelet transform for multiclass time series classification.
Trans. Large-Scale Data Knowl.-Cent. Syst. XXXII, 24–46 (2017)
17. Zhu, J., Rosset, S., Tibshirani, R., Hastie, T.J.: 1-norm support vector machines. In:
Advances in Neural Information Processing Systems, pp. 49–56 (2004)
18. Chen, Y., Keogh, E., Hu, B., Begum, N., Bagnall, A., Mueen, A., Batista, G.: The UCR time
series classification archive, July (2015)
19. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA
data mining software: an update. ACM SIGKDD Explor. Newsl. 11, 10–18 (2009)
20. Bagnall, A., Lines, J., Bostrom, A., Large, J., Keogh, E.: The great time series classification
bake off: a review and experimental evaluation of recent algorithmic advances. Data Min.
Knowl. Discov. 31, 606–660 (2017)
Analyzing and Visualizing Anomalies and
Events in Time Series of Network Traffic

Qinpei Zhao, Yinjia Zhang, Yang Shi, and Jiangfeng Li(B)

Tongji University, Shanghai, China


{shiyang,lijf}@tongji.edu.cn

Abstract. The traffic among the hosts and behaviors of the anomalous
hosts in the network is usually complex. In network traffic, there is a key
problem that is how to identify the security incidents. The corresponding
question that who have contributed to the incidents is arisen then. A
method, which detects both anomalies and events at the same time is
quite helpful. A data from network traffic can be composed of the hosts
and different attributes (traffic flow like amount of upload package and
download package) in time series. Based on the structure of the network
traffic data, we propose an anomaly and event detection method based
on the network attributes in time series. The method analyzes both the
host’s behavior and the temporal features of the network traffic.

Keywords: Network traffic · Anomaly detection · Security incidents ·


Time series

1 Introduction
More and more devices are being linked to the Internet, there is gigabytes of
traffic flowing every second through network devices. Therefore, the network
security becomes imperative and network traffic anomaly detection constitutes
an important part of the network security.
The network security is threaten by different attacks from malicious sources,
such as Denial of service attack (DoS), large-volume point-to-point flows (alpha
flows), worms, traffic from a single source to many destinations (point-to-
multipoint) and so on [1]. The attacks usually show features of anomalies in
a traffic of network. For a common network, a traffic flow with unusual and
significant changes is considered as an anomaly. The network traffic has several
kinds of attributes, for example traffic volumes, IP addresses and port num-
bers. The changes of the attributes, such as volume in traffic flows, distribution
patterns of IP source and/or destination addresses and port numbers etc. are
typically signs of anomalies. When there is a significantly changed traffic flow in
the network, it could be identified as an anomaly.
The network traffic data can be obtained at multiple levels of granularity,
such as data characterizing TCP/UDP or data gathered from network equip-
ments (routers, switches). Time as an important factor among the network needs
c Springer Nature Switzerland AG 2020
P. Boonyopakorn et al. (Eds.): IC2IT 2019, AISC 936, pp. 15–25, 2020.
https://doi.org/10.1007/978-3-030-19861-9_2
16 Q. Zhao et al.

to be included into the data [2–4]. Network traffic data with attributes of time
stamps, source and destination IP addresses, port numbers and even text, is
often related to each other and shows dependencies.
There have been a lot of research on the network traffic anomaly detection
methods [5]. Several surveys are available in the literatures. A structured and
comprehensive overview of various facets of network anomaly detection has been
given in [6]. A large number of detection methods and systems, especially tools
are compared in the paper. A tutorial focusing on statistical techniques (also
data mining techniques) is presented in [1]. Three major methods of statistical
techniques: PCA-based [7,8], sketch-based and signal-analysis-based, are com-
pared. Signal processing techniques have been successfully applied to the problem
due to their ability in point change detection and data transformation. Wavelets
are excellent tool for computing aggregates at different scales [9,10]. Combining
various anomaly detectors through consensus optimization is also a choice [11].
The authors in [12] believe that change in network traffic over time is well
suited for detecting uncommon system behaviors. The importance of time infor-
mation is enhanced in anomaly detection. Therefore, we first introduce the
S liding W indow based E vent and Anomaly detection method for the network
traffic in T ime series (SWEAT ). The sliding window is employed for getting
different resolutions of results. The novelty of the method is that not only the
time series of each host is analyzed, but the correlation among the hosts is also
considered. Therefore, the method is capable of detecting the event from time
aspect, and locating the hosts contribute to the event.

2 Preliminaries
The traffic of a network is commonly composed of a set of hosts P =
{p1 , p2 , ..., pN }. Given a raw data of network traffic, the first step is to per-
form data cleaning and feature engineering. After the preprocessing on the raw
data, the analysis method for detecting anomalies and events are employed on
the processed data.
The feature selection [13] procedure usually affects the final results. Let
the behaviors/features of the hosts F = {f1 , f2 , ..., fM } at the time t =
{t1 , t2 , ..., tT }, where N is the size of the hosts, M is the number of features
and T is the length of the time series. Each host is associated with a unique
identifier, which is the IP address in a computer network.
In a real computer network, the hosts are separated into the source IP (SIP )
and destination IP (DIP ). The traffic of the network comes from the connec-
tion among the SIP and DIP (see Fig. 1). A suspicious host (42.81.21.8) can be
detected from its behaviors in the network. The connection among the hosts may
vary over time in the network. For example, several users may share the same
host machine or connect through the same Network Address Translation (NAT )
device at some time. The hosts may change as new users to join the local net-
work and share the same public IP, and others leave. Considering the traffic in
time series, a potential event might be detected and the suspicious hosts can be
Analyzing and Visualizing Anomalies and Events 17

traced when there is heavier traffic happened at certain moment than in normal
times. We define an event in the network traffic as a subset of time stamps when
there have high activities different from usual. The anomalies are the attractive
hosts that have significantly changed behaviors as usual. A host is anomaly or
not in the whole traffic depends on different features and time periods.

Fig. 1. The network connections and traffic details can be viewed in one single graph.
The weight of the lines between the nodes indicates the traffic volume and the color of
the hosts indicates the status, e.g., normal and suspicious hosts.

Anomaly detection in network traffic aims to discover a subset of the hosts


that change significantly on their communication patterns. In most cases, the
anomalies come with the pattern of an event happens. For example, a person
visits a website at 8:00 a.m every day, it is a pattern. However, if he/she suddenly
visits the same or different website many times at around 8:00 p.m. An event can
be detected. Combining the uploading package amount for example, it is possible
to determine the person is anomaly or not. An anomaly is usually considered as
the significantly changed objects from a global aspect. However, an object may
be considered as an anomaly only in a specific context but not only globally. It is
found that almost all the traffic time-varying signal is multi-scales [14]. In order
words, the difference between anomalous traffics and typical/normal traffics is
various in different frequency bands.

3 The SWEAT

In the proposed method, a definition of a novelty score is given, which quantifies


the similarity between two time stamps. It is used for identifying events. Eigen-
vectors are analyzed to spot anomalous patterns and behaviors of the hosts.
18 Q. Zhao et al.

3.1 Event Detection


For the network with hosts P as IP address, the behaviors of the host at certain
time t is XfPt . Define the time series in linear model for a certain feature as:


XtP = aP Xt−w
P
+ bP (1)
w=1

where w is the lag, or the distance between two observations, and a is the
weights of the model, b is the noise part. The covariance of XtP and Xt−w P
,
P P
i.e., cov(Xt , Xt−w ) depends on the separation lag w instead of t.
Therefore, a data D in a matrix composed of N hosts in time series of length
T is generated as: ⎡ P1 ⎤
X1 X2P1 ... XTP1
⎢ X P2 X P2 ... X P2 ⎥
⎢ 1P 2 T ⎥
DN ×T = ⎢ 3 P3
⎢ X1 X2 ... XT ⎥
P3 ⎥
(2)
⎣ ... ... ... ... ⎦
X1PN X2PN ... XTPN
The DN ×T is taken as an input (data representation) of the method, which
is composed of the IP and its time series. An analysis on the data D within a
moving window w is performed after then. For the time series Xt as a stationary
process, (1 ≤ i ≤ w, 1 ≤ j ≤ w) in the window w, the covariance Cw is obtained
by:
w
1  P
Cw = X · XjP (3)
w i=1 i
Since the XtP is a vector with N hosts, the covariance matrix Cw for each
window is a symmetric matrix with size N . In this paper, we employ the Pearson
correlation instead of the covariance for measuring the similarity.
cov(XiP , XjP )
ρ(XiP , XjP ) = (4)
σXiP σXjP

With the moving window, there have been T − w + 1 lagged copies of the time
series, thereby T − w + 1 matrices are generated.
The eigenvectors of the matrices are sorted by decreasing order of the asso-
ciated eigenvalue λk , 0 ≤ λT −w+1 ≤ ... ≤ λ2 ≤ λ1 . As eigenvectors indicate
the directions in which the principal change happens, it is natural to detect the
events through the eigenvector at time t by comparing it with the eigenvectors
at surrounding time stamps. A typical eigenvector V̄t is calculated, which repre-
sents a typical or average behavior of all the hosts within a window w at time t.
It is meant to compare the current behavior of the host at t Vt and the average
behavior V̄t to check if any significant change has made. A novelty score N St
is then defined to estimate the difference by calculating the cosine distance of
them:
Vt  V¯t
N St = 1 − (5)
|Vt ||V¯t |
Another random document with
no related content on Scribd:
coriander seed. The breakfast before us was a most substantial one, there
being no lack either of welcome, which is the best of cheer, or of mutton, fish,
beer, coffee, milk, and stale black rye-bread. Be it remembered that this
breakfast was neither Icelandic, Danish, nor Scotch; but, exhibiting some of
the characteristics of all three, seemed marvellously adapted to our present
requirements in this distant habitat.
We stepped into the store, and saw exposed for sale hardware and soft goods
of all kinds. In a corner were standing lots of quart-bottles gaudily labelled
“essence of punch,” whatever that may be. Mr. Henderson showed me some
specimens of double refracting calc, or Iceland spar, which is obtained in the
neighbourhood. It only occurs in one place of the island, filling a fissure of
greenstone from two to three feet wide and twenty to twenty-five feet long, on
the north bank of the Reydarfiord, about a thousand feet above the sea level.
There, a cascade rushes over the rock, bringing down fragments of the spar
from time to time. The mass itself gets loosened, bit by bit, through the action
of frost on the moisture which enters edgeways between the laminae, wedging
them apart in the direction of the cleavage of the crystals. Transparent
specimens more than a few inches in size are rare and valuable. Mr. Henderson
presented me with a beautiful large semi-transparent chalcedony weighing 1 ℔
7 oz., and some pebbles.
His partner, Mr. Jacobson, an Icelander, also gave me a young raven to make a
pet of. It was this year’s bird and quite tame. I called it Odin; and, having got
hold of an old box, improvised a door from a few spars, that it might have a
sheltered place to roost in at night till it got to the end of its voyage.
I now wandered up the valley, for an hour or two, alone, and sat down on a
slope, on the right side of it, to look around me and rest. The river, near where
I sit, flashes down over a steep rock and forms a fine waterfall, the roaring of
which is echoed from the chimney-capped amphitheatre of hills opposite.
Beneath the fall, it flows peacefully along, runnelling and rippling on, to the
blue fiord, through the quiet green valley. White streamlets of water trickle
down the trap hill-sides, every forty or fifty yards; the whole producing a
continuous quiet murmur or undertone, not unlike that from the wings of an
innumerable swarm of gnats playing in the sunshine on a warm summer’s day,
but ever broken in upon by the clear liquid tinkle of the streamlets nearest us,
heard drip, dripping, with a clear metallic sound which might be compared to
the chirp of the grasshopper. This solitary glen, now lying bathed in light, is
fanned by the gentle breeze, fragrant with the smell of tedded hay, and richly
variegated with wild flowers—harebells, butter-cups, wild thyme, cotton-grass,
and forget-me-nots—a gathered bunch of which is now lying beside me on a
moss-cushioned rock. Quietly musing here on all, of strange or new, I have
seen since leaving home, and dwelling more particularly on the great kindness
I have received at all hands, I feel grateful to God, who has hitherto opened up
a way for me and given me friends amongst strangers wherever I chanced to
wander.
We saw specimens of surturbrand, which crops out on the top of a steep
mountain, at the mouth of the fiord, on the north side, and obtained a few
more geological specimens and plants.
After dinner, I strolled for a quarter of a mile up the valley with Mr.
Henderson and Dr. Mackinlay, to visit the farm behind the store. It consists of
a group of hovels, the walls are stone and turf, the gables wood, and the roofs
covered with green sod. The entrance is a dark muddy passage leading into a
ground-floor apartment as dark and muddy, where, in winter, cattle are kept.
The kitchen is a dirty, smoky, sooty hole, with fish hanging in it to smoke and
dry; a pot of seal-blubber stands steaming in a corner. The fire is raised on a
few stones above the floor, like a smithy-forge; while there is a hole in the roof
for the smoke. Picking our way through another long passage, dark and dirty,
we found a trap-ladder and ascended to a little garret, where I could only walk
erect in the very centre. The apartment was floored and fitted up with bunks
all round the sides and ends. In these box-beds, at least seven people—men,
women and children—sleep at night, and sometimes a few more have to be
accommodated. The little windows in the roof are not made to open, and no
regard whatever is paid to ventilation. Dr. Mackinlay prescribed for an old man
we found lying ill in this abominable fetid atmosphere, where his chances of
recovery were very slight. He was an old farm servant about whom nobody
seemed to care anything.
FARM HOUSE, SEYDISFIORD.

In a little apartment shut off from this one, and in the gable portion of the
building which in this case constitutes the front of the house, an old woman at
the window sits spinning with the ancient distaff,[39] precisely as in the days of
Homer.
To amuse the farmer’s daughters I showed them my sketches, with which they
seemed much interested.
SEYDISFIORD, LOOKING EAST TOWARDS THE SEA.

I understood part of their remarks, and could in some degree make myself
understood by them, with the few Danish and Icelandic words I kept picking
up. On receiving a little money and a few knick-knacks, they, all round, held
out their hands and shook mine very heartily. This, the Icelanders always do,
on receiving a present of anything however trifling.
After sketching the farm-house, I took two views of Mr. Henderson’s store;
one of them from a height behind, looking down towards the fiord, and the
other from the brink of it, looking up the valley. In the latter, a part of the
same farm-house appears, and thus indicates its exact position.[40] With the
assistance of these three sketches taken together, the reader will be enabled to
form some idea, of the appearance presented by this arm of the North Sea.
SEYDISFIORD, BY FARÖE TO LEITH
We sailed from Seydisfiord at half-past six P.M. on Saturday night, direct for the
Faröe islands.
There is a singular cone-shaped mountain called Brimnæs Fjall at the mouth of
the fiord, showing masses of clay-rock alternating with and pushing up trap,
which is deposited in thin layers of perpendicular structure. Several pillars or
shafts are left standing singly on the very summit, and present a very curious
appearance, distinctly relieved against the amber light of the sky. At Dr.
Mackinlay’s request I made a sketch of it.

BRIMNÆS FJALL.

A vessel of Mr. Henderson’s, which had been given up as lost, now


unexpectedly came in sight, which necessitated Mr. Jacobson and a young
Iceland lad, who were en route to Copenhagen, to get on board her and return
to Seydisfiord to look after her cargo, evidently much to their disappointment.
The wild scenery of the coast, especially at Reydarfiord, was strikingly
picturesque.[41]
Mr. Murray, Professor Chadbourne, Mr. Henderson and I walked the deck till
a late or rather an early hour, and watched the fast receding mountain-ranges
of Iceland—pale lilac, mauve, or deep purple—and the distant horns, shading
through similar tints from rose to indigo, all distinctly seen athwart the golden
light of the horizon which for hours has been ebbing slowly and softly away,
but is now on the turn, and about to flow again.

Sabbath, August 7. The weather is fine; no land or sail in sight all day; whales
playing about the ship. Had many pleasant deck-walks and talks, and several
quiet hours, sitting perched on the stem, reading, or watching the prow, below,
cutting and cleaving through the clear green water like a knife.

Monday morning, August 8. We are sailing between two of the Faröe islands,
bright sunshine lighting up all the regularly terraced trap-rocks, caves, and
crevices of this singular group.
I have now got a pet to look after, and, without Shakspere’s authority for it, we
know that
“Young ravens must have food.”

The last thing I did last night was to shut Odin in his box, and the first thing
this morning to let him out again and give him the freedom of the ship. The
bird knows me, is pleased when I scratch his head, and confidingly runs
hopping to me for protection when the boys about the ship teaze him more
than he likes. His fellow traveller, a young Icelandic fox brought on board at
Reykjavik to be sent to the Marquis of Stafford, also runs about the ship
during the day. At first we had some misgivings on the subject; for
“Treason is but trusted like the fox—
Who ne’er so tame, so cherished, and locked up,
Will have a wild trick of his ancestors.”

However, these fears were soon dissipated; for Odin can hold his own, and
when the fox, approaching furtively, uses any liberty with his tail feathers, he
suddenly gets a peck from the bird’s great formidable beak, which he does not
seem much to relish. The salutary fear continues for a short time, is forgotten,
and again the dab comes as a reminder. We were often greatly amused,
watching their individual habits and droll ways, when the one intruded upon
the other. It was half play, half earnest, a sort of armed neutrality with a basis
of mutual respect.
On the west coast of Stromoe is the roofless ruin of the church of Kirkuboe.
It was begun in the twelfth century, but never finished. It is built of stone, has
five large windows and several small ones below; a little farm house or hut,
with red tiles on the roof, stands near it. What a strange lonely place for a
church! Thorshavn lies on the other—the east—side of the island. It is only
five miles distant as the crow flies, but as we have to sail round the south
point, and Stromoe is twenty-seven miles long, we do not reach it till near
noon.
On landing, Mr. Haycock accompanied me to call for Miss Löbner, who has
been poorly ever since her sea voyage. Her mother presented wine, cake,
coffee &c., and was most hospitable. None of us being able to speak Faröese,
at first we felt a little awkward; but a brother of the old lady’s who speaks
English soon came to the rescue and acted as interpreter. With justifiable
pride, they again showed us their flower and kitchen garden. I got the whale-
knives, caps, shoes, gloves &c., which had been made or procured for me
during my absence in Iceland. Ere leaving, Miss Löbner appeared to say adieu!
and insisted on my accepting several other specimens of Faröese workmanship
as remembrances of Thorshavn. No people could have been kinder.
Again, wandering about, we explored the town, looked at the church, stepped
into the stores, passed the governor’s garden, and wandered a mile or two in
that direction in order to obtain a view, and get quit of the fishy smells which
superabound in Thorshavn.
On our return we called for Mr. Müller, who presented me with a copy of the
gospel of St. Matthew in Danish and Faröese, arranged in parallel columns. I
understood him to say that this was the only book ever printed in the Faröese
dialect, and that it is now out of print and very rare. It bears the date of 1823.
Here we saw an old man 76 years of age, an Icelander who has been in Faröe
for the last 40 years. He had spent several years in England, and told me that,
in 1815, he saw our regiments land at Liverpool after the battle of Waterloo.
He speaks English fluently.
A Thames fishing smack, and a sloop from Lerwick, are lying in the bay.
Piping and dancing goes merrily on, on board the latter, relieved by intervals
of music alone. In one of these, we heard “The Yellow Hair’d Laddie,”
rendered with considerable taste, although, doubtless, several “improvements
and additions” were made on the original score.
We took some Faröese boatmen into the saloon of the steamer, and I shall not
soon forget the look of wonder and utter astonishment pourtrayed on their
countenances, as they gazed on the mirrors and everything around, or were
shown things with which they were not familiar and heard their uses explained.
They were greatly pleased with my life-belt. Dr. Mackinlay showed them a
multiplying-glass, and, as it was handed from one to another—each man first
making the discovery of what had so inexplicably excited the wonder of the
last looker—the queer exclamations of amazement accompanied by inimitable
pantomimic gestures reached their culminating point, and were irresistibly
droll.

NAALSÖE—FARÖE.

The weather is all we could desire. The sailors are singing some curious Danish
songs, with the time well marked, as they heave the anchor; and at 20 minutes
past 6 o’clock P.M. we are steaming out of the bay. The evening is lovely, and
the Thermometer, on the deck, stands at 68°. Thorshavn soon disappears, and
we leave the Faröe islands astern, relieved against an amber sky, Dimon being
the most striking and conspicuous of the group. A few stars shone overhead,
and I walked the deck till midnight.
ENTRANCE TO THE SOUND LEADING TO THORSHAVN.

Tuesday, August 9. At breakfast, tasted a whale-steak which Miss Löbner had


yesterday sent on board for me, with particular instructions to the stewardess
to have it properly cooked. The flesh looked and tasted like dry tough beef,
with a slight flavour of venison. The blubber, however was too strong for any
of us to do more than merely satisfy—not gratify—our curiosity.
The day was lovely. Professor Chadbourne invited me to visit and spend a
month with him during his holiday. Indeed, cordial, pressing invitations, all
round, were the order of the day. As fellow-travellers we had been happy
together, and felt sorry at the near prospect of our little party being broken up
and scattered; for several valued friendships had been formed.
Between three and four o’clock in the afternoon, the thermometer indicated
98° in the sun and 75° in the shade. Dr. Mackinlay showed me an old Danish
dollar he had got, in change, at Reykjavik; it bore the date of A.D. 1619, the year
of the landing of the Pilgrim Fathers from the May Flower. Part of the day was
spent in writing out these pages from my diary. In the evening we saw, far to
our left, faint and dim on the horizon line, the north-west islands of Shetland;
and by a quarter to 8 o’clock P.M. were sailing twenty miles to the west of Fair
Isle, which lies between Orkney and Shetland. Both groups are in sight. We
have not seen a sail since we left Faröe, and now, what we at first fancied to be
one, off the north end of the Orkneys, turns out to be a light-house, rising
apparently from the sea, but in reality from low lying land which is yet below
the horizon.
The sunset to-night is gorgeous; cavernous recesses opening through a dense
purple cloud-bank into glowing regions of fire; while broad flashing gleams ray
out on every side athwart the sky, as if from furnace-mouths. Then we have
moonlight on a sea smooth as glass, and not even a ripple to be seen. The
Orkney light-house, now gleaming like a setting star, is left far astern. The
phosphoresence along the vessel’s side and in her wake is most brilliant; while,
seething, electric-like, from the screw, it rivals the “churned fire-froth” of the
demon steed. The moon, half-hid, is at times deep crimson and again bright
yellow. Many falling stars are shooting “madly from their spheres;” not that
our music lured them, although, “on such a night” of nights, when all is
harmonious, we cannot but sing. Mr. Murray gives us “Home, sweet home”
and “The last rose of summer,” and ere retiring at midnight, all of us join
together in singing the “Spanish Chant.”

Wednesday morning, August 10. We are off Inverness; wind a-head and rising.
Professor Chadbourne to-day gave me an oak-leaf which he plucked from the
tree, at Upsala, planted by Linnæus with his own hands. Wrote as long as the
heaving of the ship would admit of it, then arranged botanical specimens and
read Wordsworth. The wind is blowing so fresh, off Peterhead, that, with full
steam, we are not making above one and a half knots; and at times can scarcely
keep any way on. Passed the Bell-Rock; the sea still rising. Went to bed at 11
o’clock P.M.; vessel pitching a good deal.

Thursday morning, August 11. Rose at four o’clock and was on deck ere the
Arcturus dropt anchor in Leith Roads. But as we cannot get our traps on shore
till the custom-house officer comes at nine o’clock to overhaul them, we
remain and breakfast on board. The examination made, at half-past ten o’clock
A.M., we landed by a tug steamer, and made for our respective railway stations,
each, on parting, bidding the other “a bright adieu!” in the hope that it might
only be for “a brief absence!” “Odin” was in good feather: his owner sun-
bronzed and strong.
At length, comfortably ensconsed in the fast express, I lay back in the corner
of a compartment, closed my eyes and resigned myself to see pleasant pictures
and dream waking dreams—of snow jökuls, volcanoes, glaciers, and ice-fields;
of geysers, mud-cauldrons, and sulphur-pits; of lava plains, black, wierd and
blasted, or dreary wastes of ice; of deep rapid rivers, flashing waterfalls, leaping
torrents; of frightful chasms, rugged cliffs, and precipitous mountains
mirrored in deep blue fiords; of pathless stony deserts, enlivened at times with
oasis-like spots of tender green herbage and bright coloured flowers; of wild
break-neck rides, over bare rocks, among slabs and lava-blocks of all shapes
and sizes and lying in every conceivable direction; through volcanic sands and
scoriæ; by red and black vetrified craters, or across dangerous fords; of
multifarious scamperings too, and mud-plashings over hill and dale; or wild
rides down rocky steeps, not on a phantom steed, but on a sure-footed Iceland
pony; of pleasant companionship by the way; of cordial welcome and great
kindness received, in quiet homesteads, and at all hands from the people,
wherever we went; then again of Frost contending with Fire, and of all the
varied and marvellous phenomena of Iceland, that singularly interesting island
in the lone North Sea.

STROMOE—FARÖE.
APPENDIX.
I.

ICELANDIC STORIES AND FAIRY TALES


TRANSLATED INTO ENGLISH BY THE REV. OLAF PÁLSSON, DEAN
AND RECTOR OF REYKJAVIK CATHEDRAL. REVISED AND
EDITED BY DAVID MACKINLAY AND ANDREW JAMES
SYMINGTON.

STORIES OF SÆMUNDUR FRODI, CALLED THE


LEARNED.[42]

I. THE DARK SCHOOL.

Long, long ago, when Trolls and Giants lived among men, there was a famous
school where curious youths were taught the mysteries of witchcraft. France
and Germany both claim the honour of it, but no one knows where it really
was.
It was kept in a dismal cavern, deep underground, into which no ray of
sunlight ever entered. Here, the scholars had to stay no less than seven winters;
for it took them all that time to complete their studies. They never saw their
teacher from one year’s end to another. Every morning a grey grizzly hand, all
covered with hair, pushed itself through the cavern wall and gave to each one
his lesson book. These books were written all over with letters of fire, and
could be read with ease, even in the dark. The lessons over, the same grizzly
hand again appeared to take away the books and bring in the scholars’ dinner.
At the close of winter, the scholars who had then got through their seven years
apprenticeship were dismissed. The great iron door was opened, and the
master stood watching those who went out; for he had stipulated that the
scholar who walked hindmost, in passing through, was to be seized by him and
kept as a thrall. But who was this strange school-master? Why, Old Nick
himself. No wonder, then, that each of the scholars struggled hard to be first
in passing the fatal threshold.
Once on a time, there were three Icelanders at the dark school; Sæmund Frodi,
afterwards parish priest at Oddi, Kalfur Arnason, and Halfdan Eldjarnsson,
afterwards parish priest at Fell, in Slettuhlid. They were all dismissed at the
same time. Sæmund, to the great delight of his companions, offered to walk
hindmost in going out of school, so he dressed himself in a long loose cloak,
which he took care to leave unbuttoned, and bidding good bye to school-
fellows left behind, prepared to follow his countrymen. Just as he was putting
his feet on the first step of the stair which led up from the school door, Old
Nick, who was watching hard by, made a clutch at the cloak and called out,
“Sæmund Frodi, pass not the door,
Thou art my thrall for evermore.”

And now the great iron door began to turn on its hinges; but, before Old Nick
had time to slam it too, Sæmund slipt his arms out of the sleeves of his cloak,
and sprung forward out of the grasp of his enemy.
In doing so, the door struck him a heavy blow on the heel, which gave him a
good deal of pain, when he said,
“The door hath swung too near the heel,
But better sore foot than serve the Deil.”

And so Sæmund outwitted Old Nick, and got away from the dark school along
with his two friends. Since then, it has become a common saying in Iceland,
when a person has had a narrow escape from danger, that “the door swung
too near his heels.”[43]

II. SÆMUND GETS THE LIVING OF ODDI.

At the time Sæmund, Kalfur, and Halfdan came out of the dark school, there
was no priest at Oddi, for the old priest had just died. All three of them would
fain have the living, and so each went to the king to ask it for himself. The
king knew his men; and so he sent them all away with the same answer, that
whoever reached Oddi first, should be made priest of that place.
Thereupon Sæmund summoned Old Nick and said to him, “Now, I’ll make a
bargain with you, if you swim with me on your back across to Iceland, and
land me there without wetting my coat-tail, I’ll be your servant as long as I
live.” Old Nick was highly pleased with the offer and agreed at once. So, in less
than no time, he changed himself into a seal, and left Norway with Sæmund on
his back.
Sæmund took care to have his prayer book with him, and read bits out of it
every now and then while on the way. As soon as they got close to the shores
of Iceland, which they did in less time than you would think, he closed the
book and suddenly struck the seal such a heavy blow on the neck with it that
the animal went down all at once into deep water. Sæmund, now left to
himself, struck out for the shore and got easily to land. In this way Old Nick
lost his bargain, and Sæmund got the living of Oddi.

III. THE GOBLIN AND THE COWHERD.

When Sæmund was priest of Oddi, he once had a cowherd—a good servant
withal, but greatly addicted to swearing. Sæmund often reproved him for this,
but all his reproofs were of no avail. At last he told him, he really ought to
leave off his bad habits, for Old Nick and his servants lived upon people’s
curses and wicked words. “Say you so?” said the cowherd, “if I knew for
certain that Old Nick would lose his meals by it, I would never say a bad word
more.” So he made up his mind to mend his ways.
“I’ll soon see whether you are in earnest or not,” said Sæmund, and so, he
forthwith lodged a goblin in the cowhouse. The cowherd did not like his
guest, and no wonder: for he was up to every kind of mischief, and almost
worried the life out of him with his wicked pranks. The poor cowherd bore up
bravely for a time, and never let slip an oath or angry word. The goblin got
leaner day by day, to the intense delight of the cowherd, who hoped, bye and
bye, to see an end of him.
One morning, on opening the byre door, the poor cowherd found every thing
turned topsy-turvy. The milk pails and stools were broken in pieces and
scattered about the floor; and the whole of the cows—and there were many of
them—tied tail to tail, were straggling about without halters, and goring each
other. It needed but half an eye to see who had done the mischief. So the
cowherd in a rage turned round to the goblin who, shrunk and haggard, lay
crouched up in a corner of a stall, the very picture of wretchedness, and
poured forth such a volley of furious curses as would have overwhelmed any
human being in the same plight. The goblin all at once began to revive; his
skin no longer shrivelled looked smooth and plump; his eye brightened up, and
the stream of life again flowed joyously through his veins.
“O, oh!” said the cowherd, as he suddenly checked himself, when he saw the
wonderful effect his swearing had on the goblin, “Now I know for certain that
Sæmund was right.” And from that day forward he was never known to utter
an oath. As for the goblin, he soon pined away again and has long since been
beyond troubling anybody. May you and I, and all who hear this story, strive to
follow the good example of Sæmund’s cowherd!

IV. OLD NICK MADE HIMSELF AS LITTLE AS HE WAS ABLE.

Sæmund one day asked Old Nick how little he could make himself. “Why,”
replied he, “as for that I could make myself as small as the smallest midge.”
Thereupon Sæmund bored a tiny hole in the door post, and asked him to
make good his boast by walking into it. This he at once did; but no sooner was
he in, than Sæmund stopped the hole with a little plug of wood, and made all
fast.
Old Nick cursed his folly, cried, and begged for mercy; but Sæmund would not
take out the stopper till he promised to become his servant and do all that he
was told. This was the reason why Sæmund always had it in his power to
employ Old Nick in whatever business he liked.

V. THE FLY.

As might be expected, Old Nick always harboured a great ill will against
Sæmund: for he could not help feeling how much he was in Sæmund’s power.
He therefore tried to revenge himself on various occasions; but all his tricks
failed, for Sæmund was too sharp for him.
Once, he put on the shape of a little fly, and hid himself—so he thought, at
least—under the film that had gathered on the priest’s milk jug, hoping that
Sæmund would swallow him unawares, and so lose his life. But Sæmund had
all his eyes about him; so instead of swallowing the fly he wrapped it up in the
film, covered the whole with a bladder, and laid the package on the altar.
There, the fly was obliged to remain till after the service, when Sæmund
opened the package and gave Old Nick his liberty. It is told, as a truth, that old
Nick never found himself in a worse case than when lying on the altar before
Sæmund.
VI. THE GOBLIN’S WHISTLE.

Sæmund had a whistle of such wonderful power, that, as often as he blew it,
one or more goblins appeared before him, ready to do his bidding.[44] One day,
on getting up, he happened to leave the whistle under his pillow, and forgot all
about it till the afternoon when the housemaid was going to make his bed. He
charged her, if she found anything unusual about the bed, she was on no
account to touch it, or move it from its place. But he might have saved himself
the trouble of speaking; for, as soon as the girl saw the whistle, she took it up
in her hand, and looked at it on every side. Not satisfied with much handling
it, she put it to her mouth and blew it lustily. The sound of the blast had not
died away before a goblin stood before her, saying, “what will you have me to
do?” The girl was not a little startled, but had the presence of mind to conceal
her surprise.
It so happened that the hides of ten sheep, that had been killed that day, were
lying on the ground in front of the parsonage. Recollecting this, the girl replied
to the goblin, “Go and count all the hairs that are on the ten hides outside,
and, if you finish your task before I get this bed made, I’ll consent to marry
you.” The goblin thought that a task worth undertaking for such a prize; and
hurrying out, fell to counting the hairs with all his might. The girl who did not
like the idea of being the wife of a goblin, lost no time, you may be sure, in
getting through with her work; and it was well she bestirred herself; for, by the
time the bed was made, the goblin had almost finished his task. Only a few
hairs of the last hide remained uncounted, but they were enough to make him
lose his bargain. When Sæmund afterwards learned how prudently the girl had
got out of her scrape, he was very well pleased.

ICELANDIC FAIRY TALES.

BIARNI SVEINSSON AND HIS SISTER SALVÖR.[45]

Once on a time, a worthy couple, Sveinn and his wife, occupied a farm, on the
shores of the beautiful Skagafiord, in the north country. They were in easy
circumstances and were blessed with two fine children, a son and daughter,
who were the joy of their hearts. Biarni and his sister Salvör—for these were
the names of their children—were twins and greatly attached to each other.
In the spring of the year,[46] about St. John’s day, when these two had reached
the age of twenty, the people of Skagafiord were arranging a party to make a
journey to the mountains of the interior, to gather Iceland-moss for making
porridge. Sveinn promised to let his son go with the party. As soon as Salvör
knew that, she felt a great desire to go too; and so she went to her parents to
ask their consent. This was not so easily got, as they did not wish to part with
both their children at once; and besides, they knew she was ill fitted to bear the
hardships and fatigues of mountain travelling. But she fretted so much at the
thought of being left behind, that, at last, they consented to let her go.
The night before the moss-gatherers were to leave, Sveinn the farmer dreamed
that he had two beautiful white birds, of which he was very fond, and that all
at once, to his great grief, the hen-bird disappeared and could nowhere be
found. On awaking in the morning, he could not help thinking that his dream
betokened no good to his darling Salvör, so he called her to him, and after
telling her his dream, he said to her, “Salvör dear! I cannot bear to part with
you, you must stay at home with your mother and me, for I would never
forgive myself if any ill befel you by the way.” Salvör who had been in great
glee at the prospect of riding, day after day, up the romantic valleys to the
south of Skagafiord, and there tenting out amidst the mountains, was neither
to hold nor to bind, when she found that, after all, she would have to stay at
home; she wept with vexation and distressed herself so much that her father
could not bear it, and again gave an unwilling consent to let her go. So she
accompanied her brother and the rest of the party to the mountains.
The first day after getting there, she gathered Iceland-moss with the others,
but during the night she fell suddenly ill and was unable to leave her tent on
the following day. Biarni stayed with her, and did all that a brother could do to
help and comfort her. For three whole days he was her companion, but, on the
fourth day, he left her for a time in charge of a friend, while he himself joined
the moss-gatherers. After partly filling his bag, he sat himself down by a large
stone, and, resting his head on his hand, brooded over his sister’s unhappy
fate; he feared she was going to die among the mountains.
By and by he heard a great tramping of horses, and, on looking about, he saw
two men riding towards him at a quick pace. One of them wore red coloured
clothes, and had a red horse; the other who was younger, was dressed in black,
and was mounted on a black horse. On reaching the place where Biarni was
sitting, they dismounted and saluted him by name.
“What ails you Biarni,” said the elder of the two strangers. For a time Biarni
answered not a word, but on being pressed to do so, he opened up his heart to
them and told all about his sister’s illness.
“My companions are going to return home, but I must stay to watch over
Salvör; and who knows how soon she may die in my arms.”
“You are in a hard case Biarni,” said the other, “and I am sorry for you, but
won’t you leave your sister with me, and I will take good care of her.”
“No, no,” said Biarni, “that I dare not do, for I know neither who you are, nor
where you come from. But will you tell me where your home is?”
“That’s no business of yours,” said the other, rather gruffly, and then, taking
from his pocket a silver-gilt box set with precious stones, added, “Won’t you
sell me your sister for this box.”
“No,” said Biarni, “nor for a thousand like it. I would not give her to you for
any money.”
“Well! well! there is no help for it, you will at all events accept this box, as a
token that you have met with men among the mountains.”
Biarni took the offered gift with pleasure, and thanked the giver. The two men
then bade him farewell and rode away, while he returned to the tent. Next
morning his companions went away home, leaving him alone with his sister.
Though she was now a little better, he dared not sleep, for he was afraid lest
the strangers should come and steal her away. But, after watching a whole day
and night, he felt overcome with fatigue; so he lay down, and folding his arms
round her waist to protect her, fell into a sound sleep. But, when he awoke, his
sister was gone, and was nowhere to be found. He spent a whole day
sorrowfully wandering from spot to spot, looking and calling for her, but it
was all in vain. He then turned his back on the mountains, and with a heavy
heart went home, and told his parents what had happened.
“Woe is me,” said Sveinn, “what I feared most has come to pass, but God’s
will be done!”
There was great grief in Skagafiord when the news spread from farm to farm;
for Salvör, with all her way-wardness, was a promising girl, and was every
body’s favourite. A party of young men returned to the mountains to look for
her, but nowhere was the least trace of her to be found.
And now ten years had passed away. By this time Biarni was married and
settled on a farm, not far from his father’s. During autumn all his sheep went
amissing, and his shepherd could not discover what had become of them
though he searched diligently for them three whole days. On learning this,
Biarni bid his wife provide him with a week’s supply of food, and an extra pair
of shoes; “for,” said he, “I shall go to the mountains myself to look for the
sheep.” His parents, who were still alive, urged him to stay at home; for they
feared that, if he went to the mountains, they might never see his face again.
“I must go,” said he to them, “I cannot afford to lose the sheep. But be of
good heart, and do not begin to weary for me till the week is over.”
He then went away on foot, and did not leave off walking for three days. At
the end of that time he came to a cavern, where he turned in and lay down to
sleep. On waking, he could not see a yard before him; for a thick fog which
rested on the ground. He continued his journey, but soon lost his way.
Towards evening the fog cleared off, and he found himself in a spacious valley,
not far from a large well built farm house. It was the hay season, so that all the
people of the farm were busy in the meadow. On getting near the house, he
noticed, in particular, two women and a girl who were tedding the hay. “God’s
peace be with you,” said he, on reaching the spot; and then, telling them of his
mishaps, he asked permission to stay all night under their roof. They gave him
a hearty welcome, and the girl went with him to the house. She was of more
genteel appearance than the rest—young and handsome—and, as Biarni
thought, bore some resemblance to his long lost but well remembered sister.
This unexpected circumstance renewed his old griefs, but he did what he could
to seem cheerful before his young hostess. She led him through several
apartments to a large well furnished room, where everything was neat and tidy.
Here, she drew in a chair, and kindly asked him to sit down and rest, while she
brought in supper. He had not long to wait; for she soon placed upon the table
a plentiful supply of meat and wine.
After supper, she showed him to the little room where he was to sleep for the
night; she then took away his wet clothes, wished him a kind good night, and
left the room.
As Biarni lay in bed, he fell a-wondering where he was, and how the sight of
the girl should have so waked up the sad memories of the past. He fell asleep
thinking of these things, but was soon awakened by the sound of singing in a
room over his head. It was the family at evening worship, as is the custom of
the country. He heard both men and women singing, but one voice sounded

You might also like