Download as pdf or txt
Download as pdf or txt
You are on page 1of 53

Business and Consumer Analytics:

New Ideas Pablo Moscato


Visit to download the full and correct content document:
https://textbookfull.com/product/business-and-consumer-analytics-new-ideas-pablo-m
oscato/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...

Product Analytics: Applied Data Science Techniques for


Actionable Consumer Insights (Pearson Business
Analytics Series) 1st Edition Rodrigues

https://textbookfull.com/product/product-analytics-applied-data-
science-techniques-for-actionable-consumer-insights-pearson-
business-analytics-series-1st-edition-rodrigues/

Economy, Business and Uncertainty: New Ideas for a


Euro-Mediterranean Industrial Policy Jaime Gil-Lafuente

https://textbookfull.com/product/economy-business-and-
uncertainty-new-ideas-for-a-euro-mediterranean-industrial-policy-
jaime-gil-lafuente/

Business Analytics James R. Evans

https://textbookfull.com/product/business-analytics-james-r-
evans/

Business Analytics Jeffrey D. Camm

https://textbookfull.com/product/business-analytics-jeffrey-d-
camm/
Business Analytics, 5e Jeffrey D. Camm

https://textbookfull.com/product/business-analytics-5e-jeffrey-d-
camm/

Digital Analytics for Marketing (Mastering Business


Analytics) 2nd Edition A. Karim Feroz

https://textbookfull.com/product/digital-analytics-for-marketing-
mastering-business-analytics-2nd-edition-a-karim-feroz/

Emotion-Driven Innovation: A Methodology to Envision


Emotion-Focused New Product Ideas (Future of Business
and Finance) 1st Edition Alaniz

https://textbookfull.com/product/emotion-driven-innovation-a-
methodology-to-envision-emotion-focused-new-product-ideas-future-
of-business-and-finance-1st-edition-alaniz/

SAP Business Analytics A Best Practices Guide for


Implementing Business Analytics Using SAP 1st Edition
Sudipa Duttaroy (Auth.)

https://textbookfull.com/product/sap-business-analytics-a-best-
practices-guide-for-implementing-business-analytics-using-
sap-1st-edition-sudipa-duttaroy-auth/

Strategic System Assurance and Business Analytics P. K.


Kapur

https://textbookfull.com/product/strategic-system-assurance-and-
business-analytics-p-k-kapur/
Pablo Moscato · Natalie Jane de Vries
Editors

Business and
Consumer
Analytics:
New Ideas
Business and Consumer Analytics: New Ideas
Pablo Moscato • Natalie Jane de Vries
Editors

Business and Consumer


Analytics: New Ideas

123
Editors
Pablo Moscato Natalie Jane de Vries
School of Electrical Engineering School of Electrical Engineering
and Computing and Computing
The University of Newcastle The University of Newcastle
Callaghan, NSW, Australia Callaghan, NSW, Australia

ISBN 978-3-030-06221-7 ISBN 978-3-030-06222-4 (eBook)


https://doi.org/10.1007/978-3-030-06222-4

Library of Congress Control Number: 2019934369

© Springer Nature Switzerland AG 2019


This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors, and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or
the editors give a warranty, express or implied, with respect to the material contained herein or for any
errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG.
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface

It is hard to find something else to say about data science today that has not already
been said. Perhaps what has not yet been discussed is that the progress in this area is
profoundly challenging the way concepts have to be presented to newcomers. Data
science brings together concepts from disciplines like computer science, statistics
and applied mathematics, and the applications reach all possible aspects of life
and economics. Consequently, universities around the world are having difficulties
in addressing the need for a reformulation of their courses. There is a clear need
for new ways to present the material, with an emphasis in understanding the
key concepts, the novel applications and the impact of the techniques. This is,
undoubtedly, a huge endeavour as there is no universally established curriculum
for data science. In fact, we feel there is a need for students and practitioners who
have been trained in one particular field to find a “shortcut” that would allow them
to understand other areas.
This book can be seen as a first step in that direction. We aim at “bridging the
gap” between some core new ideas in data science with the application in business
and consumer analytics.
There are several reasons to choose this intersection as our first target. Advances
in data science, data analytics and data mining methodologies are bringing many
novel contributions to business and marketing applications. On the other side, the
scale of e-commerce activities and the possibility of reaching a novel understanding
of consumer behaviour are a driving force that pushes and challenges the field of
data science. It is clear that the trend is here to stay. Conducting business and
speaking out to consumers will be forever paired with data analytics. We have
gained an incredible capability for collecting large amounts of widely varied data,
and providing business insights from these data sources has become an important
and continuous task of many researchers and business professionals.
At the time of writing this work, all the techniques included are considered novel
in the area of business and consumer analytics. We are sure that more work needs
to be done for many of them to reach the scalability necessary to deal with datasets
of millions of consumers and products. That is a necessity of daily operations of
many companies; we feel that this need for scalability will be met by the natural

v
vi Preface

algorithmic progress provided by computer science. This said, we are interested in


the new ideas currently at the crossroads between developments in data science,
optimization, network analytics, computational complexity, artificial intelligence
and machine learning, evolutionary algorithms and their application to business
scenarios. We are confident that many of these techniques will soon flourish and
become more widely adopted by businesses.
There are several lessons that are normally learnt when you finish a book. For the
two editors, some of them were early ones at the beginning of the work, and they,
in turn, helped to mould the introductory section to address them. For instance,
the overzealous preoccupation of computer scientists with the provision of highly
predictive analytic learning systems often clashes with the interest of business
professionals. The latter tend to prefer models with perhaps less variables, at a cost
of having a reduced predictive capability, that nevertheless contain the necessary
levers that can lead to improved decision-making, reducing cost and maximizing
profit. For both sides of the intersection we are looking at (i.e., data scientists and
business professionals), even the word “problem” conveys a different meaning,
and the quest of “efficiency” in computer science may be misinterpreted by a
business marketer. We also aimed at presenting some fundamentals on marketing
and consumer behaviour to enlighten the “hardcore computer scientists” about some
of the needs and wants of marketing and business professionals. We quote: “Data
has been king for well over a decade by now, but the way we use it is undergoing
some serious change. Gone are the days of awe at pretty charts and heat maps.
Gone, too, is any patience for analytics unaligned to action”.1 These differences
in the use of language and purpose are discussed in a tutorial way to help engage
both communities towards a common goal. Once again, although we are certain that
more needs to be done, our intention was clear; we tried to fill this gap.
Apart from the two introductory chapters, the rest of the book is organized as
follows. Each section is centred in one specific area of fast development, which
is either methodological or application based. The clustering and pattern mining
section contains recent developments that are important for customer segmentation
and targeting. They go from an introductory tone (Chaps. 3 and 4 are of that type),
while the other two chapters relate to more advanced methods currently under
development.
The network section contains a review chapter, which again is of an introductory
nature, followed by an introduction to the area of centrality analysis (which is a
main topic for areas of computational social science and product analyses). The
remaining four chapters provide a variety of methods and applications related to
network analytics including novel applications in survey analyses and co-purchasing
network analytics.
In the 1980s, a new word was coined by Fred Glover called “metaheuristics”,
which are generic techniques aimed at guiding heuristics for a problem at hand

1 https://www.information-management.com/opinion/data-science-underlies-everything-the-

enterprise-now-does.
Preface vii

which can be posed as an optimization task. Analogously, he is proposing the


use of the word “meta-analytics” for identifying high-level techniques aiming at
guiding a multitude of methods working together to address a data analytic problem.
These ideas are introduced, and then several chapters are dedicated to specific
implementations of these methodologies: two with a focus in ensemble learning
and one in a classification problem.
Memetic algorithms are a paradigm that was championed by one of the editors,
and during the last years, it has become a highly active field of research following
the publication of the Handbook of Memetic Algorithms by Springer in 2011. While
applications in many fields of science and technology exist, there is an enormous
potential for the area of business and consumer analytics. The chapters included in
this work are just the tip of the iceberg of the activity that currently exists. It includes
applications in product and customer visualization, personalized recommendation
systems, facility location and vehicle routing, and orienting problems. We also
note that one of the contributions in the network section includes a memetic algo-
rithm application addressing overlapping community identification in co-purchasing
graphs from the Amazon group.
The final section includes a more application-oriented major theme around travel
and fashion analytics with three contributions in tourism and one in recommendation
systems for a fashion e-commerce service. While the major theme is on personal-
ization of services, the chapters are self-contained and methodologically address the
questions using different techniques (fuzzy clustering, mathematical programming
and ranking-based techniques).
We have tried, whenever possible, to use the same dataset in some of the different
chapters. This allows readers to understand the possibilities that the different
techniques gave individually and to concentrate the descriptions of all the datasets
in a single Appendix chapter at the end of this book.

Callaghan, NSW, Australia Pablo Moscato


Natalie Jane de Vries
Acknowledgements

The editors would like to thank a multitude of people who helped create this
interdisciplinary and collaborative piece of work. Firstly, our thanks go to Luke
Mathieson, Sergio Gomez, Carlos Cotta and Fred Glover who helped us to put
together the various sections in this book by inviting fellow collaborators and
sharing co-edition. We would like to thank all of the reviewers and proofreaders
of the many contributions in this volume, those who did so anonymously and those
who did so for fellow colleagues. Further, we would like to thank Ademir Cristiano
Gabardo and Mohammad Nazmul Haque for their help in a range of things from
formatting to technical issues and editing figures.
Finally, the editors would like to thank all of the data parasites2,3 around the
world who, like us, strive for a transparent and open research environment where
collaboration, not competition, is the main motivation for researchers sharing data
and where we strive towards the common goal of the advancement of science and
human knowledge. We humbly dedicate our “Open Access Data” Appendix chapter
to you!
Natalie de Vries would like to thank her family, friends and partner Simon Faint
for their personal support of all her professional and academic endeavours. She
would also like to thank fellow editor Pablo Moscato for the great experience and
memories gained from creating this book together.
Pablo Moscato thanks Regina and Anna Clara for their patience and the support
through the many stages of this project. He also thanks his fellow co-editor,

2 Data parasites is a term that came to be popular after a rather controversial editorial published

in The New England Journal of Medicine (NEJM) in January 2016 (http://www.nejm.org/doi/full/


10.1056/NEJMe1516564) criticized the sharing of data. This editorial received a huge backlash
from the research community who are in support of open access trends and data sharing activities.
The satirical “Data Parasite Award” has since also been established handing out two awards:
“the first recognizes an outstanding contribution from a junior parasite (postdoctoral, graduate, or
undergraduate trainee), and the second recognizes an individual for a sustained period of exemplary
research parasitism”.
3 http://researchparasite.com/.

ix
x Acknowledgements

Ms. de Vries, for being essential in their “largest catch” together. He also considers
that, while a few cats, but no sharks, slowed the team to get the fish, not even “the
great DiMaggio who does all things perfectly” could have been a better partner in
this boat.
Looking back, and measuring all the obstacles to their projects created by some
people over the last years, the editors say: “‘Eat that galanos. And make a dream
you’ve killed us’. This book is also dedicated to you.”
Contents

Part I Introduction
1 Marketing Meets Data Science: Bridging the Gap. . . . . . . . . . . . . . . . . . . . . . 3
Pablo Moscato and Natalie Jane de Vries
2 Consumer Behaviour and Marketing Fundamentals
for Business Data Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Natalie Jane de Vries and Pablo Moscato

Part II Segmentation, Clustering and Pattern Mining


3 Introducing Clustering with a Focus in Marketing
and Consumer Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
Natalie Jane de Vries, Łukasz P. Olech, and Pablo Moscato
4 An Introduction to Proximity Graphs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
Luke Mathieson and Pablo Moscato
5 Clustering Consumers and Cluster-Specific Behavioural Models. . . . . 235
Natalie Jane de Vries, Jamie Carlson and Pablo Moscato
6 Frequent Itemset Mining . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
Massimo Cafaro and Marco Pulimeno

Part III Networks and Community Detection


7 Business Network Analytics: From Graphs to Supernetworks . . . . . . . . 307
Pablo Moscato
8 Centrality in Networks: Finding the Most Important Nodes . . . . . . . . . . 401
Sergio Gómez
9 Overlapping Communities in Co-purchasing and Social
Interaction Graphs: A Memetic Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435
Ademir Gabardo, Regina Berretta and Pablo Moscato

xi
xii Contents

10 Taming a Graph Hairball: Local Exploration in a Global Context . . . 467


James Abello, Daniel Mawhirter, and Kevin Sun
11 Network-Based Models for Social Recommender Systems . . . . . . . . . . . . 491
Antonia Godoy-Lorite, Roger Guimerà, and Marta Sales-Pardo
12 Using Network Alignment to Identify Conserved Consumer
Behaviour Modelling Constructs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513
Luke Mathieson, Natalie Jane de Vries, and Pablo Moscato

Part IV Memetic Algorithms


13 Memetic Algorithms for Business Analytics and Data Science:
A Brief Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545
Pablo Moscato and Luke Mathieson
14 A Memetic Algorithm for the Team Orienteering Problem. . . . . . . . . . . . 609
Dimitra Trachanatzi, Eleftherios Tsakirakis, Magdalene Marinaki,
Yannis Marinakis, and Nikolaos Matsatsinis
15 A Memetic Algorithm for Competitive Facility Location Problems . . 637
Benjamin Biesinger, Bin Hu, and Günther R. Raidl
16 Visualizing Products and Consumers: A Gestalt Theory
Inspired Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 661
Claudio Sanhueza Lobos, Natalie Jane de Vries,
Mario Inostroza-Ponta, Regina Berretta, and Pablo Moscato

Part V Meta-Analytics
17 An Overview of Meta-Analytics: The Promise of Unifying
Metaheuristics and Analytics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 693
Fred Glover and Carlos Cotta
18 From Ensemble Learning to Meta-Analytics: A Review
on Trends in Business Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 703
Mohammad Nazmul Haque and Pablo Moscato
19 Metaheuristics and Classifier Ensembles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 733
Ringolf Thomschke, Stefan Voß, and Stefan Lessmann
20 A Multi-objective Meta-Analytic Method for Customer Churn
Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 781
Mohammad Nazmul Haque, Natalie Jane de Vries,
and Pablo Moscato
21 Hotel Classification Using Meta-Analytics: A Case Study
with Cohesive Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 815
Buyang Cao, Cesar Rego, and Fred Glover
Contents xiii

Part VI Data Science Applications in Travel and Fashion


Analytics
22 Fuzzy Clustering in Travel and Tourism Analytics . . . . . . . . . . . . . . . . . . . . . 839
Pierpaolo D’Urso, Marta Disegna, and Riccardo Massari
23 Towards Personalized Data-Driven Bundle Design with QoS
Constraint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 865
Mustafa Mısır and Hoong Chuin Lau
24 A Fuzzy Evaluation of Tourism Sustainability . . . . . . . . . . . . . . . . . . . . . . . . . . 911
Joseph Andria, Giacomo di Tollo, and Raffaele Pesenti
25 New Ideas in Ranking for Personalized Fashion Recommender
Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 933
Heri Ramampiaro, Helge Langseth, Thomas Almenningen,
Herman Schistad, Martin Havig, and Hai Thanh Nguyen

Part VII Appendix


26 Datasets for Business and Consumer Analytics . . . . . . . . . . . . . . . . . . . . . . . . . 965
Natalie Jane de Vries and Pablo Moscato

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 989
Contributors

James Abello DIMACS and Computer Science Department, Rutgers University,


New Brunswick, NJ, USA
Thomas Almenningen Department of Computer and Information Science,
Norwegian University of Science and Technology, Trondheim, Norway
Joseph Andria Dipartimento di Scienze Economiche, Aziendali e Statistiche,
University of Palermo, Palermo, Italy
Regina Berretta School of Electrical Engineering and Computing, The University
of Newcastle, Callaghan, NSW, Australia
Benjamin Biesinger Institute of Computer Graphics and Algorithms, TU Wien,
Vienna, Austria
Massimo Cafaro University of Salento, Lecce, Italy
Buyang Cao China Intelligent Urbanization Co-Creation Center for High Density
Region, School of Software Engineering, Tongji University, Shanghai, China
Jamie Carlson Newcastle University Business School, The University of Newcas-
tle, Callaghan, NSW, Australia
Carlos Cotta ETSI Informática, Universidad De Màlaga, Malaga, Spain
Natalie Jane de Vries School of Electrical Engineering and Computing, The
University of Newcastle, Callaghan, NSW, Australia
Giacomo di Tollo Dipartimento di Economia, Universitá Ca’ Foscari, Venezia,
Italy
Marta Disegna Accounting, Finance & Economics Department, Faculty of Man-
agement, Bournemouth University, Bournemouth, UK
Pierpaolo D’Urso Department of Social Sciences and Economics, Sapienza
University of Roma, Roma, Italy

xv
xvi Contributors

Ademir Gabardo School of Electrical Engineering and Computing, The Univer-


sity of Newcastle, Callaghan, NSW, Australia
Fred Glover Leeds School of Business and the College of Engineering & Applied
Science, University of Colorado, Boulder, CO, USA
Antonia Godoy-Lorite Department of Mathematics, Imperial College London,
London, UK
Sergio Gómez Universitat Rovira i Virgili, Tarragona, Spain
Roger Guimerà Institució Catalana de Recerca i Estudis Avançats (ICREA),
Barcelona, Catalonia, Spain
Departament d’Enginyeria, Química, Universitat Rovira i Virgili, Tarragona,
Catalonia, Spain
Mohammad Nazmul Haque School of Electrical Engineering and Computing,
The University of Newcastle, Callaghan, NSW, Australia
Martin Havig Department of Computer and Information Science, Norwegian
University of Science and Technology, Trondheim, Norway
Bin Hu AIT Austrian Institute of Technology, Mobility Department - Dynamic
Transportation Systems, Vienna, Austria
Mario Inostroza-Ponta Universidad de Santiago de Chile, Santiago, Chile
Helge Langseth Department of Computer and Information Science, Norwegian
University of Science and Technology, Trondheim, Norway
Hoong Chuin Lau Singapore Management University, School of Information
Systems, Singapore, Singapore
Stefan Lessmann Humboldt-Universität zu Berlin, School of Business and
Economics, Berlin, Germany
Magdalene Marinaki Technical University of Crete, School of Production
Engineering and Management, Chania, Greece
Yannis Marinakis Technical University of Crete, School of Production Engineer-
ing and Management, Chania, Greece
Riccardo Massari Department of Social Sciences and Economics, Sapienza
University of Roma, Roma, Italy
Luke Mathieson School of Electrical Engineering and Computing, The University
of Newcastle, Callaghan, NSW, Australia
School of Software, University of Technology Sydney, Ultimo, NSW, Australia
Contributors xvii

Nikolaos Matsatsinis Technical University of Crete, School of Production Engi-


neering and Management, Chania, Greece
Daniel Mawhirter Colorado School of Mines, Golden, CO, USA
Mustafa Mısır Nanjing University of Aeronautics and Astronautics, College of
Computer Science and Technology, Nanjing, Jiangsu, China
Pablo Moscato School of Electrical Engineering and Computing, The University
of Newcastle, Callaghan, NSW, Australia
Hai Thanh Nguyen Department of Computer and Information Science, Norwe-
gian University of Science and Technology, Trondheim, Norway
Telenor Research, Trondheim, Norway
Łukasz P. Olech Department of Computational Intelligence, Faculty of Computer
Science and Management, Wrocław University of Science and Technology,
Wrocław, Poland
Raffaele Pesenti Dipartimento di Management, Universitá Ca’ Foscari, Venezia,
Italy
Marco Pulimeno University of Salento, Lecce, Italy
Günther R. Raidl Institute of Computer Graphics and Algorithms, TU Wien,
Vienna, Austria
Heri Ramampiaro Department of Computer and Information Science, Norwegian
University of Science and Technology, Trondheim, Norway
Cesar Rego School of Business Administration, University of Mississippi, Oxford,
MS, USA
Marta Sales-Pardo Departament d’Enginyeria Química, Universitat Rovira i
Virgili, Tarragona, Spain
Claudio Sanhueza Lobos School of Electrical Engineering and Computing, The
University of Newcastle, Callaghan, NSW, Australia
Herman Schistad Department of Computer and Information Science, Norwegian
University of Science and Technology, Trondheim, Norway
Kevin Sun Rutgers University, New Brunswick, NJ, USA
Ringolf Thomschke Humboldt-Universität zu Berlin, School of Business and
Economics, Berlin, Germany
Dimitra Trachanatzi Technical University of Crete, School of Production Engi-
neering and Management, Chania, Greece
xviii Contributors

Eleftherios Tsakirakis Technical University of Crete, School of Production Engi-


neering and Management, Chania, Greece
Stefan Voß University of Hamburg, Institute of Information Systems, Hamburg,
Germany
Part I
Introduction
Chapter 1
Marketing Meets Data Science: Bridging
the Gap

Pablo Moscato and Natalie Jane de Vries

Abstract It is certain that computer science is completely reformulating the


way that business is being conducted around the world. We are witnessing the
increasing availability of large volumes of data together with the advances in
artificial intelligence, machine learning and optimization techniques. Breakthroughs
in statistics, discrete applied mathematics and new algorithms are leading to the
development of a new interdisciplinary field: data science. The purpose of this
chapter is to provide a bridge, a short-cut to understand some of the questions
that computer science deals with in a context of developing new techniques to get
knowledge from data.

Keywords Analytics · Marketing and customer behaviour analytics · Data


mining · Marketing

1.1 Data Science for Marketing

Once upon a time. . . things were a bit simpler. Indeed. Several decades ago, when
computers started to be used for marketing and business intelligence, they were
mainly used to analyse surveys, evaluate simple statistics or to produce interactive
displays to compute the results of some mathematical models that humans would
create [169]. That was a perfect marriage. The different disciplines seemed to
complement each other and would not need anything more, they satisfied the mutual
requirements very well and a natural symbiosis occurred. Marketers and business
analysts would first identify a few things to build a case for a quantitative study
and would then ask statisticians’ help. Typically it would be a triad consisting of:
(a) an assumedly important problem would be identified, (b) a solution that can be
implemented would need to be evaluated if introduced, (c) an expected outcome

P. Moscato () · N. J. de Vries


School of Electrical Engineering and Computing, The University of Newcastle, Callaghan, NSW,
Australia
e-mail: Pablo.Moscato@newcastle.edu.au; natalie.devries@newcastle.edu.au

© Springer Nature Switzerland AG 2019 3


P. Moscato, N. J. de Vries (eds.), Business and Consumer Analytics: New Ideas,
https://doi.org/10.1007/978-3-030-06222-4_1
4 P. Moscato and N. J. de Vries

would benefit the business. If this triad can lead to a testable experiment, statistics
would help to analyse the data, to draw some conclusions and the whole process
would provide some business insight.
In fact, nobody can argue against the use of statistics, they bring a necessary
quantitative dimension which is indispensable for competitiveness. However, there
are some limitations. For decision-making some people cite a few common prob-
lems with such ad hoc experimentation: small sample sizes, the use of frequencies
instead of tolerances, probabilities instead of possibilities and presence of outcome
bias. It is also true that experimentation clearly helps to get insights, identify new
problems and, when carefully planned, helps to understand the core difficulties of
the company.

1.1.1 Data Is Almost Everything Now, Enabling Change

There was something missing in that old symbiosis; discussing such problems
is not really the purpose of this chapter. Instead, here we choose to make the
reader appreciate another wave of change that of introducing computer science
techniques in marketing, business and customer analytics (without leaving statistical
methods behind). We are sure that traditional hypothesis-driven research in business
and marketing will continue, stronger than ever. Here we will concentrate on the
new changes coming from a data-driven revolution. It is fuelled by the increased
availability of data gathered, and stored, by new technologies. Today, we are moving
into the era of Data Science, and again, it all started not with products and services
but with the humble consumers and by putting them in the centre of the scene.
It is always interesting to look at the past, the signs of change are already there.
We quote:
The view that an industry is a customer-satisfying process, not a goods-producing process,
is vital for all business people to understand. An industry begins with the customer and his
or her needs, not with a patent, a raw material, or a selling skill.

We are sure that the reader will think about companies like Amazon, Google, Apple,
or Virgin almost immediately. However, we are neither quoting Bezos, Page, Brin,
Jobs, Zuckerberg, nor Branson. We are citing here an academic scholar, Theodore
Levitt, from an article titled “Marketing Myopia” written almost 60 years ago [137].
That article had a clear message, it is a manifesto for change. In it Levitt also said:
. . . the entire corporation must be viewed as a customer-creating and customer-satisfying
organism. Management must think of itself not as producing products but as providing
customer-creating value satisfactions. It must push this idea (and everything it means and
requires) into every nook and cranny of the organization. It has to do it this continuously
and with the kind of flair that excited and stimulates the people in it.

Users and customers, like decades before, are continuously generating huge
amounts of data. What is now really unprecedented that we have the capacity
to gather this data and transform it into knowledge. Even just considering the
1 Marketing Meets Data Science: Bridging the Gap 5

statistical-only approach, the time cycles between hypotheses formation and their
testing and validation have been dramatically shortened. Ron Kohavi, at the
Knowledge Discovery from Databases 2015 conference in Sydney, explained how
hundreds of online controlled statistical experiments are conducted on a daily basis
to understand user behaviour. He explained how “data trumps intuition” (see a
video from a similar talk he gave that is available in YouTube1 ). After all, another
early quote, this one attributed to W. Edwards Deming, also warned us:
In God we trust, the rest bring data.

It is then easy to recognize that the ethos of some of the companies cited above
embody a new thinking pioneered by Levitt. From solving “tiny problems”, some
companies scaled-up their skills and managed to disrupt existing business models
occurring at a global scale (and they have created new business niches in the
process). They quickly moved from “tiny problems” to “big data”, and then to world
scale, but actually they are profoundly based in a customer-centric and adaptive new
view of the role of the company business intelligence.

1.1.2 While Data Is Everything, You Have Nothing Without


Understanding It

The large volumes of data collected by companies around the world are increasingly
being exploited with methods that employ computer science techniques. However,
this is not a new field; “Why is computer science so useful now?” Well, there are the
obvious aspects of dealing with computers that have changed. We are increasingly
having them allied to the necessary skills to conduct large-scale online statistical
testing (as the ones Kohavi discusses). But computer science has established itself
as a new discipline for the past 70 years and, at its core, it has a very clear manifesto
that of being the quintessential approach to identify, characterize and solve problems
involving the processing of information. It creates knowledge from large databases
containing information.
This emphasis in problem identification and problem-solving of computer sci-
ence is interesting for data-driven companies. They directly benefit on the behaviour
of people trained in the discipline. We can quote Levitt’s article [137] (the article is
so rich that it seems an inevitable temptation to continue quoting it);
If thinking is an intellectual response to a problem, then the absence of a problem leads to
the absence of thinking.

This said, what computer science brings to companies, business and consumer
analytics is a new view, a novel perspective. Computer scientists have a proactive
role in an organization, shared by the roots of the discipline in Applied Mathematics.

1 https://www.youtube.com/watch?v=qtboCGd_hTA.
6 P. Moscato and N. J. de Vries

Questions like: “Can this be done at this price?” “Can I serve all my customers
if each of their requirements need to be met?” “What is the optimal schedule of
tasks?” and many others indicate the role of computer scientists at enabling optimal
decisions and how they can base their actions on the result of algorithms operating
on the data available.
From a pure psychological perspective, computer scientists bring “new blood”
to the mix of skills and human capital of the company. They tend to be constantly
on the hunt for “the next problem”, the “next variation” that can make a problem
seemingly intractable, tractable. Computer scientists will thrive in finding those new
problems that would create the “need” for new thinking. They will help to “bridge
the gap”, if properly motivated, to create the intellectual response required and to
open new business niches for companies. We discuss the computer science’s “world
view” and some of its primary goals later in Sect. 1.3.
In addition to their different complementary perspective, computer scientists
also constantly look for efficiencies, reductions of times and costs, increments in
prediction. More recently, adopting a customer-centric view, they were central in
the development of the wave of “personalization”. All these areas, also linked
to discrete applied mathematics, operations research and management science,
are becoming hugely useful for companies that thrive on the benefits of a data-
driven agenda to drive marketing and selling (note, en passant, that as clarified by
Levitt [137], these are two different things). He said:
The difference between marketing and selling is more than semantic. Selling focuses on
the needs of the seller, marketing on the needs of the buyer. Selling is preoccupied with
the seller’s need to convert his product into cash; marketing with the idea of satisfying the
needs of the customer by means of the product and the whole cluster of things associated
with creating, delivering, and finally consuming it.

The convergence of new methods coming from computer science and statistics is
delivering incredible breakthroughs in the areas of automated learning by machines
and computers. The field of Artificial Intelligence will dramatically change business
and marketing is no exception. However, just “predicting” events, even at the level
of predicting what an individual consumer will do/like, is not good enough without
an “understanding” of the causes of that choice. Products and services can then be
the consequence of an informed marketing process, making it deliver in its true
role [137]. This said, the increased availability of data from consumers can be
translated, via computer analysis, into both understandable and actionable insights
on their needs.

1.1.3 Data Science: Do We Need Another Name


in “Buzzland”?

An article in the Harvard Business Review popularized the idea that Data Science
is the “Sexiest Job of the 21st Century”. But looking smart, Mr. Bond, would not
1 Marketing Meets Data Science: Bridging the Gap 7

be enough. . . you will need to study hard if you want to become a true data scientist
and get one of these “sexy jobs”. For the old players in this game it is not news. But
what really is this “new field”?

100
Data Science Data Mining
90
80
70
Popularity

60
50
40
30
20
10
0
2004-01
2004-08
2005-03
2005-10
2006-05
2006-12
2007-07
2008-02
2008-09
2009-04
2009-11
2010-06
2011-01
2011-08
2012-03
2012-10
2013-05
2013-12
2014-07
2015-02
2015-09
2016-04
2016-11
2017-06
Fig. 1.1 Interest of “Data Mining” over “Data Science” from the year 2004 till 2017 (measured
by Google Trends). The chart indicates that Data Science is starting to surpass in popularity Data
Mining from the beginning of 2016

“Data Science” is increasingly being recognized as a hub, an interdisciplinary


field dedicated to transforming data into useful knowledge. To be trained in the
area you will need to master fundamental knowledge in algorithms for data mining,
predictive analytics, Machine Learning and Computational Intelligence. Aside of
this, Data Science blends techniques from Computer Science, Statistics, Applied
Mathematics, Operations Research, Management Science, Artificial Intelligence,
together with Psychology and Economics. The interdisciplinary nature of Data
Science is challenging academic institutions around the world. Countries like the
UK have opted to create a network of affiliated institutions under the umbrella of
the Alan Turing Institute,2 as an integrated national response to the need of research
and training.
The numbers in the y-axis of Fig. 1.1 represent the search interest of the term
relatively measured against the highest point for the mentioned period of time. A
value of 100 indicates the peak popularity for the term. A value of 50 means that the
term is half as popular than the most popular term in the period.

2 https://www.turing.ac.uk/.
8 P. Moscato and N. J. de Vries

The question is not if we need a new name, it is already here and it is based on
a very coherent message. There is a quest for a data-driven approach for changing
science as well as understanding consumers; thus, we expect that its relevance will
continue to grow in the next decades as “algorithmic” and “computational thinking”
extend their influence (Fig. 1.2).

100 Data Science Data Mining Consumer & Business Analytics Data Analytics
40
90
30
80 Popularity

20
70
Popularity

10
60
0
2012-01
2012-04
2012-07
2012-10
2013-01
2013-04
2013-07
2013-10
2014-01
2014-04
2014-07
2014-10
2015-01
2014-04
2015-07
2015-10
2016-01
2016-04
2016-07
2016-10
2017-01
2017-04
2017-07
50

40

30

20

10

0
2004-01
2004-06
2004-11
2005-04
2005-09
2006-02
2006-07
2006-12
2007-05
2007-10
2008-03
2008-08
2009-01
2009-06
2009-11
2010-04
2010-09
2011-02
2011-07
2011-12
2012-05
2012-10
2013-03
2013-08
2014-01
2014-06
2014-11
2015-04
2015-09
2016-02
2016-07
2016-12
2017-05
Fig. 1.2 The popularity of the search terms from the year 2004 till July 2017 (again, measured
via Google Trends). For clarity the inset shows the popularity of those terms during the period of
2012–2017

1.2 The Algorithmic Revolution

A central theme of data science is the identification of algorithms for the solution of
problems that can be solved with computers, one of the core objectives of computer
science. We can ask: “Why are they so central?”

1.2.1 If Algorithms Already Rule the World: Why Another


“Revolution”?

It is true, algorithms already rule the world; everything we do with computers is


based on them. What are they? One “computational” or “mechanical” definition is
1 Marketing Meets Data Science: Bridging the Gap 9

that they are step-by-step detailed instructions for a machine to execute. A more
philosophical view, quoting Steven Skiena, is that algorithms are “the ideas behind
computer programs”. More than ever before, algorithms are at the core of all
our activities. Some of the world’s most successful companies of the digital and
networked global economy are now entirely based on them.
Why is another “revolution” coming? There is a clear path by which algorithms
will be redefining the Knowledge Economy and will be creating a new one. This is
certainly not an understatement. For instance, the progress observed in some areas of
machine learning and Artificial Intelligence (AI) has been phenomenal since we first
discussed the possibility of editing this book in 2012. Computer-based algorithms
are now beating our world’s masters in both Chess and Go, by “training themselves”
to do so.
The revolution is coming because we are living in a period of accelerated change.
Current tablets and smartphones have performances that are comparable to those of
the world’s best supercomputers three decades ago. This means that we have now
what used to be “supercomputer power” at our fingertips, in our pockets or even on
your wrist while you rest on a sofa or go for a run through the park. A number
of companies are turning things around by thinking of clever strategies to give
users more power, by personalizing the customer experience in ways not previously
thought possible.
In terms of AI advances, the recent result product the DeepMind company is
fascinating. This company is responsible of AlphaGo the first computer program
that managed to defeat a human world champion in the game of Go. The approach
required human supervision, so the algorithm would adapt its decision based on
human expert moves. But when AlphaGo became the world master in the game,
what is next? The company put then AlphaGo in the “driver’s seat”, or, in a twist,
perhaps a better wording would be: AlphaGo became its own teacher. The new
algorithm, AlphaGo Zero, starts tabula rasa and then learns from, and develops the
capacity to predict the original AlphaGo algorithms decisions, resulting in a new
algorithm with superhuman performance that defeated the original AlphaGo 100
times without losing a single match [207].
Humankind has accumulated Go knowledge from millions of games played over thousands
of years, collectively distilled into patterns, proverbs and books. In a space of a few days,
starting tabula rasa (i.e., from scratch), AlphaGo Zero was able to rediscover much of this
Go knowledge, as well as novel strategies that provide insights into the oldest of games.

In an interesting twist, AlphaZero was “repurposed” from Go to Chess. In only


four hours the system came from just the basic knowledge of the rules of Chess to
be at a level in which it has beaten Stockfish 8 the current world champion chess
program in a 100-games match up [205]. It also learned Shogi (Japanese Chess) in
just 2 h and defeated another world-class program called Elmo [205].
This is an extraordinary acceleration, provided by the closed loop of algorithms
teaching algorithms, reaching superhuman abilities in a matter of hours. This feat
has no previous comparison in the history of our world and it is really revolutionary.
Algorithms now decide what to show to us; which books we may like, what
to watch, where to stay, how to travel, where to study, what to buy and in which
10 P. Moscato and N. J. de Vries

companies to invest. Although hype also exists, even in the medical area, algorithms
are now allowing us help to diagnose and even predict the occurrence of certain
diseases. Algorithms are creating new knowledge by teaching themselves from the
data they generate.

1.2.2 Bad Algorithms Might Go, But Good Ideas Remain

The progress with AlphaGo and AlphaGo Zero is in part due to new existing
hardware (with some versions of AlphaGo running over 176 GPUs, while other
versions and AlphaGo Zero run on 4 TPUs, the Tensor Processing Units created by
Google specially for machine learning, which are now being reported to run at 45
Teraflops, that is, they are capable of performing 45 trillion floating-point operations
per second).
It is also just to say that some algorithmic framework ideas are, in some
sense, more transcendental than hardware and the latest technological achievements.
Good algorithms, and good algorithmic ideas, remain for a long time. At the core
of AlphaGo and AlphaGo Zero there are tree search techniques [206, 207] that
have been proposed many decades ago. In some cases an algorithm may “remain
forever” once you have mathematically proved that there is no better algorithm
for a particular problem. A formal mathematical proof could bring one question
to a close: “how complex” is the computational problem? That is why the field
of computer science always strives to find the best possible algorithm for a given
problem. In some cases researchers can find them and a problem is, in some sense
to be defined later, a “case closed”.
New algorithms are also inspired by new hardware, and the core ideas will
remain, evolving with the new generations of hardware. For instance, the technical
presentations at the Parallel Computing and Transputer Applications conference
in Barcelona (PACTA ’92, 20–24 September 1992) had a general focus on the
“Teraflop Grand Challenge”. The general conclusion arrived at that conference was
that: “a Teraflop computer would cost hundreds of millions of pounds; need to be
housed in a small warehouse; and require an extensive cooling system. To operate
the machine would require upwards of three Megawatts, requiring its own electricity
sub-station to down-load off the national grid. It was noted that switching such
a machine on or off would require the approval of the local Electricity authority
on each occasion”.3 Twenty years later NVIDIA unveiled a 1.3 Teraflop GPU for
Supercomputing based on the Kepler architecture. We have now this dreamed, once
utopical supercomputing power, and it is a reality. A system built on many of these
“building blocks” achieved 27,000 Teraflop peak performance (Titan at Oak Ridge
National Laboratories).4 The new building blocks are proving even faster, NVIDIA’s

3 http://www.chilton-computing.org.uk/inf/transputers/p011.htm.
4 http://energy.gov/articles/new-titan-supercomputer-named-fastest-world.
1 Marketing Meets Data Science: Bridging the Gap 11

Volta GPU now has a peak performance at around 7 Teraflops. The game console
Xbox One X delivers 6 Teraflops. In terms of supercomputing, Sunway TaihuLight,
in Wuxi, China, maintains its leadership with a mark of 93.01 Petaflops (data from
November 2017); thus, it tops the list of the 500 fastest supercomputers in the world.
Supercomputing brings an impressive new world to business analytics which is
yet to be fully explored. Together with new algorithms it will give large corporations
(as well as data-driven and computationally wise start-ups) an unprecedented
capacity to analyse large datasets. Supercomputing will bring a transformational
capacity to small and medium enterprises which is still waiting to be properly used.
When allied to Big Data this will shake marketing [37, 64, 211].
While hardware progress and prowess is impressive, however, it is “the ideas
behind computer programs” that live on. For instance, at the same supercomputing
conference we just mentioned (PACTA ’92), a relatively new methodology named
“Memetic algorithms” was presented for the first time in the European community
[162]. It was one of the 183 papers accepted; now it is not only the most cited
paper of that conference, it has become a driving idea that has become stronger than
ever before and we dedicate an entire section to it and the current applications of
memetic computing in business analytics and data science. Originally, they found
an initial motivation in the use of these large scale computing systems based on
parallel architectures (see [160] for a historical account of the development of this
field until 2012). With the advent of “supercomputing at your fingertips”, smart
phones and other systems allow users to tailor solutions to their own needs. There is
a pending revolution, that of personalized systems, that would benefit from the user
experience and that wishes to bring the value of the wealth of data available online
with a unique perspective.
What is important about the story of memetic algorithms and their development
is that it is a clear example which shows that some methods remain in practice
many years later. While the “algorithmic revolution” is here now, its origins go back
three decades or more, when computing became personal. It may be the case, then,
that we should say that now we are living the times that what it was considered
“supercomputing” has become personal and is in our pockets.

1.2.3 Is the Disruption Real?

In any revolution there are new opportunities, challenges, as well as new problems
ahead. Nothing comes without a cost, especially in revolutionary times. The global
changes of our intellectual and economic endeavours are imminent. The clear
trouble of manufacturing industries, for many countries, may have been an early
warning of more disruption to come. A perfect storm is brewing, powered by
accelerated progress in Mathematics and Computing; its key force is the global
deployment of Artificial Intelligence (AI) techniques. In the mid-1960s, one of the
founders of Artificial Intelligence, the Nobel Laureate in Economics H.A. Simon,
predicted that “machines will be capable, within twenty years, of doing any work
a man can do”. Our current machines are not able to do “any work a man can do”;
12 P. Moscato and N. J. de Vries

instead, they are much better than the best humans in specific and certain tasks, e.g.,
in games like Go as discussed in the prior section, a trend that started with Checkers
(Chinook vs. M. Tinsley, 1994) or Chess (Deep Blue vs. G. Kasparov, 1997), as well
as Jeopardy (Watson vs. B. Rutter and K. Jennings, 2011).
The “digital revolution” should now be renamed “the algorithmic revolution”.
In reality, everything that we do with computers corresponds to a set of step-by-
step detailed instructions designed to accomplish some final outcome in a machine.
Our anthropocentric perspective “Everything we hope to do with computers requires
the design and implementation of algorithms” may still be valid, but the design of
algorithms is no longer the privilege of our race, machines now can also design
algorithms. And algorithms have become a powerful new force in the Knowledge
Economy. An algorithm presented by Brin and Page in a conference in Brisbane,
Australia, back in 1998, named PageRank [30], is credited to be the foundation
for Google. A similar algorithm was used in the 1996 search engine RankDex
designed by the CEO of Chinese search giant Baidu. Algorithms also give us great
opportunities. Algorithms are the true engine of the new Networked Knowledge
Economy.
The Digital Economy is now considered a key driver of growth for many
developed countries, it will also shape the future of some economies that will use
the news technologies in creative ways. Europe has already recognized this fact
and is working to get all possible global advantages of it. Before Brexit, Europe
was planning to merge 28 national markets to a single digital one. Europeans
estimate a global contribution to their continent of 415 billion Euros per year and the
creation of 3.8 million jobs.5 This new Digital Economy will create unprecedented
opportunities for the delivery of goods and services, boost existing skills, allow
life-long learning and facilitate investment for the creation of ICT start-ups and new
companies. The sheer size of this new affluent cohort of 500 million people can only
be matched by the growth of the Asia-Pacific economies and brings opportunities
and challenges.6

1.2.4 If “Data is Dumb”. . . Is “Big Data” Dumber?

“Algorithms are where the real value lies. Data is inherently dumb. Algorithms
define the way the world works”. These were the words of Peter Sondergaard,
Senior Vice President, Gartner Research, in his opening talk at the Symposium
ITxpo (Oct. 5, 2015). This is no overstatement. No actionable insights can be found
by looking at data without any methodological tools such as algorithms. Industry
is clear about the benefits of an algorithmic-based approach, supported by the best
Computer Science practices, to develop the economy.

5 https://ec.europa.eu/commission/priorities/digital-single-market_en.
6 https://www.gov.uk/government/publications/the-age-of-algorithms.
1 Marketing Meets Data Science: Bridging the Gap 13

There is no doubt that with the increasing availability of large datasets, there is a
huge potential delivered by scientific activities to transform the generated data into
knowledge. Peter Sondergaard estimates that an emergent “Algorithmic Economy”
will develop from the Internet of Things (IoT) [208] . He predicts that the impacts
will be ubiquitous and massive;
Products will be defined by the sophistication of their algorithms. Organizations will be
valued based not just on their big data, but the algorithms that turn that data into actions and
ultimately customer impact.

The prediction is that “by 2020, 30 billion mobile phones, tablets, computers,
wearable technology devices and other types of connected devices will be in use”
and that “the incremental revenue generated by the IoT suppliers is estimated to
reach 309 billion per year by 2020”.7
It is interesting to point out that the part of the prediction that says that
organizations “will be valued based not just on their big data, but the algorithms
that turn that data into actions and ultimately customer impact” may have already
been verified. A relatively new company called Jet.com (established in Jan. 2014)
was valued and purchased at USD 3.3 billion in 2016. Part of its success is based
on an algorithm that allows consumers to find the best deal based on the actual
contents of their digital shopping carts. With an estimated return-on-investment of
approximately 15 times for an e-Commerce company and the news that an entire
shopping mall previously valued at USD 200 million was purchased in Jan. 2017 for
just USD 100 [18], it is clear that Internet retailing based on algorithmic solutions
is having a clear impact.

1.2.4.1 Big Data vs. Large Datasets

Many people and many media reporters, perhaps for lack of understanding, neglect
the importance that algorithms have. They also tend to confuse “Big Data” with
“larger datasets”, so it would be relevant to give some clear definitions. The current
Wikipedia entry for “Big Data” is rather unsatisfactory: “‘Big data’ is a broad term
for data sets so large or complex that traditional data processing applications are
inadequate”.8 Defining something by our current “inadequacy” to do something
with it is not a good start. However, it does point at the need to have powerful
algorithms that can extract meaning from it. Any relevant online blogpost or
academic referred paper on “Big Data” will explain some common definition that
includes the “four V’s of Big Data”; Volume, Variety, Velocity and Veracity. This
is where the difference lies between simply a “larger dataset” and true “Big Data”
[39].

7 http://www.businessnewsdaily.com/5450-internet-of-things-business-opportunities.html.
8 https://en.wikipedia.org/wiki/Big_data.
14 P. Moscato and N. J. de Vries

The Volume of data of course does refer to sheer size but it is also the number of
datasets that are increasingly collected, stored and available to business leaders and
researchers which is challenging us. It is predicted that by 2020, 40 zettabytes (43
trillion gigabytes) of data will be created.9 This is a 300% increase since 2005.
Although greater amounts of data make it harder to analyse all the information
available, it is the other three V’s that truly make Big Data so hard to deal with.
Today’s data landscape has an extremely high variety and velocity. This means
that there is not just one type of big dataset that makes up the whole equation
to understanding a problem, trend or topic. Businesses need to derive data from
many different sources which come in many different formats, combine all this data,
analyse it and generate useful insights. For instance, data coming from social media
sites in the form of text or numeric values such as “likes” needs to be combined with
financial data trends, sales figures and possibly historical datasets for a manager to
make a completely informed decision about their next strategic move. Furthermore,
high Velocity means that all these high volumes and varieties of data are increasing
at an ever accelerating pace. You may have heard that nowadays, one flight of a
Boeing 787 creates half a Terabyte of data. Similarly, during one trading session
of the New York Stock Exchange, around 1 TB of data may be generated. In a lot
of cases, decisions need to be made almost instantaneously, sometimes not even
by a person, but by algorithms and machines. This means that a high velocity of
data-streams needs to be accounted for in the computing power and the algorithms
scalability aspects. Finally, the last “V” stands for “Veracity”. Basically, it refers to
the uncertainty of data. This topic has received a lot of attention in recent years as
businesses and consumers have become more aware and concerned about integrity,
privacy and data security in our digitally interconnected world. Not only does poor
data quality or poor data standards cost economies and organizations a lot of money,
it also risks the integrity of findings coming from these data sources leading to
potentially catastrophic business decisions.
This is why we need to tread especially carefully when dealing with Big Data as
we do not want to have “bigger” and “dumber” data without any extra understanding
or advancement in knowledge. What we want is to make ever more informed
decisions, use the information available to us in ever-increasingly efficient ways and
enhance the life of consumers, organizations, businesses and economies through the
use of data science techniques.

1.3 Computer Science: An Unusual but Rightful


Introduction

Firstly, we provide a brief background to the computer science discipline in general,


what it encompasses and some of the reasons for the unavoidable omnipresence
of data analytics in any transaction, purchase, or planning today. A business or

9 http://www.ibmbigdatahub.com/infographic/four-vs-big-data.
Another random document with
no related content on Scribd:
This island did not tempt us to stay long, so we steered for
Cagayan Sulu, which is a gem in the ocean; it has three peaks,
wooded, but varied by grassy glades, groves of cocoa-nuts and fruit-
trees, partly concealing and partly revealing scattered houses and
villages. It is, indeed, a picturesque island from every view. Our first
intercourse appeared likely to be unfriendly. Steering round to the
south-west side, we landed at a place where we saw some houses
close to the beach, and as we pulled ashore, we could see the
inhabitants gathering in armed groups; however, we were received
with great civility, and explained the object of our visit, which was to
inquire what fresh provisions could be procured there. We did not
stay long, as they promised to bring us down next day a good
supply.
When we landed on the following morning, we found a very large
party assembled with several fine bullocks for sale; while the
bargaining was going on I wandered inland with a companion to
have a look at the country. Wherever we went we found plantations
of cocoa-nuts and plantains, and round the houses were small
vegetable gardens, while between the dwellings were occasionally
extensive tracks of long coarse grass, on which were herds of
bullocks feeding.
At length we came to a spot which tempted us to rest. It was a
rock overhanging a tiny bay, thrown into deep shade by the tall
graceful palms which bent over it; while looking inland across the
gently sloping fields of long high grass interspersed with groves, we
could see parties of natives marching in Indian file, with their bright
spear-heads flashing in the sun, winding their way down to the
extemporized market. We sat under the shade of some areca palms;
which, though young, and not twenty feet in height, were yet covered
with fruit and freshly expanded blossoms, which shed a delicious
perfume through the whole grove.
Our bargaining prospered, as fine cattle were secured at thirty
shillings a piece, ducks for two wine bottles, fine cocks and hens for
one; as well as a couple of pretty ponies, cocoa-nut oil and nuts,
plantains, limes, ginger, onions, and fruits. This island, though
formerly a dependency of Sulu, is now independent, and is governed
by some of those half-bred Arabs who corrupt and weigh heavily on
these countries. It is finely situated in the Sulu seas, and it is both
healthy and fertile. The inhabitants appear much the same as those I
had seen about Maludu Bay, and, with the exception of some
strangers, were civil. The latter were traders who had visited
Samboañgan, Manilla, and other Spanish ports, and were there
corrupted, by intercourse with the low Europeans and dissipated
classes who usually frequent such places; and at one time these
men were so insolent that I thought their conduct would become
unbearable, till they were quieted by my shooting down a cocoa-nut,
as mentioned in my Limbang journal.
The most singular spot in this island is near the old crater-looking
harbour, mentioned by Sir Edward Belcher, which we entered over a
reef. It is almost circular, and is surrounded by lofty rocks clothed
with trees, bushes, and hanging creepers, presenting a magnificent
wall of evergreen. Rowing to the west side of the bay, and climbing
to the top of a lofty bank, we had a splendid view of a remarkable
almost circular, lake. The place where we stood was a gap between
the lofty cliffs that rose on either hand, and appeared to have been
formed by the inner waters bursting their boundary, and overthrowing
the upper defences of this natural dam. The wooded cliff’s continue
all round, forming a perfect barrier, now rising to a great height, then
sinking to some fifty or sixty feet. The waters, elevated more than
forty feet above the sea, lay in undisturbed repose, and presented
for upwards of half a mile a clear mirror, reflecting back the rays of
the sun and the deep shadow of the tall trees.
I mounted with some difficulty the top of the left-hand cliff, and had
an extensive view through the tangled bushes. On one side were the
waters of the crater, on the other the serene lake with the sea
appearing beyond. We heard from the natives that alligators swarm
in this secluded water, so that fish must be plentiful also, as few pigs
could descend here. In the interior they say there is another more
extensive lake of the same formation; this one was found to be eight
fathoms deep at the edges, and forty in the centre.
Started from Cagayan Sulu, and after encountering rougher
weather than we expected to find in this usually calm sea, lay to, as
we thought we were not far from the island we were intending to
visit.
At daylight the two peaks of Sulu were visible; and as we
approached, the summits of the other hills appeared, while all the
rest of the island lay enveloped in mist; but the sun soon dispelled
this, and showed us the west coast of the lovely island of Sulu. The
slopes of the hills presented alternate patches of cleared grassy
land, with clumps of trees scattered over its face, reminding one of a
noble English park; while a long dense line of cocoa-nut palms
skirted the beach, among which were seen many houses and groups
of inhabitants, who were no doubt watching our approach with much
anxiety, as they had already suffered from the attacks of the Dutch
and Spaniards; but the sight of the English flag no doubt restored
their confidence.
Anchored off Sugh, the capital of Sulu, which is situated at the
bottom of a bay, and appears small; but among the existing
dwellings we could distinguish blackened piles, the remains of
portions of the city burnt by the Dutch. On the right of the town is a
high hill, curiously peaked and well wooded. Farther off to the left,
and at the back, are many high mountains, some peaked, others
rounded; but, as a whole, forming a beautiful view. A white flag, with
a castle represented on it, waved over the fort, and a pilot jack
marked the residence of Mr. Wyndham, an Englishman, settled here
for the purposes of trade.
Very few natives came off, until at length a messenger arrived
from the sultan to know who we were; we returned a suitable and, no
doubt, a satisfactory reply, as the Sulus were in a state of great
excitement, having suffered considerably from the recent shots of
the Dutch ships. Having communicated with Mr. Wyndham, we went
to see the watering-place about a mile to the west of the town,
situated near the great tree to which I have alluded in my second
account of Kina Balu, where we found the water bursting out of the
sand, clear as crystal, and pleasant to the taste.
The tree is very fine, being at least thirteen feet in diameter, with a
very sinewy stem. Mr. Wyndham walked with us a little way into the
country, and showed us some of the houses, having much the same
appearance as those of the Malays. These people are better-looking
than most other inhabitants of the archipelago; but appeared to be
suspicious, watching us at every point; in fact, we afterwards heard
the fugitives from Balignini were scattered among them, and they
had had no time to forget what they had suffered from the well-
deserved attacks of the Spaniards. We then pulled to Mr.
Wyndham’s house, a mere rough building, raised near the site of his
former one, burnt by the Dutch—why or wherefore is inconceivable.
The whole night after our arrival the country was in an uproar,
reports spreading of the advent of innumerable ships, which made
the inhabitants hurry their women, children, and goods up to the
mountain. In the evening we took a friendly letter to the sultan.
28th.—Went with the watering-party, but were unable to penetrate
far, on account of heavy rain and the incivility of the natives, who
waved us back. No answer, as yet, from the sultan.
30th.—Landed at the watering-place, where the mountaineers
were assembled at a sort of market, bartering, buying, and selling.
We walked about a little, and then returned to the beach, where we
were surrounded by some scores of men, women, and children from
the mountains and neighbourhood; the former are said to be of a
different race, but we saw little signs of it. The women, on the whole,
are better-looking than the Malays, and some of the little girls were
quite pretty; they are civil enough, but anxious to know our business.
In the afternoon we went to an audience with the sultan. Having
landed at Mr. Wyndham’s, who led us by a long shaky platform to the
shore, where we found numbers of armed natives assembled, an
officer from the sultan led the way along a broad rough road with a
high stockade on the left, and houses on the right. We passed in,
through ever-increasing crowds, to a market-place, where the
women were selling fowls, fish, and vegetables, till we came to a
creek, over which a rude bridge took us to the palace. Before
crossing, we observed a large brass 24-pounder showing through an
embrasure. The stockade continued on the left, until we passed a
large gate, where on a green were assembled some hundreds of
men, armed with muskets, spears, heavy Lanun swords, and krises,
and defended by shields, and some brass armour, and old Spanish-
looking helmets.
The audience hall was on the right, and the house of prayer on
the left. The crowd opening, we mounted some steps, and crossing a
verandah thronged with armed men, found ourselves in the presence
of the sultan. The audience hall is large, but perfectly bare of
ornament, as all their valuable silks and hangings were packed up,
and mostly sent to the hills, on account of an absurd report spread
by the mate of the Spanish brig we found anchored near, which,
when we came in sight, began to get under weigh, and stand out to
sea.
On being questioned by the Chinese passengers, he said that we
had given him notice to get out of the way, as we were about to
bombard the town. The rumour having spread, the natives packed
up their valuables, and spent the night in carrying their goods off to
the hills, and in removing their women and children. This caused the
suspicious behaviour of the people, and what tends to keep up their
apprehensions a little is that the Spanish brig has not yet returned to
her proper anchorage.
But to return to the hall. In the centre stood a round table; on the
opposite side sat the sultan surrounded by his datus, and around
were a number of empty chairs, on which we took our seats. After
shaking hands, a few questions were asked, as, “What was the
news? Was France quiet?” The sultan was very like the picture in Sir
Edward Belcher’s book, and was dressed in light-flowered silk, with a
very broad gold belt round his waist, a handsome kris, and gold
bracelets, sparkling with jewels. Some of the datus were splendidly
dressed in silks, gold brocades, handsome turbans and head-
dresses, like golden tiaras; the young men were, as usual, the most
gaudily decorated, while the old were in plain white jackets, and
crowds of the better class sat behind and around us.
Observing that Sir James Brooke, who had lately suffered much
from fever, looked hot and tired, the sultan politely broke up the
audience, and we returned by the same path we came, and after
sitting a short time with Mr. Wyndham, went on board to dine with
Captain Keppel, where our Sulu acquaintance amused us with
stories about the natives.
Went on shore to Mr. Wyndham’s to meet datu Daniel; very little
conversation passed. He appears to be a quiet, good-natured man;
his brothers are very fine fellows, and very fair; with them we went to
see the race-course. Passing through a portion of the town, we came
to an open grassy field, where a few men were trying their horses by
trotting them over the sward. None of the chiefs being present, there
were no trials of speed.
2nd.—On shore early with our guide, but the people not appearing
to like our penetrating into the country, we returned to Mr.
Wyndham’s house, and as we passed the stockades, the Sulus
laughingly pointed to some indentations in the wood where the shot
from the Dutch ships had struck, but had done little damage.
3rd.—Weighed anchor and stood along the coast for Tulyan Bay;
but wind and tide being against us, we let go our anchor; sailing
again at three, we found ourselves towards seven in Tulyan Bay,
much to the discomfort of the villagers, whose shouts and screams
could be distinctly heard on board. At last a couple of men came off
and returned on shore after a few reassuring words from Mr.
Wyndham, which appeared sufficient to pacify the inhabitants, as the
noises ceased. Tulyan Bay is rather deep, and appears to be well
protected, and takes its name from a pretty-looking island in the
offing, which was ceded to the English in Dalrymple’s time—in fact,
the inhabitants thought we were come to take possession.
4th.—I must now endeavour to give the little information I
collected concerning Sulu. The government is carried on by a sultan,
with his council of datus; at the present time the principal power is
held by datu Molok, an elder but illegitimate brother of the sultan—a
shrewd-looking man with quick, inquiring eyes. The sultan is said to
be well-intentioned; but, being weak in character, is totally unable to
subdue the turbulent aristocracy by whom he is surrounded. In
despair he is reported to give way to his fondness for opium-
smoking.
The laws are but little respected, and ancient customs are fast
falling into disuse, particularly one resembling a voluntary poor-rate.
Every Sulu trader used to present five per cent. of his yearly profits
to a fund, which was divided among the poor of the island. The
mountaineers acknowledge the supremacy of the sultan, but refuse
to pay tribute, and a government which cannot enforce that is not
likely to be able to suppress feuds, or effectually to put down
disturbances. Mr. Wyndham pointed out a man who was notorious
as a murderer, and one instance he had himself witnessed of his
bloodthirstiness.
Again, two mountaineers, bargaining for a slice of fish, quarrelled;
they mutually seized each other’s weapons; one held the handle of
his opponent’s kris, the other his spear shaft; they struggled, a fight
ensued, the crowd collected, some took one side, some the other,
and in a few minutes seven men lay gasping on the ground. It is not
to be expected that the dependencies of the island will obey, when
all is in such confusion. The sultan’s power is very limited, though
the datus still send parties to raise contributions from the
neighbouring towns, villages, and islands. In all decaying states we
find religion neglected, and here, I imagine, it is held but in slight
respect; their houses of prayer being like a tumble-down barn, and
the inhabitants indulging in the use of wine, and occasionally pork.
Mr. Wyndham told us an amusing story of an old datu, who, going
on business to a Chinese trader, began to find a delicious odour
insinuatingly creeping over his senses.
“Ah,” said he, “what is this? some cooking, what is it?”
“Pork.”
“Pork?” said he; “ah!”
“Would you like to taste some?”
“Why,” he answered in a low voice, but cautiously surveying the
room to see if he were watched, “yes, bring me a little.”
On tasting it, and finding it very good, he began to eat some more.
Mr. Wyndham living next door, and hearing the old fellow’s noise,
had removed some of the partition, and was watching him. He now
coughed.
“Oh, I am ruined,” cried the datu; “who lives in the next house?”
“Signor Wyndham.”
“Then he has me in his power.”
Our informant then went in, and, laughing, shook the Mahomedan
chief by the hand, and congratulated him on his freedom from
prejudice. He ever after had much influence with the old man, who
feared being exposed. The inland inhabitants call themselves
Islamites, but are very lax and ignorant.
The Sulu language is soft; it contains, I believe, many Malay
words and expressions, but it is essentially different; though the
upper classes understand Malay, and also many of the lower, there
being here numerous slaves from Borneo. The population, they say,
is 200,000; it is probably 100,000; not less, from the numerous towns
and villages along the coast, and the number of houses detached in
twos and threes. On an extraordinary occasion, they say they could
bring some 15,000 or 20,000 men into the field; but, in general,
5,000 would be as many as they could assemble. In fact, when the
day of trouble came, they had not, perhaps, 2,000 to defend the
town; and this may be readily accounted for, as a large proportion of
the population is in servitude, which is, however, generally an easy
state of existence.
The slaves are collected from all parts of the archipelago, from
Acheen Head to New Guinea, and from the south of Siam to the
most northern parts of the Philippines: it is a regular slave market.
The Sulus themselves are a better-looking people than any I have
yet seen; they are daring and independent, and the mountaineers,
particularly, are a wild but polite people. Their young women and
little girls are dark-eyed and good-featured, with easy figures; free,
though not obtrusive, in their ways.
Bold and daring as the datus appear, they have much politeness
in their manners, particularly datu Daniel and his brother; and on
proper occasions their carriage is dignified and commanding. The
lower orders are outwardly rough, violent, and fierce, yet have an
inherent politeness, which, when inclined to show confidence, they
display to much advantage. On state occasions the young men
appear in splendid dresses, while the elder content themselves with
plain clothes. The dress is the same as the rest of the archipelago—
a jacket, trousers, sarong, and occasionally a shirt or under-vest.
They all wear krises, and most of them also carry either muskets or
spears.
The Balignini near the watering-places were the worst we met—
insolent and inclined to pilfer; indeed, there was nearly a quarrel
about some of the seamen’s clothes they tried to appropriate. To
show their dislike, they planted sharp fish-bones round the watering-
place, in the hope that our men, landing in the dark, might cut their
feet. The Dutch burnt about two hundred houses, but did little injury
to the stockades, which are, however, sadly out of repair. In proper
order, well mounted with guns, they could make a good defence, as
the walls facing the sea are about fifteen feet thick of mud and stone,
encased with teak stockades. The rampart around the sultan’s
palace is in the best repair, but not so thick as the others; and datu
Daniel’s is by no means contemptible. The men, too, would fight
bravely, as they did against the Dutch; but their guns, except the
brass ones, are mostly dismounted, and they have no carriages
ready; the iron ones are said to be those taken at Balambañgan,
when they surprised and captured our settlement.
The appearance of the country from the sea is very beautiful,
many of the hills rising to a peak some 2,000 feet above the level of
the sea; while others are lower and wooded, and form an agreeable
contrast. Several of these eminences are forest-covered to the
summit, while many present alternate patches of rice cultivation,
pasture land, groves of cocoa-nut, palms, gardens, and detached
clumps of forest trees. It is by far the most beautiful island I have
seen. Sulu, in good hands, might be made to produce every tropical
production, and become the centre of the commerce in these seas.
Ships, by staying a little time, may obtain bullocks, fowls, ducks,
vegetables, fruits, cocoa-nuts, and very fine water at a very good
watering-place. The duties on goods are high; nevertheless, Mr.
Wyndham and the Spaniards carry on a profitable commerce.
Tulyan is rather a small island, with hills to the north, but low land
on the south: the former with a few trees and some bananas, with
cocoa-nut palms at the foot; the latter a little woody. Dalrymple gives
some account of it. In his time the Spaniards had driven the natives
away, burnt their houses, and cut down their fruit-trees; but there is
now a large village along the beach, with many cocoa-nut groves.
The inhabitants are pearl fishers.
Next day anchored off two woody islets; the captain, as usual,
shelling and dredging. Islands are to be seen in the distance all
around us.
6th.—Anchored off Basilan. High hills and lowlands covered with
woods, showing but few clearings. We counted eighteen islands at
one time; among which were the late pirate haunts of Balignini and
Tonquil. Beat about, and anchored off Samboañgan after dark.
We remained seven days at Samboañgan, walking and exploring
in every direction, and enjoyed our stay there very much.
Magindanau, as far as we have seen it, is very hilly and woody, with
the exception of the neighbourhood of the Spanish settlement of
Samboañgan, which has been cleared for some miles; though, for an
old colony, not so much as might have been expected.
The town is situated on the west point of Lanun Bay, and from the
sea appears much smaller than it is in reality. It presents no very
striking features; the long, low, dark fort and whitewashed houses,
intermixed with a few groves of cocoa-nuts, with forest on either
side, and the hills, some cleared and some wooded, rising about
seven miles inland, suggest a rough idea of this pleasant little town.
The plain around is very well cultivated; as you walk along the roads
—very much like English country ones—you have a continual series
of large rice fields, cocoa-nut groves, now swelling into extensive
plantations, then a few round a detached cottage, and intermixed
with these are great quantities of bananas. Many small streams
intersect the plain, adding much to its fertility, and are spanned by
covered bridges.
The fort is to the right of the town, and has rather low walls,
mounted with a good many guns; against a native force it must be
impregnable, but a little shelling would soon subdue it. It is
garrisoned by about two hundred and fifty native soldiers. Leaving its
gate, you cross a large green, beyond which lies the principal portion
of the town, laid out in a rectangular shape, with streets intersecting
each other at equal distances. The houses are in general mere
native ones; others a little superior; and perhaps a couple of dozen
of a better class, in which reside the Governor, Lieutenant-Governor,
Commandant, and other officers. Almost every other house has a
shop, in which cigars, spirits, chocolate, sugar, and various other
articles are sold. The most respectable class keep retail shops, as
well as the little traders.
Beyond this portion of the town is a little green, with the church—a
long barn-like building. Seen within by the imperfect light of evening,
it appeared destitute of ornaments, except round the altar, and
beyond this are some more streets. The houses I entered had very
little furniture; a small table, some chairs, a bedstead, and a kind of
drawers, with a few shelves, complete the whole.
The people themselves are well worthy of notice. They are a
mixture of Spaniards and natives; a few of pure blood, in the
Government offices; the rest half-castes, mestiches, and natives.
The men exhibit no remarkable features, except the Governor,
Colonel Cayelano de Feguaroa, who was an agreeable man, and
spoke French. We were much pleased with his liberal ideas and
gentlemanly manners, and the other officers were equally polite and
attentive. The generality of the native men are kind and civil, but with
fat, unmeaning faces. The women are much better. The Lieutenant-
Governor’s wife, the only pure Castilian in the place, was a very
pretty woman, with fine eyes and regular features. The mestiches
have in general good eyes and dark hair; but, with the exception of a
few, their faces are too broad.
There was a famous corner shop opposite the church, which
contained good specimens of the race:—Gabriella, likewise called
Romantica, one of the handsomest women in the town, with pleasing
features, and her sister, with the usual flat face. Every one visiting
Samboañgan made that shop their place of call, as the staid old
mother and the girls were very civil and hospitable.
In the country one could always obtain cocoa-nuts, and many of
us were invited in to eat fruit and drink chocolate and gin by the
obliging residents, whose pretty houses, embowered in fruit-trees,
were an ornament to the road side.
The officers of the Meander gave a ball. The quarter-deck was
cleared of guns and surrounded with flags on all sides, and
ornamented with green boughs. All the Spanish officers came, and
about fifty of the townswomen, some young, others old. We danced
quadrilles, waltzes, and polkas: the first caused much confusion, the
second was well danced, while the third was quite new to most of
them. The commandant gave one in return, which was kept up with
greater spirit. Many of the girls were fairer and better-looking than
those we saw on board, and a few were in European costumes, with
shoes and stockings, while the rest had Malay petticoats, and little
jackets with scarfs. Dancing the polka with them was found to be
very difficult, as, few having chemises on, the hand constantly
coming in contact with the skin, it was impossible to obtain a hold;
and their little slippers were flying in every direction.
Their own band played waltzes very well by ear; but nothing else.
Indeed, it is almost the only dance they care for, as the girls find it
difficult to try any other, on account of their wearing slippers without
heels, some of which are very prettily ornamented with gold and
silver embroidery.
Supplies were scarce, though I saw a great many oxen and cows,
some goats, fowls, and ducks; but its being a penal settlement, trade
is obstructed and carefully watched to prevent the escape of
convicts, and none could come to the ship without a pass. The
ponies are very good, except the hacks; the water buffaloes are
large, and employed to draw a peculiar sledge along the smooth
roads. The chief amusement of the men on Sundays is cock-fighting:
crowds assemble to witness this cruel sport; and then they show
some money, which at other times appears so scarce that few shops
could give change for a dollar. We observed that the rice was
trodden out by the buffaloes, on hard beaten ground. Washing was
dear, being eleven dollars a hundred.
16th.—Started on our return voyage. Lay-to off a sand-bank not
marked in the chart. A grave was found there with four bamboos
stuck round, one at each corner, in the midst of thousands of birds,
with immense numbers of eggs, some of which were brought off, and
proved good eating. I will omit the ground we went over on our
return, and give an account of a second visit we paid this
archipelago.
CHAPTER IX.
THE SULU ISLANDS.

SECOND VISIT.

Reach Sugh—Mr. Wyndham comes on board—His News—


Commercial Rivalry—The Stockades—Visit the Audience Hall—
Appearance of the Sultan—Visit Datu Daniel’s Stockaded House
—Guns—Datu Daniel—Appearance of the principal Chamber—
The Bed—Boxes—Property—General look of Discomfort—
Spittoons—Dismounted Iron Guns—Taken from the English—
Excitement in the Town—Hereditary Hatred of the Sulus to the
Spaniards—Their Treaty with Spain—Sandakan Bay—Supplies—
Variegated Wood—Salute—English popular—An Exception—
Death of a Sulu Lady from Grief—The Rumah Bechara—A Ship
taken—Interview with the Sultan—Hope—Character of Datus—
The Balignini—Capture an English Ship—Captives brought to
Sulu—Result of the Action of the Nemesis—The Lanuns—At
Magindanau and Cape Unsang—A narrow Escape—Mate to Lord
Cochrane—Capture of the Maria Frederica—Cold-blooded
Murder of the Captain—Jilolo Prahus—Their Rencontre with Sir
Edward Belcher—Pirates off the Arru Group—Sulu Justice—
Appearance of the People—Attack on the Spanish Gunboats—
Public Audience with the Sultan—Private Visit to the House of his
Bride—The Women—Opium-smoking—Invitation to revisit Sulu—
The Spanish Gunboats—Samboañgan—The Corner Shop—
Sunday’s Amusements—Appearance of the Neighbourhood—A
Breakfast in the Country—Long Walks—People comfortable—
Story of the Loss and Re-capture of the Dolphin—The Dolphin
sails for Maludu Bay—Quarrels—Surprised—Death of Mr. Burns
and the Captain—Murder of a Woman—Injury to Trade—Datu
Badrudin’s Monopolies—The Tungku Pirates—Visit the chief
Town of Maludu—Sherif Husin’s Account of the Surprise of the
Dolphin—The Re-capture of the Vessel by Sherif Yasin—Arrival
at Benggaya—Dolphin delivered up—Visit the Village of Sherif
Yasin—His Appearance—His Account of the Re-capture—His
Position—Smoking over Powder—Delivery of the Cargo—Return
to the Ship—Argus Pheasants—Meet with Baju Boats—Pearl
Fishers—Retaliation—Fishing for Pearls—Mr. Edwardes’s Pearl
—Story of the Datu, and his great good Fortune—The Mermaid
Pearl—Present State of Piracy on the North-West Coast—Cruise
of the Balignini in 1861—Ransom of Inchi Ngah—Names of
present Haunts of the Balignini—A Mangrove Swamp—Present
System of Balignini—Escape of a Native—The Lanuns—A
Dayak’s Experience—A Slave Mart—Spanish Attack on Sugh—
Severe Fight—Bravery of the Sulus—Capture of the Town—The
Sulu Government retire to the Hills, and refuse to submit—A Teak
Forest burnt—No Teak in Borneo—Elephants extinct in Sulu.
Started from Labuan in the steamer Nemesis; and passing over
our old ground soon found ourselves in the Sulu seas. It is difficult
navigation, but we passed safely among the shoals, steering south
of Cagayan Sulu, and between the islands of Ubian and Peñgaturan,
where there is a deep channel. The latter is a long and low island, of
great extent, with a few small villages, palm groves, and near it we
observed many fishermen’s canoes. The sea is studded with shoals
and little islands, and I counted eleven at one time from the deck. It
was misty weather when the island first appeared; but as we
approached Sugh the weather cleared, and by the time we had
anchored it was tolerably fine. Since our last visit many houses have
been built, and the town is gradually assuming its ancient
appearance.
A quarter of an hour after our arrival Mr. Wyndham, whom we had
met on our previous visit, came on board, and brought us the little
news he possessed, that Sulu was perfectly tranquil, and that the
Dutch, with two small vessels, were gradually extending their claims
along the east coasts of Borneo, that the Governor of Manilla had
protested against the Dutch interference in Sulu, and that the pirates
of Balignini were utterly rooted out of their old haunts on the islands
of Tonquil and Balignini. But it was a mistake to imagine that the
Spaniards had killed or taken prisoners the whole number, for very
many escaped and retired to Sulu. Mr. Wyndham was startled on
hearing of the expected arrival of an European enemy, and uttered
many exclamations as to the injustice and barbarity of attacking an
inoffensive people, and asked many questions as to the profit of
making this aggression, though he was scarcely surprised, as he
had known they had long desired the eastern coast of Borneo, and
that they are endeavouring to obtain a show of right for its seizure,
as by them our ancient claim is no longer remembered.
At four, we left the Nemesis with Mr. Wyndham, to visit datu
Molok, the prime minister. We pulled in for the little creek in the
centre of the town, passing many new houses lately built by the
Chinese. Mr. Wyndham told us that he found them very troublesome
competitors; as, spreading themselves over all the neighbouring
islands, they offered apparently higher prices for produce than he
could possibly do; so he obtained from the sultan an order for their
recall to the capital. I say apparently higher prices, for by means of
false weights and the tricks usually practised by the Chinese, they
were enabled to outbid the honest European. Yet I fear that in this
there is much commercial jealousy, and that the injury done to the
general trade by the restriction is much greater than the profit to the
individual.
As we passed before the fort I could see numerous guns
displaying their adamantine lips through the embrasures; and I
learned from our companion that the Sulus possess about 150 brass
guns, besides innumerable iron ones. In the market-place, crowds of
armed men were assembled, looking eagerly and anxiously at us.
We landed at the little bridge, and learning that Molok was at the
sultan’s palace, we proceeded thither, escorted by about half a
dozen men armed with long spears, and followed by a crowd of men,
women, and children. We soon arrived at the fort, and then entering
the old audience hall, found it in much the same condition as before,
quite as bare of ornament, with the old round table and white cloth,
and the chairs arranged around. Datu Molok was present, with a few
others; and we had nearly arranged about the salute when the sultan
entered, and it was settled that it should be given the next day.
The news of the Dutch having taken Bali made them all look at
each other with marks of great disquietude, and when they heard of
the amount of the Dutch force which was expected to visit Sulu, their
uneasiness was clearly to be seen under their assumed quietude.
The sultan appeared in better health, but the heaviness of the lower
part of his face gives him a stupid look, and his long jacket of white
silk did not suit his pale appearance. Having taken our leave, we
retired, and returned to the ship; but Sir James Brooke, being
anxious to see some of the datus that evening, sent us on shore
again to arrange a meeting with Mr. Wyndham: but Molok, being
then with the sultan, smoking opium, we went to datu Daniel’s.
For a couple of hundred yards from the shore the water was so
shallow, that we were compelled to be dragged along over the sand
in Mr. Wyndham’s flat-bottomed boat. Walking a few hundred yards
inland to the left of the town, on the road to the race-course, we
came to the house. It is surrounded by a stockade some thirty feet
high; there were two long iron eighteen-pounder guns to defend the
approaches, but only one was mounted, though there were two
brass ones ready for service in his verandah. A wretched-looking
pony, and a man with a chain round his neck pounding rice, were the
most remarkable features of the place.
Having mounted some high steps, we entered the house, where
we found datu Daniel lying down on his bed, suffering from an attack
of intermittent fever. He sat up, however, and talked a little,
expressed his regret that he was unable to come on board and see
the Rajah, but sent one of his people to Molok to arrange a meeting.
The room we entered was large, some fifty feet long by twenty
broad, with a raised recess on one side fifteen feet square. A native
bed, thirty feet long and twelve broad, occupied the greater portion of
the room; it was divided in two by a curtain, and resembled more a
large raised room than a bed. At the end were long shelves filled
with packets of goods, small boxes, and innumerable articles which I
could not distinguish, while around were hung the datu’s handsome
jackets, sarongs, trousers, krises, and other finery for state
occasions. Four gun-cases were piled at the side of the bed, while in
the recess above mentioned, and about the room, were forty or fifty
boxes, containing his valuable property; there was a ladder, also,
which led to the loft.
Chairs were provided for us; but, with the exception of the bed
covered with beautiful mats and the handsomely-ornamented
pillows, the whole place had an air of discomfort, a sort of musty
look. The bed itself is no doubt the datu’s chief residence; we could
only just see the head of the farther half, as a blue curtain was drawn
across, concealing its beauties and its treasures. Around the room
were arranged several dozen brass spittoons, as if the chief were
accustomed to give large parties. Some of the women and young
girls came to the doors to look at us; a few were tolerably good-
looking, with their dark eyes and black hair, but the generality were
like the Malays.
Returning, we were, as usual, very much stared at, but little
followed. We walked behind some of the stockades, but saw that
nothing had been done either to repair them, or to mount the guns,
which lay grass-covered in every direction. As they were those
captured from the English when Balambañgan was taken, it is
probable they were too honeycombed to be of any use. We
remained at Mr. Wyndham’s house waiting for Molok’s answer, which
came at length, mentioning nine as the hour.
The news of the intended visit of their last European enemy must
have spread very rapidly; boats were plying to and fro from the
palace and outside houses, bringing all the valuables on shore. As
Molok sent a message that he could not come, we had a long talk
with Mr. Wyndham, who frankly confessed he was a Spaniard in
heart, as he thought they were more likely to settle the affairs of
these countries than the English, but added, for some reason he
could not fathom, there was an unconquerable prejudice among the
Sulus against the Spaniards. It is no doubt an hereditary feeling, for
these islanders and the Castilians have been constantly at war for
the last three hundred years; the latter endeavouring to subject
them, and the former supporting their independence with great
determination.
By the treaty of September 23rd, 1836, the Spaniards offered their
protection and the assistance of their army and navy to support the
sultan and repel any enemies who might attack him, and the sultan
of Sulu accepted their friendship and protection. This article the
Sulus considered the Spaniards had failed to carry into effect, as the
spirit and letter of the treaty bound Spain and Sulu to consider their
respective enemies as perpetual foes, except that the Sulus were
not required to support Spain in an European war.
Mr. Wyndham said that Sandakan Bay is exceedingly unhealthy,
and that the whole eastern coast bears the same character during
the south-west monsoon; but nearly every tropical place is
insalubrious until cleared and drained. Sandakan is reported to be
one of the noblest harbours in the world, perfectly sheltered, with
eight fathoms of water. Innumerable boats came around the steamer
with vegetables, cocoa-nuts, eggs, fowls, fruit, krises, Sulu knives,
Dutch 32-pounder shot, cattle, and excellent fish, particularly the red
mullet; eggs on the whole very good; mangoes and plantains; but
krises of indifferent quality; the Sulu knives, of very peculiar shape,
are mostly made in China for this market; a tolerable one, with a
sheath of clouded wood, and a handle with an Arabic inscription, I
tried to buy, but they asked too much for it. The upper portion of the
sheath was of the Kayu Kamuning, a beautiful wood from the island
of Magindanau, which, if it could be obtained of large size, would be
invaluable for furniture; but the natives said it was only the knots of
the trees which were beautifully marked.
At one, we saluted the sultan with twenty-one guns. The echo
seemed to commence at the first hill, and gradually, now with a
deeper, then with a lighter sound, rolled round over the whole circle
of hills and valleys. The sultan returned the salute from his various
stockades.
The English appear very popular in Sulu, their only enemy being
datu Boyak, the rajah Mudu, who was away. He felt aggrieved with
the English on account of Sir Thomas Cochrane’s attack on Maludu
Bay. His sister had married sherif Usman, its chief, who, during the
fight, was shot in the stomach with grape, as he, being one of those
whom they deem invulnerable, exposed himself to every fire, and
fought to the last. His wife was inconsolable for his loss, refused to
return to Sulu, retired into the country, fell ill, and died. Her brother
keeps this as a sort of canker in his bosom. The way these men
prepare themselves to be invulnerable is different from that practised
in Sarawak: here they rub their whole bodies with some preparation
of mercury.
I asked Mr. Wyndham about the Rumah Bechara, or House of
Discussion, mentioned by the American navigator Wilkes, but he

You might also like