Professional Documents
Culture Documents
Applied Machine Learning 1st Edition M. Gopal full chapter instant download
Applied Machine Learning 1st Edition M. Gopal full chapter instant download
M. Gopal
Visit to download the full and correct content document:
https://ebookmass.com/product/applied-machine-learning-1st-edition-m-gopal/
More products digital (pdf, epub, mobi) instant
download maybe you interests ...
https://ebookmass.com/product/source-separation-and-machine-
learning-chien/
https://ebookmass.com/product/machine-learning-in-microservices-
productionizing-microservices-architecture-for-machine-learning-
solutions-abouahmed/
https://ebookmass.com/product/distributed-machine-learning-with-
pyspark-1st-edition-abdelaziz-testas/
https://ebookmass.com/product/practical-machine-learning-on-
databricks-1st-edition-debu-sinha/
Machine Vision Inspection Systems, Machine Learning-
Based Approaches Muthukumaran Malarvel
https://ebookmass.com/product/machine-vision-inspection-systems-
machine-learning-based-approaches-muthukumaran-malarvel/
https://ebookmass.com/product/machine-learning-on-geographical-
data-using-python-1st-edition-joos-korstanje/
https://ebookmass.com/product/machine-learning-techniques-for-
space-weather-1st-edition-enrico-camporeale-editor/
https://ebookmass.com/product/machine-learning-for-beginners-
aldrich-hill/
https://ebookmass.com/product/adversarial-robustness-for-machine-
learning-chen/
APPLIED
MACHINE
LEARNING
ABOUT THE AUTHOR
Dr. M. Gopal
Ex-Professor
Indian Institute of Technology Delhi
1 23456789 D102744 22 21 20 19 18
Printed and bound in India.
Information contained in this work has been obtained by McGraw Hill Education (India), from sources believed to be reliable.
However, neither McGraw Hill Education (India) nor its authors guarantee the accuracy or completeness of any information
published herein, and neither McGraw Hill Education (India) nor its authors shall be responsible for any errors, omissions,
or damages arising out of use of this information. This work is published with the understanding that McGraw Hill Education
(India) and its authors are supplying information but are not attempting to render engineering or other professional services.
If such services are required, the assistance of an appropriate professional should be sought.
Typeset at TVH Scan, WZ 391, Madipur, New Delhi 110 063 and printed at
Cover Printer:
Cover Designer: The Computer Minds
Visit us at: www.mheducation.co.in
Write to us at: info.india@mheducation.com
CIN: U22200TN1970PTC111531
Toll Free Number: 1800 103 5875
Dedicated
to my granddaughter
Over the past two decades, the field of Machine Learning has become one of the mainstays of
information technology. Many successful machine learning applications have been developed, such
as: machine vision (image processing) in the manufacturing industry for automation in assembly
line, biometric recognition, handwriting recognition, medical diagnosis, speech recognition, text
retrieval, natural language processing, and so on. Machine learning is so pervasive today that
you probably use it several times a day, without knowing it. Examples of such “ubiquitous” or
“invisible” usage include search engines, customer-adaptive web services, email managers (spam
filters), computer network security, and so on. We are rethinking on everything we have been doing,
with the aim of doing it differently using tools of machine learning for better success.
Many organizations are routinely capturing huge volumes of historical data describing their
operations, products, and customers. At the same time, scientists and engineers are capturing
increasingly complex datasets. For example, banks are collecting huge volumes of customer data
to analyze how people spend their money; hospitals are recording what treatments patients are on,
for which periods (and how they respond to them); engine monitoring systems in cars are recording
information about the engine in order to detect when it might fail; world’s observatories are storing
incredibly high-resolution images of night sky; medical science is storing the outcomes of medical
tests from measurements as diverse as Magnetic Resonance Imaging (MRI) scans and simple blood
tests; bioinformatics is storing massive amounts of data with the ability to measure gene expression
in DNA microarrays, and so on. The field of machine learning addresses the question of how best
to use this historical data to discover general patterns and improve the process of making decisions.
Terminology in the field of learning is exceptionally diverse, and very often similar concepts are
variously named. In this book, the term machine learning has been mostly used to describe various
concepts, though the terms: artificial intelligence, machine intelligence, pattern recognition, statis-
tical learning, data mining, soft computing, data analytics (when applied in business contexts), also
appear at various places.
There have been important advances in the theory and algorithms that form the foundations of
machine learning field. The goal of this text book is to present basic concepts of the theory, and a
viii Preface
wide range of techniques (algorithms) that can be applied to a variety of problems. There are many
machine learning algorithms not included in this book, that can be quite effective in specific situa-
tions. However, almost all of them are some adaptation of the algorithms included in this book.
Self-learning will easily help to acquire the required knowledge.
Basically, there are two approaches for understanding machine learning field. In one approach,
we treat machine learning techniques as a ‘black box’, and focus on understanding the problems
(tasks) of interest: matching these tasks to machine learning tools and assessing the quality of the
output. This gives us hands-on experience with machine learning from practical case studies. Subse-
quently, we delve into the components of this black box by examining machine learning algorithms
(a theoretical principle-driven exposition is necessary to be effective in machine learning). The
second approach starts with the theory; this is then followed by hands-on experience.
The approach into the field of machine learning taken in this book has been the second one.
We have focussed on machine learning theory. For hands-on experience, we propose to provide a
platform through self-study machine learning projects.
In this book on “Applied Machine Learning”, the reader will get not only the theoretical under-
pinnings of learning, but also gain the practical know-how needed to quickly and powerfully apply
these techniques to challenging problems: learning how to conceptualize a problem, knowing how
to represent the data, selecting and tuning algorithms, being able to interpret results properly, doing
an effective analysis of results to make strategic decisions. Recognizing that most ideas behind
machine learning are wonderfully simple and straightforward, the book presents machine learning
concepts and techniques in a non-rigorous mathematical setting, with emphasis on effective method-
ology for using machine learning to solve practical problems. It is a comprehensive textbook on
the subject, covering broad array of topics with more emphasis on the techniques (algorithms) that
have been profitably employed, thus exploiting the available knowledge base.
Machine learning draws on concepts and techniques from many fields, including computational
statistics (a discipline that aims at the design of algorithms for implementing statistical methods
on computers), artificial intelligence, information theory, mathematical optimization, biology,
cognitive science, and control theory. The primary goal of this book is to provide a broad-based
single source introduction to the field. It introduces basic concepts from various fields as the need
arises, focussing on just those concepts most relevant to machine learning. Though the required
material has been given in the book, some experience with probability, statistics, and linear algebra
will be useful.
The first-generation machine learning algorithms covered in this book, have been demonstrated
to be of significant value in a variety of real-world applications with numeric features. But these
algorithms also have significant limitations, for example, although some learning algorithms are
available to classify images, text or speech, we still lack effective algorithms for learning from data
that is represented by a combination of these various media. Also most learning algorithms perform
acceptably well on datasets with tens of thousands of training examples, but many important datasets
are significantly larger. The volume and diversity (structured/unstructured) of data available on
the Internet and corporate Intranets is extremely large and growing rapidly. Scaling to complex,
extremely large datasets—the big data analytics—is probably the most debated current issue.
Given these and other limitations, and the strong commercial interest despite them, we might well
Preface ix
expect the next decade to produce an order of magnitude advancement in the state of the art. Deep
learning algorithms are emerging as very powerful next-generation tools. Like most other areas
of technology, data mining exists on a shifting landscape; not only is the old part of the landscape
being redefined, but new areas of interest always loom ahead.
All learning algorithms are explained so that the student can easily move from the equations in
the book to computer programs. Proliferation of free software that makes machine learning easier
to implement, will also be helpful in the project work. The diversity of machine learning libraries
means that there is likely to be an option available of what language or environment a student uses.
There are many machine learning websites that give information on available machine learning
software. Some of the popular software sources are R, SAS, Python, Weka, MATLAB, Excel, and
Tableau.
This book does not promote any specific software. We have included a large number of examples,
but we use illustrative datasets that are small enough to allow the reader to follow what is going
on without the help of software. Real datasets are far too large to show this. Datasets in the book
are chosen not to illustrate actual large-scale practical problems, but to help the reader under-
stand what the different techniques do, how they work, and what their range of application is. This
explains why a heavy focus on project work is a necessity. Each project must handle a large-scale
practical problem. Use of domain knowledge to formulate the problem in machine learning setting,
and interpretation of the results given by machine learning algorithms are important ingredients
of training the students, in addition to the training on machine learning software. This book on
‘Applied Machine Learning’ provides necessary ingredients for practice—the concepts and the
techniques —but the actual practice will follow through project work on real-life problems.
In a university setting, this book provides an introductory course for undergraduate students in
computer science and all engineering degree programs. Such an introductory course will require a
properly selected subset of techniques covered in the book. The course design must have a heavy
focus on project work, so that when a student has completed the course, he/she should be fully
prepared to attack new problems using machine learning.
Postgraduate students and Ph.D. research scholars will find in this book a useful initial exposure
to the subject, before they go for highly theoretical depth in the specific areas of their research.
The book is aimed at a large professional audience as well: engineers, scientists, and business
managers; with machine learning and deep learning predicted to be the next ‘grand slam’ in
technology, professionals in almost all fields will need to know at least the basics of machine
learning.
I hope that the reader will share my excitement on the subject of machine learning, and will find
the book useful.
M. Gopal
mgopal.iitd@gmail.com
ACKNOWLEDGEMENTS
I am deeply indebted to many people who, directly or indirectly, are responsible for this book
coming into being. I am grateful to Dr. Ashwani Kumar (Professor, Indian Institute of Management,
Lucknow) for his support in my endeavor and for helping me in including an overview of Business
Decision Making concepts in the book.
I thank my undergraduate and postgraduate students from Indian Institute of Technology Delhi,
especially my former PhD students with research projects in machine learning: Dr. Laxmidhar
Behera (Professor, IIT Kanpur), Dr. Rajen Bhatt (Director, Machine Learning Division, Qeexo,
Pittsburgh), Dr. Alok Kanti Deb (Associate Professor, IIT Kharagpur), Dr. Rajneesh Sharma
(Associate Professor, Netaji Subhash Institute of Technology, Delhi), Dr. M. Arunkumar (Lead
Scientist, Target Corporation India, Bengaluru), Dr. Hitesh Shah (Professor, G.H. Patel College of
Engineering and Technology, Vallabh Vidyanagar, Gujarat), and Dr. Ehtesham Hassan (Research
Scientist, Innovation Labs, Tata Consultancy Services India), whose desire for new knowledge was
both a challenge and an inspiration to me.
Ashish Kushwaha, currently working with me for his PhD, has been of great help in completing
this project; my sincere thanks to him.
M. Gopal
mgopal.iitd@gmail.com
CONTENTS
Preface vii
Acknowledgements xi
1. Introduction 1
1.1 Towards Intelligent Machines 1
1.2 Well-Posed Machine Learning Problems 5
1.3 Examples of Applications in Diverse Fields 7
1.4 Data Representation 12
1.4.1 Time Series Forecasting 15
1.4.2 Datasets for Toy (Unreastically Simple) and Realistic Problems 17
1.5 Domain Knowledge for Productive use of Machine Learning 18
1.6 Diversity of Data: Structured/Unstructured 20
1.7 Forms of Learning 21
1.7.1 Supervised/Directed Learning 21
1.7.2 Unsupervised/Undirected Learning 22
1.7.3 Reinforcement Learning 22
1.7.4 Learning Based on Natural Processes: Evolution,
Swarming, and Immune Systems 23
1.8 Machine Learning and Data Mining 25
1.9 Basic Linear Algebra in Machine Learning Techniques 26
1.10 Relevant Resources for Machine Learning 34
3. Statistical Learning 73
3.1 Machine Learning and Inferential Statistical Analysis 73
3.2 Descriptive Statistics in Learning Techniques 74
3.2.1 Representing Uncertainties in Data: Probability Distributions 75
3.2.2 Descriptive Measures of Probability Distributions 80
3.2.3 Descriptive Measures from Data Sample 83
3.2.4 Normal Distributions 84
3.2.5 Data Similarity 85
3.3 Bayesian Reasoning: A Probabilistic Approach to Inference 87
3.3.1 Bayes Theorem 88
3.3.2 Naive Bayes Classifier 93
3.3.3 Bayesian Belief Networks 98
3.4 k-Nearest Neighbor (k-NN) Classifier 102
3.5 Discriminant Functions and Regression Functions 106
3.5.1 Classification and Discriminant Functions 107
3.5.2 Numeric Prediction and Regression Functions 108
3.5.3 Practical Hypothesis Functions 109
3.6 Linear Regression with Least Square Error Criterion 112
3.6.1 Minimal Sum-of-Error-Squares and the Pseudoinverse 113
Contents xv
Problems 567
References 613
Index 623
Another random document with
no related content on Scribd:
His unconscious sex-bias apparently overlooked the pregnant and
very pertinent caution which he had himself uttered in a previous
work:—“Useful organs, however little they may be developed, unless
we have reason to suppose that they were formerly more highly
developed, ought not to be considered as rudimentary. They may be
in a nascent condition, and in progress towards further development.
Rudimentary organs, on the other hand, are either quite useless,
such as teeth which never cut through the gums, or, almost useless,
such as the wings of an ostrich, which serve merely as sails.... It is,
however, often difficult to distinguish between rudimentary and
nascent organs, for we can judge only by analogy whether a part is
capable of further development, in which case alone it deserves to be
called nascent.”—(“Origin of Species,” Chap. XIV.).
But surely Darwin would admit that experiment in capacity of
education and development was as worthy evidence as “analogy,”
and would further acknowledge how little effort in this direction had
ever been made with woman. Buckle would seem to be far nearer the
truth in ascribing to woman an unconscious deductive form of
reasoning, as against the slow and studied inductive process to which
man is so generally trained to be a slave.—(See Buckle’s Essay on the
“Influence of Women on the Progress of Knowledge,” as quoted from
in Note XIII., 8.)
Japan.
India.
The Bombay Guardian calls attention to an extraordinary book
which is being circulated (early in 1891) broadcast, as a prize-book in
the Government Girls’ School in the Bombay Presidency. The
following quotations are given as specimens of the teachings set forth
in the book:—
“If the husband of a virtuous woman be ugly, of good or bad
disposition, diseased, fiendish, irascible, or a drunkard, old, stupid,
dumb, blind, deaf, hot-tempered, poor, extremely covetous, a
slanderer, cowardly, perfidious, and immoral, nevertheless she ought
to worship him as God, with mind, speech, and person.
“The wife who gives an angry answer to her husband will become a
village pariah dog; she will also become a jackal, and live in an
uninhabited desert.
“The woman who eats sweetmeats without sharing them with her
husband will become a hen-owl, living in a hollow tree.—(Conf. Note
VI., 8.)
“The woman who walks alone without her husband will become a
filth-eating village sow.
“The woman who speaks disrespectfully to her husband will be
dumb in the next incarnation.
“The woman who hates her husband’s relations will become from
birth to birth a musk-rat, living in filth.
“She who is always jealous of her husband’s concubine will be
childless in the next incarnation.”
To illustrate the blessed result of a wife’s subserviency, a story is
told of “the great reward that came to the wife of an ill-tempered,
diseased, and wicked Brahmin, who served her husband with a
slavish obedience, and even went the length of carrying him on her
own shoulders to visit his mistress.”
So quotes the Woman’s Journal of Boston, Mass., and says in
comment thereon:—“The British Government in India has bound
itself not to interfere with the religion of the natives, but it certainly
ought not to inculcate in Government schools the worst doctrines of
heathenism.”
Yet, again, are these Hindoo, or Japanese, or Chinese doctrines
simply the precepts of “heathenism” alone? Buckle quotes for us the
following passage from the Nonconformist “Fergusson on the
Epistles,” 1656, p. 242:—“There is not any husband to whom this
honour of submission is not due. No personal infirmity, frowardness
of nature, no, not even on the point of religion, doth deprive him of
it.”
Much the same teaching is continued a century later in the noted
Dr. Gregory’s “A Father’s Legacy to his Daughters”; and again,
hideously true is the picture which Mill has to draw, in 1869:
—“Above all, a female slave has (in Christian countries) an admitted
right, and is considered under a moral obligation to refuse to her
master the last familiarity. Not so the wife; however brutal a tyrant
she may unfortunately be chained to, though she may know that he
hates her, though it may be his daily pleasure to torture her, and
though she may feel it impossible not to loathe him, he can claim
from her and enforce the lowest degradation of a human being, that
of being made the instrument of an animal function contrary to her
inclinations.... No amount of ill-usage, without adultery superadded,
will in England free a wife from her tormentor.”—(“The Subjection of
Women,” pp. 57, 59.)
As to how far public feeling, if not law, has amended some of these
conditions, see Note XXXVI., 6. Meanwhile, as an evidence of what is
the “orthodox” opinion and sentiment at this present day, it may be
noted that Cardinal Manning wrote in the Dublin Review, July, 1891:
—“A woman enters for life into a sacred contract with a man before
God at the altar to fulfil to him the duties of wife, mother, and head
of his home. Is it lawful for her, even with his consent, to make
afterwards a second contract for so many shillings a week with a
millowner whereby she becomes unable to provide her husband’s
food, train up her children, or do the duties of her home? It is no
question of the lawfulness of gaining a few more shillings for the
expenses of a family, but of the lawfulness of breaking a prior
contract, the most solemn between man and woman. No arguments
of expediency can be admitted. It is an obligation of conscience to
which all things must give way. The duties of home must first be
done” (by the woman) “then other questions may be entertained.”
Are not these English injunctions to womanly and wifely slavery as
trenchant and merciless as any ascribed to so-called “heathenism”?
And is it not the fuller truth that the spirit of the male teaching
against woman is the same all the world over, and no mere matter of
creed—which is nevertheless made the convenient vehicle for such
teaching; and that, in brief, the precepts of womanly and wifely
servitude are blind, brutal, and universal?
See also Note XXXIV., 8.
XVIII.
“We need a new ethic of the sexes, and this not merely, or even
mainly, as an intellectual construction, but as a discipline of life, and
we need more. We need an increasing education and civism of
women.”—P. Geddes and J. A. Thomson (“The Evolution of Sex,” p.
297).
Newnham and Girton, Vassar and Zurich, are already rendering
account of woman’s scope of mental power; while the circus, the
gymnasium, swimming and mountaineering are showing what she
might do corporeally, apart from her hideous and literally impeding
style of clothing. As for some other forms of utilitarian occupation,
read the following concerning certain of the Lancashire women:—
“Mr. Edgar L. Wakeman, an observant American author, is at
present on a visit to this country, and is giving his countrymen the
benefit of his impressions of English life and social conditions.
“The ‘pit-brow’ lasses of the Wigan district will not need to
complain, for he writes of them not only in a kindly spirit, but even
with enthusiasm for their healthy looks, graceful figures, and good
conduct. We need not follow his description of the processes in
which the women of the colliery are employed, but we may say in
passing that Mr. Wakeman was astonished by the ‘wonderful
quickness of eye and movement’ shown by the ‘screeners,’ and by the
‘superb physical development’ and agility of the ‘fillers.’ He had
expected to find them ‘the most forlorn creatures bearing the image
of women,’ and he found them strong, healthy, good-natured, and
thoroughly respectable. ‘English roses glow from English cheeks. You
cannot find plumper figures, prettier forms, more shapely necks, or
daintier feet, despite the ugly clogs, in all of dreamful Andalusia. The
“broo gear” is laid aside on the return home from work, and then the
“pit-brow” lass is arrayed as becomingly as any of her class in
England, and in the village street, or at church of a Sunday, you could
not pick her out from among her companions, unless for her fine
colour, form, and a positively classic poise and grace of carriage
possessed by no other working women of England. Altogether,’ he
says, ‘I should seriously regard the pit-brow lasses as the
handsomest, healthiest, happiest, and most respectable working
women in England.’—(Manchester Guardian, Aug. 28, 1891.)
Id.... Concerning the question of male and female dress, evidence
as to how far woman has been hindered and “handicapped” by her
conventional attire, and not by her want of physical strength or
courage, is reported from time to time in the public prints, as witness
the following, published generally in the English newspapers of 14th
Oct., 1891:—
“Not long since a well-known European courier, having grown grey in his
occupation, fell ill, and like others similarly afflicted, was compelled to call in a
doctor. This gentleman was completely taken by surprise on discovering that his
patient was a female. Then the sick woman—who had piloted numerous English
and American families through the land of the Latin, the Turk, and others, and led
timid tourists safely through many imaginary dangers—confessed that she had
worn men’s clothes for forty years. She stated that her reasons for this masquerade
were that having, at the age of thirteen, been left a friendless orphan, she had
become convinced, after futile struggling for employment, that many of the
obstacles in her path could be swept away by discarding her proper garments and
assuming the rôle and attire of masculine youth. This she did. She closely cut her
hair, bought boy’s clothes, put them on, and sallied forth in the world to seek her
fortune. With the change of dress seems to have come a change of luck, for she
quickly found employment, and being an apt scholar, and facile at learning
languages, was enabled after a time to obtain a position as courier, and, but for her
unfortunate illness, it is tolerably certain that the truth would never have been
revealed during her lifetime.”
In the early days of April, 1892, the Vienna correspondent of the
Standard reported that—
“On the 30th ult., there died in Hungary, at about the same hour, two ladies who
served in 1848 in the Revolutionary Army, and fought in several of the fiercest
battles, dressed in military uniform. One of them was several times promoted, and,
under the name of Karl, attained the rank of First Lieutenant of Hussars. At this
point, however, an artillery major stopped her military career by marrying her. The
other fought under the name of Josef, and was decorated for valour in the field.
She married long after the campaign. A Hungarian paper, referring to the two
cases, says that about a dozen women fought in 1848 in the insurrectionary ranks.”
Somewhat more detailed particulars concerning “Lieutenant Karl”
were afterwards given by the Manchester Guardian (June 6, 1892),
as follows:—
“The Austrian Volkszeitung announces the death of Frau Marie Hoche, who has
had a most singular and romantic career. Her maiden name was Lepstuk. In the
momentous year of 1848 Marie Lepstuk, who was then eighteen years of age,
joined the German legion at Vienna; then, returning home, she adopted the name
of Karl and joined the Tyroler Jager Regiment of the revolutionary army. She
showed great bravery in the battlefield, received the medallion, and was raised to
the rank of lieutenant. A wound compelled her to go into hospital, but after her
recovery she joined the Hussars. As a reward for exceeding bravery she was next
made oberlieutenant on the field. Soon after this her sex was discovered, but a
major fell in love with her, and they were married. At Vilagos both were taken
prisoners, and while in the fortress she gave birth to her first child. After the
major’s death she was remarried to Oberlieutenant Hoche. For the past few years
Frau Hoche has been in needy circumstances, but an appeal from Jokai brought
relief.”
All of which goes far to discredit M. Michelet’s theory that women
are “born invalids,” an assertion which Dr. Julia Mitchell
“stigmatises naturally enough as ‘all nonsense,’” and is thus
approved—with a strange magnanimity—by the British Medical
Journal.—(See Pall Mall Gazette, April 29, 1892.)
The “incapacity of women for military service” has been of late
days continually quoted as a bar to their right of citizenship, as far as
the Parliamentary Franchise is concerned. In the face of the
foregoing cases, and of the fact that every mother risks her life in
becoming a mother, while very few men, indeed, risk theirs on the
battlefield, it might be thought that the fallacious argument would
have perished from shame and inanition long ago. But the
inconsistencies of partly-cultivated, masculine, one-sexed intellect
are as stubborn as blind.
See also Note XLV., 6.
XIX.
George Eliot had yet to say, “Heaven was very cruel when it made
women”; and Georges Sand, “Fille on nous supprime, femme on
nous opprime.”
XXI.