The Context for the Explosion

in
Machine Learning

The City College of New York

CSc 59929 – Introduction to Machine Learning
Spring 2020 – Erik K. Grimmelmann, Ph.D.
Drivers of recent progress in ML

• Faster and less expensive computing

• More and larger data sets
• Greater need
• Better algorithms

These drivers work together in a virtuous cycle

The City College of New York

CSc 59929 – Introduction to Machine Learning 2
Spring 2020 – Erik K. Grimmelmann, Ph.D.
Growth in computing power

The City College of New York

CSc 59929 – Introduction to Machine Learning 3
Spring 2020 – Erik K. Grimmelmann, Ph.D.
 Growth in data

EE Times, 9/14/2016 The City College of New York

CSc 59929 – Introduction to Machine Learning 4
Spring 2020 – Erik K. Grimmelmann, Ph.D.
My first method for calculating trig functions

The City College of New York

CSc 59929 – Introduction to Machine Learning 5
Spring 2020 – Erik K. Grimmelmann, Ph.D.
My first input method

The City College of New York

CSc 59929 – Introduction to Machine Learning 6
Spring 2020 – Erik K. Grimmelmann, Ph.D.
My first scientific computer - 1967

IBM 1620

The City College of New York

CSc 59929 – Introduction to Machine Learning 7
Spring 2020 – Erik K. Grimmelmann, Ph.D.
 Running a program
Source
Type Card Punch Code
Program

Manual Step

Automated Step

The City College of New York

CSc 59929 – Introduction to Machine Learning 8
Spring 2020 – Erik K. Grimmelmann, Ph.D.
 Running a program
Source
Source Card Reader
Type Card Punch Printout
Code Printer
Program

Manual Step

Automated Step

The City College of New York

CSc 59929 – Introduction to Machine Learning 9
Spring 2020 – Erik K. Grimmelmann, Ph.D.
 Running a program
Source
Source Card Reader
Type Card Punch Printout
Code Printer
Program

Compiler
Pass 1 Manual Step
Source
Code
Automated Step

Card Reader
Processor
Card Punch

Pass 1
Output

The City College of New York

CSc 59929 – Introduction to Machine Learning 10
Spring 2020 – Erik K. Grimmelmann, Ph.D.
 Running a program
Source
Source Card Reader
Type Card Punch Printout
Code Printer
Program

Compiler Compiler
Pass 1 Pass 2 Manual Step
Source Pass 1
Code Output Automated Step

Card Reader
Processor
Card Punch

Pass 1
Object Code
Output

The City College of New York

CSc 59929 – Introduction to Machine Learning 11
Spring 2020 – Erik K. Grimmelmann, Ph.D.
 Running a program
Source
Source Card Reader
Type Card Punch Printout
Code Printer
Program

Loader

Compiler Compiler
Pass 1 Libraries
Pass 2 Manual Step
Source Pass 1 Object
Code Output Code Automated Step

Card Reader
Processor
Card Punch

Pass 1
Object Code Output
Output

The City College of New York

CSc 59929 – Introduction to Machine Learning 12
Spring 2020 – Erik K. Grimmelmann, Ph.D.
 Running a program
Source
Source Card Reader
Type Card Punch Printout
Code Printer
Program

Loader

Compiler Compiler
Pass 1 Libraries
Pass 2 Manual Step
Source Pass 1 Object
Code Output Code Automated Step

Card Reader
Processor
Card Punch

Pass 1 Card Reader Output

Object Code Output Printout
Output Printer

The City College of New York

CSc 59929 – Introduction to Machine Learning 13
Spring 2020 – Erik K. Grimmelmann, Ph.D.
My second scientific computer - 1968

IBM 360/44
Model G
Funded by a NSF Grant

Used by three colleges

Haverford
Bryn Mawr
Swarthmore

The City College of New York

CSc 59929 – Introduction to Machine Learning 14
Spring 2020 – Erik K. Grimmelmann, Ph.D.
My second scientific computer - 1968

IBM 360/44
Model G
Funded by a NSF Grant

Used by three colleges

Haverford
Bryn Mawr
Swarthmore

The City College of New York

CSc 59929 – Introduction to Machine Learning 15
Spring 2020 – Erik K. Grimmelmann, Ph.D.
The first scientific calculator - 1972

HP-35
First calculator with
transcendental functions

The City College of New York

CSc 59929 – Introduction to Machine Learning 16
Spring 2020 – Erik K. Grimmelmann, Ph.D.
My first scientific calculator

HP-35
First calculator with
transcendental functions

$395

The City College of New York

CSc 59929 – Introduction to Machine Learning 17
Spring 2020 – Erik K. Grimmelmann, Ph.D.
My first scientific calculator

HP-35
First calculator with
transcendental functions

$395 (≈ $2,400 in 2020 dollars)

The City College of New York

CSc 59929 – Introduction to Machine Learning 18
Spring 2020 – Erik K. Grimmelmann, Ph.D.
The commercial Internet prior to 1992

The NSFNET Backbone Services Acceptable Use Policy, June 1992

Unacceptable Uses
10. Use for for-profit activities, unless covered by the General Principle or
as a specifically acceptable use.
11. Extensive use for private or personal business.
https://en.wikipedia.org/wiki/National_Science_Foundation_Network

The City College of New York

CSc 59929 – Introduction to Machine Learning 19
Spring 2020 – Erik K. Grimmelmann, Ph.D.
Global Internet traffic
Global Internet Traffic (IP Traffic)
1.E+09

1.E+08

1.E+07

1.E+06

1.E+05
TB/month

1.E+04

1.E+03

1.E+02

1.E+01

1.E+00
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
https://en.wikipedia.org/wiki/Internet_traffic
The City College of New York
CSc 59929 – Introduction to Machine Learning 20
Spring 2020 – Erik K. Grimmelmann, Ph.D.
Notable programming languages

Year Language Year Language Year Language

1951 Regional Assembly Language
1952 Autocode 1970 Pascal 1990 Haskell
1954 IPL (forerunner to LISP) 1970 Forth 1991 Python
1955 FLOW-MATIC (led to COBOL) 1972 C 1991 Visual Basic
1957 FORTRAN (first compiler) 1972 Smalltalk 1993 Ruby
1957 COMTRAN (precursor to COBOL) 1972 Prolog 1993 Lua
1958 LISP 1973 ML 1993 R
1958 ALGOL 58 1975 Scheme 1994 CLOS (part of ANSI Common Lisp)
1959 FACT (forerunner to COBOL) 1978 SQL (a query language, later extended) 1995 Ada 95
1959 COBOL 1995 Java
1959 RPG 1995 Delphi (Object Pascal)
C++ (as C with classes, renamed 1995 JavaScript
1980
in 1983) 1995 PHP
1962 APL 1983 Ada 1997 Rebol
1962 Simula 1984 Common Lisp 1999 D
1962 SNOBOL 1984 MATLAB
1963 CPL (forerunner to C) 1985 Eiffel
2000 ActionScript
Speakeasy (computational 1986 Objective-C
1964 2001 C#
environment) LabVIEW (Visual Programming
1986 2003 Apache Groovy
1964 BASIC Language)
2003 Scala
1964 PL/I 1986 Erlang
2005 F#
1966 JOSS 1987 Perl
2006 Windows PowerShell
1967 BCPL (forerunner to C) 1988 Tcl
2007 Clojure
1968 Logo Wolfram Language (as part of
1988 2009 Go
1969 B (forerunner to C) Mathematica)
1989 FL (Backus)
2010 Rust
2011 Dart
https://en.wikipedia.org/wiki/History_of_programming_languages 2011 Kotlin
2012 Julia
2014 Swift
2016 Ring

The City College of New York

CSc 59929 – Introduction to Machine Learning 21
Spring 2020 – Erik K. Grimmelmann, Ph.D.
Introduction of programming languages
6

4 An average of more than one a year

0
1950

1955

1960

1965

1970

1975

1980

1985

1990

1995

2000

2005

2010

2015
https://en.wikipedia.org/wiki/History_of_programming_languages

The City College of New York

CSc 59929 – Introduction to Machine Learning 22
Spring 2020 – Erik K. Grimmelmann, Ph.D.
Virtuous cycle of progress in ML
Greater
Need

More More
Compute
ML Data
Power

Better
Algorithms

The City College of New York

CSc 59929 – Introduction to Machine Learning 23
Spring 2020 – Erik K. Grimmelmann, Ph.D.
Take aways
• Your career opportunities are effectively limitless.

The City College of New York

CSc 59929 – Introduction to Machine Learning 24
Spring 2020 – Erik K. Grimmelmann, Ph.D.
Take aways
• Your career opportunities are effectively limitless.
• You will have to run to stay in place.

The City College of New York

CSc 59929 – Introduction to Machine Learning 25
Spring 2020 – Erik K. Grimmelmann, Ph.D.
Take aways
• Your career opportunities are effectively limitless.
• You will have to run to stay in place.
• You will have to retrain yourself again and again
throughout your career.

The City College of New York

CSc 59929 – Introduction to Machine Learning 26
Spring 2020 – Erik K. Grimmelmann, Ph.D.
Take aways
• Your career opportunities are effectively limitless.
• You will have to run to stay in place.
• You will have to retrain yourself again and again
throughout your career.
• You will probably never get bored.

The City College of New York

CSc 59929 – Introduction to Machine Learning 27
Spring 2020 – Erik K. Grimmelmann, Ph.D.
What makes ML so interesting?

Algorithms

Probability
Programming &
Statistics

ML
Real
World Math
Problems
Rapid
Progress

The City College of New York

CSc 59929 – Introduction to Machine Learning 28
Spring 2020 – Erik K. Grimmelmann, Ph.D.