DATA SCIENTIST

REPORT 2018

Figure Eight Data Scientist Report 2018

INTRODUCTION
Figure Eight has been taking
the pulse of the data science
community for quite a while
now. But a lot has changed
since our original Data Science
Report in 2015 (run back
when we were still known as
CrowdFlower). Machine learning
projects are multiplying and
more and more data is required
to power them. Data science
and machine learning jobs are
LinkedIn’s faster growing jobs.
And the internet is creating 2.5
quintillion bytes of data each
day to power all of it.
Figure Eight Data Scientist Report 2018
Another thing that’s changed officers–how they felt about
since 2015? The community AI and contrast their opinions
is grappling with more ethical with our core constituency of
issues than ever before. Data data scientists near the end
privacy, of course, has always of the report.
been a paramount concern.
But as AI is increasingly used to Without further ado, let’s get
make big decisions like medical to our findings.
diagnoses and courtroom
sentencing, these ethical
considerations require careful
debate. Understanding what
those involved think about the
technology they’re pioneering
felt important. In fact, we asked
500 ethical professionals–like
doctors, clergy, and police
Introduction 3
DATA SCIENTISTS

DON’T LIKE THEIR

JOB, THEY LOVE IT

DATA SCI E N T I STS W HO AR E “HAP PY ” O R “ VERY H A PPY ”

89%
88%

We’re often told that if you

67%
can find a way to get paid
doing what you love, you’ve
made it. If that’s true, it’s hard
to think of a group of people
who’ve been better at this than
data scientists.

We’ve been asking this

question for years and every
time out we discover that data
scientists absolutely love what
they do. Somehow, they love
2018
2015

2017

it even more than they did

last year (although a true data
scientist might argue that + 1%
is statistically insignificant).

Figure Eight Data Scientist Report 2018 Job Satisfaction 5

IF YOU LOVE YOUR DATA
SCIENTIST, DO NOT SET
THEM FREE
Data (and data science) inspire
a lot of breathless headlines.
Peter Norvig famously outlined
the Unreasonable Effectiveness
of Data. The Harvard Business
Review dubbed data scientist
as the sexiest job of the 21st
century. The Economist
jumped on the “data is the
new oil” bandwagon a year
back. We’d bet plenty of you
remember “Big Data” being a
thing people realized might
suddenly be useful.
 DATA SCI E N T I ST J OB DE M AN D
H OW OFT E N DO YOU G E T CON TACT ED FO R N EW J O B O PPO RT U N IT IES?

29.70%

26.73%

19.31%
P ER CE NTAG E OF R ESPO N D ENTS

11.39%
9.41%

3.47%

SEVE RAL ON CE A S E VERA L O NC E A SEVERA L RA RELY

TIM ES A WEEK T IM ES A MONTH T IM ES A
WE E K M O NT H Y EA R

Figure Eight Data Scientist Report 2018 Data Scientist Demand 7

But it’s worth remembering it
wasn’t always like this. After all,
some of the same companies
that now hoard and guard
proprietary data didn’t track
and save user interactions
ten or fifteen years ago. The
explosion of cheap server space
and the realization of what
could be done with copious
amounts of information helped
make all those headlines in the
last paragraph a reality.
HOW OFTE N DO YOU G E T CON TACT E D
FO R NEW JOB OP P ORT UN I T I E S ?
85%
AT LEAST ON CE
A M ONT H
With so much data to process,
and with so much of it going
towards business processes
and initiatives that create real
value for their organizations,
it’s no wonder that data
scientists are in such high
demand. When we asked data
scientist how often they get
contacted about new gigs, the
results speak for themselves.
50%
AT L EAST O NC E
A W EEK
Data scientists are front
and center in all of this. And
when we asked them how
often they get contacted
about new gigs, the results
speak for themselves:
Nearly 50% of data scientists
get contacted at least once
a week about a new job
opportunity, with about
30% getting contacted
several times each week.
85% get contacted at least
once a month.
In other words, data science
talent is still in big demand. So
if you’ve got good ones in your
organization, keep them happy.
They’ve certainly got options.
30%
SEVERA L T IM ES
A W EEK
WHAT HOLDS DATA
SCIENTISTS BACK? THE
DATA, NOT THE SCIENCE
The thing about data scientists
is they’re greedy. Not in a bad
way, mind you. We know plenty
and they give fabulous holiday
gifts. But no matter how much
data they have, chances are,
they need more.
We’ve been running this survey
for a few years now and this is
always the biggest challenge
for the community. In fact,
last year, about 50% of data
scientists cited this as one
of their top three snags in
their day-to-day. This year,
that number has increased
even more to 55% cited it as
their biggest.
55%
CITE D QUALI T Y/QUAN T I T Y
OF TRAIN I N G DATA AS BE I N G
TH E IR BIG G E ST CHALLE N G E
Figure Eight Data Scientist Report 2018
Here’s what data pros know:
high quantities of high quality
data are what build accurate
models and inform smart
decisions. And the more of it
you have, the more confident
your model will be.
Anything your organization
can do to give your data and
machine learning teams the
fuel they need to make a real
difference should be a big
priority. But remember: they
need good data and, as we’ve
found in years past, they don’t
particularly love munging and
cleaning data either. It’s a waste
of their skills to be polishing the
materials they rely on.
Roadblocks 9
FEED THE MACHINE
(LEARNING)
One question we’d never asked before was what exactly data scientists
do with all that data anyway. But as our platform has grown and we’ve
been able to lift the hood on a lot of projects, more and more of the data
that comes through Figure Eight directly informs AI and machine learning
projects. So we thought we’d just come right out and ask: what percentage
of their work is used for AI?

P E R CE N TAGE O F WO R K U SED FO R A I

10%

none
of the time 28%

1% to 25%
of the time 23%

25% to 50%
of the time 21%

50% to 75%
of the time 17%

75% or more
of the time
90
ARE INFORMING
%
SOME SORT OF
ML PROJECT
While about 1 in 10 data
scientists say none of their
work informs AI projects,
nearly 40% say a majority
of their work does.
With massive increases in
investment for these exact
types of initiatives, we’re
excited to see where this
Figure Eight Data Scientist Report 2018
number goes next year. But
it’s undoubtedly a positive
development. Instead of
being tasked with cleaning
log files, data scientists are
seeing their work inform
cutting edge solutions
across their organizations.
No wonder they’re so happy.
Data in Machine Learning 11
HOW MU CH O F YOU R
WO R KI NG T I M E G OE S
TO R & D ( AS O P P OS E D
TO PR O DU CT I ON )?

N ON E

<25%
5%
25% -50%
30%
50 % -75%
12%
>75%

PERCEN TAGE
OF WORK I N G
TI ME FOR R&D

29%

24%
THE TOOLS
BEHIND THE TALENT
Back in 2015, one of our more
interesting findings centered
around the tools data scientists
used. It wasn’t that Excel was
still a crucial part of their day-
to-day but rather that the
breadth of tools and approaches
they used was very wide. In fact,
the folks at Partially Derivative
picked up on this fact in a
podcast episode they titled
“Keep Data Science Weird.”
Their point? Because data
science was still a young field,
there wasn’t an overwhelming
consensus on what languages,
Figure Eight Data Scientist Report 2018
tools, and frameworks were
best. Data scientists got to stay
creative, finding the specific
suite of tactics that worked
best for their particular projects.
You could argue that machine
learning is in a similar place
right now. There isn’t a codified
set of strategies but a wide
range of approaches to solve
what were once intractable
problems. While the community
has largely coalesced around
Python (61%), they’ve done
nothing of the sort around
machine learning frameworks:
Tools and Frameworks 13
P OP ULAR M ACHI N E LE AR N I N G FRAM E WO RKS

Pandas

N u m py

S c i k i t- l e a r n

Matplotlib

Te n s o r F l o w

Ke r a s

Seaborn

P y t o r c h & To r c h

AW S D e e p L e a r n i n g A M I

Google Cloud ML Engine

Theano
Microsof t A zure
Machine Learning
I B M Wa t s o n
Machine Learning
Amazon SageMaker

C a f f e/C a f f e 2

Mxnet

Salesforce Einstein

Boken

CNTK

Gluon

Deeplearning4j

Paddle

BigDL

Licensing

Chainer

DyNet

0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150

N UM BER O F RESPO N D ENTS U SIN G M L F RA M EWO RK

What really jumps out here is the community. And since a lot
the preponderance of open of these frameworks have been
source tools. Pandas and around for quite some time,
Numpy have been around early adopters are intimately
for a while. Same goes for familiar with them. It’ll take
Scikit-learn and Matplotlib. time, effort, performance (and
TensorFlow’s origins are Google, a little marketing budget) to
but that was open-sourced too. unseat those open-source
frameworks.
Notably, a lot of the more white
label solutions are further
down the list. This isn’t to judge
quality mind you–some of these
tools are tremendous–but
it does speak to a particular
fondness for open-sourced,
community-driven software in

Figure Eight Data Scientist Report 2018 Tools and Frameworks 15

THE DATA THAT DATA
SCIENTISTS ARE WORKING
WITH IN 2018
With all the media focus on
sexy machine learning use
cases like autonomous vehicles
and home assistants, it’s
worth remembering that most
data scientists aren’t actually
working with LIDAR or audio
utterance data.
We asked data scientists to
select all the types of data
they frequently work with and
found that, yes, text and time-
series data remain the most
common ingredients for their
projects. Sensor, audio, and
video data were the least used,
respectively, while about a
fourth of data scientists were
working with still images.
 COM M ON LY US E D DATA T Y P E S

180

160

140

120
NUM BE R OF RES PO ND EN TS U SING DATA T Y PE

100

80

60

40

20

TEXT T I M E- PRODUCT ST IL L SEN SO R AU D IO VID EO

SERIES OR SKU IM AGES DATA
DATA INFORMATION

Figure Eight Data Scientist Report 2018 Data Types 17

DO YOU WORK P R I M AR I LY
W I TH STRUCTU R E D OR
UNSTRUCTU RED DATA?

71%
ST RU CT U RE D U N STRU CTU RED

VS. 29%
ETHICS 101
The ethical issues around
both building and deploying AI
have been magnified in recent
years. We’ve seen countless
examples of algorithmic bias
in sub-disciplines like facial
recognition, employment
application review, audio
assistants, and more. Just
last year, the Supreme Court
had the opportunity to
adjudicate a matter involving
algorithmic sentencing (Loomis
v. Wisconsin). And while the
court chose not to hear that
case, it’s a safe assumption
that legal precedents will be set
around machine learning in the
coming decade.
These are the sort of ethical
issues we were most concerned
with in our survey. Not the
long-term, science-fiction-
tinged ethical issues around
Figure Eight Data Scientist Report 2018
agency, general intelligence,
or defining the boundaries of
consciousness, but real-world
issues the field–and the public–
are grappling with right now.
Keep in mind that we also ran a
survey capturing the opinions
of ethical professionals
like doctors, clergy, and
law enforcement. We’ll be
contrasting the viewpoints
of data scientist with those
professionals throughout
this section.
For starters, data scientists
are really optimistic about AI
in general.
Ethics
While both groups felt that
there would be more attendant
good than ill, the big outlier here
is that ethical professionals are
fairly blasé about the potential
changes AI will bring to society.
Which makes sense. We know
that data scientists are far
more intimately involved in the
space than, say, a judge might
be. They’re more informed
about and more invested in this
technology, so are less likely to
feel that work won’t make much
of a difference at all.
19
 9%

16%

DATA SCIENTISTS

75%

A I W IL L B E...

GO O D I N WO RLD

BA D I N WO R LD

N O C H A N GE
AI WILL BE . . .

GOOD IN WOR LD

BAD IN WOR LD

N O C H ANG E

15%

39%

ETHICS PROFESSIONALS

45%

Figure Eight Data Scientist Report 2018 Ethics 21

ARE WE STILL IN DENIAL
ABOUT ALGORITHMIC BIAS?
We mentioned a handful of
fairly well-known cases of
algorithmic bias in the last
section. In fact, a recent MIT
Technology Review cautioned
that “biased algorithms are
everywhere and no one seems
to care.” But when we asked
both data scientists and ethical
professionals if they believed
AI was more or less biased
than people, our respondents
thought otherwise:
IS AI M OR E OR LE S S BI AS E D T HAN PEO PL E?
DATA SC IENT ISTS
ETHIC S P R OF ESSIONALS
75%
73%
Now, we understand that
comparing technological bias
to human bias is inherently a
slippery idea. It really depends
on how optimistic your view of
human nature is and whether
you blame algorithmic bias on
human programmers, data, or
some other, vaguely ineffable
reason. But it was interesting
to see how many responses
coalesced around both “less”
biased and, especially, “not
biased at all.” After all, we have
copious examples of algorithmic
bias actually happening.
Figure Eight Data Scientist Report 2018
N OT B IASE D
AT AL L
9%
14%
L E S S B I AS E D
Of course, why this happens
is the real issue here. And
oftentimes, it’s not the inherent
nature of, say, a deep learning
model, but the data that
powers that model. The bias is
latent and unintentional. But it
does exist. Solving it requires
real effort, asking the right
questions, annotating data
conscientiously, collecting
data to repair bias, iterating on
models, and showing empathy
for end-users (among others).
Ethics 23
WHAT ABOUT
REAL-WORLD
AI IMPLEMENTATIONS?

Most internet users deal with In fact, let’s start there.

machine learning on a daily basis.
Product and entertainment
recommendations, search
engines, new feeds, you name
it: all are increasingly being
powered by ML.
Because when it comes
to decision-making, most
data scientists have no
problem with the scenarios AI’s
already sewn in. The more crucial
the situation, the less likely
they’re comfortable:

How much of these opinions have to do with

the seriousness of the decision versus the
fact we’ve already seen success applying AI to
those less vital scenarios isn’t something we
can necessarily say for sure. What we can say
is that implementing AI across all sectors of
society isn’t something the community has a
big appetite for. When the people behind the
technology are preaching a slower, more sober
approach, we’d all do well to sit back and listen.
 NU MB ER O F R ESP O NSES

0
10
20
30
40
50
60
70
80
90
1 00
1 10
120
1 30
14 0
1 50
S e l e c t in g Ente r t ai n m e nt

P o pu l at in g So c i al Me d i a Fe e d s

G ra nt i n g B ui l d in g Ac c e s s

Figure Eight Data Scientist Report 2018

E TH ICS: AI DECI S I ON M AK I N G

A p prov i n g Mo r tg a ge A ppl i cati on s

P roc e ss in g He a lt h I n su ra n ce Cl a i ms

P re sc r i bi n g Me di c at i on

S e l e c t in g Orga n
Tra n sp la nt Re c ip i e nt s

Ethics
FOR AI TO M AK E DECI S I ON S W I T HO U T H U M A N IN T ERACT IO NS?

D ete r m in i n g C o ll e g e
Adm i tt an c e
IN WH ICH OF T HE FOLLOW I N G S CEN A R IO S WO U L D IT B E A PPRO PRIAT E

H ir in g Jo b
C an d id ate s

D ete r m in i n g
Ja il Se nte n c e s

No n e o f
T h e se

25
TO RELEASE

OR NOT TO

RELEASE,

THAT IS THE

QUESTION
 Audio interfaces are becoming
more and more prevalent with
each passing day. Comscore
predicts that 50% of all searches
will be voice searches by 2020
and there are already roughly
a billion voice searches each
and every month. But even
the most advanced voice-
activated assistants struggle
with regular, everyday speech.
This is especially true when the
speakers themselves aren’t
native speakers, have regional
accents or dialects, or code
switch between languages.
Figure Eight Data Scientist Report 2018
We asked data scientists about
this exact issue. Specifically, we
wanted to know what should
be done about releasing a
home assistant that couldn’t
understand accents or dialects.
Should the company just release
the product? Should they have
to include a warning label that
notes certain people may not
be able to use the product? Or
should there be regulations in
place preventing the release of
a product in this state.
Ethics 27
Frankly, we expected the TO REL EASE O R NOT TO R EL EASE,

community would want to release T H AT IS T H E Q U EST IO N

the product as is. After all, once

it’s out in the world, it will be
collecting audio utterances that
can be used to iterate on the
48%
model to make it so more and
more users will be understood.

DATA S C I E N TI STS
But that’s not what we found:

Release
with
warning
label

39%

69%
E TH ICAL PR OF ES SI ON ALS

Release

12% 12%

19% Regulations
should prevent
While we were a bit surprised
by this finding, it dovetails fairly
well with the previous section.
The data science community
is cautious about these
implementations. They’re in
Figure Eight Data Scientist Report 2018
favor of transparency. And when
you think about the community’s
fondness for open source
platforms and open data, a lot of
this really starts to make sense.
Ethics 29
DRIVING HOME
A BIG DIFFERENCE
The last state-of-AI we
posed to both our ethical
professionals and data
scientists was a simple one: if
a self-driving car were proven
statistically safer than your
average human driver, would
you rather drive yourself or use
an autonomous vehicle?
For most of our survey, both
groups tracked pretty similarly.
They largely felt that AI was a
force for good, that products
should have labels informing
who they work for and who
they don’t, and were more
comfortable with AI-driven
product recommendations than
AI-driven mortgage approvals
or judicial sentencing.
They were polar opposites here:
Members of the data science
community certainly know
more about the work that’s
going into autonomous vehicle
technology than your local
clergy, but we certainly weren’t
anticipating opposite reactions.
Why were those reactions so
completely different? It’s tough
to say. But at the very least, if
you’re working on self-driving
cars, you certainly know who to
market to first.
WOULD YOU RAT HE R DR I V E YOURSEL F O R U SE A N AU TO N O M O U S VEH IC L E

WOU LD R I DE
75% IN CAR

WO U L D DR I VE
T H E M S E LVE S 75%

DATA SC IENT ISTS

ETH IC S P R OF ESSIONALS

Figure Eight Data Scientist Report 2018 Ethics 31

SURVEY
INFORMATION
For this year’s report, we
surveyed 240 data scientists
over Survey Monkey via email
and at conference booths. To
get copies of older versions of
this report, dating back to 2015,
please head to the resource
center on figure-eight.com.
 Figure Eight is the essential human-in-the-loop AI plat-
form for data science teams. Figure Eight helps customers
generate high quality customized training data for their
machine learning initiatives, or automate a business
process with easy-to-deploy models and integrated
human-in-the-loop workflows. The Figure Eight software
platform supports a wide range of use cases including
self-driving cars, intelligent personal assistants, medical
image labeling, content categorization, customer support
ticket classification, social data insight, CRM data enrich-
ment, product categorization, and search relevance.

Headquartered in San Francisco and backed by Canvas

Venture Fund, Trinity Ventures, and Microsoft Ventures,
Figure Eight serves data science teams at Fortune 500
and fast-growing data-driven organizations across a
wide variety of industries.

figure-eight.com

Figure Eight Data Scientist Report 2018 33