Lab Overview

Last updated - September 2019

 2009
Harvard NASA Tournament Lab established at IQSS

2010
Research Cooperation with HMS/Harvard Catalyst

LISH’s 2011

Evolution Helped establish US Government Center for Excellence in Collaborative Innovation

2012
NASA Tournament Lab Now Called Crowd Innovation Lab

2016
Crowd Innovation Lab Completes Over 700 Challenges & Pioneers Field Experiments for Innovation

2017
Laboratory for Innovation Science at Harvard Established
 The Laboratory for Innovation
Science at Harvard (LISH) is
spurring the development of the
science of innovation through a
systematic program of solving real-
world innovation challenges while
simultaneously conducting
rigorous field-based social science
Lab Mission research.
 • Pioneer use of field experiments and field methods for
research on innovation

• Build innovation research community across Harvard

University and beyond
LISH • Connect with practice in using research-driven
Objectives approaches to improve innovation outcomes

• Educate students and practitioners on cutting edge

topics related to innovation

• Develop next generation of innovation scholars

grounded in field methods
LISH is a cluster of faculty and researchers that push the boundaries of research
through field experiments.

FIELD SETTINGS &

FACULTY LISH RESEARCHERS
INDUSTRY PARTNERS
 Professor Karim R. Lakhani (HBS)
Founder & Co-Director
Professor Marco Iansiti (HBS)
Co-Director

LISH Professor Eva C. Guinan (HMS)

Co-Director
Leadership Jin Paik (HBS)
General Manager

Professor David C. Parkes (SEAS)

Co-Director
 LISH Core Staff – September 2019
Andrea Blasco, Ph.D.
Alexandra Kesick Shreyas Sekar, Ph. D.
Research Scientist
Research Associate | Project Coordinator Postdoctoral Fellow

Nina Cohodes
Assistant Director Michael Menietti, Ph. D. Rinat Sergeev, Ph. D.
Senior Research Scientist Senior Data Scientist

James Dana
Hirotaka Miura Thomaz Teodorovicz, Ph.D.
Program Associate
Research Associate | Data Scientist Postdoctoral Fellow

Haylee Ham Jackie Ng Lane, Ph. D. Wei Yang Tham, Ph.D.

Data Scientist Postdoctoral Fellow Postdoctoral Fellow

Jennifer Hoffman Steven Randazzo Ruihan Wang

Assistant Director Assistant Director Research Associate | Data Scientist
 Crowdsourcing & Data Science & Science of Science
Areas of Open Innovation AI Development

application

Technology Business of Sports

Commercialization
Crowdsourcing &
Open Innovation
JOY’S LAW HAUNTS MOST CREATIVE EFFORTS

“NO MATTER WHO YOU

ARE, MOST OF THE
SMARTEST PEOPLE WORK
FOR SOMEONE ELSE”

Bill Joy, Computer Scientist,

Cofounder of Sun Microsystems
EXPERIMENT: ONLINE CONTEST TO TEST JOY’S LAW

CAN THE CROWD BEAT HARVARD

MEDICAL SCHOOL?
• Objective: Make the NIH MegaBlast algorithm for
nucleotide sequence alignment for genomics be faster
and more accurate
• Why Important? Genomic sequencing is now a key tool
in life sciences and medical research — a data explosion
that is creating research and clinical bottlenecks
• Approach: Design a two week long contest with $6000
prize pot on Topcoder platform
 WHAT HAPPENED?
Crowd

• 89 different approaches to solve HMS

problem identified NIH

• 122 coders submitted 654 submissions

• Winners from Russia, France, Egypt,
Belgium & US
• Annotate 10 million sequences in < 3
mins; Quarter billion sequences ~1
hour on laptop

Lakhani, K. R., Boudreau, K. J., Loh, P.-R., Backstrom, L., Baldwin, C., Lonstein, E., … Guinan, E. C. (2013). Prize-Based Contests Can

Provide Solutions to Computational Biology Problems. Nature Biotechnology, 31(2), 108–111. http://doi.org/10.1038/nbt.2495
WHAT ABOUT
OTHER TYPES
OF PROBLEMS?
FISHING FOR FISHERMAN
• Background: Individuals engage in illegal, unreported,
and unregulated fishing that circumvent conservation
and management measures, avoiding costs associated
with sustainable fishing practices, and derive
economic benefits from exceeding harvesting limits
• Goal for Contest 1: Use existing data sets to develop a
predictive algorithm to effectively identify if a vessel is
fishing, based on observable behavior regardless of
vessel type or declared purpose
• Goal for Contest 2: Use existing data sets to develop a
predictive algorithm to effectively identify if a vessel is
fishing, and what type of fishing they are doing
FISHING FOR FISHERMAN CONTEST OUTCOMES
Contest 1 Details Contest 2 Details

14 Days 16 Days

$18K Prize pool

+ $1750 for supporting $26K Prize pool
contests & tasks

489 Registered from 62 countries 131 Registered from 31 countries

77 Competitors from 26 countries 56 Competitors from 21 countries

Winners from Poland, Brazil, Winners from Brazil, Poland,

Austria, Japan, & Romania Slovakia, & Russia

1206 Submissions, 807 Submissions,

298 uses of example tests 152 uses of example tests
FISHING FOR FISHERMAN CONTEST OUTCOMES
Two algorithms were developed and
are being tested in the Naval Research
Lab
• Contest 1 Algorithm can distinguish
fishing from not fishing with 97%
predictive power
• Contest 2 Algorithm can distinguish type
of fishing activity with 91-98%
predictive power depending on class of
vessel
The project has been awarded an
Honorable Mention for the 2017
SECNAV Innovation Award
 Better

Faster

WHAT HAVE
Cheaper WE FOUND?
 EXTREME
VALUE
OUTCOMES
AND THE
EXPERT
SPACE
 Can the problem
be defined for
external solver?

QUESTION Do you have

REMAINS:
Can the data &
an in-house
tools be made
expert who
accessible to

WHEN DO
can implement
the solution? Questions to solvers?

ask before
YOU GO TO going to the

THE crowd

CROWD?
Do you have
Can the outcome
budget, staff &
be objectively
time to allocate to
evaluated?
this project?
 • Well-formulated question

WHAT • Easily identifiable ground truth and

benchmarking
MAKES A • Implementation and execution plan prior to
GOOD challenge launch
• Type of solver matches problem in scale and
CHALLENGE? scope
• Pick the crowd and the platform that best matches your
solver needs
• e.g., optimization problem needs someone with background
in coding
 NASA - CoECI Challenges
NASA open Total Challenges/Projects 267

innovation Solution Resolution

Problem Was Solved 39%
research Significantly Advanced Towards a Solution 20%
Incrementally Advanced Towards a Solution 34%
No Significant Value Was Provided 6%
Not Applicable 1%

Implementation
Solutions Planned or Implemented 84%

Cost Savings
% of Challenges with Cost Savings 89%
Average % Cost Savings 37%
Average Cost Savings $248,000
Total Cost Savings $22,775,207

Forthcoming Paper: Jin Paik, Steven Rader, Lynn Buquo, and Karim Lakhani, “The Economics of Scalable Open Innovation: Cost Savings and Value Generation at NASA”
Forthcoming Paper: Jeff Davis, Elizabeth Richard, Jin Paik, and Karim Lakhani, “Establishing a Center of Excellence to Scale and Sustain Open Innovation”
Integrating crowds in academic labs: SEAS

SEAS GeoChemistry (Atmospheric Modeling)

Computational contest to develop a machine learning solution to predict
chemical reactions in NASA's global air quality simulations, to replace the most
computationally resource intensive and slow part of the physics-based
simulator.

SEAS Robotics: Circuit Board Power Out (Robobee's Cousin - Spider)

Ideation contest to design a high efficiency lightweight and small power
conversion circuit.

SEAS Robotics: Material Science Claw

Ideation contest to design actuator for ultra-soft robot hands for handling
extremely fragile organisms, e.g. marine life.
 Integrating crowds in academic labs

Single Cell / MGH

Computational contest to develop a new, automated single cell trajectory method
that surpasses current benchmarks and is a plug-and-play solution for researchers.
In the development of the contest, LISH and MGH are collaborating with over 25
labs worldwide in the development of contest solution parameters with goal of
adoption of the solution at all the partner labs.

Broad CMAP D-Peak

Computational contest to improve the current D-peak algorithm to produce higher
quality data more efficiently. The D-peak algorithm is part of the Broad Institute’s
Connectivity Map technology which is used to measure gene expression changes in
cells.
 Partnering with Linux Foundation to establish the Core Infrastructure
Core Initiative (CII) - a network of researchers and practitioners from across
academia, government, nonprofit, and industry working to advance the field
Infrastructure of open source software. The CII will examine the prevalence, impact, and
durability of OSS in the economy via research and engagement with the
initiative community that will engender the adoption of beneficial norms and
effective practices to enhance the security and efficacy of all OSS projects
Data Science & AI
Development
Published: April 18, 2019
Science of Science
 Science Production Function Society
PI and lab member interviews for deep dive reports characterizing the micro-
organization of scientific production in scientific and technical fields. Currently
developing a continuation of this project into a longitudinal study

Basis for Scientific Influence: The Value of Scientific Citations

Science of Large scale survey on citation behavior

science Social Influence in Collective Decision-Making

Experiment on scientific grant applications—displaying other reviewer ratings to
examine peer influence

projects Understanding Research Collaborations: Sociometric Data from Symposium

To examine how interactions at a research symposium then impacts future co-
authorship and collaboration networks
Technology
Commercialization
 Launching Breakthrough
Technologies

• 6-week MOOC through HarvardX

(EdX)
• Teaching a systematic process for
commercializing technology
• Technologies sourced from Air
Force Research Labs for first run
Business of Sports
 LISH and affiliate researchers are looking to
have real-world impact on the organized
professional team sports in the following fields:
• Studying on-field player performance with
advance analytics
Business
• Using AI and ML techniques for off-field
of Sports market predictions based on consumer
data
projects • Experimentation with new sporting
models
• Empirically-validated research
 QUESTIONS?
lish.harvard.edu | innovationscienceguide.org

@LISHarvard