Africa’s

Artificial Intelligence
Initiative
Outline
Rest of the world
Impediments
Addressing impediments
Cirrus overview
Cirrus FOUNDRY overview
Cirrus FOUNDRY Fund overview
Example application
Going forward
Rest of the world
 United States
AI-IA
(Artificial Intelligence Initiative Act)
- 5 Artificial Intelligence Research Centres
- 300 million USD per centre over 5 years (60 million USD a
year)
- Higher education, industry and national laboratories

Link:
https://bit.ly/2I0KEbE
 UK/Europe
CLAIRE:

Confederation of Laboratories for Artificial Intelligence Research in Europe

- ~600 top scientists (visiting for periods of time)

- +~400 world-class support staff (permanent)

- ~1 billion EUR a year budget (hub + network)

- https://claire-ai.org

ELLIS:

European Laboratory for Learning and Intelligent Systems

- ELLIS Institutes which need at least 200 million EUR each for the first ten
years

- 10 million EUR a year for co-funding about 200 students / postdocs initially

- https://ellis.eu
Africa

Cirrus:
- > 50 million USD over 6 years (excluding revenue,
grants, gifts and sponsorships)
- Brings together academia and industry
Impediments
Technical Debt
“If you are anything like me, 99 percent of the time you
will be in a situation where your code is not working…”
– Jeremy Howard, Fast.ai

https://dawn.cs.stanford.edu/assets/dawn-overview.pdf

https://xkcd.com/1319/
 Compute Infrastructure and Engineering
The twilight of Moore’s Law and the impact of computing resources on the performance of AI
applications requires optimal hardware and software deployment

“The biggest lesson that can be read from 70 years of

AI research is that general methods that leverage
computation are ultimately the most effective, and by
a large margin.”

The Bitter Lesson

Rich Sutton
March 13, 2019

http://www.incompleteideas.net/IncIdeas/BitterLesson.html

A New Golden Age for Computer Architecture

John L. Hennessy, David A. Patterson
February, 2019

https://cacm.acm.org/magazines/2019/2/234352-a-new-golden-age-for-computer-architecture/fulltext
 Energy
The amount of compute used to train deep learning
models has increased 300,000x in 6 years.

Since 2012, the amount of compute used in the largest AI

training runs has been increasing exponentially with a 3.5
month doubling time (by comparison, Moore’s Law had an 18
month doubling period).

Dario Amodei and Danny Hernandez

https://openai.com/blog/ai-and-compute

The total energy cost of producing a model increases linearly

with each of three variables:
- the size of its training data set
- the number of hyperparameter experiments
- the cost of executing the model on a single example.

https://arxiv.org/abs/1907.10597

https://openai.com/blog/ai-and-compute
Data Infrastructure and Engineering

https://wallpaperstock.net
Addressing impediments
Teams
Bring together the necessary specialised expertise

Distinct tones symbolize the distinct possible tasks for an individual or team to perform. A. Separating distinct types of tasks, specialists can address more conditions
than one individual; the numbers add. What is shown is that two specialists can do twice as many tasks as one. For example, if each one can do 10,000 tasks,
together they can do 20,000. B. Teams can address an even more diverse set of conditions because the numbers multiply. For a two member team it would be
10,000 x 10,000 = 100,000,000.

http://necsi.edu/research/overview/why-teams
 Compute

Graphcore Rackscale IPU-POD NVIDIA DGX-2

https://www.graphcore.ai/posts/introducing-the-graphcore-rackscale-ipu-pod https://www.nvidia.com/content/apac/gtc/ja/pdf/2018/1050.pdf
 Data Management Platform

Conventional ML workflow and data silos. MLdp system architecture showing control plane and data plane and hybrid
storage model.

https://dl.acm.org/citation.cfm?doid=3299869.3314050
The Cirrus Components

Cirrus

Cirrus FOUNDRY

Cirrus
FOUNDRY
Fund
 Cirrus: Capability
Infrastructure:
- Hardware infrastructure
- Software infrastructure
- Data infrastructure
Cirrus
Focus areas:
- AI research
- AI application
- Particularly the “hard sciences” Technology collaboration
- Co-development program
External collaboration: - Hardware
- Cooperation agreements with universities and - Software
industry Cirrus FOUNDRY
Research to Communication (RtC) program
Revenue generation: - Program to aid in the communication of
- Partner and Affiliate programs with industry research results
- In person seasonal programs

Open Learning:
- Open and free access to learning products Cirrus
- Tea time sessions and salons FOUNDRY
Fund
 Cirrus: Resources
Engineering and research core:
- Hardware engineers
- Software engineers Cirrus
- Data engineers
- Machine learning engineers

Cirrus FOUNDRY

Cirrus
FOUNDRY
Fund
Academically Speaking

https://www.thetimes.co.uk/article/is-the-new-dyson-institute-
the-answer-to-the-university-funding-crisis-55qcdblhs
 Cirrus: Resources
Engineering and research core:
- Hardware engineers
- Software engineers
- Data engineers Cirrus
- Machine learning engineers

Cirrus Residency program:

- 12 month program Cirrus Postdoc program:
- Open to undergrad, masters and PhD - Minimum 12 months
- Paid program - PhD’s
- Paid program
Cirrus FOUNDRY
Cirrus PhD internship program:
- 5 month program Cirrus Assistantship program:
- PhD’s in their penultimate or final year of study - Duration varies
- Paid program - Primarily masters and PhD’s
- Paid program

Cirrus
FOUNDRY
Fund
 Cirrus: Revenue
Revenue from:
- Partner program
- Affiliate program Cirrus
- In person Summer and Winter programs
- Grants
- Gifts
- Sponsorships

Cirrus FOUNDRY

Cirrus
FOUNDRY
Fund
 Cirrus FOUNDRY
Cirrus FOUNDRY Residency program:
Bridges the “Valley of Death”
- 12 month program
- Challenge of turning a start-up idea or scientific
- Open to undergrad, masters and PhD
research into large-scale commercial application
- Paid program
Cirrus
Build and foster the ecosystem of:
Cirrus FOUNDRY PhD internship program:
- Serial entrepreneurs
- 5 month program
- Financiers
- PhD’s in their penultimate or final year of study
- Coaches
- Paid program
- Academic bridges
- Infrastructure builders

Building businesses around Cirrus Cirrus FOUNDRY

- Ideation
- Product creation
Results:
- Ability to deploy machine learning
Monthly classes, events and workshops products with significantly more ease and
affordability
Cirrus - Foster new start-up opportunities for the
region and beyond
FOUNDRY
Fund
 Cirrus FOUNDRY: Revenue

Cirrus

Revenue options
- Successful exits of start-up business
- Run the product as a revenue generating Cirrus FOUNDRY
project
- Sell the code/IP without spinning it out

Opportunity to generate a return on the

investment and sustain the FOUNDRY fund
Cirrus
FOUNDRY
Fund
 Stage of Business

Start-up Company

Cirrus
Product/Market Fit
Discovery
FOUNDRY

Cirrus
No Business Activity

No tech Scholarly Translational New Product Commercial Product

R or D Research Research R&D (1.0)

Stage of Technology
Where The Money Goes
2017 Early Stage Venture Deals
Physical Sciences
3%

Life Sciences
9%
Other
7%

Online/Software
81%

Online/Software Other Life Sciences Physical Sciences www.pitchbook.com

FOUNDRY Advantage
Research:
Knowing what facilities and programs exist and where and how to leverage them.

Innovation:
Knowing what innovations are in the pipeline and what industries are most likely to be affected.

Industry:
Knowing what types of innovation industry is looking for and what constitutes technical validation - to create technology
pathway maps to success.

https://static1.squarespace.com/static/543fdfece4b0faf7175a91ec/t/5d1169950985c600011d354e/1561422231115/Hardtech+Investing+White+Paper.pdf
Cirrus FOUNDRY Fund
 Cirrus FOUNDRY Fund
Cirrus FOUNDRY FUND
- Provides the Cirrus FOUNDRY with the capital to
support the start-up portfolio
Cirrus
- ~ R100 million in capital
- No management fees
- No performance fees

Assists the Cirrus FOUNDRY to:

- Maximise its ownership stake in the start-up
Cirrus FOUNDRY
company
- Maximise the valuation of the start-up

Ensures the Cirrus FOUNDRY start-ups:

- Are not wholly dependent on outside capital
- Increases the short term success of the start-up Cirrus
FOUNDRY
Fund
 Summary
Global Engagement: Program managers:
- Full time persons responsible for establishing - Partner and Affiliate program
and managing collaborative engagements in: - RtC program
- North America
Cirrus
- Open Learning Program
- UK/Europe
- Residency, PhD Internships, Postdocs
- Asia Pacific
and Assistantships programs
- Africa and South America
- Engineering projects
- Cirrus FOUNDRY
Collaborative engagements include:
- Grants, Gifts and Sponsorships
- Partner and Affiliate programs Cirrus FOUNDRY
- Academic collaboration
- Cirrus FOUNDRY Fund capital raising
- Technology co-development

Governance
Cirrus
- Equity owned by:
- Staff
FOUNDRY
- Funders Fund
- Board of Directors
- Advisory Council
Example Application
Matter Still Matters…

https://www.bloomberg.com/features/2019-periodic-table-elements-issue
Illustration of Early ML Approaches in Materials Science

https://doi.org/10.1016/j.jmat.2017.08.002
Machine Learning Matters in Matter…

- Structure determination and imaging

- Drug design, drug screening, chemical and fuels production, discovery of new scientifically designed materials.
- ML is already being applied to a large suite of scientific techniques such as:
- phase identification via the ICSD database
- Rietveld refinement, and
- pair distribution function analysis [1] of total scattering experiments
- Density functional theory [2]
- Modelling of new chemically relevant systems
- Materials discovery and design
- Energy storage, space-age materials, environmental solutions
- Automating synchrotron data-analysis [3]
- Detectors in synchrotrons generate 9 Gbps each (over 100 detectors running simultaneously)
- Simulation and prediction of properties of materials [4]
- Massive savings on hit-and-miss type R&D
- Processing of research papers in experimental materials science to extract structural data on huge classes of novel materials [5]
- Extraction of rules from known manufacturing processes to guide synthesis of materials, fuels, drugs
[1] [4]
Using a machine learning approach to determine the space group of a structure from the atomic pair Scientists Use Artificial Intelligence To Discover New Materials
distribution function (PDF) https://www.forbes.com/sites/meriameberboucha/2018/04/22/scientists-use-artificial-
Chia-Hao Liu, Yunzhe Tao, Daniel Hsu, Qiang Du, Simon J.L. Billinge intelligence-to-discover-new-materials/#1b0b6cf138c4
https://arxiv.org/abs/1902.00594
New Lithium-Rich Battery Could Last Much Longer
[2] https://www.mccormick.northwestern.edu/news/articles/2018/01/new-lithium-rich-battery-
A density functional theory parameterised neural network model of zirconia could-last-much-longer.html
Chen Wang, Akshay Tharval, John R. Kitchin
https://doi.org/10.1080/08927022.2017.1420185 Generative models for fast simulation
Sofia Vallecorsa
Deep Learning and Density Functional Theory https://indico.cern.ch/event/567550/papers/2656673/files/5841-
Kevin Ryczko, David Strubbe, Isaac Tamblyn SofiaVallecorsa_Plenary.pdf
https://arxiv.org/abs/1811.08928
Accelerated discovery of metallic glasses through iteration of machine learning and high-
[3] throughput experiments
Automated Data Analysis Strategy for Synchrotron Experiments Fang Ren, Logan Ward, Travis Williams, Kevin J. Laws, Christopher Wolverton, Jason Hattrick-
Lifen Yan Simpers, and Apurva Mehta
http://www.imedpub.com/articles/automated-data-analysis-strategy-for-synchrotron-experiments.pdf https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5898831

Researchers get most comprehensive view yet of lithium-ion battery electrode damage [5]
https://www6.slac.stanford.edu/news/2019-06-03-researchers-get-most-comprehensive-view-yet- With little training, machine-learning algorithms can uncover hidden scientific knowledge
lithium-ion-battery-electrode-damage https://techxplore.com/news/2019-07-machine-learning-algorithms-uncover-hidden-
scientific.html
Collaboration

DeepMind: AlphaFold: Using AI for scientific discovery

https://deepmind.com/blog/alphafold

Google AI: Healthcare and biosciences

https://ai.google/healthcare

Toyota Research Institute (TRI): Researchers Use AI to Accurately Predict the Useful Life of Batteries
https://www.tri.global/news/researchers-use-ai-to-accurately-predict-the-usefu-2019-3-25

Machine learning collaborations accelerate materials discovery:

https://physicsworld.com/a/machine-learning-collaborations-accelerate-materials-discovery
Where We Are Headed

AI and Robotics in the Lab: Accelerating Materials Discovery

https://events.technologyreview.com/video/watch/jill-becker-accelerating-materials-discovery
Jill Becker, Kebotix
Science: AI Transforms Science

Science:
07 July 2017
Vol 357, Issue 6346

https://science.sciencemag.org/content/357/6346
 Going Forward
Get in touch
- Contact details on the website
www.CirrusAI.net

Letters of support
- Individuals
- Research groups
- CoE’s
- Institutes
- Universities
- Corporations

Cooperation Agreement
- Announcements coming soon
Reference
Slide 3:
NASA. (1969). Apollo 11 mission image - earth view over Africa, the Mediterranean and the Middle East. [image]. Retrieved from
https://www.flickr.com/photos/nasacommons/5052744618
Slide 8:
Stanford Dawn. (2017). Hidden technical debt in machine learning systems. [figure]. Retrieved from Dawn: infrastructure for usable machine learning. [pdf].
Retrieved from https://dawn.cs.stanford.edu/assets/dawn-overview.pdf
Slide 9:
Hennessy, J. Patterson, D. (2019). A new golden age for computer architecture. [figure]. Retrieved from Communications of the ACM, Vol. 62 No. 2, Pages 48-60.
Retrieved from https://cacm.acm.org/magazines/2019/2/234352-a-new-golden-age-for-computer-architecture/fulltext
Sutton, R. (2019). The bitter lesson. [blog]. Retrieved from http://www.incompleteideas.net/IncIdeas/BitterLesson.html
Slide 10:
Amodei, D. Hernandez, D. (2019). AI and compute. [blog]. Retrieved from https://openai.com/blog/ai-and-compute
Schwartz, R. Dodge, J. Smith, N. Etzioni, O. (2019). Green AI. [pdf]. Retrieved from https://arxiv.org/abs/1907.10597
Slide 13:
Bar-Yam, Y. (2017). Fig. 2. [figure]. Retrieved from NECSI. (2017). Why teams?. [webpage]. Retrieved from http://necsi.edu/research/overview/why-teams
Reference
Slide 14:
Graphcore. (2018). Graphcore Rackscale IPU-POD. [image]. Retrieved from Introducing the Graphcore Rackscale IPU pod. [webpage]. Retrieved from
https://www.graphcore.ai/posts/introducing-the-graphcore-rackscale-ipu-pod
NVIDIA. (2019). NVIDIA DGX-2. [image]. Retrieved from NVIDIA DGX-2 APAC. [pdf]. Retrieved from https://www.nvidia.com/content/apac/gtc/ja/pdf/2018/1050.pdf
Slide 15:
Agrawal, P. et al. (2019). Data platform for machine learning. [pdf]. Retrieved from International Conference on Management of Data (SIGMOD ’19). Retrieved from
https://dl.acm.org/citation.cfm?doid=3299869.3314050
Slide 20:
The Times. (2019). Is the new dyson institute the answer to the university funding crisis. {webpage]. Retrieved from https://www.thetimes.co.uk/article/is-the-new-
dyson-institute-the-answer-to-the-university-funding-crisis-55qcdblhs
Slide 26:
Adapted from:
UC Berkeley. (2018). UCB I&E Ecosystem: Tech Stage vs Biz Stage. [image]. Retrieved from Lab to Market. [pdf]. Retrieved from
https://bpep.berkeley.edu/sites/default/files/shared/docs/BPEP_Idea_TO_MARKET.pdf
Slide 27:
U.S. Department of Energy. (2019). 2017 Early Stage Venture Deals. [image]. Retrieved from A New Mechanism to Fund Hardtech Innovation. [pdf[. Retrieved from
https://static1.squarespace.com/static/543fdfece4b0faf7175a91ec/t/5d1169950985c600011d354e/1561422231115/Hardtech+Investing+White+Paper.pdf
Reference
Slide 28
U.S. Department of Energy. (2019). A New Mechanism to Fund Hardtech Innovation. [pdf[. Retrieved from
https://static1.squarespace.com/static/543fdfece4b0faf7175a91ec/t/5d1169950985c600011d354e/1561422231115/Hardtech+Investing+White+Paper.pdf
Slide 34:
Liu, Y. Zhao, T. Ju, W. Shi, S. (2017). Graphical abstract. [figure]. Retrieved from Materials discovery and design using machine learning. [webpage]. Retrieved from
https://www.sciencedirect.com/science/article/pii/S2352847817300515?via%3Dihub#undfig1