NEWS

(/) (/)

Big Data (http://venturebeat.com/category/big-data/)

Why a deep-learning genius left Google & joined Chinese tech shop
Baidu(interview)

Image Credit: Jordan Novet/VentureBeat

July 30, 2014 8:03 AM

Jordan Novet (http://venturebeat.com/author/jordan-novet/)

Gaming execs:Join 180 selectleaders (http://events.venturebeat.com/event/gamesbeatsummit2015/)

from King, Glu, Rovio, Unity, Facebook, and moreto plan your pathto global domination in
2015.GamesBeat Summit (http://events.venturebeat.com/event/gamesbeatsummit2015/) is invite-only - apply here (http://events.venturebeat.com/event/gamesbeatsummit2015/).Ticket prices increase on
April 3rd!

SUNNYVALE, California Chinese tech company Baidu has yet to make its popular search engine and other web
services available in English. But consider yourself warned: Baidu could someday wind up becoming a favorite among
consumers.
The strength of Baidu lies not in youth-friendly marketing or an enterprise-focused sales team. It lives instead in Baidus
data centers, where servers run complex algorithms on huge volumes of data and gradually make its applications
smarter, including not just Web search but also Baidus tools for music, news, pictures, video, and speech recognition.
Despite lacking the visibility (in the U.S., at least) of Google and Microsoft, in recent years Baidu has done a lot of work
on deep learning, one of the most promising areas of artificial intelligence (AI) research in recent years. This work
involves training systems called artificial neural networks on lots of information derived from audio, images, and other
inputs, and then presenting the systems with new information and receiving inferences about it in response.
Two months ago, Baidu hired Andrew Ng away from Google (http://venturebeat.com/2014/05/16/baidu-snatchesgoogles-deep-learning-visionary-andrew-ng-as-its-chief-scientist/), where he started and led the so-called Google Brain
project. Ng, whose move to Baidu follows Hugo Barras jump from Google to Chinese company Xiaomi

SIGN UP (/SIGN-UP/)

| LOGIN (/LOGIN/)

(http://venturebeat.com/2013/08/28/hugo-barra-leaves/) last year, is one of the worlds handful of deep-learning rock

stars.
Ng has taught classes on machine learning, robotics, and other topics at Stanford University. He also co-founded
massively open online course startup Coursera.
He makes a strong argument for why a personlike him would leave Google and join a company with a lower public
profile. His argument can leave you feeling like you really ought to keep an eye on Baidu in the next few years.
I thought the best place to advance the AI mission is at Baidu, Ng said in an interview with VentureBeat.
Baidus search engine only runs in a few countries, including China, Brazil, Egypt, and Thailand. The Brazil service was
announced just last week (http://venturebeat.com/2014/07/18/baidu-brings-its-search-engine-to-brazil/). Googles
search engine is far more popular than Baidus around the globe, although Baidu has already beaten out Yahoo and
Microsofts Bing in global popularity, according to comScore figures (http://searchengineland.com/google-worlds-mostpopular-search-engine-148089).
And Baidu co-founder and chief executive Robin Li, a frequent speaker on Stanfords campus, has said he wants Baidu
to become a brand name in more than half of all the worlds countries. Presumably, then, Baidu will one day become
something Americans can use.

(http://venturebeat.com/2014/07/30/andrew-ngbaidu/robin-li-baidu/)
Above: Baidu co-founder and chief executive Robin Li.

Image Credit: Baidu

Now that Ng leads Baidus research arm as the companys chief scientist out of the companys U.S. R&D Center here, its
not hard to imagine that Baidus tools in English, if and when they become available, will be quite brainy perhaps
even eclipsing similar services from Apple and other tech giants. (Just think of how many people are less than happy
with Siri.)

A stable full ofAI talent

But this isnt a story about the difference a single person will make. Baidu has a history in deep learning.
A couple years ago, Baidu hired Kai Yu, a engineer skilled in artificial intelligence. Based in Beijing, he has keptbusy.
I think Kai ships deep learning to an incredible number of products across Baidu, Ng said. Yu also developed a system
for providing infrastructure that enables deep learning for different kinds of applications.
That way, Kai personally didnt have to work on every single application, Ng said.
In a sense, then, Ng joined a company that had already built momentum in deep learning. He wasnt starting from
scratch.

(http://venturebeat.com/2014/07/30/andrew-ng-baidu/kai-yu/)
Above: Baidus Kai Yu.

Image Credit: Kai Yu (http://www.dbs.ifi.lmu.de/~yu_k/)

Only a few companies could have appealed to Ng, given his desire to push artificial intelligence forward. Its capitalintensive, as it requires lots of data and computation. Baidu, he said, can provide those things.
Baidu is nimble, too. Unlike Silicon Valleys tech giants, which measure activity in terms of monthly active users
(http://venturebeat.com/2014/07/23/mobile-saves-the-day-facebook-surpasses-revenue-expectations/), Chinese
Internet companies prefer to track usage by the day, Ng said.
Its a symptom of cadence, he said. What are you doing today? And product cycles in China are short; iteration
happens very fast, Ng said.
Plus, Baidu is willing to get infrastructure ready to use on the spot.
Frankly, Kai just made decisions, and it just happened without a lot of committee meetings, Ng said. The ability of
individuals in the company to make decisions like that and move infrastructure quickly is something I really appreciate
about this company.
That might sound like a kind deference to Ngs new employer, but he was alluding to a clear advantage Baidu has over
Google.
He ordered 1,000 GPUs [graphics processing units] and got them within 24 hours, Adam Gibson, co-founder of deeplearning startup Skymind (http://venturebeat.com/2014/06/02/skymind-launches-with-open-source-plug-and-playdeep-learning-features-for-your-app/), told VentureBeat. At Google, it would have taken him weeks or months to get
that.
Not that Baidu is buying this type of hardware for the first time. Baidu was the first company to build a GPU cluster for
deep learning, Ng said a few other companies, like Netflix (http://venturebeat.com/2014/02/10/netflix-moves-intodeep-learning-research-to-improve-personalization/), have found GPUs useful for deep learning and Baidu also
maintains a fleet of servers packing ARM-based chips (http://venturebeat.com/2014/01/29/arm-releases-standards-tounite-server-vendors-in-fight-with-intel/).

(http://venturebeat.com/2014/07/30/andrew-ng-baidu/baidu-headquarters/)
Above: Baidu headquarters in Beijing.

Image Credit: Baidu

Now the Silicon Valley researchers are using the GPU cluster and also looking to add to it and thereby create still bigger
artificial neural networks.
But the efforts have long since begun to weigh on Baidus books and impact products. We deepened our investment in
advanced technologies like deep learning, which is already yielding near term enhancements in user experience and
customer ROI and is expected to drive transformational change over the longer term, Li said in a statement
(http://ir.baidu.com/phoenix.zhtml?c=188488&p=irol-newsArticle&ID=1951199&highlight=) on the companys earnings
the second quarter of 2014.

Next step: Improving accuracy

What will Ng do at Baidu? The answer will not be limited to any one of the companys services. Baidus neural networks
can work behind the scenes for a wide variety of applications, including those that handle text, spoken words, images,
and videos. Surely core functions of Baidu like Web search and advertising will benefit, too.
All of these are domains Baidu is looking at using deep learning, actually, Ng said.
Ngs focus now might best be summed up by one word: accuracy.
That makes sense from a corporate perspective. Google has the brain trust on image analysis, and Microsoft has the
brain trust on speech, said Naveen Rao, co-founder and chief executive of deep-learning startup Nervana
(http://venturebeat.com/2014/04/17/nervana-takes-600k-to-build-hardware-for-deep-learning/). Accuracy could
potentially be the area where Ng and his colleagues will make the most substantive progress at Baidu, Rao said.
Matthew Zeiler, founder and chief executive of another deep learning startup, Clarifai, was more certain. I think youre
going to see a huge boost in accuracy, said Zeiler, who has worked with Hinton and LeCun and spent two summers on
the Google Brain project.
One thing is for sure: Accuracy is on Ngs mind.

(http://venturebeat.com/2014/07/30/andrew-ng-baidu/baidu-sunnyvale-office-novet/)
Above: The lobby at Baidus office in Sunnyvale, Calif.

Image Credit: Jordan Novet/VentureBeat

Heres the thing. Sometimes changes in accuracy of a system will cause changes in the way you interact with the
device, Ng said. For instance, more accurate speech recognition could translate into people relying on it much more
frequently. Think Her (https://www.youtube.com/watch?v=WzV6mXIOVl4)-level reliance, where you just talk to your
computer as a matter of course rather than using speech recognition in special cases.
Speech recognition today doesnt really work in noisy environments, Ng said. But that could change if Baidus neural
networks become more accurate under Ng.
Ng picked up his smartphone, opened the Baidu Translate app, and told it that he needed a taxi. A female voice said
that in Mandarin and displayed Chinese characters on screen. But it wasnt a difficult test, in some ways:This was no
crowded street in Beijing. This was a quiet conference room in a quiet office.
Theres still work to do, Ng said.

The future heroes of deep learning

Meanwhile, researchers at companies and universities have been hard at work on deep learning for decades.
Google has built up a hefty reputation for applying deep learning to images from YouTube videos
(http://static.googleusercontent.com/media/research.google.com/en/us/archive/unsupervised_icml2012.pdf), data
center energy use (http://venturebeat.com/2014/05/28/google-data-centers-get-smarter-all-on-their-own-no-humansrequired/), and other areas, partly thanks to Ngs contributions. And recently Microsoft made headlines for deeplearning advancements with its Project Adam (http://research.microsoft.com/en-us/news/features/dnnvision071414.aspx) work, although Li Deng of Microsoft Research has been working with neural networks for more than 20
years.
In academia, deep learning research groups all over North America and Europe. Key figures in the past few years
include Yoshua Bengio at the University of Montreal, Geoff Hinton of the University of Toronto (Google grabbed him
last year through its DNNresearch acquisition (http://venturebeat.com/2013/03/12/google-dnnresearch/)), Yann LeCun
from New York University (Facebook pulled him aboard (http://venturebeat.com/2013/12/09/zuckerberg-to-talk-aboutfacebooks-artificial-intelligence-plans/) late last year), and Ng.
But Ngs strong points differ from those of his contemporaries. Whereas Bengio made strides in training neural
networks, LeCun developed convolutional neural networks (http://yann.lecun.com/exdb/lenet/), and Hinton
popularized restricted Boltzmann machines (http://www.scholarpedia.org/article/Boltzmann_machine), Ng takes the
best, implements it, and makes improvements.

Andrew is neutral in that hes just going to use what works, Gibson said. Hes very practical, and hes neutral about
the stamp on it.
Not that Ng intends to go it alone. To create larger and more accurate neural networks, Ng needs to look around and
find like-minded engineers.
Hes going to be able to bring a lot of talent over, Dave Sullivan, co-founder and chief executive of deep-learning
startup Ersatz Labs (http://venturebeat.com/2014/06/11/ersatz-labs-launches-with-hardware-and-software-for-all-yourdeep-learning-needs/), told VentureBeat. This guy is not sitting down and writing mountains of code every day.
And truth be told, Ng has had no trouble building his team.
Hiring for Baidu has been easier than Id expected, he said.
A lot of engineers have always wanted to work on AI. My job is providing the team with the best possible
environment for them to do AI, for them to be the future heroes of deep learning.

More information:
Stanford University

Google

Baidu

Andrew Ng

Stanford University is a research university that offers bachelors, masters, doctoral and professional degrees. Stanford
includes seven schools: Business, Earth Sciences, Education, Engineering, Humanities and Sciences, Law and M... read
more (http://www.vbprofiles.com/companies/3e122f809e597c1000000466)
Powered by VBProfiles (http://www.vbprofiles.com)

VentureBeats VB Insight team isstudying email marketing tools. Chime in here, and well share the
results (http://insight.venturebeat.com/email-marketing).

Promoted Stories

Recommendedby

(http://ad.doubleclick.net/ddm/clk/289569229;116605329;u)

The CAPE Ratio Can Be Useful

But Do Not Take It At Face (The
Value Perspective - The Value
Perspective - Schroders)

Scaling New Customer

Acquisition - Should You Use
Freemium (Zuora)

(https://www.zuora.com/academy/guides/is-freemium-right-business-model/?
utm_source=outbrain&utm_medium=CPC&utm_campaign=academy_launch&utm_content=7C)
(https://business.yell.com/knowledge/6-seo-myths-you-need-to-ignore/?

Six SEO myths you need to

ignore in 2015 (Yell Business)
utm_source=zemanta&utm_medium=native&utm_campaign=zemanta&_z1_adgid=446&_z1_msid=outbrain)
(http://www.etoro.com/partners/aw.aspx?

The last Apple product

envisioned by the late Steve

Jobs (eToro Blog)

A=46599&Task=Click&SubAffiliateID=AppleWatch1ENTxt1Img2&targetURL=http%3A%2F%2Fwww.etoro.com%2Fblog%2F64582%2F19032015%2F
watch%2F) (http://wanderingpioneer.com/23-stunning-rock-formationsyou-need-to-see/?
utm_source=Outbrain&utm_medium=CPC&utm_campaign=desktop_UK)
(http://www.viralnova.com/earthoddities-gallery/?mb=lmgout)
23 Mysterious Rock Formations
That'll Loeave You Dumbstruck
(Wandering Pioneer)

Just When The World Couldn't

Get Any Crazier #5 Is Unreal
(Viral Nova)

More Promoted Stories

14 Extinct Animals We Could Clone Right Now

5 Trends you need to know to optimize your

(LOLWOT) (http://www.lolwot.com/14-extinct-animals-

social media strategy this year (Social Ads Tool)

we-could-clone-right-now/?

(http://www.77tracking.com/Redirect.ashx?

utm_source=ob.da1&utm_medium=referral)

website=49a42682&77tadunit=1a11343b&77tentry=outbrain_post&redirect=http://www.socialadstool

10 of the Most Mysterious Places on Earth (Travel

Blog) (http://realfactgist.com/10-of-the-most-

mysterious-places-on-earth/)

Bitcoin might just be on the verge of a quiet

media-marketing-trends-for-2015/?
utm_source=outbrain&utm_medium=referral&utm_campaign=outbrain_satarticles)

Make Your Revenues Bigger By Emotional

Targeting (PageWiz)

comeback. [Bitcoin review] (eToro Blog)

(http://www.pagewiz.com/blog/landing-pages/using-

(http://www.etoro.com/partners/aw.aspx?

emotional-targeting?outbrain)

news%2F)

of The Ocean (UdderlyPettable)

A=46599&Task=Click&SubAffiliateID=Bitcoin3ENTxt2Img2&targetURL=http%3A%2F%2Fwww.etoro.com%2Fblog%2F64582%2F03112014%2Fbitcoin 11 Of The Scariest Things Found At The Bottom

(http://udderlypettable.com/11-horrific-deep-seacreatures-that-look-unearthly/?
utm_source=Outbrain&utm_medium=CPC&utm_campaign=desktop_UK)

Topics >

Andrew Ng (http://venturebeat.com/tag/andrew-ng/)
artificial intelligence (http://venturebeat.com/tag/artificial-intelligence/)
Baidu (http://venturebeat.com/tag/baidu/)

deep learning (http://venturebeat.com/tag/deep-learning/)

deep neural networks (http://venturebeat.com/tag/deep-neural-networks/)
editor's pick (http://venturebeat.com/tag/editors-pick/)
Google (http://venturebeat.com/tag/google/)

GPUs (http://venturebeat.com/tag/gpus/)

neural networks (http://venturebeat.com/tag/neural-networks/)

top-stories (http://venturebeat.com/tag/top-stories/)

BIGdata:Stay
tuned
Successstories,tips,
backgroundinfo,visualization

0 comments
Sign in

(http://livefyre.com)
4 people listening

+ Follow

Share

Post comment as...

Newest | Oldest | Top Comments

Home (/)

About (/about/)

Contact (/contact/)

Events (http://events.venturebeat.com)

Careers (/careers/)

Advertise (/advertise/)

Privacy Policy (/privacy-policy/)

Terms of Service (/terms-of-service/)

Copyright 2015 VentureBeat (http://venturebeat.com). All rights reserved.