Import AI: Issue 2: Microsoft’s AI chips, George Hotz’s bandwidth bill, and Spy! VS Spy!
by Jack Clark
Welcome to Import AI, a newsletter about artificial intelligence. Subscribe here.
Deep learning’s dirty data secret: modern deep learning approaches require a ridiculous quantity of data. How much data? A hacker house in SF used 3.8 terabytes of bandwidth in July. No coincidence that one of its residents is George Hotz, founder of self-driving car startup Comma.ai. The house has now upgraded to an ‘unlimited’ bandwidth plan, making Hotz foot the bill for the bit-flurry.
Free data! Comma.ai released seven and a quarter hours of highway driving data so world+dog can make their own AVs, and François Chollet, creator of the Keras deep learning framework, has made three image classification models available for free. That means you can now classify an image in as little as three lines of code.
( model = VGG16(weights=’imagenet’)
preds = model.predict(imgs)
print(decode_predictions(preds)) ).
How many lines to drive a car badly?
Show me the research: Spotting the difference between an AI startup with genuine technology and one that has re-implemented commodity systems is tricky. It’s easier to judge if the company participates in the wider AI research community. Therefore points should be awarded to robotics company Brain Corporation, text analysis startup Maluuba, and image recognition outfit Curious AI, which have all published papers recently. Keep them coming! (Extra kudos to Curious AI for publishing code.)
Spy VS Spy! Seven autonomous computer systems battled each other at the Darpa Cybersecurity Challenge in Las Vegas last week. ‘Mayhem’, the winner, was built by CMU startup For All Secure. The system performed automated program analysis (via symbolic execution) to find and exploit weaknesses in running programs. It appears to use a smart scheduler to let it run multiple checks on a piece of software in parallel, then selectively pause certain jobs to throw resources at promising vulnerabilities. This lets it efficiently explore the vast underbelly of the program and selectively focus on weak points, like a sophisticated, thieving octopus. (Unfortunately, some of the papers are paywalled.) Podcast with more information here.
Care for a little AI with your global government, sir? Google DeepMind would eventually like to donate its technologies to the United Nations, according to CEO Demis Hassabis.
Deep learning chips: Typical x86 processors are a bad fit for (most) modern AI tasks. GPUs are a bit better, but still not optimal. So expect new hardware substrates for AI. Microsoft will launch a new cloud service within a few months that lets people accelerate their neural network workloads with FPGA co-processors, said Qi Lu at the Scaled Machine Learning Conference. Google has already indicated it will offer its TPUs to people to accelerate TensorFlow workloads. Startup Nervana plans to produce its own chip to accelerate its ‘Neon’ software further. There are also numerous stealth chip and hardware startups that are re-thinking systems around deep learning (think: tens of thousands to millions of processors for low-precision matrix multiplication, etc).
GoodAI, a European AI research group founded by game developer Marek Rosa, has fleshed out its research and development roadmap and begun work on analyzing the overall AI landscape. It’s also developed some software for prototyping AI systems “with highly dynamic neural network topologies” named Arnold. It’s very pretty! The company will release it as open source software in a few months months, Rosa tells me.
Open VS Closed AI development? “Developing a joint set of openness guidelines on the short and long term would be a worthwhile endeavor for the leading AI companies today,” says Victoria Krakovna, co-founder of the Future of Life Institute.
Imagine that, computer. Generative models are gathering interest (as mentioned last week) and have a huge number of potential applications. For more information check out these two slide decks from Russ Salakhutdinov at CMU and Shakir Mohamed from DeepMind, from their sessions at the Deep Learning Summer School this week.
You have questions, they have answers: Google’s Brain team will be taking questions on Reddit and OpenAI’s Ian Goodfellow on Quora. Both on August 11th.
Thanks for reading. If you have suggestions, comments or other thoughts you can reach me at jack@jack-clark.net or tweet at me@jackclarksf
Import AI 
Tens of millions of processors? Sure. The expertise to manage dynamic range and corral data through such a processor efficiently? Pretty rare.
Yes, I’m sure it’ll be reminiscent of when Intel did a big (largely unsuccessful) parallel programming push a few years ago?
It’s easy to put 1M+ cores(tm) on a die, but it’s hard to design a processor and a programming model that lets one utilize them efficiently across a wide variety of parallel workloads. That’s what CUDA represents.
But until Deep Learning, IMO CUDA was just a better mousetrap in search of a mouse. While it scored multiple stunning and surprising HPC victories in the past decade, its marketing was focused on a brutal head to head with Intel to win the LINPACK benchmark and thus win supercomputer contracts.
But as Tim Dettmers has described eloquently in a 2015 blog entry (http://timdettmers.com/2015/07/27/brain-vs-deep-learning-singularity/), LINPACK was and remains the wrong benchmark. NVIDIA’s GPUs were capable of much more, and now they have a decade of legacy CUDA code they can exploit to design future processors and maybe even ASICs. No one else has this advantage, and further, only Intel and AMD can fund processor development by selling to gamers and consumers. I call that advantage the Gaming/Industrial Complex.
As such, even if NVIDIA were to develop a CUDA ASIC (and arguably GP100 with its exclusive FP16 and FP64 support is a small step in that direction), these advantages will stay with them. And their competition in the consumer space keeps dropping the ball lately.
If I were an NVIDIA competitor, I’d stop trying to lure people away from CUDA and its decade of useful code, and I’d start trying to run CUDA better than NVIDIA GPUs. That’s a tall order, but until Moore’s Law runs out, I don’t see any other way to kick sand in the face of said Gaming/Industrial Complex.
[…] on specially designed chips. The company first announced the Catapult system years ago and said in August of this year that it would make it available as a service other developers could rent. Meanwhile, a new company […]
[…] more free data: Self-driving startup Comma.ai released 80GB of driving data a couple of months ago. ‘Pah! That’s nothing,’ I imagine Udacity’s Oliver […]