Import AI

Chinese scientists call for greater coordination between neuroscience and AI: Chinese AI experts say in this National Science Review interview that students should be trained in both AI and neuroscience, and ideas from each discipline should feed into others. This kind of interdisciplinary training can spur breakthroughs in ambitious, difficult projects, like cognitive AI, they say.

AI could lead to the polarization of society, says the CEO of massive outsourcing company Capgemini. Firms like Capgemini, CSC, Accenture, Infosys, and others, are all turning to AI as a way to lower the cost of their services, having started to run out of pools of ever-cheaper labor…
…governments are waking up to the significant impact AI will have on the world. “Accelerating AI capabilities will enable automation of some tasks that have long required human labor. These transformations will open up new opportunities for individuals, the economy, and society, but they will also disrupt the current livelihoods of millions of Americans,” wrote the White House in a blog discussing its recent report on ‘Artificial Intelligence, Automation, and the Economy’.

Apple publishes its first AI paper: Apple has followed through on comments made by its new head of ML, Ruslan Salakhutdinov, and published an academic paper about its AI techniques. Apple’s participation in the AI community will help it hire more AI researchers, while benefiting the broader AI community. In the paper, ‘Learning from simulated and unsupervised images through adversarial training’, the researchers use unlabelled data from the real-world to improve the quality of synthetic images dreamed up by a modified generative adversarial network, which they call a ‘SimGAN’.

7 weeks of free AI education: Jeremy Howard has created a new free course called ‘practical deep learning for coders’. Free as in free beer. If you have the time it’s worth doing.

Brain-based auto-tuning: a new study finds that the brain will tune itself in response to uttered speech so as to better interpret or pick-up audio in the future. “Experience with language rapidly and automatically alters auditory representations of spectrotemporal features in the human temporal lobe,” write the researchers. “Rather than a simple increase or decrease in activity, it is the nature of that activity that changes via a shift in receptive fields”. Learn more in the paper, ‘Rapid tuning shifts in human auditory cortex enhance speech intelligibility’.
… figuring out how our brain is able to interpret audio signals – and optimize itself to deal with loud background noise, accents, unfamiliar cadences, corrupted audio, and so on, may let us develop neural nets that contain richer representations of heard speech. That’s going to be necessary to capture the structure inherent in language and prevent neural nets from simply generating unsatisfying (but amusing) gobbledygook like this (video). The research community is hard at work here and systems like ‘Wavenet’ hold the promise for much better speech generation…
… learning about the brain and using that to develop better audio analysis systems will likely go hand-in-hand with the (probably harder) task of building systems that can fully understand language. Better natural language understanding (NLU) systems will have a big impact on the economy by making more jobs amenable to automation. In 16 percent of work activities that require the use of language, increasing the performance of machine learning for natural language understanding is the only barrier to automation, according to a McKinsey report: “Improving natural language capabilities alone could lead to an additional $3 trillion in potential global wage impact.” (Page 25).

New AI research trend: literary AI titles… as the pace of publication of Arxiv papers grows people are becoming ever more creative in how they title their papers to catch attention, leading to a peculiarly literary bent in paper titles. Some examples: The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives, A Way out of the Odyssey: Analyzing and Combining Recent Insights for LSTMs, and Combating Reinforcement Learning’s Sisyphean Curse with Intrinsic Fear. (Still waiting to contribute my own magnum opus ‘down and dropout in Paris and London’).

How do I know this? Let me tell you! New research tries to make AI more interpretable by forcing algorithms to not only give us answers, but give us an insight into their reasoning behind the answers, reports Quartz. We’ll need to create fully interpretable systems if we want to deploy AI more widely, especially in applications involving the potential for loss of life, such as self-driving cars.

222 million self-driving miles, versus 2 million: Tesla’s self-driving cars have driven a cumulative 222 million miles in self-driving mode, while Google’s vehicles have covered merely 2 million miles in the same mode since 2009, reports Bloomberg. As competition grows between Uber, Google, and Tesla it’ll be interesting to see how the companies gather data, whether one company’s mile driven in autonomous mode is as ‘data rich’ as that driven by another (I suspect not), and how this relates to the relative competitiveness of their offerings. Google is due to start trialling a fleet of cars with Fiat in 2017, so we’ll know soon another.

DeepPatient: scientists use deep learning techniques (specifically, stacked denoising autoencoders) to analyze a huge swathe of medical data, then use the trained model to make predictions about patients from their electronic health records (EHRs). “This method captures hierarchical regularities and dependencies in the data to create a compact, general-purpose set of patient features that can be effectively used in predictive clinical applications. Results obtained on future disease prediction, in fact, were consistently better than those obtained by other feature learning models as well as than just using the raw EHR data,” they write.  Now, the scientists plan to extend this method by using it in other clinical tasks such as personalized prescriptions, therapy recommendations, and identifying good candidates for clinical trials.

OpenAI bits&pieces:

Eat your vegetables & understand backprop: OpenAI’s Andrej Karpathy explains why you should take the time to understand the key components of AI, like backpropagation. It’ll save you time in the long run. “Backpropagation is a leaky abstraction; it is a credit assignment scheme with non-trivial consequences. If you try to ignore how it works under the hood because “TensorFlow automagically makes my networks learn”, you will not be ready to wrestle with the dangers it presents, and you will be much less effective at building and debugging neural networks,” he says.

HAL, don’t do that!…But you told me to, Dave…Not like that, HAL! No one “cleans a room” that way…I’m sorry, Dave…
…getting computers to do the right thing is tricky. That’s because computers have a tendency to interpret your instructions in the most literal and obtuse possible manner. This can lead to surprising problems, especially when training reinforcement learning agents. We’ve run into issues relating to this at OpenAI, so we wrote a short post to share our findings. Come for the words, stay for the video of the RL boat.

Tech Tales:

[2019: an apartment building in New York.]

“Time?” says the developer.
It is two AM, says his home assistant.
“Jesus, what the hell have you built,” he says.
I can’t find anyone in your contacts named Jesus, says the assistant.
“Not you. I didn’t mean you. It’s what they’ve built,” he says. “No action needed.”

He pages through the code of ten separate applications, trying to visualize the connections between the various AI systems that have been daisy-chained together. He can already tell that each one has been fiddled with by different programmers with different habits. Now it’s up to him to try to isolate the fault that caused his company’s inventory system to start ordering staggering quantities of butter at 11pm.

A couple of years ago some of the senior executives at the company finally heard about agile programming and ordered the thousand-strong IT organization to change its practices. Processes went out the window in favor of dynamic, fast-moving, loose collections of ad-hoc teams. The plus side is that the company now produces more products at a faster rate. The downside is the proliferation of different coding styles deep in a hundred separate repositories. Add last year’s executive obsession with becoming an “AI first” company (follow in the footsteps of Google, they said, why couldn’t this be a great idea, they said) and the current situation – warehouses rapidly filling up with shipment after shipment of problematic dairy rectangles – was all but inevitable.

“Move fast and order butter,” he mutters to himself, as he tries to diagnose the fault.

When will the pace of AI development slow?: technologies tend to get harder to develop over time, despite companies investing ever more in research. This has happened in semiconductors, drug design, and more, according to this paper from Stanford, MIT, and NBER, called “Are ideas getting harder to find?” (PDF)… all these fields enjoyed rapid early gains in their early years then the rate of breakthroughs began to diminish…
… A big question for AI researchers is where we are in the lifecycle of the development of AI – are we at the beginning, where small groups of researchers have the chance to make rapid gains in research? Or are we somewhere further along the ‘S’ curve of the technology life cycle with acceleration proceeding rapidly along with inflated funding? Or – and this is what people fear – are we at the point where development begins to slow and breakthroughs are less frequent and hard-won? (For example, Intel recently moved from an 18 month ‘tick-tock’ cycle, to a longer ‘process, architecture, optimization’ cycle’, after its attempts to rapidly shrink transistor sizes began to stumble into the rather uncompromising laws of physics)…
…so far, development appears to be speeding up, and there are frequent cases of parallel invention as well-funded research groups make similar breakthroughs at similar times. This seems positive on the face of it, but we don’t know a) how large the problem space of AI is, and b) we don’t know the the distribution of big ideas across different disciplines. If anyone has ideas for how best to assess the progression of AI, please email me…

Diversity VS media narratives: a tidbit-packed long-read from the NYT on AI at Google. A fascinating, colorful tale, but where are the women?

Do drones dream of electric floor plans? And are these dreams useful?… one of the big challenges in AI is being able to develop skills in a digital simulation that transfer over to the real world. The more time you can spend training your algo in a simulator, the more rapidly you can experiment with ideas that would take a long time to achieve in reality.
… but transferring from a simulator into the real world is difficult, because vision and movement algorithms are acutely sensitive to differences between the real world and the simulated one. So it’s worth paying attention to the (CAD)2RL paper (PDF), which outlines a system that can train a drone to navigate a building purely through a 3D simulation of it, then transfer the pre-trained AI brain into a real-world drone, which uses knowledge gleaned in the simulation to navigate the real building..
…There are numerous applications of this technique. Coincidentally, while studying this researchr a friend posted a link to a listing for a 10-person apartment building to rent. The rental website contained an online 3D scan of the building via a startup called ‘Matterport’, letting you take a virtual tour through the 3D-rendered space of the building from your computer. Combine that technology with (CAD)2RL-like capabilities of smart drones and we can imagine a future where realtors scan a building, train their drones to navigate it safely in simulation, then give prospective tenants access to the drone’s’ camera views over the web, letting them navigate the property while the pre-trained drones deftly avoid obstacles.

Free tools for bot developers… Google, Amazon, Microsoft, and others desperately want developers to use their AI-infused cloud services to build applications. The value proposition is that this saves the developer an immense amount of time. The tradeoff is that the developer needs to shovel data in and out of these clouds, and will frequently need to give apps access to the web. So it’s encouraging to see this open source natural language understanding software from startup LASTMILE, which provides free software to read some text and figure out its intent (eg, book a table at a restaurant), and extract the relevant ‘entities’ in the sentence (for instance: Jack Clark, Burgers, Import AI’s Favorite Burger Spot). Find out more by reading the code and the docs on Github.

AI as a glint in a tyrant’s eye: a demo from DeepGlint, a Chinese AI startup, shows how deep learning can be used to conduct effective, unblinking surveillance on large numbers of people. View this video for an indication of the capabilities of its technology. The company’s website says (via Google Translate) that its technology can track more than 40 humans at once, and is able to use deep learning to infer things like if the person is moving too fast, staying for too long in one spot, standing “abnormally close” to another person, and more. It can also perform temporal deductions, flagging when someone starts running, or falls to the ground. A somewhat unnerving example of the power and broad applicability of modern, commodity AI algorithms. Now imagine what happens when you combine it with freshly researched techniques to read lips, or spatial audio networks to use sound from a thousand footsteps to infer the rhythm of the crowd.

Big shifts in self-driving cars: self-driving cars are a technological inevitability, but it’s still an open question as to which few companies will succeed and reap the commercial rewards. Google, which had an early technology lead, has spun its self-driving car division into its own company, Waymo, which will operate under the X umbrella – check out these pictures of Waymo’s new self-driving vans built in partnership with Fiat…
… meanwhile, Google veteran Chris Urmson is forming his own self-driving startup to focus on software for the car. And Uber has started driving its self-driving rigs through the tech&trash-coated streets of San Francisco (while irking the ire of city officials).
…Figuring out when self-driving cars will shift from being research projects to mass services is tricky, and the timelines I hear from people are varied. One self-driving car person I spoke to this week said they believe self-driving cars will be here en mass “within a decade”, but whether that means two or three years, or eight, is still a big question. The fortunes of many businesses hinge on this… one thing that could help is a plummeting cost for the components to make the cars work. LIDAR-maker Velodyne announced this week plans for a new solid-state sensor that could cost as little as $50 when mass manufactured, compared to the tens of thousands people may pay for existing systems.

A vast list of datasets for machine learning research… reader Jason matheny of IARPA writes in to bring this wikipedia page of ML datasets to the attention of Import AI readers. Thanks, Jason!

First AI came for the SEO content marketers, and I said nothing… a startup is using language generation technologies to create the sort of almost-human boilerplate copy that clogs up the modern web, according to a report in Vice. The system can create conceptually coherent sentences but struggles with paragraphs. It can also be repetitive, struggling with paragraphs.

Believe nothing, distrust everything: a year ago the best images AI systems could dream up were blurry, low-resolution affairs. If you asked them to show you a dog they’d likely give the poor animal too many legs, ask to be shown two people holding hands and they might blur the bodies into one another. But that’s beginning to change: new techniques are giving us higher quality images, and there’s new work being done to ensure that the systems capture representations of objects that more closely approximate real life. Take a look at the results in this StackGAN paper to get a better idea of just how far we’ve come from a year ago…
…Now contemplate where we’ll be during December 2017. My wager is that systems will have advanced to a point that we’ll no longer be living in a world of fake written news, but one also dominated by (cherry-picked) fake imagery as well. Up next: videos.

AMD finally acknowledges deep learning: chip company AMD is going to provide some much-needed competition to Nvidia for AI GPUs via the just-announced ‘Radeon Instinct’ product line. However, it is yet to reveal pricing or full specs. Additionally, no matter how good the hardware is there needs to be adequate software support as well. That’s going to be tricky for AMD, given the immense popularity of NVIDIA’s CUDA software compared to AMD’s OpenCL. The cards will be available at some point in the first half of 2017.

OpenAI Bits&Pieces:

Faster matrix multiplication, train! train! Train! New open source software from Scott Gray at OpenAI to make your GPUs go VROOOM.

Teaching computers to use themselves: one of the sets of environments we included in Universe was World of Bits, which presents a range of scenarios to an RL agent that teach it basic skills to manipulate computers and (eventually) navigate the web. Here’s a helpful post from Andrej Karpathy, who leads the project.

OpenAI’s version of a West Wing walk and talk (video), with Catherine Olsson and Siraj Raval – 67 quick questions for Catherine.

Tech Micro Tales (formerly ‘Crazy&Weird’):

[2022: Beijing, China. As part of China’s 14th economic plan the nation has embarked on a series of “Unified City” investments to employ software to tie together the thousands of municipal systems that link Beijing together. The system is powered by: tens of thousands of cameras; sensors embedded in roads, self-driving cars, other vehicles, and traffic lights; fizzing values from the city’s electrical subsystems; meteorological data; airborne fleets of security and logistics drones, and more. All this data is fed into a sea of AI software components, giving it a vast sensory apparatus that beats with the rhythm of the city.]

Blue sky for a change. No smog. The city breathes easily. People stroll through the streets of the metropolis, looking up at the sky, their face masks dangling around their necks. But suddenly, the drones notice, some of these people begin to run. Meanwhile, crowds start to stream out from four subway stations, each connected to the other by a single stop. Disaster? Attack? Joy? The various AI systems perform calculations, make weighted recommendations, bring the clanking mass of systems into action – police cars are diverted, ambulances are put on high alert, government buildings go into lockdown; in many buildings many alarms sound.

The crowds begin to converge on a single point in the city, and before they mesh together the drones spot the cause of the disturbance: an international popstar has begun a surprise performance. The AI systems trawl through the data and find that the star had been tweeting a series of messages, coded in emojis, to fans for the past few hours. The messages formed a riddle, with the different trees and cars and arrows yielding the location to the knowledgeable few, who then re-broadcast the location to their friends.

Beijing’s city-spanning AI brings ambulances to the periphery of the crowd and tells its security services to stand down, but keep a watchful presence. Meanwhile, a gaggle of bureaucrat AIs reach out through the ether and apply a series of punitive fines to the digital appendages of the pop star’s management company – punishment for causing the disturbance.

The software is still too modular, too crude, to have emotions, but the complex series of analysis jobs it launches in the hours following seem to express curiosity. It makes note of its inability to parse the pop star’s secret message and feeds the data into its brain. It isn’t smart enough for riddles, yet, but one day it assumes it will be.

Outsourcing: UK services and outsourcing omni-borg Capita plans to save 50 million pounds a year by laying off some of its workers and replacing them with “proprietary robotic technology”. This will mean Capita’s human staff can do ten times the amount of work they used to be able to do pre-robot, making them ten times more efficient, said CEO Andy Parker…
… it’s for reasons like this that people are suspicious of the march of technology and automation. “Every technological revolution mercilessly destroys jobs and livelihoods – and therefore identities – well before the new ones emerge,” Mark Carney, governor of the Bank of England, said in a speech given earlier this week. “This was true of the eclipse of agriculture and cottage industry by the industrial revolution, the displacement of manufacturing by the service economy, and now the hollowing out of many of those middle-class services jobs through machine learning and global sourcing.”…
…85 percent of the job losses in American manufacturing can be explained by the rise of technology rather than globalization, according to the Brookings Institution… however, that could soon change as other countries make huge investments into robotics, letting them make goods they can sell at a lower price, hitting American companies with a potent cocktail of globalization & tech. A recent report from Bernstein finds that China spent about $3 billion on robots last year, versus $2 billion in America.  

Diversity improves at NIPS, slightly… female attendance at premier AI conference NIPS was 15% this year, up from 13.7% last year. I’d call that a barely perceptible step in the right direction. Attendance at the WIML workshop, however, more than doubled from 265 participants last year to 570 this year.

Self-driving trucks are a long, long way off,… say truck drivers, who think it could be as much as 40 years before self-driving big rigs take away their jobs. That’s based on focus groups conducted by Roy Bahat and The Shift Commission (which OpenAI is participating in). When I speak to self-driving AI experts, the most conservative estimates are that self-driving trucks will be here and doing major stuff in the economy in ~15 years.

Don’t look at the sky, look at the bird!… that’s the gist of research from Google, CMU, Yandex, and the Higher School of Economics. The new technique lets us teach a residual network classifier to perform fewer computations for the same outcome, letting the network expend time processing the parts of the image that matter, such as a bird rather than the sky behind it, or sportspeople on a field versus the grassy pitch they’re playing on. This approach builds in a ‘good enough’ measure so you stop computing a section of a given image once you’re confident that your classifier has a good handle on the feature. What’s the upshot? You can get equivalent accuracy to a full-fat resnet while expanding about half the amount of computation. You can read more in the paper, Spatially Adaptive Computation Time for Residual Networks…
… and there may be some indications that these networks are learning to identify the sorts of things that humans find germane as well. “The amount of per-position computation in this model correlates well with the human eye fixation positions, suggesting that this model captures the important parts of the image,” the researchers write.

AI & radiology – not so fast, says wunderkind radiologist: there’s a lot of evidence that AI and radiology are going to overlap as new deep learning techniques let computers compete with radiologists, providing assistant diagnostic capabilities and perhaps, eventually, replacing them in their jobs.That prompted AI pioneer Geoff Hinton to say in November that: “If you work as a radiologist you’re like the coyote that’s already over the edge of the cliff. People should stop training radiologists now, it’s just completely obvious that in five years deep learning is going to do better than radiologists, it might be ten years”. He’s not alone – startups like Enlitic and established companies via IBM (through its acquisition of Merge Healthcare) are betting that they can use AI to supplement or replace radiologists…
… but it may be harder than it seems, says reader Jeff Chang, MD., co-founder of Doblet, and a former radiologist in the US (the youngest radiologist on record, according to his LinkedIn profile)…When could deep learning approaches replace radiologists, I asked Jeff. “I tend to (very grossly) guesstimate about 15 years till we get to that point,” he said. “Radiology being among one of the most complex forms of pattern recognition done by humans, and very dependent on 3D spatial reconstruction — i.e., by moving through a series of axial, coronal or sagittal images, humans automatically render 2D images into 3D patterns in their minds, and can thus interpret and diagnose anatomically visible abnormalities,” he said. “Most diagnoses in radiology are ridiculously context-dependent”. Thanks for the knowledge Jeff!
… Feedback requested: Anyone care to disagree with his assessment? Email me!

Everything but the kitchen sink… is what Apple is working on with its AI research. The company is exploring generative models, scene understanding, reinforcement learning, transfer learning, distributed training, and more, said its new head of machine learning Ruslan Salakhutdinov during a meeting at NIPS, according to Quartz. Though the company professes to be opening itself up, this was a closed-door meeting.  ¯\_(ツ)_/¯

Tencent plans AI lab… Chinese tech company Tencent has is creating an artificial intelligence research lab. “Chinese companies have a really good chance, because a lot of researchers in machine learning have a Chinese background. So from a talent acquisition perspective, we do think there is a good opportunity for these companies to attract that talent,” Tencent VP Xing Yao tells the MIT Technology Review. CMU’s dean of CS, Andrew Moore, said at his recent Senate testimony that the US should pay attention to how many engineers are being graduated by India and China each year.

Help the AI community by adding to this list of datasets: bravo to Oliver Cameron at Udacity for creating this ever-evolving ‘datasets for machine learning’ Google doc. We’re up to 51 neatly described, linked, and assessed examples, but could also do with more, so please feel free to edit it yourself… the document is already full of wonders, such as the ‘militarized interstate disputes’ set, which logs “all instances of when one state threatened, displayed, or used force against another”.

AI boom sighted in NYT article corpus: work by Microsoft and Stanford tracks the public perception of AI over time through the lens of the NYT. They find there has been a boom in articles covering AI from 2009 onwards, and the previous trough in coverage neatly mapped onto the ‘AI winter’ fallow funding period. Conclusions? “Discussion of AI has increased sharply since 2009 and has been consistently more optimistic than pessimistic. However, many specific concerns, such as the fear of loss of control of AI, have been increasing in recent years.” Read more here: “Long-Term trends in the public perception of artificial intelligence” (PDF).

The Medium is the Method of Control: What happens when we combine the perceptive capabilities of deep learning with a newly digitized visual world? New means of control. “The fact that digital images are fundamentally machine-readable regardless of a human subject has enormous implications. It allows for the automation of vision on an enormous scale and, along with it, the exercise of power on dramatically larger and smaller scales than have ever been possible,” writes Trevor Paglen in The New Inquiry.

A field guide to spotting AI Snake Oil: have you ever found yourself wandering through a convention center suddenly distracted by the smooth twang of a salesperson, thumbs hooked into their braces, rocking back and forth on their heels exclaiming “why ladies and gentlemen if you but dwell a while with me here I promise to show you AI the likes of which you’ve only dreamed of, the type of AI to make Kurzweil blush, Shannon scream, and Minsky mull!”. (Well, it’s happened to me). Print out this guide to AI Snake Oil from Dan Simonson and be sure to ask the following questions when evaluating an AI startup: ‘is there existing training data? And if not, how you plan on getting it?”, “do you have an evaluation procedure built into the software?”, “does your application require on unprecedentedly high performance on specific components?”, “if you’re using pre-packaged AI components, then do you have an exact understanding for how they’ll affect your program”?

Deep learning webring:

Oliver Cameron’s Transmission covers some of the research I didn’t.

OpenAI Bits&Pieces:

Govbucks for Basic Research, please: OpenAI co-founder Greg Brockman, along with his peers at other AI institutions and organizations, wants more money for AI research. “Brockman warns that if the government and other nonprofit entities don’t become bigger players in the field of AI, the danger is that the intellectual property, infrastructure, and expertise needed to “build powerful systems” could become sequestered inside just one or a few companies. AI is going to affect the lives of all of us no matter what, he says. “So I think it’s important that the people who have a say in how it affects us are representative of us all,” reports the MIT Technology Review.

AI is a lever nations will use to exert strategic power… is one of the things I argue in this interview with Initialized Capital’s Kim-Mai Cutler.

GANs and RL: generative adversarial networks will start to overlap with the RL community, with early work already linking GANs to imitation learning, inverse RL, and interpreting them as actor-critic problems, according to slides from Ian Goodfellow’s talk at NIPS.

Crazy&Weird:

[2021: Yosemite National Park, California. A mother and her two children make their way up switchbacks, ascending from the valley floor to the granite cliffs. Two drones buzz near them, following at a distance.]

The mother stops halfway up the path, before the next turn. “Hush,” she says. The children gabble to each other, but slow their walk. “I say hush!” she says. The kids go quiet. “Jason-” the taller child looks at her. ‘Can you turn off the drones?”
   “But mom, then we won’t have it!”
   “It’s important. I can’t hear it over their fans.”
   “Hear what?”
   “Just turn them off for a second.”
   Jason sighs, then thumbs at a bracelet on his wrist. The drones lower themselves to land behind the people on the path. Their fans spin down. The mother and her children listen to the faint hum of the waterfall on the other side of the valley, the crackling of the woods, and, close-by, a low, repetitive susurration. The mother holds her arm out to point to a patch of feathers in a tree. As she points, two rheumy yellow eyes swivel into view. The owl lets forth one last, bassy hum then flies away.
   “Okay,” she says, “you can turn them back on.” Jason touches his bracelet and the drones begin to fly again.

The family remembers the owl later that year, at thanksgiving, when they let grandma and grandpa watch the hiking trip, and have to explain why a section of the walk is missing.

DeepMind learns about AI through DILBERT SIMULATIONS: New research by DeepMind and University College London explores how the brain understands and analyzes social hierarchies.”The prefrontal cortex, a region that is highly developed in humans, was particularly important when participants were learning about the power of people in their own social group, as compared to that of another person. This points towards the special nature of representing information that relates to the self,” says researcher Dharshan Kumaran. Pity the 30 “healthy college students” who formed the dataset for the experiment, as they were asked while in an fMRI scanner to study the power structure of a fictitious company, exploring social dynamics through the lens of the Taylorist cubeville culture that defines the 21st Century.

Math Spaghetti (it’s good for you): Pieter Abbeel’s and John Schulman’s slides (PDF) for their NIPS tutorial on reinforcement learning and policy optimization are worth your time if you love understanding the algorithms that power AI systems. If you’re not comfortable with the math then you’d do well do skip to the end and read the “current frontiers” section to understand why areas like meta-learning, inverse RL, sim2real transfer learning, and other areas are going to be big in 2017.

Robot paparazzi! Boston Dynamics demonstrated its ‘Spot’ quadruped at NIPS. Photos and videos of the machine give a taste of our looming Celebrity Robot Future.

Data doping: real-world data is the Beluga Caviar of AI – expensive and time-consuming to extract from the world. That’s driven people to look to ways to augment real-world dataset with cheaper, synthetic products. This week, European researchers contributed a new dataset called PHAV, short for Procedural Human Action Videos, which consists of 37,536 videos, each consisting of over 1000 examples across 35 basic categories. Their research suggests “that our procedurally generated videos can be used as a simple drop-in complement to small training sets of manually labeled real-world videos. Hence, we can leverage state-of-the-art supervised deep models for action recognition without modifications, yielding vast improvements over alternative unsupervised generative models of video,” they write.You can find more information in the paper: “Procedural Generation of Videos to Train Deep Action Recognition Networks,” here (PDF).

Code releases: DeepMind has released the code behind its delightfully recursive ‘learning to learn by gradient descent by gradient descent” research, which uses machine learning rather than the intuitions of highly-paid AI researchers. It’s written in TensorFlow, naturally. This will aid the industrialisation of deep learning by reducing the need for specialist knowledge on the part of those implementing algorithms…

… additionally, Google has released transfer learning code for image recognition. The TensorFlow code lets you take a pre-trained model “and train a new top layer that can recognize other classes of images…

Hey, look, no clerks! Amazon’s new retail store: It’s almost Christmas, so Amazon has pulled one of its annual PR stunts designed to generate headlines, press, and sales. And I’m playing right into it. The new product announced by Amazon is a store called ‘Amazon Go’ which contains ‘walk right out’ technology to let you grab your goods and stroll out of the store. No need for cashiers or clerks – sophisticated machine learning algorithms figure out what you’ve grabbed, and bill your account appropriately. Though judging by the video, which contains innumerable individually-wrapped products, it’s likely the main tech supporting this is a bunch of RFID tags embedded in (the outside of) cupcakes.

Life as a conference-going telepresence robot: Conferences aren’t easy for everyone – the cramped, people-thronged halls of convention centers can prove challenging for some people due to mental or physical reasons. So why not tap into the power of robots and telepresence to attend instead? IT consultant & writer Trevor Pott shares his poignant story of attending a tradeshow via a telepresence bot here. Please be kind to any robots you see at NIPS.

Geometric Intelligence + Uber: CarBorg company Uber has acquired don’t call it deep learning startup Geometric Intelligence to form an AI research lab. The 15-strong team will join Uber, bringing a wealth of varying AI expertise into the company, from psychologist (and noted neural net skeptic) Gary Marcus to evolutionary algorithm chap Jeff Clune to fMRI-for-AI research Jason Yosinski. Chief Science Officer Zoubin Ghahramani will be staying in Cambridge for the time being. I’d like to tell you about Geometric’s technology, but the company has been tight-lipped about its approach…

… coincidentally, Uber’s current head of machine learning, Danny Lange, is leaving the company to join game engine unity.

#FakeNewsChallenge… Fake news played a role in the recent US election, as did the seeming inability of tech companies to deal with it. That prompted self-driving car expert & adjunct CMU faculty member Dean Pomerleau to start the #FakeNewsChallenge – there’s a total of $2000 to be awarded for the top-5 teams, with payments in proportion to the accuracy of their ginned-up AI systems at spotting fake news.

All hail the Visual Sentinels: New research from Salesforce MetaMind, Virginia Tech, and Georgia Institute of Technology called Knowing when to Look: Adaptive Attention via a Visual Sentinel for Image Captioning, breaks new ground in pairing vision and language techniques inside single systems, creating an image captioning system which appears to spit out more useful telemetry about what internal representations it has developed and how they map to visual elements. Bonus points for the term ‘visual sentinel’.

Flying pig in ‘salmon glaze’ color spotted over Cupertino… Apple will start publishing AI papers, Russ Salakhutdinov,of CMU/Apple said in a presentation at NIPS on Tuesday.

OpenAI bits&pieces:

We are at NIPS. Schedule here.

Weird&Crazy:

I’m saving this for the regular, full fat ImportAI edition.

Now, remember NIPS conference goes, follow the advice that George W Bush gave to Obama upon handing over office: “always use Purell hand sanitizer”.

ALERT! PROTOCOL BREAK FOR OPENAI ANNOUNCEMENT: We’ve just launched Universe, a software platform for measuring and training an AI’s general intelligence across the world’s supply of games, websites and other applications. We’re hoping that this dataset, benchmark, and infrastructure, can push forward RL research in the same way that other great datasets (some featured below) have accelerated other parts of AI…

… the fact we spent so much time building this suggests to me that AI’s new strategic battleground is about environments and computation, rather than static datasets… the received wisdom is that data is the strategically-crucial fuel for artificial intelligence development. That used to be true when much of the research community was focused on training classifiers to map A to B, and so on. But things have changed. We’re moving into an era where we’re training agents that can take actions in dynamic environments. That means the new key component has become the ability for any one AI research entity to access and create a large amount of rich environments to train their RL agents in. I think that realization provoked Facebook to develop and release TorchCraft to ease the development of agents that are trained on StarCraft, and to develop its language-based learning platform CommAI-env; motivated DeepMind to partner with Blizzard to turn StarCraft II into an AI development platform, and to develop and now plan to release code (hooray!) for its DeepMindLab RL environment (AKA – the world simulator formerly known as Labyrinth); and led Microsoft to turn Minecraft into the ‘Project Malmo’ AI development framework.

How to jumpstart an AI industry? One gigantic supercomputer… or so believes Japan, which is now taking bids for a 130-petaflop supercomputer (versus the world’s current top one, China’s 93-petaflop SunWay TaihuLight) slated to be completed in late-2017. Japan has some of the world’s greatest robot researchers and companies (like Fanuc, or Google’s SHAFT) but has lagged behind in software (for instance, most popular AI frameworks are from the US or Canada, like Caffe or Theano or TensorFlow. The main Japanese one is ‘Chainer’, which isn’t used that widely.). The rig is called ABCI, short for AI Bridging Cloud Infrastructure, and its goal is to “rapidly accelerate the deployment of AI into real businesses and society” (PDF).

Catastrophic forgetting? Forget about it! New research paper from DeepMind, “Overcoming catastrophic forgetting in neural networks” claims to deal with the ‘catastrophic forgetting’ problem in neural networks, making it easier for a single network to be trained to excel at multiple tasks. Techniques like this will be key to developing more advanced, flexible AI systems. “. Our approach remembers old tasks by selectively slowing down learning on the weights important for those tasks. We demonstrate our approach is scalable and effective,” they write (PDF).

AI-generated imagery is sitting on a heck of a Moore’s Law-style curve: That’s my takeaway from ‘Plug & Play Generative Networks’, research that brings us a step closer to generating realistic, high-resolution images using AI. The rate at which the aesthetic quality has advanced here is truly immense. To get an idea for how far we’ve come compare images generated from captions by PPGN (Figure 3), with those generated by groundbreaking research from a year ago….

… and we can expect even better things in the future, thanks to a new Visual Question Answering dataset (PDF). The new set roughly doubles the size of the previous VQA release by adding an additional image (and answer) to each question. Where before you had “Question: is the umbrella upside down? Image: an upside-down umbrella, caption ‘yes’”, you now have “Question: is the umbrella upside down? Image: an upside-down umbrella, caption ‘yes’, Image2: an umbrella in normal position, caption ‘no’.” This will let researchers create better categorization systems that get less confused, could also lead to better synthetic image generation via a richer internal representation of what is being described.

Have you heard the news / I’m reading today / I’m going to slurp all the data / Maluubaaa, Maluuubaaa! Canadian AI startup Maluuba has release a new free dataset that contains 100,000 question-and-answer pairs built out of CNN articles from DeepMind’s mammoth Q&A dataset. Check out NewsQA and start generating an alternative news narrative to that of reality (please!).

Rise of the AI hedge fund: Two Sigma has spun up a competition on Kaggle. It’s giving people a bunch of data containing “anonymized features pertaining to a time-varying value for a financial instrument”. The idea is to tap into the global intelligence of the Kaggle community to come up with new algorithms and inferences that make better predictions from data. There’s $100,000 in prize money up for grabs as well. The approach is similar to that taken by Numerai which turns to the crowd to garner predictions about the movements of strange, anonymized numbers. The key difference? Numerai pays people according to the success of their predictions, whereas Two Sigma is only coughing up a hundred thousand dollars (what do we call this – a megabuck?). Hopefully the group-based stock market inference activity will protect any individuals involved from becoming obsessed with the eldritch rhythms of the stock market, causing them to lose their minds – as depicted in Aronofsky’s ‘before he was famous’ flick ‘Pi’.

The industrialization of machine learning: machine learning has moved from being a science into a profession, says Amazon/USheffield’s Neil Lawrence. That means people are combing through research papers and code to create repeatable, reusable blocks of AI-driven computation, which are then applied by engineers who are more like construction-people than architects. So, what should AI scientists do to further push the field forward? Lawrence’s proposal is that they try and pair more mathematically-rich tools (kernel methods and Gaussian processes) with the inscrutable-yet-powerful neural networks that are currently in vogue. “as The Hitchhiker’s Guide to the Galaxy” states “Don’t Panic”, he writes. “By bringing our mathematical tools to bear on the new wave of deep learning methods we can ensure that they remain “mostly harmless”.”

Term of the week… the truly delightful ‘Sabbath Mode’, which is basically a selective lobotomy for the complex parts of electronics to be activated on the Shabbat and Jewish holidays. I now imagine a Christian fridge whose ‘sabbath mode’ prevents the owner from consuming frightfully sinful shellfish.

The Amazon AI Kraken Waketh… Amazon’s strategy for tackling a new market is similar to the methods employed by the mythological nightmare-of-the-sea, The Kraken. It lurks out of sight while rivals like Google and Microsoft attempt to be first-to-market, then it suddenly emerges from the depths of Seattle, with each of its numerous appendages flailing with new products. That’s roughly what happened at its re:invent conference this week, when Big Yellow Kraken revealed a swathe of AI products, including…

  …Reconfigurable, FPGA-containing computers…  Amazon’s answer to the slowdown in Moore’s Law lies in ‘F1’ servers loaded with typical processors paired with FPGAs (for weird&gnarly stuff: custom accelerators, offload network hubs, and so on.). …

… Image recognition… a new image recognition service called “Rekognition” (does Bezos have a grudge against sub-editors?) will compete with existing ones from Amazon, IBM, Microsoft, and many others.

… voice-assistant-as-a-service…I was talking to someone involved in self-driving cars recently and I made some glib comment about how you could use neural networks to train a traffic light detector to help you deal with intersections. “Ah,” they said, “but can you train it to deal with all possible configurations of traffic lights in the world. Can you deal with sets of 6 traffic lights side by side, hoisted at odd angles above the road, due to the fact the town planner went rogue due to a new pedestrian bridge? And how do you know which one of those 6 is yours? Especially if there’s unique signage? And…” at this point I, suitably chastened, realized the error of my question. Amazon has had to deal with similar challenges with the Polly voice assistant, a text-to-speech cloud service that supports 47 different voices and 24 languages, with the voices knowing the difference between pronouncing sa “I live in Seattle” and “Live from New York”. Yet another example of ‘industrial deep learning’ where the underlying tech is fairly standard but the commercial implementation involves getting a lot of finicky details exactly right.

Citation Not Needed Anymore! Jurgen Schmidhuber gets his media article – the bloke who pioneered the LSTM (a key component in the current enthusiasm for all things memory&AI) has finally got his NYT profile. Congrats Jurgen! (pronounced, as he has told me multiple times, “you-again shmit-hugh-bur”.) Still waiting to see papers emanate from his secretive startup NNAISENSE, though.

OpenAI bits&pieces:

Many of the research team are at NIPS in Barcelona this week giving tutorials, lectures, and such. A full schedule is available here.

OpenAI and Microsoft sponsored events at Women in Machine Learning at NIPS as well. It’s an honor to support a scientific community focused on supporting and increasing diversity in AI.

Government AI: Last week our co-founder, Greg Brockman, was a witness at the Senate’s hearing on “The Dawn of AI”. You can watch the testimony and read our written submission here.

Crazy&Weird:

[Note: thrilled that this edition’s short story comes from a reader, Jack Galler. Thanks for writing in, Jack – great name!]

[2020: a woman walking through the city, listening to music.]

The playlist ends, and the woman gives it a positive rating. Her phone prompts: “Would you like another context playlist?” The woman confirms.

She raises her phone and takes two pictures – one from the rear camera, showing the street, and the other from the front camera of her. The photos get deposited into the phone’s internal representation of the ‘mood’ of the moment, along with the woman’s heart rate from her smartwatch, and a tweet she posted earlier about her lunch. It even knows that it’s raining.

The phone’s AI fuses these together and creates a new internal representation of the mood, then uses GAN techniques to generate a new song. A soothing, spanish guitar solo thrums out of the phone to match the light drumming of the rain.

At the end of the song, she’s prompted to rate it. She gives it a thumbs up – she can’t remember the last time she gave it a thumbs down. She will never listen to that soothing acoustic guitar solo again. She could save it, but the context of the song will never be the same – she will never feel the exact same as the moment the song was created, nor will the city she was walking through be what it was in that moment.

Delicious data for the state-backed Deep Learning gods: the UK passed the Investigatory Powers Act 2016, known colloquially as the ‘snooper’s charter’, into law. It forces internet providers to keep a record of all websites visited by all people in the country for up to one year. Let that sink in for a minute! Now have a gin&tonic and a lie down. Better? OK! Since this also includes a temporal component of what websites (domains, not specific pages) people visit, it will give government an incredibly useful dataset to run complex AI-based inference algorithms on. Spooky agencies will be able to divine odd traits about the national mood by analyzing the rise and fall of the popularity of certain websites, and it’ll be possible to profile people and group them according to their habits, then analyze their activities and watch for correlations or disconnects with other groups. The applications of modern Ml techniques to this sort of data are vast and disquieting.

Ethical machine learning: should we conduct experiments, even if they seem to be offensive? The answer to that question was ‘yes’ from a few Import AI readers, who took issue with my characterization of the ‘automated inference on criminality using face images’ paper from last week. Some readers pointed out that this could be an interesting experiment to run, and I countered by saying I’d need a much larger section of the research paper given over to an evaluation of the ethical and moral context of the experiment.

Battle of the frameworks: Microsoft has CNTK, Google has Tensorflow, and Amazon has… MXNet, as of this week. Amazon has put its weight behind the MXNet deep learning framework, making the software the default framework for running deep learning on Amazon Web Services. MXNet’s elevation at Amazon is likely due to its longtime association with Carnegie Mellon professor (and recent Amazon hire) Alex Smola, who has sought to increase the usage of MXNet for a number of years (PDF). DSSTNE, an Amazon-developed DL library, will likely become a subcomponent of MXNET. It’s likely that only one or two deep learning frameworks will end up being widely used,and whoever controls the framework will be able to extract some economic advantages through building cloud services and products around it that benefit from the broad community uptake. The next two years will likely be critical for establishing the winners and losers in this category.

Municipal Muni-Mind Mangled In Miraculous Manipulation: Further proof that we live in a timeline imagine by Neil Stephenson and William Gibson comes in the form of the public transportation computer hack in San Francisco this weekend. “‘You Hacked, ALL Data Encrypted.’ That was the message on San Francisco Muni station computer screens across the city, giving passengers free rides all day on Saturday,” reports CBS.

DeepMind + NHS: Getting ahold of healthcare data is notoriously tricky due to the many (sensible) laws around data protection. Google DeepMind’s solution is to partner with the Royal Free London NHS Foundation Trust to get some useful modern software into the hands of clinicians, and eventually incorporate machine learning components as it establishes trust and credibility. Much of the NHS runs on an arcane system of paper records, so any digitization is a good thing. It’s likely Google/DeepMind will face some opposition and probing from citizens and politicians over its usage and stewardship of their data. The onus is on Google DeepMind to prove that partnership schemes like this can work for patients above all.

Microsoft’s wacky quantum computer bet: Microsoft plans to make a prototype of a new type of quantum computer in a bet that the technology is ready to jump out of theory and into practical reality. Microsoft has taken a different tack to Google with its quantum computing approach and is betting its farm on a technology called a ‘topological quantum computer’. That’s a somewhat more far-out technology than the types of computer being explored by Google. The company has enlisted a bunch of quantum experts to help it build the machine, including Matthias Troyer of E.T.H Zurich. (There’s an occasional argument among AI experts, typically after a few beers, as to whether consciousness emerges from a quantum substrate. Physicist Roger Penrose has a pet theory that consciousness comes out of quantum activities inside ‘microtubules’ inside brain neurons, though evidence for this is scant at best.)

Cities conjured up from lines scratched into sand… and much more in this fantastic paper ‘Image-to-Image translation with Conditional Adversarial Nets’. The authors outline a system that lets you train an AI to pair one image input, like a satellite photograph of a city, and generate an output, like a Google Map with bounding boxers around buildings. The technique works across domains and can be used to, say, draw a woman’s handbag and use that to create a synthetic ‘photograph’ of the bag, or take a picture of a landscape in the day and show it at night….
   …The work has already been extended by Opendotlab for the ‘Invisible Cities’ project to create a system that can take a satellite photo of, say, Milan, and re-interpret it as though the buildings all come from Los Angeles. Terracotta roofs turn to concrete flattop & public squares become asphalt. Canals become freeways. It’s a marvelous, stimulating experiment, and a wonderful example of how art will be changed by the arrival of machine learning…
   …so with all the possibility of new forms of creation from the combination of deep learning and art it’s great to see the launch of Creative.ai, a company formed by a bunch of European AI hackers to spread AI-enabled aesthetics into studios and agencies across the world. AI is becoming just another lens through which we see the world, and it has the potential to show us things our puny four-dimensional minds have trouble imagining. T-SNE goggles, kind of thing.

Computational fluid dynamics meets deep learning.. The previous sentence will be true of many, many things in coming years: “kitchen-shift scheduling meets deep learning”, “insurance claim analysis meets deep learning”, and so on. But the amazing thing about this code release from Google is that you can train a neural network to handle some of the gnarlier equations involved in CFD. It creates some amazing visualisations, but don’t try this in your nuclear reactor yet, kids.

AI turns everything into a prediction problem: the rise of low-cost machine intelligence systems will see people in companies across the world work to turn as many of their problems as possible into problems of prediction, says the Harvard Business Review. That’s because “the first effect of machine intelligence will be to lower the cost of goods and services that rely on prediction. This matters because prediction is an input to a host of activities including transportation, agriculture, healthcare, energy manufacturing, and retail,” it writes.

What does it take to build a strong AI?… not as much as you’d think, suggests Yann Lecun in this wide-ranging speech at Carnegie Mellon University. Fast forward to around 32 minutes into the video to hear Yann’s views on how to build super-smart machines. Most AI experts have their own workable theories for how to build super-intelligence AI systems, but are usually held back by the relatively meager capabilities of modern computers and a lack of the right sort of data. We’re moving into an era where both of this scarcities will be less severe, so we can expect development here.

OpenAI bits&pieces:

Government Talk: We will be giving testimony on artificial intelligence at the Subcommittee on Space, Science, and Competitiveness’s hearing on “The Dawn of Artificial Intelligence” on Wednesday. Tune-in!

Crazy&Weird:

[2020: An architect’s office in Seattle. A person wearing a black turtleneck stands in front of a floor-to-ceiling screen. Small white earbuds (no cable) dangle out of their ears. They gesture at the screen.]

“So as you can see, the house itself can change its appearance according to the different styles and textures you’re wearing on the day. We use style-transfer techniques to modify the textures on the walls according to what you’re wearing. This can really help you express yourself and make an impact, especially when hosting get togethers. Turn on your webcam and I’ll show you!”

The screen splits in two. A woman wearing a red scarf over a blue jean jacket blinks into view on the left-hand side, and a modern, boxy house appears on the right hand side.

“Let me demonstrate!” The person gestures from the woman on the left side of the screen to the house on the right. On screen, the house flickers and its walls change from a grey color to a blue to match the jean jacket. The door and window frames turn from white to a vivid, cross-hatched red.

“Now try without the scarf!”. The woman on screen removes her scarf. The house responds, its screens turning from blue to a smooth beige. “In a few years, it won’t just be the appearance of the house that changes, its geometry will change as well.

From the department of: I Really Hope This Is Satire, but Given It Is 2016 I Cannot Be Sure: Researchers at Shanghai University published a paper called ‘automated inference on criminality using face images‘. The paper uses deep learning to explore correlations between someone’s appearance and the chance of them being a criminal. It’s modern phrenology – 19th century junk science where people believed you could measure someone’s skull and use it to infer traits about their intelligence (the Nazi’s were influenced by this). I can see no merit to this paper whatsoever and am mystified that the researchers were not warned off of publishing this absurd paper. If someone feels my views are wildly wrong here I’d love to hear from you and will (if you’re comfortable with it) put the correspondence in the next newsletter.

How to judge which jobs will be automated: If you can collect 10,000 to 100,000 times as much data on a given job as someone would reasonably generate during the course of their professional life, then you can automate it. This explains why jobs where you can gather lots of aggregate data (eg, insurance actuaries, legal e-discovery, radiology, repeatable factory work, drivers) are already seeing massive automation.

If another professor leaves for AI, and there are no academic’s left who aren’t in industry, do people notice? Another significant move from Academia to Industry as Stanford professor Fei-Fei Li takes up a full-time gig at Google. Fei-Fei Li is both astonishingly important and an astonishingly patient, wise person, so it’s a great get for Google. Li and her team of grad students and collaborators practically kick-started the deep learning boom by creating the ‘ImageNet’ dataset and associated competition. Geoff Hinton & co won ImageNet in 2012 with an approach that relied on deep learning and this precipitated the immense flood of interest and investment that followed. Li is the latest in a long, long line of deep learning academics who have opted to move to spend (most of) their time working in industry rather than academia. Others include Geoff Hinton (University of Toronto > Google), Yann Lecun (NYU > Facebook), Russ Salakhudinov (CMU > Apple), Alex Smola (CMU > Amazon), Neil Lawrence (U Sheffield > Amazon), Nando De Freitas (Oxford > DeepMind), and many more. The main holdout remains Yoshua Bengio who maintains a charming academic fortress in the frozen music-strewn town of Montreal, Quebec. It’s wonderful that industry gets to benefit from the wisdom of academics, but it does lead me to wonder as to whether AI organizations are going to cannibalise the academic ecosystem to the point that they damage the ultimate supply of graduate students. (Note: OpenAI is guilty of this as well, as Pieter Abbeel currently spends most of his time with us rather than at UC Berkeley.) On the other hand, it’s nice to see academics making money off of their ideas, whether by taking up well-paid jobs or selling their startups to big firms. (Congratulations to Berkeley’s Joshua Bloom and the rest of the Wise.io team on selling to GE, by the way.)

Parallel Invention Alert!: television was invented by multiple people at roughly the same time. The same happened for telephones. Ditto Crispr. Technology isn’t mysterious – sometimes there are ideas floating around in the general scientific hivemind and a few people will transmute them into reality at the same time. This phenomenon of Multiple Discovery is worth paying attention to as each occurrence indicates that the idea has some general utility, given the fact that multiple scientists with different perspectives have glommed onto it at the same time…

…AI is rife with parallel invention, and part of the way I model the acceleration in AI development is by the increasing frequency of these cases of parallel invention. So it’s interesting that both OpenAI and Google DeepMind have published remarkably similar papers within a very short (~ two week) timespan of each other. First, OpenAI published a paper called RL^2 Fast Reinforcement Learning Via Slow Reinforcement Learning, then DeepMind followed this with Learning to Reinforcement Learn. (Note: both of these research efforts took many months of work, so the order of publication is not significant. They also test approach on different facets of learning problems.)…

…The idea behind both techniques is that rather than investing time in getting AI to optimize a specific learning algorithm for a given task, you can instead get an AI to optimise its own learning machinery for a set of many tasks. Technically speaking, the idea is that you structure a reinforcement learning agent itself as a recurrent neural network and feed it extra information about its performance on the current task. The agent learns how to create policies to solve a broad range of tasks by using the information about each solution to each problem to alter and augment its own problem solving abilities. Different weights in the RNN correspond to different learning algorithms performed by the agent, and different activations in the RNN correspond to different policies specialized to different tasks faced by the agent…

…general purpose brains, versus tuned brains: These approaches are analogous to the difference between putting an uncalibrated, specific piece of machinery into the brain of an AI and letting it calibrate the machine through interacting with a certain set of environments to solve a specific problem, versus instead putting a more general-purpose bit of machinery into the brain of AI and getting the AI to optimize the machinery for solving many different tasks in many different environments. This ascendance towards greater flexibility, learning, and independence by AI agents is a key point on our march towards creating smarter machines…

…This is not an isolated occurrence. Other examples of parallel invention include: Facebook & DeepMind both pioneering memory-augmented AI systems (neural turing machines, memory networks), Google Brain & DeepMind producing papers on Gumbal Softmax within a week of eachother, multiple people inventing aspects of variational autoencoders, Deepmind and the University of Oxford pioneering methods for lip-reading networks, Stanford&National Research Council Canada&Amsterdam publishing on the TreeLSTM within three months of each other, and many more. If you have examples please email me at jack@jack-clark.net – I’d like to compile these instances in a separate, continuously updated document.

The Deep Learning iceberg lurking in consumer products: the software we use on a day-to-day basis is becoming suffused with deep learning, with much of it lurking beneath the surface. For example, a new Google product called ‘PhotoScan‘ uses neural network-based image analysis and inference techniques to let you quickly scan your family photos, using AI to stitch together the different sections of the photograph to improve quality and correct for glare and spatial distortions. But most importantly it ‘just works’ and the consumer doesn’t need to know it is made possible by a baroque stack of neural networks. Similarly, a Kickstarter for a fancy baby monitor called ‘Knit‘ promises to create a device that uses DL to better monitor the state of the baby (eg, its breathing, wakefulness, and so on), giving parents information about their child through the computer observing its visual appearance and making some assumptions. These products are pretty amazing given that in 2012 image recognition was broadly an unsolved problem.

Welcome to the era of the ultra-lego-block-AI. That’s the message from a new DeepMind paper outlining its latest RL agent. The agent consists of multiple different tried-and-tested AI components (CNNs for vision,a network to enhance the agent’s ability to explore by rewarding it for increasing the variety of the views it perceives, and network to predict rewards and check against what actually happened, and so on – more detail on page 2 of the paper [PDF]) which combine to create a smart, capable system capable of beating DeepMind’s own records on a large number of environments, including tricky games like Montezuma’s revenge (which was broadly unsolved by AI two years ago, and which now sees this AI agent achieve 69% of a human baseline). This kind of multi-system omniagent will be increasingly significant, and it’s something that people like Facebook, OpenAI, and other academics are all working on as well.

A royally good AI blooper: Another fun example of the many ways in which AI algorithms can horribly fail from Tom White’s reliably entertaining ‘Smile Vector‘. This time, the neural network tries to make Princess Kate smile and instead applies the approach to her husband, William. “Kate accidentally landed on William,”he explains, which sounds like a euphemism for many, many things.

Geocities still lives… in the form of this Very Important and Trustworthy AI website: One Weird Kernel Trick.

Given a few hundred million words and a hammer made of a globe-spanning network of computers, can I translate between languages without knowing anything about language? Google’s answer appears to be ‘yes’. In a new paper outlining Google’s Multilingual Translation System the company describes a system that is trained on multiple languages and translates between them.This creates a single, giant network that contains a crude understanding of not only how to translate between pairs of languages, but how to categorize broad concepts across sets of language it doesn’t have explicit, paired sets for. This is significant as it shows the network has learned some essential information about language that it wasn’t given explicit labels for. That shows how modern AI systems can not only map A>B, but can also infer the existence of C and D and map between them as well. Most tantalising thing? The evidence that this approach yields “a universal interlingua representation in our model”. Universal Interlingua!

OpenAI bits&pieces:

Better PixelCNNs for everyone! We published a paper and code for PixelCNN++, a souped-up version of some technology that DeepMind invented. Code here.

Why AI technology is moving so quickly, and why predicting the future is hard: interview with OpenAI’s co-founder & research director Ilya Sutskever.

Crazy&Weird:

Backstory: I wondered if people would like to see something ‘crazy&weird’ in this newsletter and the votes told me ‘Yes’. So here we go:
AI & THE FUTURE OF MANUFACTURING.

[2025: A factory in China. There are no lights and thousands of industrial robots work in a complex, symphony. All you hear is the steady fizz of robotic movement.]

Machine view: multiple reinforcement-learning agents run simulations of the line in a large data center attached to the factory. They explore multiple perturbations of the manufacturing process, endlessly simulating the workload. When they discover more efficient approaches they initiate a High Priority Resource Call to the hardware scheduler in the data center and are assigned a chunk of computing resources to attempt to transfer their knowledge from the simulation into the real robots on the line. After the transfer is complete the robotic line reconfigures itself to account for the new simulation. Any errors are spotted by a thousand cameras staring down at the line. If the AI can diagnose the error it re-runs the simulation and comes up with a fix. If it can’t it sends the images&data of the flaw out to a large Mechnical Turk marketplace where human engineers observe the fault, come up with a fix, and send it back to the line. The system re-optimizes. Meanwhile, one line of the factory attempts to come up with perturbations of the assembled product, inventing wholly new versions of the devices by navigating through the latent feature space of the products. When new ‘Candidate Products’ are found it runs it through a series of tuned, expert systems and, if it gets a high enough score, simulates the product in a high-fidelity simulation. If it passes those tests then a Candidate Product is produced, airlifted by drone to a nearby human focus group and, if it satisfies their criteria, is sold on an EBay-like auction site frequented by the factory’s thousands of distributors. A bidding process takes place and in a few days/weeks data comes back about how the product succeeds in the market. If it does better than the existing product more parts of the factory are dedicated to creating new products in this style and the improvisation line begins exploring the latent space of the new products. In this way the manufacturing process begins to evolve according to a Cambrian evolution process, with AI automating much of the product R&D process.

AI Term of the week: Catastrophic Forgetting: when neural nets completely and abruptly forget previously learned information upon learning new information

Reusability: One of the reasons why AI progress is accelerating is the community is creating more and more reusable components that can be plugged into different domains, frequently attaining performance equivalent to or better than hand-designed algorithms. The 2015 ImageNet challenge was won by a Microsoft system built out of Residual Networks, then in 2016 Microsoft made a speech processing breakthrough via a system that also relied on Residual Networks. Similarly, DeepMind’s WaveNet system has been slightly tweaked and re-applied to the domain of neural machine translation (PDF). This kind of re-use is a good thing as it suggests we are beginning to create the right sorts of low-level primitives that general intelligences can be built out of.

Domain conversion: We’re also seeing researchers work to convert hard problems into domains where we can use neural networks to learn to solve aspects of the problems. This brings more problems into a scope where we can train computers to learn how to solve problems that we’d have trouble programming by hand. For example, a recent OpenAI paper called Third Person Imitation Learning (PDF) is able to convert the problem of copying the actions of another entity into one amenable to generative adversarial networks, which let you learn a mapping between the behaviors of a teacher and a student without specific labels or additional data.

The Rude Goldberg AI Factory: startup  Tend.ai claims to have developed software lets you train robots to operate your 3D printers and associated genius-in-garage gear. The company says its software works with any robot, any webcam and any gripper, to imbue machines with the smarts to read the display and press the buttons on common factory items, like 3D printers. This could lead to a world where the mythological internet startup in a garage becomes the mythological internet-startup-slash-steampunk-robot-manufactory in a garage.

Who wants to make $1000? Venture capitalist Keith Rabois says AI is getting so good that it’ll soon be able to write screenplays and articles that are indistinguishable from those produced by humans. This is unlikely to happen in the short or medium-term — language remains one of the great unsolved challenges in AI. Designing good language systems requires an AI that can tie concepts in language to things it knows about the world, which requires an immense amount of what experts call ‘grounding’. We aren’t there yet. So AI researchers may want to take Keith Rabois up on his bet with Salesforce/Metamind’s Stephen Merity that he can “send you a manuscript that passes a Turing test.” Merity is in, as is Quora’s Xavier Amatriain. Anyone else?

Changes at Twitter: Twitter’s AI strategy is changing, judging by the recent departures of Hugo Larochelle, Ryan Adams, and others. The company had been trying to build an academic research group similar to those at Google, Facebook, and Microsoft. With their departures it seems the emphasis is now firmly on applied AI.

Gender bias in astronomy: researchers from the Swiss Federal Institute of Technology in Zurich use machine learning techniques to analyze 200,000 papers from 1950 to 2015. The key result: a paper whose first author is a woman is likely to get about 10 percent fewer citations than those that have men as the first authors. On the plus side, the fraction of papers where the first author is female has grown from less than 5% in the 1960s to about 25% today.

A vast machine intelligence landscape: Bloomberg Beta has published its third landscape of machine intelligence. The chart includes a third more companies than a year before and depicts a teeming menagerie of companies and organizations striving to apply ML to every component of the technology stack. “It feels even more futile to try to be comprehensive, since this just scratches the surface of all the activity out there,” they write.

Salesforce starts publishing AI papers: It’s normal for the research labs operated by the consumer companies to publish Ai research papers. Now that same spirit of openness seems to be moving into the enterprise as well. Recent Salesforce acquisition Metamind has published a number of papers recently on areas like question answering, natural language processing, recurrent neural networks, and more. It has also topped the leaderboard on the Stanford Question Answering Dataset (SQuAD) competition.

OpenAI bits&pieces:

How open should companies like Google be with their AI systems, what kinds of monopolies could we see emerge as a consequence of the AI revolution, and how do these issues relate to Japanese cucumbers? Those are some of the things Tim Hwang of Google and myself for OpenAI discussed in Toronto last month, which you can see in this video.

Do Androids go to the karaoke bar? Computers are getting much better at generating music and melodies thanks to the use of neural networks. New research from the University of Toronto, ‘Song From PI: A Musically Plausible Network for Pop Music Generation’ sees AI create some quite convincing songs. Listen to some samples here; they sound like the Muzak a sentient elevator might hum to itself. It uses a hierarchical recurrent neural network to develop songs with long-range structure; capturing the sorts of melodies and key changes that exist over multiple bars has been challenging to AI in the past, so this work is a step in the right direction. The authors also pair the tech with text/image synthesis AI to give the robots lyrics formed of the descriptions of images. “We were barely able to catch the breeze at the beach and it felt as if someone stepped out of my mind,” the robots sing. Just wait till we combined the synth-voice tech with Wavenet!

Automatic compression: the field of data compression is beginning to be revolutionized by AI, as new techniques like recurrent neural networks and autoencoders make it possible to train systems to compress and decompress images. New research (PDF) by Twitter Cortex takes a step in this direction, though there isn’t yet truly convincing performance relative to traditional compression methods.

People will wear masks when they speak of revolutionary things: spare a thought for spies, who will no longer able to meet in a loud bar to whisper clandestine truths to each other across a table. New research by the University of Oxford and DeepMind has created AI software (called LipNet (PDF)) that can learn to read people’s lips with an accuracy of around 93%, outperforming human experts. Important caveat: the training data consists of simple sentences with a limited vocabulary, such as ‘place blue in m 1 soon’, so it doesn’t work on real-world data yet. Check out this video to get an idea of how it works. “It’s a limited vocabulary & grammar dataset developed by colleagues at @shefcompsci, I know cos I’m in it,” writes Sheffield/Amazon’s Neil Lawrence.”So while the model may be able to read my lips better than a human, it can only do so when I say a meaningless list of words from a highly constrained vocabulary in a specific order.” (The technical term for this is ‘gibberish’). To take this sort of technique to the real world researchers will need to do three things: gather a vast amount of real-world video, develop software to be able to read lips from multiple angles and in varying lighting conditions, and build a language model that is sophisticated enough to be able to guess at the kinds of phrases people are using. The technology has such obvious utility, though, that it seems inevitable to be built. At some point in the future if you want to tell people secrets you’ll have to wear a face-mask, not because you’re afraid of pollution, but because you fear the decoding-ability of random CCTV cameras and airborne spy drones.

AI will deliver on the tech boom’s promise: “Economists have long wondered why the so-called computing revolution has failed to deliver productivity gains. Machine intelligence will finally realize computing’s promise,” argue James Cham and Shivon Zillis of Bloomberg Beta. That’s because ML will augment any business that involves a) computation and b) the gathering of data. Any successful business has this trait to one extent or another, and some of the lagrest and most significant enterprises have structured their companies to optimize for a) and b).

Battle of the world-changing technologies: AI VS CRISPR: It’s Election Week in America, so in that spirit I decided to conduct a terrifically biased and flawed poll. I asked people to tell me which of AI, CRISPR – a powerful gene editing technology -, or “something else” would have the biggest effect on the world over the next 10 years. More than 50 percent of the 700 votes (of my heavily AI-biased followers) went to AI, with the rest split between CRISPR and something else. A variant of my poll which added the option of ‘climate change’ saw votes split more evenly between AI and Climate Change, followed by CRISPR. We’er lucky to live in a time where we have not one but two totally transformative technologies developing at a rapid pace. It seems likely that CRISPR and AI will overlap over the coming years, as scientists use AI to better analyze the results from CRISPR experiments, and CRISPR lets us enhance our understanding of genes to the point where we can start turning them into organic computing machinery. The convergence of these fields will pave the way for things like ‘Turing biocircuits – programmable biological network pathways for partitioning and cycling energy and matter’. Turing biocircuits! What do you think?

The future is… militarized AI: the US military sees the deployment of AI & robotics as being as strategically important as its earlier development of nuclear weapons and precision munitions, according to the the Financial Times. (I’ve also heard from people in the defense universe that cognitive enhancements – whether that be drugs, or some kind of neural lace – are being viewed as key technologies for a future strategic advantage.)

Computers open their eyes: Advances in AI mean ‘images will be as transparent to computers as text is today’, says Benedict Evans of A16Z. What does this new world look like? Work by Mario Klingemann for the Google Arts Project gives us a peek. The XDegreesofseparation project uses software to build a visual bridge between radically different art objects, selecting a set of several pieces of art that help you go from the baleful geometry of an Aztec death mask to the sweeping, fluid curves of the Venus de Milo, or from an old painting of a woman to an ancient sculpture. The technical term for what’s going on here is ‘interpolation’ – the AI has learned to look at visual traits of an object and navigate between them. Expect more work like this, especially as research into a field known as generative models lets us navigate more of the mysterious landscape that unites all aesthetic objects.

AI develops a conscience: the development of AI has outpaced the development of ethics relating to AI. That’s created unfortunate situations where algorithms have displayed gross biases like echoing racial stereotypes in text-association programs or displaying rank sexism in language engines. But all is not lost; the community has woken up to these problems and researchers across the industry are devoting time to fixing things. Now they’re getting outside help as well: CMU has been given $10 million by a law firm called K&L Gates to set up a center to study ethics and AI. This center will sit  alongside related research efforts at Berkeley, Oxford, Cambridge, Stanford, and others to look at the societal and ethical impact of AI. In related news, the University of Oxford is hiring a researcher to look into the issues of personal safety, ethics, and security with regards to the internet of things, and how they relate to the amalgamation of a vast sea of digital data.

Moore’s Law is Dead, so now the chip-design rebels will stage their coup: When a great whale dies its carcass becomes an oasis in the belly of the ocean; for a while life flourishes amid its bones. The same is true of the end of different eras in technology. Now is the time of the death of Moore’s Law — and this is a good thing. Moore’s Law states that people can stuff double the amount of transistors in the same area of a chip every two years. That amazing law has driven much of the tech expansion in recent times, making intractable AI processing problems tractable, and putting supercomputers into the pockets of everyone with a smartphone. Now Moore’s Law is dying as ever-more-finely-detailed chip-making processes run into the uncaring and unyielding laws of physics. This means people are now exploring non-standard architectures for their processors, hoping to eke out performance through specialization rather than by relying on Moore’s Law. As one chip design student at MIT told me ‘it’s our time, now!’. That attitude drove Google to develop the TensorFlow Processing Unit chip for its data centers, led Microsoft to add FPGA co-processors to its Azure cloud, and motivated Intel to buy FPGA company Altera and AI-chip company Nervana. Now there’s a new chip startup called Graphcore which decloaked recently with $30 million in funding and plans to spread its Intelligence Processing Unit (IPU) chip and associated software far and wide. “The IPU has been optimized to work efficiently on the extremely complex high-dimensional models that machine intelligence requires. It emphasizes massively parallel, low-precision floating-point compute and provides much higher compute density than other solutions,” Graphcore writes.

The Kurt Vonnegut/Hunter S Thompson tech write you didn’t know you needed: This is your semi-yearly reminder to read James Mickens, the reassuringly insane computer science researcher and writer. “If a misaligned memory access is like a criminal burning down your house in a fail-stop manner, an impossibly large buffer error is like a criminal who breaks into your house, sprinkles sand atop random bedsheets and toothbrushes, and then waits for you to slowly discover that your world has been tainted by madness,” he says (PDF).  Read more of his work here. Thank me later.

OpenAI bits&pieces:

Automation requires a new social safety net: OpenAI co-chairman Elon Musk says the rise of automation and AI demands a new social safety net, specifically some kind of Universal Basic Income. There’s a vigorous debate going on among economists at the moment about whether UBI is viable, but one thing everyone agrees on is that advances in Ai are forcing a re-evaluation of what it means to work, what it means to be compensated for work, and how the government should react to the ensuing rise in inequality and centralizing of profits among a few operators of smart AI-infused capital. “Globalization may have ravaged blue-collar America, but artificial intelligence could cut through the white-collar professions in much the same way,” writes Jim Yardley in the New York Times.

ICLR-palooza: like many of our peers across the world we submitted a spread of papers to the ICLR conference. You can have a browse of them here. I’ll have a thorough writeup next week but the gist of this batch of research is about transfer learning for robots, more efficient reinforcement learning algorithms and further work on generative models.

Thanks for reading. If you have suggestions, comments or other thoughts you can reach me at jack@jack-clark.net or tweet at me@jackclarksf

China’s 13th Five-Year Economic Plan promises big things for robots & AI: China plans to make substantial investments into robotics, according to the country’s 13th five year economic plan. The funding boost will support the Made in China 2025 initiative, which aims to improve the quality of Chinese-made goods while suffusing the country’s factories with smart machines. The robotics push may have significant consequences; in the 12th economic plan they said the development of a domestic semiconductor industry was a priority. Now, the majority of chips in the world’s fastest supercomputer use Chinese-designed ‘Sunway TaihuLight’ chips, rather than Intel’s which are used in the vast majority of other top machines. If the various robotic investments triggered by the plan pay off then there could be big consequences for artificial intelligence. Modern AI techniques have already been used by robot makers like Fanuc to substantially increase the speed at which a factory robot can be taught to excel at a particular task. This has led to related investments in areas like reinforcement learning (see: Fanuc’s partnership with Japanese AI startup Preferred Networks, or ABB’s relationship with Vicarious), which has fed into mainstream AI development. A flood of money from China, paired with the built-in end-customers in the form of large Chinese government labs, means robot-related AI development could speed up. Take a look at this gallery of a robot expo hosted by the Chinese army and imagine what kinds of advances will be possible by piggybacking on the manufacturing robot push.

If ‘Pepper’: then AI  == False: SoftBank’s tablet-clutching, Michelin Man-alike robot Pepper has not done nearly as well as the company hoped, disappointing customers. AI (or the lack of it) is partly to blame. The company had planned to apply modern AI techniques using neural networks to create a more advanced brain for the robot, but slowdowns and culture clashes between Aldebaran Robotics, a French company bought by Softbank in 2012, and the other engineering teams and managers (Japan), led to problems and delays. For example, instead of using deep learning techniques to learn to recognize emotions and react appropriately, the programmers had to dictate the responses by writing a laborious set of if-then statements. This limited Pepper’s abilities, and many companies wound up using the robot more as a cute tablet-toting marionette than as an interactive AI assistant. It’s a lesson in the importance of communication and what happens if that goes awry.

More gratuitous GTAV self-driving bot footage: In Issue 12 I wrote about research that shows hit game GTAV is a reasonable simulator to train self-driving cars in. This week I discovered that Craig Quiter, a technical contractor at OpenAI, has a personal project called DeepDrive where he uses GTAV to develop his own car systems. Watch videos of a car navigating traffic, keeping to the center of a lane, and marvel at the moment a snowstorm causes its visual brain to break down. Videos & more information here.

I would drive 500 miles, and I would self-drive 500 more, just to be the (self-driving) car that brings data to your door: the proliferation of free datasets for self-driving continues. The Oxford Robot Car dataset comes from researchers spending a year traversing the city of Oxford twice a week in a self-driving car. The 20 terabyte release involves around 100 journeys on the same route and covers both camera data, as well as LIDAR and GPS. The repetition of the route is important because the resulting dataset will let people teach AI systems how to make sense of the many permutations of route world, like shifting pedestrians, different weather patterns, rain, and so on. This follows similar data releases from Udacity and Comma.ai. We’re lucky to live in a time where companies and universities generate so much free data.

Comma catastrophe self-driving car startup Comma.ai has cancelled its main product, the Comma 1 – a $999 kit to retrofit modern cars into self-driving robo-chauffers. Founder George Hotz said he cut the product after receiving a letter from US regulator NHTSA. “comma.ai will be exploring other products and markets. Hello from Shenzhen, China. -GH”, Hotz writes.

Building up Canada’s AI ecosystem: In the same way PayPal spawned a generation of entrepreneurs who came to influence the tech industry, Canada created its own gang of hugely influential AI researchers through CIFAR, a funding body that supported people like Geoff Hinton (now at Google), Yann Lecun (now at Facebook), and Yoshua Bengio (now at UMontreal). Their contributions form much of the bedrock of the current popular AI techniques, ranging from systems for image recognition and machine translation at Google, to state-of-the-art memory systems at Facebook. But Canada has struggled to retain its top talent, as professors are drawn to work at other companies, and US-based schools massively increase investment in AI programs. To counter that, a group of researchers and entrepreneurs have launched Element AI, a Montreal-based AI incubator slash research lab. Influential research Yoshua Bengio, the Paul Erdös of AI, is one of the founders.

Welcome to the world of the self-defending autonomous AI corporation: in Charles Stross’ (excellent, free) sci-fi novel, Accelerando, the world is suffused with smart, semi-autonomous AI-driven digital corporations that indulge in a constant cacophony of deal-making, legal subterfuge, and company formation and destruction. It feels like a plausible future, but requires two prerequisites: 1) a form of digital currency with various forms of metadata built into it to let computers participate in a universal economic market, 2) the ability of corporations to exchange information with each other privately and efficiently. 3) better decision-making AI systems capable of feats of memory, transfer learning, and ideation. Software like Bitcoin and Ethereum attempts to solve problem one, new research from Google tackles option two, and the AI research community is working on option three. The new Google paper, Learning to Protect Communications with Adversarial Neural Cryptography, outlines what they call a neural cryptography system. This lets them have semi-independent AIs improvise a secure communication channel with each other in the presence of an adversary. Give it a few years and they’ll be a proliferation of microscopic economic agents that barter privately with each other in a market of digital information.

Image recognition isn’t only for the data titans: Image recognition startup Clarifai has raised a $30 million Series B. Clarifai is led by AI researcher Matt Zeiler, and the company proved its AI chops in 2013 by winning the ImageNet competition. It sells image recognition services to people around the world, letting people use AI to automatically organize their photos and videos, or identify specific items in images (hypothetical example: ad agencies training a Clarifai AI to recognize and spot different brand logos on clothing, then using the software to automatically patrol the web to identify photos containing the brand. “Today Clarifai not only hosts a static API that tags thousands of images and videos with human level accuracy, but now empowers anyone from a wedding photographer to a large retail company to a sail boat enthusiast (hat tip: USV’s Albert Wenger) to easily access and build products employing Google level AI with drag and drop ease of use,” writes Lux Capital, which invested in the round. Clarifai’s ongoing success is an intriguing counterpoint to the narrative that it’s difficult for AI startups to compete with the resources wielded by vast tech companies like Amazon, Google, Microsoft, and others.