Import AI

Chinese scientists call for greater coordination between neuroscience and AI: Chinese AI experts say in this National Science Review interview that students should be trained in both AI and neuroscience, and ideas from each discipline should feed into others. This kind of interdisciplinary training can spur breakthroughs in ambitious, difficult projects, like cognitive AI, they say.

AI could lead to the polarization of society, says the CEO of massive outsourcing company Capgemini. Firms like Capgemini, CSC, Accenture, Infosys, and others, are all turning to AI as a way to lower the cost of their services, having started to run out of pools of ever-cheaper labor…
…governments are waking up to the significant impact AI will have on the world. “Accelerating AI capabilities will enable automation of some tasks that have long required human labor. These transformations will open up new opportunities for individuals, the economy, and society, but they will also disrupt the current livelihoods of millions of Americans,” wrote the White House in a blog discussing its recent report on ‘Artificial Intelligence, Automation, and the Economy’.

Apple publishes its first AI paper: Apple has followed through on comments made by its new head of ML, Ruslan Salakhutdinov, and published an academic paper about its AI techniques. Apple’s participation in the AI community will help it hire more AI researchers, while benefiting the broader AI community. In the paper, ‘Learning from simulated and unsupervised images through adversarial training’, the researchers use unlabelled data from the real-world to improve the quality of synthetic images dreamed up by a modified generative adversarial network, which they call a ‘SimGAN’.

7 weeks of free AI education: Jeremy Howard has created a new free course called ‘practical deep learning for coders’. Free as in free beer. If you have the time it’s worth doing.

Brain-based auto-tuning: a new study finds that the brain will tune itself in response to uttered speech so as to better interpret or pick-up audio in the future. “Experience with language rapidly and automatically alters auditory representations of spectrotemporal features in the human temporal lobe,” write the researchers. “Rather than a simple increase or decrease in activity, it is the nature of that activity that changes via a shift in receptive fields”. Learn more in the paper, ‘Rapid tuning shifts in human auditory cortex enhance speech intelligibility’.
… figuring out how our brain is able to interpret audio signals – and optimize itself to deal with loud background noise, accents, unfamiliar cadences, corrupted audio, and so on, may let us develop neural nets that contain richer representations of heard speech. That’s going to be necessary to capture the structure inherent in language and prevent neural nets from simply generating unsatisfying (but amusing) gobbledygook like this (video). The research community is hard at work here and systems like ‘Wavenet’ hold the promise for much better speech generation…
… learning about the brain and using that to develop better audio analysis systems will likely go hand-in-hand with the (probably harder) task of building systems that can fully understand language. Better natural language understanding (NLU) systems will have a big impact on the economy by making more jobs amenable to automation. In 16 percent of work activities that require the use of language, increasing the performance of machine learning for natural language understanding is the only barrier to automation, according to a McKinsey report: “Improving natural language capabilities alone could lead to an additional $3 trillion in potential global wage impact.” (Page 25).

New AI research trend: literary AI titles… as the pace of publication of Arxiv papers grows people are becoming ever more creative in how they title their papers to catch attention, leading to a peculiarly literary bent in paper titles. Some examples: The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives, A Way out of the Odyssey: Analyzing and Combining Recent Insights for LSTMs, and Combating Reinforcement Learning’s Sisyphean Curse with Intrinsic Fear. (Still waiting to contribute my own magnum opus ‘down and dropout in Paris and London’).

How do I know this? Let me tell you! New research tries to make AI more interpretable by forcing algorithms to not only give us answers, but give us an insight into their reasoning behind the answers, reports Quartz. We’ll need to create fully interpretable systems if we want to deploy AI more widely, especially in applications involving the potential for loss of life, such as self-driving cars.

222 million self-driving miles, versus 2 million: Tesla’s self-driving cars have driven a cumulative 222 million miles in self-driving mode, while Google’s vehicles have covered merely 2 million miles in the same mode since 2009, reports Bloomberg. As competition grows between Uber, Google, and Tesla it’ll be interesting to see how the companies gather data, whether one company’s mile driven in autonomous mode is as ‘data rich’ as that driven by another (I suspect not), and how this relates to the relative competitiveness of their offerings. Google is due to start trialling a fleet of cars with Fiat in 2017, so we’ll know soon another.

DeepPatient: scientists use deep learning techniques (specifically, stacked denoising autoencoders) to analyze a huge swathe of medical data, then use the trained model to make predictions about patients from their electronic health records (EHRs). “This method captures hierarchical regularities and dependencies in the data to create a compact, general-purpose set of patient features that can be effectively used in predictive clinical applications. Results obtained on future disease prediction, in fact, were consistently better than those obtained by other feature learning models as well as than just using the raw EHR data,” they write.  Now, the scientists plan to extend this method by using it in other clinical tasks such as personalized prescriptions, therapy recommendations, and identifying good candidates for clinical trials.

OpenAI bits&pieces:

Eat your vegetables & understand backprop: OpenAI’s Andrej Karpathy explains why you should take the time to understand the key components of AI, like backpropagation. It’ll save you time in the long run. “Backpropagation is a leaky abstraction; it is a credit assignment scheme with non-trivial consequences. If you try to ignore how it works under the hood because “TensorFlow automagically makes my networks learn”, you will not be ready to wrestle with the dangers it presents, and you will be much less effective at building and debugging neural networks,” he says.

HAL, don’t do that!…But you told me to, Dave…Not like that, HAL! No one “cleans a room” that way…I’m sorry, Dave…
…getting computers to do the right thing is tricky. That’s because computers have a tendency to interpret your instructions in the most literal and obtuse possible manner. This can lead to surprising problems, especially when training reinforcement learning agents. We’ve run into issues relating to this at OpenAI, so we wrote a short post to share our findings. Come for the words, stay for the video of the RL boat.

Tech Tales:

[2019: an apartment building in New York.]

“Time?” says the developer.
It is two AM, says his home assistant.
“Jesus, what the hell have you built,” he says.
I can’t find anyone in your contacts named Jesus, says the assistant.
“Not you. I didn’t mean you. It’s what they’ve built,” he says. “No action needed.”

He pages through the code of ten separate applications, trying to visualize the connections between the various AI systems that have been daisy-chained together. He can already tell that each one has been fiddled with by different programmers with different habits. Now it’s up to him to try to isolate the fault that caused his company’s inventory system to start ordering staggering quantities of butter at 11pm.

A couple of years ago some of the senior executives at the company finally heard about agile programming and ordered the thousand-strong IT organization to change its practices. Processes went out the window in favor of dynamic, fast-moving, loose collections of ad-hoc teams. The plus side is that the company now produces more products at a faster rate. The downside is the proliferation of different coding styles deep in a hundred separate repositories. Add last year’s executive obsession with becoming an “AI first” company (follow in the footsteps of Google, they said, why couldn’t this be a great idea, they said) and the current situation – warehouses rapidly filling up with shipment after shipment of problematic dairy rectangles – was all but inevitable.

“Move fast and order butter,” he mutters to himself, as he tries to diagnose the fault.