Import AI

Half-price neural networks thanks to algorithmic tweaks: new research, Distributed Second-Order Optimization Using Kronecker-Factored Approximations (PDF), creates a new optimization method for training AI systems. The approach is flexible and can be dropped into pre-existing software relatively easily, its creators say. Best of all? “We show that our distributed K-FAC method speeds up training of various state-of-the-art ImageNet classification models by a factor of two compared to an improved form of Batch Normalization”. Quite rare to wake up one day and discover that your AI systems have just halved in price to train.

Bayesian Program Synthesis – bunk or boon? Startup Gamalon has decloaked with a new technology – Bayesian Program Synthesis – that claims to be able to do tough AI tasks like learning to classify an images from a single digit handful of examples, rather than a thousand. The work has echoes of MIT research published in late 2015 (PDF), which showed that it is possible to use Bayesian techniques similar to this one to perform ‘one shot learning’ – which lets computers learn to recognize something, say, a cat, from only a single glimpse. The research was shown to work on a specific test set that had been implemented in a specific way. Gamalon is claiming that its tech has more general purpose utility. However, the startup has published no details about its research and it is very difficult to establish how staged the press interview demos were. If Gamalon has cracked such a hard problem then I’m sure the scientific community would benefit from them sharing their insight. This would also help justify their significant claims.

Income tax for robots: Bill Gates says that people should consider taxing robots to generate revenues for government to offset the jobs destroyed via automation. Small query I’d like to see someone ask Bill: in hindsight, should governments also have taxed software like Excel to offset the jobs it destroyed?

AirSim: because it’s cheaper to crash a drone inside software: Microsoft has released AirSim, an environment based on the Unity game engine providing a reasonably high-fidelity simulation of reality, giving developers a cheap way to train drones and other robots via techniques like reinforcement learning, then transfer those systems into the real world (which we already know is possible, thanks to research papers such as CAD2RL). This is useful for a couple of reasons: 1) you can run the sim much faster than real life, letting you make an order of magnitude more mistakes while you try to solve your problem, and 2) this reduces the cost of mistakes – it’s much cheaper to fire up a new simulation than try to repair or have to replace the drone that just bumbled into a tree. (Well, research from MIT and others already suggests you won’t need to worry about the tree, but you get my point.)
…Simulators have become a strategic point of differentiation for companies as each battles to craft the perfect facsimile of the real world to let them train AI systems that can then be put to work in reality. The drawback: we don’t yet have a good idea for how real simulators need to be so it’s tricky to anticipate the correct level of fidelity at which to train these systems. In other words, we don’t know what level of simulation is sufficient to ensure that when we arrive in reality we are able to achieve our task. That’s because we haven’t derived an underlying theory to help guide our intuitions about the difference between the virtual and the real – Baudrillard eat your heart out!

Skeptical about The Skeptic’s skepticism: We shouldn’t worry about artificial intelligence disasters because they tend to involve a long series of “if-then” coincidences, says Michael Shermer, publisher of The Skeptic magazine.

Enter the “vision tunnel” with Jeff Bezos: When goods arrive at Amazon’s automated fulfillment center they pass through “a “vision tunnel,” a conveyor belt tented by a dome full of cameras and scanners”, where algorithms analyze and sort each box. “What takes humans with bar-code scanners an hour to accomplish at older fulfillment centers can now be done in half that time,” Fast Company reports…. There’s also a 6-foot tall Fanuc robot arm, which works with a flock of Kiva robots to load goods into the shifting robot substrate of the warehouse. The million-square foot plus facility employs around a thousand people, according to the article. A similarly sized Walmart distribution center employs around 350 (though this doubles during peak seasons)  – why the mis-match in scale, given the likelihood of Amazon having a larger degree of employee automation?

8 million Youtube bounding boxes sitting on a wall, you take one down, classify it and pass it around, 7 million 900 and 99 thousand and 900 and 99 Youtube bounding boxes on a wall: Google has updated its 8 million video strong YouTube dataset with twice as many labels as before…
…. and it’s willing to pay cash to those who experiment with the dataset, and has teamed up with Kaggle to create a series of competitions/challenges based around the dataset, with a $100,000 prize pool available. (This also serves as a way to introduce people to its commercial cloud services, as the company is providing some credits for its Google Cloud Platform as well for those that want to train and run their own models. And I imagine there’s a talent-spotting element as well.)
… I’ve been wondering if the arrival of new datasets, or the augmentation of existing ones, is a leading indicator about AI progress – it seems like when we sense a problem is becoming tractable we release a new dataset for it, then eventually solve the problem. Thoughts welcome!

Deep Learning papers – curated for you. The life of an AI researcher involves sifting through research literature to identify new ideas and ensure there aren’t too many overlaps between yet-to-be-published research and what already exists. This list of curated AI papers may be helpful.

When does advanced technology become DIY friendly?: warzones are a kind of primal soup for (mostly macabre) invention. This Wired article on robot builders in the Middle East highlights how a combination of cheap vision systems, low cost robots, and software, has allowed inventive people to repurpose consumer technology for war machines, like little moveable defense platforms and gun turrets. Today, this technology is very crude and both its effectiveness and use are unknown. But it highlights how rapidly tech can be repurposed and reapplied. Something the AI community should bear in mind as it publishes research and code of its ideas.

Neural architecture search VERSUS interpretability: a vivid illustration from Google Brain resident David Ha of the baroque topologies of neural networks created through techniques like neural architecture search.

Google researcher handicaps AI research labs: Google Brain research engineer Eric Jang has ranked the various AI research labs. He ranks Deepmind and… Google Research in joint first place, followed by OpenAI & Facebook, followed by MSR (3rd) and Apple (4th). He puts IBM at 10 and doesn’t specify the intervening companies. “Given open source software + how prolific the entire field is nowadays, I don’t think any one tech firm “leads AI research” by a substantial margin”, he writes…
…That matches comments made by Baidu’s Andrew Ng, who has said that any given AI research lab has a lead of at most a year on others…
…IBM Watson benched: MD Anderson has ended its collaboration with IBM on using AI technology marketed under the company’s “Watson” omnibrand. The strangest part? MD Anderson paid IBM for the privilege of trialing its technology – an unusual occurrence, usually it’s the other way round, Forbes reports. The project was suffused with delays and it’s still hard to establish whether things ended because of IBM’s tech, or because of a series of unfortunate bureaucratic events within MD Anderson.

OpenAI bits&pieces:

Adversarial examples – why it’s easier to attack machine learning rather than defend it. New OpenAI post about adversarial examples, aka optical illusions for computers, delves into the technology and explains why it may be easier to use approaches like this to attack machine learning systems, rather than defend them.

Ilya Sutskever talks at the Rework Summit: if you weren’t able to see Ilya’s talk at the Rework deep learning summit in person, then you can catch a replay here.

Tech tales:

[A boardroom at the top of one of London’s increasingly HR Geiger-esque skyscrapers.]

“So as you’ll see the terms are very attractive, as I’m sure your evaluator has told you,” says Earnest, palms placed on the table before him, looking across at Reginald, director of the company-to-be-acquired.
“I’m afraid it’s not good enough,” Reginald says. “As I’m sure your own counter-evaluator has told you.”
“Now, now, that doesn’t seem right, let’s-”
“Enough!” Reginald says. “Leave it to them.”
“As you wish,” you say, leaning back.

Earnest and Reginald stare at each other as their evaluators invisibly hash out the terms of a new deal, each one probing the other for logical weaknesses, legal loopholes, and what some of the new PsychAIs are able to spot – revealed preference from past deal-making. As the company-to-be-acquired, Reginald has the advantage, but Earnest’s corporation has invested more heavily in helped agents, which have spent the past few months carefully interacting with aspects of Earnest’s business to provide more accurate True Value Estimates.

Eventually, a deal is created. Both Earnest and Reginald need to enlist translator AIs to render the machine-created legalese into something the both of them can digest. Once Reginald agrees to the terms the AIs begin another wave of autonomous asset-stripping, merging, and copying. Jobs are posted on marketplaces for temporary PR professionals to write the press release announcing the M&A, and design contracts are placed for a new logo. This will take hours.

Reginald and Earnest look at eachother. Earnest says, “pub?”
“My evaluator just suggested the same,” says Reginald, and it’s tough to tell if he’s joking.