Import AI

Constraints as the key to intelligence: Machine learning whiz & long-distance runner Neil Lawrence has published a research paper, Living Together: Mind and Machine Intelligence, that explores the idea that intelligence is intimately related to the constraints imposed on our ability to communicate.
…the gist of Neil’s argument is that intelligence can be distilled as a single number, which he calls an Embodiment Factor. This expresses the relationship between how much raw compute an intelligence can make use of at once, and how much it can communicate information about that computation during the same time frame. Humans are defined by being able to throw a vast amount of compute at any given problem, but then we can only communicate at a couple of words a second at most.
…The way Neil Lawrence puts it is that a computer with a 10 Gigaflop processing capacity and a communication capacity of about 1 gigabit per second has an embodiment factor of 10 (computation / communication), versus a human brain which can handle about an exaflop of compute with a communication limit of about 100 bits per second – representing an astonishing embodiment factor of 10^16. It is this significant compression which leads to many of the useful properties in our own intelligence, he suggests.
…(Open access note: Lawrence was originally going to publish this commentary through a traditional academic channel, but balked at paying fees and put it on Arxiv instead. Thanks, Neil!)

SelfieNightmareGAN: For a truly horrifying time I recommend viewing this experiment where artist Mario Klingemann uses CycleGAN to transpose doll faces onto Instagrammable-selfies.

G.AI.VC: Google has launched an investment arm specifically focused on artificial intelligence. It’s unusual for the company to focus on individual verticals and likely speaks to the immense enthusiasm Google feels for AI. The fund will make investments with a check size of between $1 and $10 million, reports Axios’s Dan Primark.

Treasury secretary walks back AI skepticism: US Treasury Secretary Steve Mnuchin said a few months ago that problems related to AGI and AI-led automation were “50-100 years away” and these issues weren’t “on the radar screen” of federal government.
…He has changed his tune. Now, he says: “When I made the comment on artificial intelligence — and there’s different views on artificial intelligence — I was referring to kind of like R2D2 in Star Wars. Robotics are here. Self-driving cars are something that are gonna be here soon. I am fully aware of and agree that technology is changing and our workers do need to be prepared.”

iAI – Apple said to work on ‘neural chip’: Apple is developing a custom chip for its mobile devices specifically designed for inference tasks like speech and face recognition, according to Bloomberg. Other chipmakers such as Qualcomm have already taken steps in this direction. It’s likely that in the coming years we’ll see most chips get dedicated neural network bits of logic (basically matrix multiplication stuff with variable precision), given the general adoption of the technology – Nvidia is already designing certain GPU components specifically for AI-related tasks.

AI prizes, prizes everywhere! Real estate marketplace Zillow has teamed up with Google-owned Kaggle to offer a $1 million dollar data science competition. The goal? Improve its ability to predict house prices. Submitted predictive models will be evaluated against real house prices over first three months following closure of the competition.
…if this sort of thing works then, in a pleasing Jorge Luis Borges-manner, the predictions of these services could feasibly become a micro-signal in actual home prices, and so the prediction and reality could compound on each other (infinitesimally, but you know the story about butterflies & storms.)
…Next up – using the same sort of competitive model to build the guts of a self-driving car: AI-teaching operation Udacity and wannabe-self-driving company Didi (a notable competitor to troubled Uber) have partnered to create a prize for the development of open-source self-driving car technology. Over 1000 teams will compete for a $100,000 dollar prize.
…The goal? “Automated Safety and Awareness Processing Stack (ASAPS), which identifies stationary and moving objects from a moving car, and uses data that includes Velodyne point cloud, radar objects, and camera image frames. Competitors are challenged to create a redundant, safe, and reliable system for detecting hazards that will increase driving safety for both manual and self-driving vehicles,” according to Udacity.

AlphaGo’s surprisingly efficient success: AlphaGo beat the world champion Kie Jie 3-0 at The Future of Go Summit in China. But local spectators were stymied after the state ordered streams of the match shut down, as AlphaGo demonstrated prowess against the human champion. Still, the games continued. During the second game Demis Hassabis, DeepMind’s founder, said AlphaGo evaluated many of human champion Kie Jie’s moves in the second game to be “near perfect”. Still, he resigned, as AlphaGo created a cautious, impenetrable defense…
…later, DeepMind revealed more details about the system behind AlphaGo. In its original incarnation AlphaGo was trained on tens of thousands of human games and used two neural networks to plan and evaluate moves, as well as Monte Carlo Tree Search to help with planning. Since earning a cover of Nature (via beating European Go expert Fan Hui) and then beating seasoned player Le Sedol in Korea last year, DeepMind has restructured the system.
…the version of AlphaGo that was shown in China ran on a single TPU board – that’s a computer full of custom AI training&inference processors made by Google. It consumed a tenth of the computation at inference time as its previous incarnation, suggesting that its underlying systems have become more efficient – a crucial mark of both earnest optimization by DeepMind’s engineers, as well as dawning intelligence from greater algorithms.
…But you might not be aware of this if you were trying to watch the game from within China – the state cut coverage of the event shortly after the first game began, for nebulous hard-to-discern political reasons.
…China versus the US in AI: While the US and Europe investments in AI either reduce or plateau, China’s government is ramping up spending as it tries to position the country to take advantage of the AI megatrend, partially in response to events like AlphaGo, reports The New York Times.

Could AI help healthcare? The later you wait to treat an ailment, the more expensive the treatment will be. That’s why AI systems could help bring down the cost of healthcare (whether that be for governments that support single-payer systems, or in the private sector). Many countries have spent years trying to digitize health records and, as those projects come to fruition, a vast hoard of data will become available for AI applications – and researchers are paying attention.
…“Many of us are now starting to turn our eyes to social value-added applications like health,” says AI pioneer Yoshua Bengio in this talk (video). “As we collect more data from millions and billions of people around the earth we’ll be able to provide medical advice to billions of people that don’t have access to it right now”.

Reading the airy tea leaves: AWS GPU spot price spike aligns with NIPS deadline: prices for renting top-of-the-range GPU servers for Amazon spiked to their highest level in the days before the NIPS deadline. That synced with stories of researchers hunting for GPUs both within companies and at cloud providers.
…The evidence, according to a tweet from Matroid founder Reza Zadeh: a dramatic rise in the cost to rent ‘p2.16xlarge’-GPU Instances on Amazon Web Services’s cloud:
…Baseline: $2 per hour.
…May 18th-19th (NIPS deadline): $144 per hour.
…Though remember, correlation may not be causation – there are other price spikes in late April that don’t seem to be correlated to AI events.

Imagining rules for better AI: When you or I try to accomplish tasks in our day we usually start with a strong set of biases about how we should go about completing the tasks. These can range from common sense beliefs (if you need to assemble and paint a fence, it’s a bad idea to paint the fence posts before you try to assemble them), to the use of large pre-learned rulesets to help us accomplish a task (cooking, or doing mathematics.)
…This is, funnily enough, how most computer software works: it’s a gigantic set of assumptions, squeezed into a user interface, and deployed on a computer. People get excited about AI because it needs fewer assumptions programmed into it to do useful work.
…But a little bit of bias is useful. For example, new research from the Georgia Institute of Technology and other researchers, shows how to use some priors fruitfully. In Game Engine Learning from Video (PDF) the authors come up with an AI system that plays a game while having the parallel goal of trying to successfully approximate the underlying program of the game engine, which it only sees through pixel inputs – aka what the player sees. It is given some priors – namely, that the program it is trying to construct contain game mechanics eg, if a player falls then the ground will stop them, and a game engine which governs the larger mechanics of the world. The researchers feed it example videos of the game being played, as well as the individual sprites of the images used to build the game. The AI then tries to learn to align sprites with specific facts or precepts, ranging from whether a sprite is animated, how its spatial arrangement changes over time, whether it is related to any other sprites, its velocity, and so on. The AI then learns to scan over the games and align specific sprite actions with rules it derives, such as whether the Sprite corresponding to Mario can move right if there is nothing in front of him, and so on. The system can focus on trying to learn specific rules by rapidly paging through the stored play images that correspond to the relevant sprite actions.
…It uses a fusion of this sort of structured, supervised learning, to iteratively learn how to play the game by reconstructing its inner functions and projecting forward based on its learned mechanistic understanding of the system. They show that this approach outperforms a convolutional neural network trained for next-frame prediction. (I’d want to also see baselines for traditional reinforcement learning algorithms as well to be convinced further.)
…This approach has numerous drawbacks from the need for a human in the loop to load it up with specifically specified priors, but it hints at a future where our AI systems can be given slight biases and interpret the world according to them. Perhaps we could create a Manhattan Project for psychologists to enter numerous structured entries about human psychology, and feed them to AIs to see if they can help the AIs predict our own reactions, just like predicting the movement of a mushroom in Super Mario.
…Components used: OpenCV, Infinite Mario

Pix2code: seeing the code within the web page: at some point, we’re going to want our computers to be able to do most programming for us. But how do you get computers to figure out how to program stuff that you don’t have access to the source for?
…In pix2code, startup UIzard creates a system that lets a computer look at a screenshot of a web page and then figure out how to generate the underlying code which would produce that page. The approach can generate code for iOS and Android operating systems, with an accuracy of 77%. In other words, it gets the underlying code right four times out of five.

OpenAI bits&pieces:

OpenAI Baselines: release of a well-tuned implementation of DeepMind’s DQN algorithm, plus three of its variants. Bonus: raw code, trained models, and a handy tips and tricks compendium for training and debugging AI algorithms. There will be more.

Tech Tales:

2025: Russia deploys the first batch of Shackletons across its thinly-populated Eastern flanks. The mission is two-fold: data gathering, and experimental research into robotics and AI. It drops them out of cargo planes in the night, hundreds of them falling onto the steppes of Siberia, their descent calmed by emergency-orange parachutes.

Generation One could traverse land, survive extremely low temperatures, swim poorly (float with directional intent, one officer wrote in a journal), and consistently gather and broadcast data. The Shackletons beamed footage of frozen lakes and bare horizon-stretching foxes back to TV and computer screens around the world and people responded, making art from the data generated by Russia’s remote parts. The robots themselves became celebrities and, though their locations were unknown, sometimes roving hunters, scavengers, and civil servants would find them out there in the wastes and take selfies. One iconic photo saw a bearded Russian soldier with his arm slung over the moss-mottled back of an ageing Shackleton. He had placed a pair of military-issue dark glasses on one of the front sensor bulges, giving the machine a look of comedic detachment.
“Metallicheskaya krysa”, the Russians affectionately called them – metal rats.

2026: Within a year, the Shackletons were generating petabytes of raw data every day, ranging from audio and visual logs, to more subtle datapoints – local pollen counts, insect colonies, methane levels, frequency of bubbles exploding from gas escaping permafrost, and so on. Each Shackleton had a simple goal: gather and analyze as much data as possible. Each one was capable of exploring its own environment and the associated data it received. But the twist was the Shackletons were able to identify potentially interesting data points they hadn’t been pre-loaded with. One day one of the machines started reporting a number that scientists found correlated to a nearby population of foxes. Another day another machine started to output a stream of digits that suggested a kind of slow susurration across a number line, and the scientists eventually realized this data corresponded to the water levels of a nearby river. As the years passed the Shackletones became more and more astute, and the data they provided was sucked up by the global economy, going on to fuel NGO studies, determine government investment decisions and, inevitably, give various nebulous financial entities a hedge in the ever-more competitive stock markets. Russia’s selectively declassified more and more components of the machines, spinning them off into state-backed companies, which grew to do business across the world.

2029: Eventually, the Shackletons became tools of war – but not in the way people might expect. In 2029 the UN started to drop batches of improved Shackletons into contested borders and other flashpoints around the world – the mountains of east Afghanistan, jungles in South America, even, eventually, the Demilitarized Zone between South and North Korea. At first, locals would try to sabotage the Shackletons, but over time this ebbed. That was because the UN mandated that the software of the Shackletons be open and verifiable – all changes to the global Shackleton operating system were encoded in an auditable system based on blockchain technologies. They also mandated that the data the Shackletons generated be made completely open. Suddenly, militaries around the world were deluged in rich, real-world data about the locations of their foes – and their foes gained the same data in kind. Conflict ebbed, never quite disappearing, but seeming to decline to a lower level than before.

Some say the deployment of the Shackletons can be correlated to this decline of violence around the world. The theory is that war hinges on surprise, and all The Shackletons do is turn the unknown parts of the world into the known. It’s hard to be in a Prisoner’s Dilemma when everyone has correct information.

Technologies that inspired this story: Ethereum / Bitcoin, unsupervised auxiliary goal identification, Boston Dynamics, hierarchical temporal memory