Import AI

DeepMind’s Memory Task Suite makes it easier to build agents that remember:
…To live is to remember and to remember is to live within a memory…
Memory is mysterious, important, and frustratingly hard to build into AI systems. For the past few years, researchers have been experimenting with ways of adding memory to machine learning-based systems, and they’ve messed around with components like separate differentiable storage sub-systems, external structured knowledge bases, and sometimes cocktails of curricula to structure the data and/or environments the agent gets fed during training, so it develops memory capabilities. More recently, people have been using attention mechanisms via widely-applied things like transformers to supplement for memory, by instead having systems that can be primed with lengthy inputs (e.g., entering 1000 characters of text into a GPT-2 model), and the system then ends up using attentional mechanisms to perform things that require some memory capabilities.

How do we expect to build memory systems in the future? Who knows. But AI research company DeepMind thinks the key is to develop sufficiently hard testing suites that help it understand the drawbacks of existing systems, and let it develop new ones against tasks that definitely requie sophisticated memory. To that end, it has released the DeepMind Memory Task Suite, a collection of 13 diverse machine-learning tasks that require memory to solve. Eight of the tasks are based in the Unity game engine, and the remaining ones are based on PsychLab, a testing sub-environment of DeepMind Lab.
  Get the code for the tasks here: DeepMind Memory Task Suite (DeepMind GitHub).
  Read the background research: Generalization of Reinforcement Learners with Working and Episodic Memory (Arxiv).

####################################################

Google has photographed 10 million miles of planet Earth:
Google’s “Street View” vehicles have now photographed more than 10 million miles of imagery worldwide, and Google’s satellites have covered around 36 million square miles of satellite imagery.

50% of the world’s roads: The world has about 20 million miles of roads, according to the CIA’s World Factbook. Let’s assume this estimate lowballs things a bit and hasn’t been updated in a while, and lets also add in a big chunk of dirt paths and other non-traditional roads, since we know (some) Google Street View vehicles go there… call it an extra 10 million? Therefore, Google has (potentially) photographed a third of the roads in the world, ish.

Why this matters: Along with plugging into Google’s various mapping services, the Street View imagery is also a profoundly valuable source of data for the training of supervized and unsupervised machine learning systems. Given recent progress in unsupervised machine learning approaches to image recognition, we can expect the Street View data to become an increasingly valuable blob of Google-gathered data, and I’m especially curious to see what happens when people start training large-scale generative models against such data, and the inevitable creations of imaginary cities and imaginary roads that’ll follow.
   Read more: Google Maps has now photographed 10 million miles in Street View (CNET).

####################################################

FAA makes it easier to create DRONEFORESTS:
The FAA has given tree-planting startup DroneSeed the permission to operate drones beyond visual line of sight (BVLOS) in forested and post-forest fire areas. “The last numbers released in 2018 show more than twelve hundred BVLOS exemption applications have been submitted to the FAA by commercial drone operators and 99% have failed to be approved,” DroneSeed wrote in a press release. The company “currently operates with up to five aircraft simultaneously, each capable of delivering up to 57 lbs. Per flight of payload. Payloads dropped are “pucks” containing seeds, fertilizers and other amendments designed to boost seed survival.”

Why this matters: DroneSeed is a startup that I hope we see many more of, as it is using modern technology (drones and a little bit of clever software) to work on a problem of social importance (forest maintenance and growth).
  Read more: FAA Approves Rare Permit to Replant After Wildfires (PR Newswire).

####################################################

Google expands BERT-based search to 70+ languages:
In October, Google announced it had integrated a BERT-based models into its search engine to improve how it responds to user queries (Import AI: 170). That was a big deal at the time as it demonstrated how rapidly AI techniques can go from research into production (in the case of BERT, the timetable was a ~year ish, which is astonishingly fast). Now, Google is rolling out BERT-infused search to 70 languages, including Afrikaans, Icelandic, Vietnamese, and more.

Why this matters: Some AI models are being used in a ‘train once, run anywhere’ mode, where companies like Google will do large-scale pre-training on vast datasets, then use these pre-trained models to improve a multitude of services and/or finetune against specific services. This also stresses the significant impact we’re starting to see NLP advances have in the real world; if the mid-2010s were about the emergence and maturation of computer vision, then the early 2020s will likely be about the maturation of language-oriented AI systems. (Mid-2020s… I’d hazard a guess at robots, if hardware reliability improves enough).
  Read more: Google announces expansion of BERT to 70+ languages (Google ‘SearchLiaison’ twitter account). 

####################################################

Drones learn search-and-rescue in simulation:
…Weather and terrain perturbation + AirSim + tweaked DDQN =
Researchers with Durham University, Aalborg University, Newcastle University and Edinburgh University are trying to build AI systems for helping search and rescue drones navigate to predefined targets in cluttered or distracting environments. Preliminary research from them shows it’s possible to train drones to perform well in simulated* environments, and that these drones can generalize to unseen environments and weather patterns. The most interesting part of this research is that the drones can learn in real-time, so they can be trained in simulation on huge amounts of data, then update themselves in reality on small samples of information.
  “This is the first approach to autonomous flight and exploration under the forest canopy that harnesses the advantages of Deep Reinforcement Learning (DRL) to continuously learn new features during the flight, allowing adaptability to unseen domains and varying weather conditions that culminate in low visibility,” they write. They train the system in Microsoft’s ‘AirSim’ drone simulator (Import AI: 30), which has always been used to train drones to spot hazardous materials (Import AI: 111).

Maps and reasoning: The UAV tries to figure out where it is and what it is doing by using two maps to help it navigate: a small 10X10 meter one, which it uses to learn how to navigate around its local environment, and a large map of arbitrary size which the small one is a subset of. This is a handy trick, as it means “regardless of the size of the search area, the navigational complexity handled by the model will remain the same”, they write.

Two algorithms, one with slight tweaks: In tests, the researchers show that Deep Recurrent Q-Learning for Partially Observable MDPs (DRQN, first published in 2015) has the best overall performance, while their algorithm, Extended Dueling Double Deep-Q Networks (EDDQN) has marginally better performance in domains with lots of variation in weather. Specifically, EDDQN fiddles with the way Q-value assignments work during training, so that their algorithm is less sensitive to variations in the small sample amounts they can expect their drones to collect when they’re being trained in-flight. “Although training and initial tests are performed inside the same forest environment, during testing, we extend the flight to outside the primary training area”, they write.

Why this matters: Eventually, drones are going to be wildly used to analyze and surveil the entire world for as many purposes as you can think of. If that happens with today’s technologies, then we can also expect to see a ton of domain-specific use-cases, like search and rescue, which will likely lead to the development of specialized sub-modules within systems like AirSim to simulate increasingly weird situations. It’s likely that in a few years we’ll be tracking progress in this domain via a combination of algorithmic advances, and keeping a log of the different environments the advances are working in.
  Read more: Online Deep Reinforcement Learning for Autonomous UAV Navigation and Exploration of Outdoor Environments (Arxiv).

####################################################

Tech Tales:

Invisible Cities Commit Real Crimes
[Court, USA, 2030]

“We are not liable for the outcomes of the unanticipated behavior. As the documents provided to the court show, we clearly specify the environmental limits of our simulation, and explicitly provide no guarantees for behavior outside them.”
“And these documents – are they easy to find? Have you made every effort to inform the client of their existence?”
“Yes, and we also did some consulting work for them, where we walked them through the simulation and its parameters and clearly explained the limitations.”
“Limitations such as?”
“We have IP concerns”
“This is a sealed deposition. You can go ahead”
“OK. Limitations might include the number of objects that could be active in the simulation at any one time. An object can be a person or a machine or even static objects that become active – a building isn’t active, but a building with broken windows is active, if you see what I mean?”
“I do. And what precisely did your client do with the simulator.”
“They simulated a riot, so they could train some crowd control drones to respond to some riots they anticipated”
“And did it work?”
“Define work.”
“Were they able to train an AI system to complete the task in a way that satisfied them?”
“Yes. They conducted in-depth training across several hundred different environments with different perturbations of crowd volume, level of damage, police violence, number of drones, and so on. At the end, they had developed an agent which could generalize to what they termed New Riot Environments. They tested the agent in a variety of further simulations, then did some real-world pilots, and it satisfied both our and their testing and evaluation methodologies.”
“Then how do you explain what happened?”
“Intention.”
“What do you mean? Can you expand.”
“What we do is expensive. It’s mostly governments. And if it’s not a government, we make sure we, how shall I put this… calibrate our engagement in such a way it’d make sense to the local government or governments. This is cutting-edge technology. So what happened…. we don’t know how we could have anticipated it.”
“You couldn’t anticipate the protestors using their own AI drones?”
“Of course we model drones in our simulator – quite smart ones. They trained against this.”
“It’d be better if you could stop being evasive and give me an expansive answer. What happened?”
“The protestors have a patron, we think. Someone with access to extremely large amounts of capital and, most importantly, computational resources. Basically, whoever trained the protest drone, dumped enough resources into training it that it was almost as smart as our drone. Usually, protest drones are like proverbial ants to us. This was more like a peer.”
“And so your system broke?”
“It took pre-programmed actions designed to deal with an immediate threat – our clients demand that we not proliferate hardware.”
“Say it again but like a human.”
“It blew itself up.”
“Why did it do that?”
“Because the other drone got too close, and it had exhausted evasive options. As mentioned, the other drone was more capable than our simulation had anticipated.”
“And where did your drone blow itself up?”
“Due to interactions with the other drone, the drone detonated at approximately ground-level, in the crowd of protestors.”
“Thank you. We’ll continue the deposition after lunch.”

Things that inspired this story: Sim2Real transfer; drone simulators such as AirSim; computer games; computer versus computer as the 21st C equivalent of “pool of capital versus pool of capital”.