Import AI

How Skydio made its NEURAL DRONE:
…Why Skydio built a ‘deep neural pilot’, and what this tells us about the maturity of deep RL research…
Drone startup Skydio has become quite notorious in recent years, publishing videos of its incredible flying machines that can follow, chase, film, and track athletes as they carry out performative workouts. Now, in a post on Medium, the company says it has recently been exploring using deep reinforcement learning techniques to teach its drones to move around the world, a symptom of how mature this part of AI research has become.

How can you make a neural pilot? Skydio has built some fairly complicated motion planning software for its drones, and initially the company tried to train a neural system off of this, via imitation learning. However, when they tried to do this they failed: “Especially within our domain of flying through the air, the exact choice of flight path is a weak signal because there can be many obstacle-free paths that lead to cinematic video,” they write. “The average scenario overwhelms the training signal”.

Computational Expert Imitation Learning: They develop an approach they call Computational Expert Imitation Learning (CEILing), where their drone learns not only from expert trajectories generated by the simulator, but also gets reward penalization according to the severity of errors made, which helps the drone efficiently learn how to avoid doing ruinous things like crashing into trees. However, they don’t publish enough information about the system to understand the specifics of the technical milestone – the more interesting thing is that they’re experimenting with a deep learning-based RL approach at all.
  “Although there is still much work to be done before the learned system will outperform our production system, we believe in pursuing leapfrog technologies,” they write. “Deep reinforcement learning techniques promise to let us improve our entire system in a data-driven way, which will lead to an even smarter autonomous flying camera”.

Why this matters: At some point, learning-based methods are going to exceed the performance of systems designed by hand. Once that happens, we’ll see a rapid proliferation of capabilities in consumer drones, like those made by Skydio. The fact companies like Skydio are already experimenting with these techniques in real world tests suggests the field of RL-based control is maturing rapidly, and may soon break out of the bounds of research into the real, physical world.
  Read more: Deep Neural Pilot on Skydio 2 (Medium).
  Watch a video about the Deep Neural Pilot on Skydio 2 (Hayk Martiros, YouTube).

####################################################

Turning Drones into vehicle surveillance tools, with the DroneVehicle dataset:
…All watched over by flying machines of loving grace…
Researchers with Tianjin University, China, have released a dataset of drone-collected overhead imagery. The DroneVehicle dataset is designed to help researchers develop AI systems that can autonomously analyze the world from the sorts of top-down footage taken via drones.

The dataset: DroneVehicle consists of 15,532 pairs of RGB and infrared images, captured by drone-mounted dual cameras in a variety of locations in Tianjin, China. The dataset includes annotations for 441,642 object instances across five categories: car, bus, truck, van, and freight car. The inclusion of infrared imagery is interesting – it’s rare to see this modality in datasets, and it could let researchers develop thermal-identifiers alongside visual identifiers. 

The DroneVehicle challenge: The challenge consists of two tasks: object detection, and object counting and is self-explanatory: try and identify any of the five categories of object in different images and, as a stretch goal, count the number of them.

Why this matters: One of the craziest aspects of recent AI advances is how they build on the past two decades of development and miniaturization of consumer electronics systems for sensing (think, the tech that underpins digital cameras and phone cameras) and motion (think, quadcopters). Now that deep learning approaches have matured, we can build software to utilize these sensors, letting us autonomously map and analyze the world around us – an omni-use capability, that yields new applications in surveillance (scary!) as well as more socially beneficial things (automated traffic and environment analysis, for instance).
  Read more: Drone Based RGBT Vehicle Detection and Counting: A Challenge (arXiv).

####################################################

Better language AI research via Jiant:
…Giant progress in NLP research requires a Jiant system to test the progress…
Jiant is a software wrapper that makes it trivial to implement various different experimental pipelines into the development of language models. The software depends on Facebook’s PyTorch deep learning software, as well as the AllenNLP and HuggingFace’s Transformers software libraries (which provide access to language models).

Why jiant is useful: jiant handles a couple of fiddly parts of a language model evaluation loop: first, users can define a given experiment via a simple config file (e.g., config = { input_module = “roberta-large-cased”, pretrain_tasks = “record,mnli”, target_tasks = “boolq, mnli”,), and also handles task and sentence encoding in the background. You can run jiant from the command line, so developers can integrate it into their usual workflow.

What jiant ships with: jiant supports more than 50 tasks today, ranging from natural language understanding tasks like CommonsenseQA, to SQuAD, to the Schema Challenge. It also ships with support for various modern sentence encoder models, like BERT, GPT-2, ALBERT, and so on.

Why this matters: In the past two years, research in natural language processing has been moved forward by the arrival of new, Transformer-based models that have yielded systems capable of generating human-grade synthetic text (for certain short lengths), as well as natural language understanding systems that are capable of performing more sophisticated feats of reasoning. Tools like jiant will make it easier to make this research reproducible by providing a common environment in which to run and replicate experiments. As with most software packages, the utility of jiant will ultimately come down to how many people use it – so give it a whirl!.
Read more: jiant: A Software Toolkit for Research on General-Purpose Text Understanding Models (arXiv).
  Find out more about jiant at the official website.
  Get the code for jiant here (jiant GitHub). 

####################################################

Microsoft thinks AI will change security, so it wants to pay researchers to help it figure out how:
…$300,000 in funding for better AI<>security research…
Microsoft is providing funding of up to $150,000 for projects that “spark new AI research that will expand our understanding of the enterprise, the threat landscape, and how to secure our customer’s assets in the face of increasingly sophisticated attacks,” Microsoft wrote. Microsoft has $300,000 in total funding available for the program, and “will also consider an additional award of Azure cloud computing credits if warranted by the research”.

What is Microsoft interested in? Microsoft is keen to look at research proposals in the following areas (non-exhaustive list):
– How can automatic modeling help enterprises autonomously understand their own security?
– How can we identify the risk to the confidentiality, integrity, and availability of ML models?
– How do we meaningfully interrogate ML systems under attack to ascertain the root cause of failure?
– Can we build AI-powered defensive and offensive agents that can stay ahead of adversary innovation?
– How can AI be used to increase the efficacy and agility of threat hunter teams?
– And so much more!

Why this matters? The intersection of AI and Security is going to be an exciting area with significant potential to alter the dynamics of both cybercrime and geopolitical conflict. What might it mean if AI technologies yield dramatically better systems for defending our IT infrastructure? What about if AI technologies yield things that can aid in offensive applications, like synthetic media, or perhaps RL systems for fine-tuning phishing emails against target populations. Grants like this will help generate information about this evolving landscape, letting us prepare for an exciting and slightly-more-chaotic future.
  Read more: Microsoft Security AI RFP (official Microsoft blog). 

####################################################

Could you understand Twitter better by analyzing 200 million tweets?
…$25,000 in prizes for algorithms that can understand Twitter…
How chaotic is the omni-babble service known as Twitter? A new competition aims to find this out, by challenging researchers to build systems that can predict how people will respond to tweets on the social network. The RecSys Challenge 2020 “focuses on a real-world task of tweet engagement prediction in a dynamic environment” and has $25,000 in prizes available – though the biggest prize may be getting a chance to work with a massive dataset of real-world tweets.

200 Million tweets: To support the competition, Twitter is releasing a dataset of 200 million public engagements on Twitter, spanning a period of two weeks, where an engagement is a moment when a user interacts with a tweet (e.g., like, reply, retweet, and retweet with comment). Twitter says this represents the “largest real-world dataset to predict user engagements”, and is likely going to be a major draw for researchers.

The challenge: Entrants will need to build systems that can correctly predict how different users will interact with different tweets – a tricky task, given the different types of possible interactions and the inherent broadness of subjects discussed on Twitter.

Why this matters: Do humans unwittingly create patterns at scale? Mostly, the answer is yes. Something I’m always curious about is the extent to which we create strong patterns via our own qualitative outputs (like tweets) and qualitative behaviors (like how we interact with tweets). I think challenges like this will highlight the extent to which human creativity (and how people interact with it) has predictable elements.
  Read more and register for the data: Twitter RecSys Challenge 2020 (official competition website).

####################################################

Do we need different publication rules for AI technology?
…Partnership on AI project tries to figure out what we need and why…
What happens if, in a few years, someone develops an AI technology with significant cyber-offense relevance and wants to publish a research paper on the subject – what actions should this person take to maximize the scientific benefit of their work while minimizing the potential for societal harm? That’s the kind of question a new project from the Partnership on AI (PAI) wants to answer. PAI is a multi-stakeholder group whose members range from technology developers like Microsoft, OpenAI, and DeepMind, to civil society groups, and others. Over 35 organizations have worked on the initiative so far.

What the project will do: “Through convenings with the full spectrum of the AI/ML research community, this project intends to explore the challenges and trade-offs in responsible publication to shape best practices in AI research,” PAI writes.

Key questions: Some of the main questions PAI hopes to answer with this project include:
– What can we learn from other fields dealing with high-stakes technology, and from history?
– How can we encourage researchers to think about risks of their work, as well as the benefits?
– How do we coordinate effectively as a community?

Key dates:
– March – June 2020: PAI will collect feedback on publication norms via a Google Form.
– June 2020: PAI will host a two-day workshop to discuss publication norms within AI research.
– Fall 2020: PAI will publish information based on its feedback and workshops.

Why this matters: Most fields of science have dealt with publication challenges, and some fields – particularly chemistry and materials science – have ended up exploring different types of publication as a consequence. Work like this from PAI will help us think about whether publication norms need to change in AI research and, if so, how.
  Read more: Publication Norms for Responsible AI (arXiv).

####################################################

Tech Tales:

[A virtual school playground, 2035]
Play, Fetch

My keeper once got so tired looking for me it fell out of the sky. I got in so. much. trouble!

My keeper said that it would protect me if my Dad started hitting my Mom again. It said it’d take pictures so my Dad wouldn’t be able to do that anymore. And it did.

My keeper once played catch with me for four hours when I was sad and it told me I did a good job.

My keeper got hit by a paintball when we were out in the park and I tried to fix it so I could keep playing. It told me I needed special equipment but I just grabbed a load of leaves and rubbed them on its eye till the paint came off and it was fine.

My keeper helped my family get safe when the riots started – it told us “come this way” and led us into one of the bunkers and it helped us lock the door.

My keeper once told me that another girl liked me and it knew that because her keeper told it, and it helped me write a valentine.

Things that inspired this story: Consumer robots; a prediction that most children will grow up with some kind of companion AI that initially does surveillance and later does other things; the normalization of technology; children as narrative canaries, as naive oracles, as seers.