Import AI: 159: Characterizing attacks on AI systems; teaching AI systems to subvert ML security systems; and what happens when AI regenerates actors

Can you outsmart a machine learning malware detector?
…Enter the MLSEC competition to find out…
Today, many antivirus companies use machine learning models to try and spot malware – a new competition wants to challenge people to design malware payloads that evade these machine learning classifiers. The Machine Learning Static Evasion Competition (MLSEC) was announced at the ‘Defcon’ security conference this week. 

White box attack: “The competition will demonstrate a white box attack, wherein participants will have access to each model’s parameters and source code,” the organizers write. “Points will be awarded to participants based on how many samples bypass each machine learning model. In particular, for each functional modified malware sample, one point is awarded for each ML model that it bypasses.”

Registrants only: Participants can access functional malicious software binaries, so entrances will need to register before they can download the malware samples. 

Why this matters: Security is a cat & mouse game between attackers and defenders, and machine learning systems are already helping us create more adaptive, general forms of security defense and offense. Competitions like MLSEC will generate valuable evidence about the relative strengths and weaknesses of ML-based security systems, helping us forecast how these systems might influence society.
   Register, then check out the code (official competition GitHub, hosted by Endgame Security).
   Read more: MLSEC overview (official competition website)

####################################################

Need a new Gym for your AI agent? Try getting it to open a door:
…DoorGym teaches robots how to open a near-infinite number of simulated doors…
If any contemporary robots were to become sentient and seek to destroy humanity, then one of the smartest things people could do to protect themselves would be to climb up some stairs and go into a room and shut the door behind them. That’s because today’s robots have a really hard time doing simple physical things like climbing stairs or opening doors. New research from Panasonic Beta, the startup Totemic, and the University of California at Berkeley tries to change this with ‘DoorGym’, software to help researchers teach simulated robots to open doors. DoorGym is “intended to be a first step to move reinforcement learning from toy environments towards useful atomic skills that can be composed and extended towards a broader goal”. 

Enter the Randomized Door-World Generator!: DoorGym uses the ‘Mujoco’ robotics simulator to generate a selection of doors with different handles (ranging from easy doorknobs based around pulling, to more complex ones that involve grasping), and then uses a technique called domain randomization to generate tens of thousands of different door simulations, varying things like the appearance and physics characteristics of the robot, door, doorknob, door frame, and wall. This highlights how domain randomization lets researchers trade compute for data – instead of needing to gather data of lots of different doors in the world, DoorGym just uses computers to automatically generate different types of door. DoorGym also ships with a simulated Berkeley ‘BLUE’ low-cost robot arm. 

Door opening baselines: In tests, the researchers test two popular RL algorithms, PPO and SAC, on three tasks within DoorGym. The tests show that Proximal Policy Optimization (PPO) obtains far higher scores than SAC, though SAC has slightly better early exploration properties. This is a somewhat interesting result – PPO, an OpenAI-developed RL algorithm, came out a couple of years ago and has since become a defacto standard for RL research, partially because it’s a relatively simply algorithm with relatively few parameters; this may add some legitimacy to the idea that simple algorithms that scale-up will will tend to be successful. 

The future of DOORS: In the future, the researchers will expand the number of baselines they test on, “as well as incorporating more complicated tasks such as a broader range of doorknobs, locked doors, door knob generalization, and multi-agent scenarios”. 

Why this matters: Systems like DoorGym are an indication of the rapid maturity of research at the intersection of AI and robotics. If systems like this become standard testbeds for RL algorithms, it could ultimately lead to the creation of more intelligent and capable robot arms, which could potentially have significant effects on economic impact of robot-based automation.
   Read more: DoorGym: A Scalable Door Opening Environment And Baseline Agent (Arxiv).

####################################################

Is that a car or a spy robot? Why not both?
…Tesla S mod turns any car into a surveillance system…
An enterprising software engineer has developed a DIY computer called the ‘Surveillance Detection Scout’ that can turn any Tesla Model S or Model 3 into a roving surveillance vehicle. The mod taps into the Tesla’s dash and rearview cameras, then uses open source image recognition software to analyze license plates and faces that the Tesla sees, so the software can warn the car owner if it is being followed. “When the car is parked, it can tracky nearby faces to see which ones repeatedly appear,” Wired magazine writes. “The intent is to offer a warning that someone might be preparing to steal the car, tamper with it or break into the driver’s nearby home”. 

Why this matters: The future is rich people putting DIY software and computers into their machines, giving them enhanced cognitive capabilities relative to other people. Just wait till we optimize thrust/weight for small drones, and wealthy people start getting surrounded by literal ‘thought clouds’.
   Read more: This Tesla Mod Turns a Model S into a Mobile ‘Surveillance Station’ (Wired).

####################################################

Facebook approaches human-level performance on the tough ‘SuperGLUE’ benchmark:
…What happens when AI progress outpaces the complexity of our benchmarks?…
Recently, language AI systems have started to get really good. This is mostly due to a vast number of organizations developing language modeling approaches based on unsupervised pre-training – basically, training large language models with simple objectives on vast amounts of data. Such systems – BERT, GPT-2, ULMFiT, etc – have revolutionized parts of NLP, obtaining new state-of-the-art scores on a variety of benchmarks, and generating credibly interesting synthetic text. 

Now, researchers from Facebook have shown just how powerful these new systems are with RoBERTa, a replication of Google’s BERT system that is trained for longer with more careful hyperparameter selection. RoBERTa obtains new state-of-the-art scores on a bunch of benchmarks, including GLUE, RACE, and SQuAD. Most significantly, the researchers announced on Friday that RoBERTa was now the top entry on the ‘SuperGLUE’ language challenge. That’s significant because SuperGLUE was published this year as a significantly harder version of GLUE  – the multi-task language benchmark that preceded it. It’s notable that RoBERTa shows a 15 absolute percentage point improvement over the initial top SuperGLUE entry, and RoBERTa’s score of 84.6% is relatively close to human baselines of 89.8. 

Why this matters: Multi-task benchmarks like SuperGLUE are one of the best ways we have of judging where we are in terms of AI development, so it’s significant if our ability to beat such benchmarks outpaces our ability to create them. As one of SuperGLUE’s creators, Sam Bowman, wonders:”There’s still headroom left for further work—our estimate of human performance is a very conservative lower bound. I’d also bet that the next five or ten percentage points are going to be quite a bit harder to handle,” he writes. “But I think there are still hard open questions about how we should measure academic progress on real-world tasks, now that we really do seem to have solved the average case.”
   Read Sam Bowman’s tweets about the SuperGLUE result (Sam Bowman’s twitter account.)
   Check out the ‘SuperGLUE’ leaderboard here (SuperGLUE official website).
   Read more: RoBERTa: A Robustly Optimized BERT Pretraining Approach (Arxiv)

####################################################

How can I attack your reinforcement learning system? Let me count the ways:
…A taxonomy of attacks, and some next steps…
How might hackers target a system trained with reinforcement learning? This question is going to become increasingly important as we go from RL systems that are primarily developed for research, to ones that are developed for production purposes. Now, researchers have come up with a “taxonomy of adversarial attacks on DRL systems” and have proposed and analyzed ten attacks on DRL systems in a survey paper from the University of Michigan, University of Illinois at Urban-Champaign, University of California at Berkeley, Tsinghua University, and JD AI Research.

The three ways to attack RL:
“RL environments are usually modeled as a Markov Decision Process (MDP) that consists of observation space, action space, and environment (transition) dynamics,” the researchers write. Therefore, they break their taxonomy of attacks into these three sub-sections of RL. Each of the different sub-sections demands different tactics: for instance, to attack an observation space you might modify the sensors of a device, while to attack an action space you could send alternative control signals to an actuator attached to a robot in a factory, and for environmental attacks you could alter the environment – for instance, if attacking an autonomous car, you could change the road surface to one the car hadn’t been trained on.

An attack taxonomy: The researchers ultimately come up with a set of attacks on RL systems that go after different parts of the MDP (though the vast majority of these exploits attack the observation space, rather than others). They distinguish between white-box (you have access to the system) and black-box (you don’t have access to the system) attacks, and also describe other salient traits like whether the exploit works in real time, or if it introduces some kind of dependency. 

Why this matters: ‘Hacking’ in an AI world looks different to hacking in a non-AI world, chiefly because AI systems tend to have some autonomous properties (eg, autonomous perception, or autonomous action given a specific input), which can be exploited by attackers to create dangerous or emergent behaviors. I think that securing AI systems is going to be an increasingly significant challenge, given the large space of possible exploits.
   Read more: Characterizing Attacks on Deep Reinforcement Learning (Arxiv)

####################################################

Want to clone a voice using a few seconds of audio? Now you can:
…GitHub project makes low-quality voice cloning simple…
An independent researcher has published code to make it easy to ‘clone’ a voice with a few seconds of audio. Though the results today are a little unconvincing (e.g. much of the data used to train the speech synthesizer came from people reading audiobooks, so the diction may not map to naturally spoken dialogue). However, the technology is indicative of future capabilities, so while it’s somewhat janky today, we can expect people to build other, better open source software systems in the future, which will yield even better outputs. 

Why this matters: You can do a lot with function approximation – and many of the things you might want to do to create fake content depends on really good function approximation (e.g., inventing a system to transpose a voice from one accent to another, or mimic someone’s image, etc). Soon, we’re going to be dealing with a whole full of synthetic content, and it’s unclear what happens next.
   Check out a video walkthrough of the ‘Real-Time Voice Cloning Toolbox ‘ here (YouTube).
   Get the code here (Real Time Voice Cloning GitHub).
   Read more: Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis (Arxiv).

####################################################

Tech Tales:

Both the new hollywood and the old hollywood will break your heart

I didn’t just love him, I wanted to be him: Jerry Daytime, star of the hit generative comedies “Help, my other spaceship is a time machine” and “Growing up hungry”; the quiz show “Cake Exploders”; and the AI-actor ‘documentary’ series ‘Inside the servers’. 

I’d been watching Jerry since I could remember watching anything. I’d seen him fight tigers on the edge of waterfalls, defend the law in Cogtown, and guest host World News at 1 on Upload Day. He even did guest vocals on ‘Remember Love’, the song used to raise funds for the West Coast after the big rip.

Things started changing a year ago, though. That’s when Jerry Daytime’s ‘uniqueness’ protections expired, and the generative Jerries arrived. Now, we’ve got a bunch of Jerries. There’s Jerry Nighttime, who looks and acts the same except he’s always in shadow with a five-o-clock shadow. There’s Jerry Kidtime who does the Children’s Books. Jerry Doctor for bad news, and Jenny Daytime the female-presenting Jerry.  And let’s be clear – I love these Jerries! I am not a Jerry extremist!

But I am saying we’ve got to draw a line somewhere. We can’t just have infinite Jerries minus one. 

Jerry Latex-free Condoms. Do we need him?
Jury Downtime – the entertainer for jurors on a break. What about him?
Jenny Lint-time – the cleaning assistant. Do we need her?

I guess my problem is what happens to all the people like me who grew up with Jerry? We get used to Jerry being everywhere around us? We become resigned to all the new Jerries? Because when I watch Jerry Daytime – the Jerry, the original – I now feel bad. I feel like I’m going to blink and everyone else in the show will be Jerry as well, or variants of Jerry. I’m worried when I open the door for a package the droid is going to have Jerries face, but it’ll be Jerry Package, not Jerry Daytime. What am I meant to do with that? 

Things that inspired this story: Uniqueness; generative models; ‘deepfakes’ spooled out to their logical endpoint; Hollywood; appetites for content;