Import AI

Welcome to Import AI, a newsletter about artificial intelligence. Subscribe here.

No code neural networks: Another year brings new companies trying to let people build neural networks without having to do any programming. This time it is Aetros which has an online drag-and-drop interface people can use. It’s got a nice Bladerunner-meets-Aliens-meets-Ikea aesthetic. However, if you’re knowledgeable enough to be able to specify the fine-tuned settings and architecture, then you might prefer the precision of writing code rather than manipulating a GUI.

The secret history of Dropout: Dropout is to neural networks as fat is to cooking; it improves pretty much everything. The technique helps guard against overfitting, which is when your neural network has learned some patterns peculiar to its training data, and hasn’t learned the larger patterns present in previously unseen data. Dropout was invented by a bunch of people at the University of Toronto including Geoff Hinton, who was inspired by the tedium of queuing in a bank. “I went to my bank. The tellers kept changing and I asked one of them why. He said he didn’t know but they got moved around a lot. I figured it must be because it would require cooperation between employees to successfully defraud the bank. This made me realize that randomly removing a different subset of neurons on each example would prevent conspiracies and thus reduce overfitting,” said Hinton, in aGoogle Brain AMA on Reddit.

Regulate this: The legal profession may want to regulate companies that build its AI systems, says Wendy Wen Yun Chang, a member of the American Bar Association’s Standing Committee on Ethics and Professional Responsibility. “Lawyers must understand the technology that they are using to assure themselves they are doing so in a way that complies with their ethical obligations,” she writes. “The industry is moving along without us. Very quickly. We must act, or we will be left behind.” Some of the issues she talks about could be solved by making AI software more interpretable, as a lot of her concerns stem from the black box nature of most  AI software.

OK computer, tell me why you did that? one of the perpetual concerns people have about AI systems is their inscrutability. It’s hard to figure out exactly why a neural network has classified such-and-such a thing in such-and-such a way. But is this that big of a deal? “I think interpretability is important, but I don’t think it should slow down the adoption of machine learning. People are not very interpretable either, because we don’t really know what our brains are doing. There is a lot of evidence in the psychology literature that the reasons we give for why we decided to do things are not the real reason,” saysOpenAI’s Ian Goodfellow in a Quora AMA. Some European data protection regulations already look to be on a collision course with oblique AI.

Synthetic Pokemon: In the past couple of years we’ve worked out how to get AI tools to generate synthetic images. Researchers have since published papers and released open source code. Now people are using these techniques to generate new Pokemon.That’s a step up from some of the humble beginnings of this approach, like the creation of imaginary toilets in imaginary fields.

Care for some Keras? François Chollet, the creator of the Keras deep learning framework, Is doing a Quora AMA on Monday. Keras makes it trivial to design neural networks and is relatively easy to pick up compared to other frameworks. Self-driving startup Comma.ai – as mentioned in last week’s newsletter – is built partially on Keras.

Brainy, gloopy categorization: modern neural networks bear as much resemblance to the neurons in our heads as wind-up dolls do to living things. So it’s worth keeping an eye on other techniques that draw a bit more from biology. Researchers at the University of Pennsylvania and Michigan State University recently published a paper on ‘evolution of active categorical image classification via saccadic eye movement’. This system is able to scan a small part of an image and correctly guess what it is about three quarters of the time, when run on the standard (and basic) MNIST handwritten digit dataset. It can also do this without having to look at all of the image, instead it starts in a random position, expands until it finds something that looks like the image it’s looking for, then scans the rest of the nearby pixels from there. This is promising because of the greater efficiency but its performance is nowhere near state of the art. It’s.similar to the ‘attention’ techniques used in neural networks, though the implementation is different. Keep your eyes peeled for more links between biology and AI.

Thanks for reading. If you have suggestions, comments or other thoughts you can reach me at jack@jack-clark.net or tweet at me@jackclarksf