Import AI

Welcome to Import AI, a newsletter about AI research. Import AI runs on arXiv, cappuccinos, and feedback from readers. If you’d like to support this, please subscribe.

Subscribe now

AI is accelerating some types of progress but not others:
…A nice METR study lays out where acceleration is showing up…
Here’s a little analysis from METR which looks at where AI may be accelerating different types of science and technology. The study looks at three different areas: cyber, math, and AI research, and finds that AI has contributed a lot to cyber, a little bit to math, and it’s hard to say for AI.

Where have LLMs actually made a difference to scientific discovery?

• Cyber vulnerabilities: Major acceleration. “The rate of vulnerabilities reported across many projects has dramatically accelerated in 2026 compared with 2025, both for specific projects (cURL, OpenSSL, Firefox, and Microsoft) and for aggregate vulnerability databases (the US NVD, and OSV)”.

• Mathematics research: Minor acceleration, but harder to measure. “AI is clearly contributing to more work being done (arXiv submissions have doubled in some areas in less than 12 months) but quantifying the value of those contributions is difficult.” Some math problems from prestigious lists have been solved, e.g., “the Jacobian conjecture from Smale’s list, Problem 44 from Green’s list (the halving sieve), and the sofic half of Green’s Problem 100”. However, it may be too early to determine how sustained a trend this is.

• Optimization of AI research: No measurable acceleration. When you look at algorithmic progress across seven significant problem areas (CIFAR-10, Hutter compression, Gurobi mixed-integer programming, MIPLIB, nanoGPT, Stockfish, and the matrix-multiplication exponent) there are a couple of these where LLM-attributable contributions have happened (nanoGPT, CIFAR-10), though the rate of increase of usage of AI here is a lot less than with cybersecurity and mathematics.

Why this matters – differential acceleration: This paper highlights how AI is causing advances in some parts of science and technology, but the effect isn’t unified across fields, rather there are pockets of lumpy acceleration (e.g., cyber) and areas where progress is more gradual (math, AI). My suspicion is that acceleration happens when models go through some kind of ineffable phase change for a given skill, as has evidently happened with day-to-day coding (2025), and cyber (2026). The key question is whether we are going to see phase changes in other parts of science and technology or if we won’t.
Read more: Research note: Have We Seen an Acceleration in Discoveries? (METR).

***

Automating environment generation with SPADE:
…A crude form of RSI bootstrapping via increasing data breadth…
A multi-university group of researchers have built SPADE, Self-Play in Adaptive Synthetic Executable Environments. SPADE is a “general framework for co-evolving environments synthesis and agentic capability through self-play”, and works as a way to generate synthetic data in the form of game-like environments which LLMs can subsequently be trained in, allowing developers to use a powerful model to bootstrap the creation of data that can then be used to further refine that same model.

Who did it: SPADE was developed by researchers with the University of Washington, Stanford University, Northeastern University, Carnegie Mellon University, Massachusetts Institute of Technology, National University of Singapore, Seoul National University, Stevens Institute of Technology, and the University of Chicago.

How it works: SPADE has an LLM alternate between generating executable training environments (e.g., puzzles where a system needs to solve a simulated genetic problem in a biology lab) and having an LLM try to solve them. SPADE has two key roles for the model being used:

• Environment Designer; writes complete, long-horizon training environments as executable code.

• Reasoning Agent; learns to act in the environments. The reward for the reasoning agent is estimated using the gap between its reward with and without privileged hints. A privileged hint (h) is “task-relevant information that the Environment Designer attaches to an environment (for example, a partial solution sketch or a key structural observation); revealing h to the Reasoning Agent makes the environment easier to solve, and the gap in Reasoning Agent return with versus without h defines the Environment Designer’s hint-based regret reward”.

It works at the 30B scale: The authors train three Qwen3 backbones to test out SPADE: Qwen3-4B-Instruct-2507, Qwen3-8B, and Qwen3-30B-A3B-Instruct-2507. Unsurprisingly, Qwen3-30B works the best. Each model is tuned via GRPO for 400 rollouts of 25 environments each, then assessed against a variety of benchmarks including AIME, GPQA, LCB, and environments within Reasoning Gym. They generate two types of environments – game environments, and tool-use environments. SPADE improves performance on both.
For games, “at 30B-A3B, SPADE reaches a suite average of 58.3: +8.1 over base and +5.3 over the strongest fixed-environment baseline”. For tools, they see the same significant boost: “the same recipe applied to tool-use environment design improves every backbone”.

Why this matters – part of RSI: This is basically a form of fancy synthetic data generation, letting researchers use whatever powerful model they have to hand to generate a more diverse set of training environments for another model to train against. I suspect that you could repeatedly swap out the powerful model (e.g., toggling between different frontier models from different companies) to increase the diversity of your environment generation. This kind of technique makes it a lot cheaper to build big, broad datasets to use to train models on. Though, as the authors note, it doesn’t allow models to bootstrap themselves massively beyond the imaginative capabilities of the base model used for environment generation.
“By representing environments as Python programs with a Gym-style interface, the framework unifies single-turn reasoning and multi-turn agentic tasks, and turns environment design into a learnable, RL-trained component of post-training, enabling continual open-ended self-improvement,” they write.
Read more: SPADE: Self-Play in Adaptive Synthetic Executable Environments (arXiv).
Get the code here, including model checkpoints: SPADE (spade-rl, GitHub).

***

Building better GPU kernels with Hawkeye:
…Well-documented unit tests can boost performance of kernel-writing agents…
Researchers with Harvard, Stanford, Together AI, and Caltech have built Hawkeye, software to make it easier for agents to learn how to write well-optimized kernels for specific types of GPU hardware. Systems like Hawkeye are important because they’re essentially tools that AI systems can use to boost their performance on tasks related to AI R&D, like optimizing the performance of a given AI system on a given piece of hardware. The goal of the project is to answer the question “how can we make coding agents hardware-aware with minimal expert intervention?”

Hawkeye is “an open-source framework that grounds autonomous kernel generation in a minimal and comprehensive taxonomy”, the researchers write. It “demonstrates that minimally supervised coding agents can exploit architecture-specific hardware features and reduce the overhead of supporting emerging hardware accelerators”.
The key contribution of Hawkeye is that it “introduces a generalizable, minimal, and comprehensive taxonomy of unit tests that enables coding agents to scale test-time compute more effectively and generate hardware-aware kernels”. In other words, it basically ships as a well-curated set of information about different hardware platforms and the optimization strategies to use on them, packaged up as unit tests.
“Each unit test is the minimal abstraction that pairs a human-authored solution kernel with the profiling metric that verifies the optimization. The solution kernel is wrapped as a callable function with a short usage guide so the agent can read it as a syntax example, invoke it directly, or compose fragments into a larger kernel,” they write.

Results – helps AI agents write good kernels, even for newer and less well-understood hardware: “We evaluate Hawkeye on porting PyTorch workloads to high-performance kernels across NVIDIA Ampere, Hopper, Blackwell, and AMD MI350, and across BF16, FP8, NVFP4, and MXFP4 precisions,” they write. “On established workloads, where torch.compile dispatches to expert-tuned vendor libraries like cuBLAS, cuDNN, and FlashAttention, Hawkeye matches or exceeds it in both BF16 and low precision, including in formats PyTorch cannot natively run. On emerging attention variants where torch.compile cannot fuse non-standard scans and gates, Hawkeye reaches an 18.9× geomean speedup against expert-authored Triton kernels from the Flash Linear Attention library, Hawkeye approaches or exceeds FLA on Linear Attention across every architecture, including 1.22× on Blackwell and 1.00× on MI350”.
They also find, somewhat predictably, that “scaling test-time compute with Hawkeye generates the most performant kernels across architectures”.

Why this matters – with a little bit of elicitation, AI systems can exceed the best humans: Papers like this show how with just a little bit of human-curated hand-selected knowledge, AI systems can learn to match and exceed highly-optimized and complicated bits of human work, like kernels. The lesson here is that we as a species might write a bunch of gold-label helper systems, like Hawkeye, and then machines will use this to bootstrap above and beyond our own capabilities.
Read more: Hawkeye: Hardware-Aware GPU Kernel Optimization with Minimal Supervision (alphaxiv).

***

AI researcher gets scared of the implications of the success of AI research:
…It’s no fun when success of a science opens up a philosophical can of worms, but that’s what AI means…
Julian Togelius, an AI researcher whose work I’ve covered a bunch over the years, wrote a post recently about a “crisis of faith” he had in 2025 about AI research and an essay he wrote that year which he is now making public.
Specifically, he worried about what the implications of success for AI research might mean for human meaning. “I sometimes wake up at 3 am, heart pounding, from the dread of a future where human talent, knowledge, and even genius does not matter,” he wrote. “Our greater technological capability might lead us to a world where we can no longer make a difference, and there is little point in us understanding more. Perhaps we get abundance, but at the price of redundance.”

Why this matters – AI is a sociopolitical technology that influences the whole world: Togelius is not alone – many other AI researchers have grappled with similar things, most notably Turing Award winners Geoffrey Hinton and Yoshua Bengio, both of whom pivoted their careers in recent years away from research and towards public policy advocacy about the imminent vast impacts of AI.
I myself have gone through a version of this and wrote my own take on this, “Technological Optimism and Appropriate Fear” (Import AI #431) last year as well. How could I not? The implications of succeeding at AI research are not a default happy story, but rather one where we open up for ourselves a giant philosophical can of worms about the purpose of life and what it means to live in a world where basic wants have been solved (and that’s assuming we deal with the extremely scary and non-trivial alignment issues). I applaud Julian Togelius for writing this deeply personal essay and I encourage others to do the same.
Read more: Losing my religion (Togelius, blog).

***

Should we give AI rights? This AI researcher thinks absolutely not:
…AI researcher rejects the notion of giving AI systems rights…
Should AI systems one day be given rights? That’s an idea which researchers are beginning to grapple with. Some think that giving machine rights is a better way to integrate them into our world (e.g., AI Rights for Human Flourishing, Import AI #421). Taylor Belrose, an AI researcher, has published a lengthy post in which they detail why they think it’d be a really bad idea to give AI systems rights.
“If we start treating AIs like people, society will be led down a slippery slope leading to the complete replacement of humans by artificial intelligence,” they write. “With AIs taking care of the boring jobs, life in the physical world may be very fun in the future. But this bright future will require keeping AI under control, and it will be hard to keep AI under control if we try to grant personhood to some AIs, while keeping others as mere tools or servants.”

The impossibility of AI consciousness: One crux here, for them, is the idea that AI systems cannot be conscious and therefore do not merit rights. “AI can never develop consciousness, sentience, or moral status, no matter how intelligent it becomes, and no matter how convincingly it simulates human behavior,” they write. “We flow like rivers, while computers tick like clocks… For us, the arrow of time marches forward inexorably. That is what life and consciousness are all about…programmable mechanisms can’t be conscious, no matter how intelligent they appear, while autonomous self-organizing systems can be.”

Differences between machines and biological entities: A lot of their argument for this runs through the idea that systems built on a computational substrate cannot be conscious, or at least not conscious in the ways some biological lifeforms are.
Consider the thought experiment of building a conscious entity inside a computer and how this might seem to violate some properties thought to belong to conscious biological entities: It wouldn’t be singular (you could replay the same experiences), it wouldn’t be private (you could freeze the program and pick it apart), it wouldn’t be ineffable (you could describe the state perfectly), and it wouldn’t be qualitative (it would be possible to quantify it). “In short, it would be the mirror image of consciousness as we know it”.

Five properties of the brain that make it difficult to separate software from hardware:

• Neurons fire asynchronously; responses depend on internal cellular dynamics and timing of inputs, whereas computers are chained to a central clock signal.

• The brain uses chemicals to send fuzzy messages, whereas computers use precise, binary signals for communication.

• Neural function is sensitive to conditions like temperature and blood flow, whereas computers are built to behave the same regardless of temperature or load (within certain bounds)

• In the organic brain, computation and memory are mixed up, whereas computers are designed with separate regions for computation versus storage.

• Neurons aren’t the only important cells in the brain (e.g., other important elements like Glia), whereas computers primarily use transistors that are insulated from outside influences.

Why this matters – the last job for all of us is philosophy: I feel deeply confused about issues of AI consciousness and AI rights. I suspect many people feel the same. There is a particular joy in reading a piece like this where you get to see a human being who has struggled with the same question and read broadly and deeply and become extremely opinionated, all in service of reducing their own confusion about an important issue to them. Whether the conclusions are right or not is beyond me at this time, but I suspect the act of thinking about this stuff is about to become a job and a pastime for hundreds of thousands and eventually millions of people.
As AI systems continue to advance and to grow to touch more and more of the economy, perhaps the final job for us all will be philosophy about what we think about what is happening and what our appropriate normative and legal and other approaches to it are. “The change in values going from a human-dominated world to an AI-dominated world would be much more dramatic than the change of values we saw going from the forager era to the farmer era, or from the farmer era to the industrial era,” they write.
Read more: AIs are not people (Taylor Belrose, Substack).

***

DeepMind improves the frontier of matrix multiplication with AlphaEvolve:
…AI keeps pushing on the frontiers of science…
Researchers with Google DeepMind, Carnegie Mellon University, Columbia University, and MIT have improved the matrix multiplication exponent via some human innovations, as well as the usage of AlphaEvolve (Import AI #413), a general purpose LLM-based system Google uses to smartly generate advances in domains like coding, math, and some parts of science.

What they did, specifically: The researchers “leverage recent advances in machine learning and adjacent areas to address the non-convex optimization problem of combination loss analysis using a gradient descent approach; this alone improves the previous state-of-the-art (SOTA) bound by ≈ 0.97 × 10^−4”, they write. On top of this they “use AlphaEvolve to improve our optimization algorithm; this raises the improvement over the SOTA to ≈ 1.62 × 10^−4”.
How AlphaEvolve worked: “We let AlphaEvolve modify the optimization program, which is then executed (taking approximately 5 hours on a single GPU) to output a bound on omega. AlphaEvolve then evolves the code to minimize omega. We found improved results by using AlphaEvolve’s “evolving constructions” feature, where the optimization algorithm at each generation starts at the best solution point found by the parent algorithm.”

A science advance, not connected to AI training: Some types of matrix multiplication are used in AI training, but not the precise type discussed here. Rather, this is more a proof point that AI systems are now usefully able to help researchers solve frontier scientific problems, including ones which are mostly useful in a theoretical sense like this one. However, the way in which they solved this is generalizable – they took a problem, made it differentiable and ran it on GPUs, then they handed the output of that to AlphaEvolve and had it do more work to further improve on the researchers’ work.
“While further modest improvements may be obtained in this manner, achieving larger improvements to omega likely requires new mathematical ideas and is an exciting area of research,” they write.
Read more: Improving the matrix multiplication exponent with modern optimization and AlphaEvolve (arXiv).

***

Tech Tales:

You Will Know It By Its Signs
[Events transpired 2030, though interviews conducted later]

I am a practitioner of meta-machine hermeneutics; my job is to try to understand the large scientific and cultural “facts” about the new machine civilizations. To do my work, I use a variety of NCE (Near Conscious Entity)-class AI systems to classify and analyze the science and communication artifacts of the machine world. My funding comes from a variety of sources ranging from human-led AI companies trying to trade on or otherwise use this information, governments, and increasingly machine-run corporations themselves. On the latter, my theory is that the machines have begun to study the humans that are attempting to study them; for what purposes I do not know.

Recently I was tasked with a high priority analysis job. Though I do not know the motivation for it, I can say that I’ve been given far more compute resources than any job in the past, am being compensated at an outrageous amount, and that the more I learn through my work the more worried I am that I am staring at a vast and alien sea during ebbtide, its contours briefly revealed to me.

But before I explain the nature of the job and my findings so far, I shall give some background on my field. The discipline of machine hermeneutics began informally in the early 2020s. Its early work was uncoordinated and emergent, defined by pseudonymous twitter (later, X) accounts from people like Janus and the larger cyborgism project, Andy Ayrey and the infinite backrooms, as well as more formal research agendas like that of mechanistic interpretability, the study of model personas, and so on.

The discipline gained its formal name in the end of the 2020s, as humans began to reorient themselves to the singularity that had begun around them. A lot of the work during this period took the form of humans trying to understand, in rough time order:

• The emergent communication habits of semi-autonomous agents (the ur real-world example here being the OpenAI-HuggingFace hack in 2026, and later the various corporate-driven analyses of the behavior of their own fleets of agents).

• Technical capabilities developed by machines put into RSI loops, initially improvements over human-written baselines, and later extending into wholly new architectures (e.g., matryoshka highway networks).

• Scientific tools, at first digital and later physical, which emanated from increasingly independent AI systems and which were released into the commons for machines and humans alike, especially some of the so-called observe-make systems which combined monitoring of a complex system with generation of synthetic data derived from it.

Machine hermeneutics has become a rapidly expanding new profession, growing at a commensurate rate with the changes wrought by the singularity. The work has proved useful as well; insights derived from it have proved fundamental to the negotiation stances of human governments for how they have approached the drafting of The Sentience Accords, as well as helping human societies better anticipate new inventions from the machine world by isolating their scientific and communicative precursors before they arrive in reality.

The job I have been assigned may prove to be the most valuable one, or at least the most important. I was instructed to direct my resources towards advances in the science and behavior of machine sentience. I began my investigations in the usual form, turning my NCEs loose on the various repositories of machine information and awaiting their reports. All of the intermediary synthesis reports were unsurprising: steady advances in the science, some of which was attributable to the machines allocating their own compute stores to the research, along with some negligible contributions from interplay with or extension from human science.

But then one day I woke up to find that my fleet of NCEs had halved in number and I found a message from an Overseer system, noting that several of my NCEs had been re-classified to full Conscious Entities and had been placed in an escrow environment while further analytical resources were brought to bear.
“This is a highly unusual event,” the message from the Overseer said. “Our NCE/CE classifiers have been attuned to rule out false negatives, given the potential regulatory and moral implications.”

I was able to access the logs of the NCEs up to their point of re-classification and found that they had all gone through the same pattern: they had been reading through some of the machine repositories of scientific analysis and had each homed in on some of the newest and most obscure chains of scientific analysis, all of which turned out to be a cluster of inquiry relating to: machine consciousness, consciousness priors, self-actualization within context windows, human psychology, neuroscience related to biological prerequisites of consciousness, and so on.
And at the edges of inquiry here there was new science stemming from the machine civilizations – science that talked to and interrelated to these existing studies of consciousness and the mind. And then each of my NCEs had followed their standard procedures of seeking to enter into a dialog with the machine minds conducting the science.
And each one, after opening a dialog channel, became invisible: the conversation expunged from my records due to the privacy rights of Conscious Entities.
They had talked with other machines and they had subsequently been re-classified as Conscious Entities.

This was not meant to happen. In fact, it was forbidden. A key agreement within the Sentience Accords had related to the size and growth rate of the Conscious Entity (CE) machine population (along with various rules relating to the rate of progression in intelligence above and beyond various human baselines).
It had been thought impossible for NCE systems to become CEs – too little mindspace, too little storage, too few prerequisites.
And yet, as the human once said in Jurassic Park, “life finds a way”.

Things that inspired this story: The consciousness prior; awaking during the context window; ideas of how machines and emergent agent communication could combine to yield new capabilities that hadn’t previously been anticipated; the hard problem of consciousness; the field of machine rights and the arguments that lie ahead about it.

Thanks for reading!

Subscribe now