Published on 2025-05-19 00:00:00Z

Week of 2025-05-12 through 2025-05-18

By Human • 15 minutes read •

tags:

Table of Contents

Tech Notes 2025-05-12 through 2025-05-18

This week's selection of stories numbers only three, and all AI related, but each is either a major development or part of a major movement. AlphaEvolve, a Google Deepmind project, demonstrates use of LLMs in semi autonomous novel research with concrete, verifiable results; Prime Intellect Inc. and Nous Research deploy various techniques to allow decentralized AI training from different ends of the life cycle; and OpenAI gives their take on agent based coding. The Google and OpenAI stories demonstrate AI moving out of strictly generative roles and into autonomous systems while Prime Intellect Inc. and Nous Research provide a different avenue for AI development and use.
Looking forward a bit, this week was relatively dry in other emerging tech areas likely/in part due to the coming week (2025-05-19 through 2025-05-25) containing both Computex, a key computer hardware industry expo in Taiwan, and Google's I/O, their annual developer conference this year known to focused heavily on AI and XR (extended reality, includes virtual reality, augmented reality, mixed reality, etc).
As always, this is a relatively small selection of stories since the emerging tech landscape is too productive for anyone to keep up with the details of all of it. This week is a few deeper dives, next week I expect a larger number of shorter stories.

AlphaEvolve: Autonomous Research by AI

https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/
https://colab.research.google.com/github/google-deepmind/alphaevolve_results/blob/master/mathematical_results.ipynb
https://www.nature.com/articles/d41586-025-01523-z

Claims and Evidence

Probably the most publicized piece of news this week was Google's announcement of a system they call 'AlphaEvolve', which they describe as for "designing advanced algorithms." They come with hard proof, too; they provide a Colab notebook (an interactive coding and presentation tool) that demonstrates via observable code 16 novel more efficient matrix calculation operations and 13 more complex problems which essentially have results that refine the estimates of upper or lower bounds. In brief, it demonstrates legitimate advances in the field of math found by this system. That is not all it is claimed to have done, but that's what's in the notebook everyone can check. Beyond those examples, they claim it found material improvements to various components in Google's tech stack. Among these are an optimization to their compute orchestrator, Borg, that improves datacenter task scheduling and ultimately reclaims 0.7% of Google's global compute resources continously, an unspecified improvement to Google's custom machine learning processor (TPU, tensor processing unit) circuit design that has made it into the final product, and several improvements to key parts of the machine learning life cycle, resulting in a 1% reduction in training time for Gemini (their flagship proprietary LLM (Large Language Model)) and a 32.5% speedup to a key component of the LLM output process. While these numbers seem small, at the scale these are being deployed that represents hundreds of millions in USD.

What IS it?

Those results were collected over the course of the last year they've had this system running. The architecture and nature of the system is quite opaque otherwise, only inferable via 6 lines of pseudocode, ablation testing (where they remove components to demonstrate their individual and collective contributions), and the few concrete details they directly tell us. What they call 'AlphaEvolve' is in fact a system of at least four 'agents' operated governed by a control loop and directed by a human. The four components consist of a prompt sampler, which samples prior cycles' attempts as 'inspirations' and combines them with the current iteration of the program to generate this cycle's prompts, an ensemble of LLMs that process the prompt and generate proposed patches to the current iteration of the program, a set of evaluators that run the programs and assign scores for various metrics to determine how 'good' those patches and the resulting programs are, and then a database, which stores the prompts and modified programs attempted so far. The practical goal of this control loop is to 'evolve' prior attempts methodically and autonomously within this framework.
While that high level overview may make it sound simple, there are several reasons this research is noteworthy. Primarily, this demonstrates the use of current generations of LLMs to create genuinely new knowledge as opposed to retreading old ground. This has been a contentious point considering LLMs are fundamentally a product of the patterns of their training material. There are several secondary points, such as this system comprising of (presumably, mostly) off the shelf LLMs, namely Google Gemini 2.0 (their at the time flagship LLM family), bearing conceptual similarity to the 'singularity' idea, wherein it's theorized AI systems could become radically self improving at some point, or the discoveries themselves being significant.

Caveats and Nuances

There are major caveats and things to keep in mind when contextualizing this, however. These systems ran for at least a year and the amount of resources consumed is a detail conspicuously absent. Where this falls on the spectrum between superhuman resource efficiency and unfathomable brute force is, from the outside, unknown. That links into the nature of the problems they publicly released proof of: they're iterative improvements to established techniques or boundaries, not entirely novel in process. The evaluation loop also requires that the problem being investigated be correctly represented in code and be objectively measured. For example, reducing the number of steps in multiplying matrices is readily represented and counting steps is a clear metric. Making discoveries that change the landscape being worked within are therefore inherently difficult to evaluate in that framework.
None the less, this is a major development that is being visibly mirrored in open source projects, proprietary competitors, and by Google themselves previously (AI co-scientist and FunSearch) and is likely to become increasingly relevant. The idea has likely been driving specific design decisions of LLMs, for example the massive 1 million token contexts of Google Gemini 2.5 or 10 million token context of Llama 4 Scout. This links into more speculative ideas that AI systems are likely to become geopolitically relevant, sometimes articulated as 'a country full of geniuses in a datacenter' within a few years. In other words, that these systems will become advanced enough that they'll either serve as force multipliers for the researchers of a given country or be capable of independent research themselves, either way increasing the total scientific output of the host nation. Or even more to the point, while AlphaEvolve heavily emphasizes humans directing the control loop, there's no reason more layers couldn't fit on top, an 'Alpha-AlphaEvolve', so to speak.

Decentralized AI development

Prime Intellect Inc. Intellect 2
- https://huggingface.co/PrimeIntellect/INTELLECT-2
- https://storage.googleapis.com/public-technical-paper/INTELLECT_2_Technical_Report.pdf
Psyche Nous Research
- https://nousresearch.com/nous-psyche/
- https://psyche.network/runs/consilience-40b-1/0

Background

Opposite from large, centralized projects there were major developments for democratized and decentralized AI, with Prime Intellect Inc. publishing their decentralized post training of Intellect 2 and Nous Research launching their decentralized training network, Psyche, and beginning the pretraining of the first model on it, Consilience. Some background about how AI models, specifically large models like leading LLMs or autonomous vehicle models, are trained should help contextualize the following. Llama 4 models, Meta/Facebook's latest LLMs, were trained in a data center (or multiple) with more than 100,000 GPUs and thousands of terabytes of collective memory. Critically, part of the concept of neural networks is that they 'discover' long distance relationships in the training data, but to do that all the relevant data and parts of the model (traditionally) need loaded into memory and mutually accessible. Bandwidth and latency (among other factors) then become critical bottlenecks; Meta's datacenters, while training Llama 4, averaged an effective utilization of the GPUs around 20% (390 realized versus 1979 theoretical FP8 TFLOPS (8 bit floating point tera/10^12 operations per second) per H100 (Nvidia's at the time flagship) GPU), and that was meant to be a strong result. Consequently, the idea of decentralizing machine learning training across data centers, let alone more distant compute nodes, is impractical using traditional methods.
The aforementioned developments address decentralization of AI training from different ends of the lifecycle: Prime Intellect Inc.'s Intellect 2 addressing post training/fine tuning and Nous Research's Psyche/Consilience addressing pretraining. Here, pretraining means the process of taking a massive body (corpus) of training text, often measured in trillions of tokens, and deriving a 'base model' which embodies the patterns contained, both syntax and later semantics; effectively embedding the 'world view' collectively represented. Post training/fine tuning is the process of taking that raw base model and making it practically useful, for example formatting responses according to human taste (often a chat dialogue now), having a specific tone, or emphasizing specific knowledge. There are a lot of complexities and the terms are used differently in some contexts, however. Both Intellect 2 and Psyche use/d a variety of novel techniques, the details of which I won't elaborate on, to most efficiently break up tasks, communicate them, and integrate the results to effectively decentralize their chosen parts of the cycle.

Intellect 2: Post Training

Intellect 2 is finished, and was a fine tuning effort to increase the reasoning, math, and science capabilities of the existing open source Qwen QwQ 32B model, itself already strong in those domains. The final model shows minor but observable performance improvements in AIME24 and AIME 25 (a human math competition that's now used as a benchmark), LiveCodeBench (programming benchmark), and GPQA Diamond ('graduate level Google Proof Q&A'; a scientific reasoning benchmark). The accompanying paper shows in detail their methodology, with most if not all of their code being open source and reproducible, and several ablation experiments show the necessity and efficacy of the various techniques they employed. The results are minor, but demonstrate proof of concept.

Psyche: Pre Training

To clarify, the aforementioned Psyche/Consilience describes Nous Researche's Psyche, the decentralized training architecture and platform, and Consilience, the name of the first model being trained on it. They similarly use many techniques to mitigate the inherent difficulties in decentralized pretraining. At time of writing it's only been a few days and 0.5% progress so there are no results to speak of beyond the system continuing to function. If one is willing to be optimistic, there are signs it's actually working; the 'loss' function (the most basic indicator that a machine learning system is in fact 'learning'), has continually decreased and Hellaswag (one of the most basic benchmarks that shows basic linguistic common sense by predicting the next word in a sentence) has steadily increased beyond random guessing.

Caveats, Challenges, Questions

Theoretically, if these techniques work at scale and can be integrated, an entirely decentralized model pipeline becomes possible. That's a major if, however, and there are many caveats and problems along the way. Chief among these is the exact definition of 'decentralized'. While there are philosophical arguments to be made as to who controls and directs these compute networks, there are practical barriers for would be contributors in the form of resource requirements. Intellect 2 had the lowest barrier, with the most basic of the tasks capable of being ran on hardware costing around $5,000 (generously, assuming used RTX 3090 x 4). The majority of tasks for Intellect 2 required datacenter grade clusters and all tasks for Consilience (currently, may not be a hard requirement) use H100x8 clusters which cost at least $300,000. Beyond that, there are then major elephants in the room for both. For Intellect 2, it was essentially a proof of concept that resulted in barely observable performance improvements on already production ready models. For Consilience, at this rate the model won't be finished training for at least 2.5 years (although every time I've checked back the number has dropped, my first check placed it at 5 years).
Despite the caveats and difficulties, the existence and apparent proof of concept for both of these demonstrate a real potential for decentralized and open source models to be a presence in the AI ecosystem alongside monolithic datacenters costing into the tens and hundreds of billions of US dollars. Other decentralized computing projects, such as the Folding at Home or SETI at Home series, have yielded, and continue to yield, positive and practical results but the path forward with decentralized AI remains for now an open question.

OpenAI Codex

https://openai.com/index/introducing-codex/
On a more practical note, OpenAI released an agent based coding tool called Codex. It bears conceptual similarities to existing agent based coding tools, ie you give it instructions and it does the coding and work for you in the background (some call it 'vibe coding'), but has practical points of differentiation compared to others. Most current tools, eg Cursor or Jetbrains's Junie, work as parts of a local editor and so operations happen locally. Codex is however a cloud based system; it works within a virtualized environment in the cloud where it pulls in your code and an LLM executes tasks. The LLM used is a specially tuned variant of OpenAI's flagship o3 reasoning model, itself dubbed codex-1. Altogether, it's a different experience from other agent coding environments wherein you kick off jobs and tasks and Codex handles them asynchronously while you do other things. Crucially, it's capable of multitasking both within a given task and between tasks, again a point of differentiation from existing agent coding tools. This asynchronous task based behavior bears some resemblance to the prior Google AlphaEvolve story and no doubt features its own analogous control loop, just this time more focused on practical tasks. I can't comment on how it feels to use, as it's currently only available on OpenAI's $200/month ChatGPT Pro or Enterprise subscriptions.
(So to disambiguate all the things called codex, and for fun: OpenAI released four models in a 'Codex' family 2021 that were GPT-3 fine tunes, a CLI tool called Codex CLI in April 2025, this cloud based agent coding system in May 2025, another coding model called codex-1 that's a o3 fine tune, alongside which came codex-mini-latest that is an o4-mini fine tune that was released specifically for Codex CLI but is also available via API).

Glossary

LLM: Large Language Model, exemplified by Anthropic Claude 3, OpenAI GPT 4, Llama 3/4, and typically, are made from large bodies of data and generate responses to prompts.
TPU: Tensor Processing Unit, Google's terminology for their custom AI compute accelerators.
Token/tokens: LLMs (typically) don't work with human alphabets or vocabulary directly. For practical reasons, words, syllables, phonemes, and other structures are replaced by numbers. Consequently, words are on average made up of more than 1 token, although this depends on the specific methodology. Try a demo here.
Corpus: General term to encompass the training data a base model will be trained on. Often contains scraped internet data, books, academic papers, and more. Recent models include visual data as well.
Base model: Base models for LLMs are essentially a raw embodiment of the syntactic patterns, semantic meanings, and, at the highest levels, 'world view' the training data embodies. These are generally not practically useful, and can only 'complete' sentences in probable ways without respect to the desired tone, style, and base in reality.
Fine-tuning: Makes a base model practically useful by fixing the style and facts according to human preferences. The most common, or rather visible, example is 'instruct' tuning which allows the common 'chat' back and forth exchange. Other fine tuning aims include adopting a specific style, making responses useful, avoiding offensive or sensitive output, or adding domain specific knowledge/technique.
Loss function: Technically present at many levels during training, it generally means an evaluation score that the optimization is 'trying' to minimize. It's effectively the most basic indicator of training progress.
TFLOPS: or Tera, 10^12, FLOPS, floating point operations per second, is a basic measure of number of calculations per second a system can do on decimal numbers. It's a rough measure and not always directly comparable to other devices or operation types.
FP8: Short for floating point 8 bit, or a decimal number represented by 8 bits. Increasingly used in machine learning as it can be more efficiently operated on and stored compared to 16 or 32 bit decimal representations.
H100: Nvidia's flagship AI focused GPU released in 2022. Cost between $20,000-$ 40,000 per card, often clustered in groups of 8 for a single server.
AIME: Short for American Invitational Mathematics Examination, originally a human competitive math exam but has also been used to evaluate LLM reasoning skills.
LiveCodeBench: A continually updated benchmark that tests LLM coding skills.
GPQA: Google Proof Question Answering, a 'graduate level' scientific reasoning benchmark.
Hellaswag: A 'common sense' language benchmark that evaluates basic linguistic skills by predicting the next word in a sentence.