Week of 2025-05-26 through 2025-06-01

Table of Contents

1. Anthropic Open Sources Circuit Tracing Library
2. Deepseek R1 0528
3. Open Source Robot Hardware
4. Transformer Based Rendering: RenderFormer
5. Interesting Datasets
   1. SynLogic
   2. Open R1 Mixture of Thoughts
6. Operation Spiders Web
7. Meta Privacy Policy via AI
8. Flock

• This weeks findings turned out quite broad, with a relatively small amount being focused on commercial LLMs and applications. Anthropic open sourced a hopefully influential library that may help the community better understand how LLMs 'think', absolutely crucial to humans staying meaningfully behind the wheel. Also in open source news, Deepseek's R1 got an update in R1 0528 while Huggingface is moving to open source a robotics kit to spur embodied AI development. To round out these encouraging developments, we have several stories that may pave the way to horrors beyond imagination; covert drone warfare, mass surveillance, and ethical decision making surrendered to AI. Enjoy!

Anthropic Open Sources Circuit Tracing Library

• https://www.anthropic.com/research/open-source-circuit-tracing
• https://github.com/safety-research/circuit-tracer
• https://transformer-circuits.pub/2025/attribution-graphs/biology.html
• https://www.neuronpedia.org/gemma-2-2b/graph
• https://www.darioamodei.com/post/the-urgency-of-interpretability
• Anthropic has for some time advocated for advancing the field of 'interpretability' for AI models, considering it key to ensuring AI is "helpful, harmless, and honest." A few months ago they published a fascinating article titled "On the Biology of a Large Language Model" where they identified discrete structures within an LLM's internal representation (sometimes called latent space) and could, to some extent, trace patterns in the execution, which they termed 'circuits.' Later, their CEO wrote a blog urging governments, companies, and academics to focus heavily on interpretability, lest AI capabilities outpace human understanding to the extent the models can't be meaningfully concerned.
• To their credit, they seem to have now released as open source, under the MIT license, the library they used to make those earlier findings. They appear to have included demo notebooks and documentation so it should, in theory, be easy for the community to jump in and see how their research applies to other models. The demonstrations they included are from Gemma 2, an older model from Google, and Llama 3.2 1B, a relatively recent and performant small model from Meta/Facebook; the smaller and simpler models are supposed to make the process of teasing out structure easier, and Anthropic themselves analyzed their smallest Haiku series.
• The impact of this will be interesting but potentially slow to play out. The way it's implemented essentially swaps out some of the model layers with more easily interpretable layers, and then the replacing layers are trained to provide as close as possible output to the original layers (technically, the multilayer perceptron, MLP, layers are replaced by cross layer transcoders, CLTs, which minimize the loss for mean squared error in prediction and a penalty for sparsity. So in some sense 'decompressing' the MLP latent space.) So in practical terms, applying this technique to other models requires per architecture adaptation and training to replicate the weights. They also mention the errors compounding, so it may be increasingly expensive or infeasible to apply this method to larger models.

Deepseek R1 0528

• https://huggingface.co/deepseek-ai/DeepSeek-R1-0528
• https://huggingface.co/deepseek-ai/DeepSeek-R1-0528-Qwen3-8B
• Deepseek released an update to their R1 reasoning model, creatively named 0528 after the date. This update improved usability, since thinking model use has converged to different conventions since the original, and significantly improved performance on the more advanced benchmarks, such as GPQA Diamond, MMLU Pro, AIME 2024, and LiveCodeBench (scientific reasoning, reasoning and general knowledge, competition math, and software engineering benchmarks respectively). The results generally put R1 0528 as going toe to toe with the latest generation of state of the art models, namely OpenAI o3 and Google Gemini 2.5 Pro. The release was relatively sparse in terms of methodology, however, and it's not clear what they did differently between now and the previous version; the architecture and parameter counts are (almost, 671B originally, 685B now) the same, so that mostly leaves either higher quality training data or more training data from platform interactions. The model weights in at 685B parameters and 700-800GB (depending on the quantization), and is MIT licensed.
• Like the previous release they included a distill variant, R1-0528-Qwen3-8B, which post trained Qwen 3 8B, itself a performant, but small, reasoning capable model, on R1 0528 reasoning traces. The results appear very good; on reasoning based tasks, as opposed to factual recall, it becomes competitive with models like Qwen3 235B-A22B, Phi 4 reasoning plus, Gemini Flash 2.5, and OpenAI o3-mini. This variant is only 8B parameters as the name suggests, only 16GB in its raw form, tool capabilites, and has a context length of 131k; quite capable for a local model on most consumer hardware

Open Source Robot Hardware

• https://arstechnica.com/ai/2025/05/hugging-face-hopes-to-bring-a-humanoid-robot-to-market-for-just-3000/
• https://x.com/RemiCadene/status/1928015436630634517
• While details are sparse, it appears that Huggingface, a popular open source AI hub and software company, is progressing on releasing an open source humanoid robot sometime in 2025 and targeting a price point of $3,000 for a kit. They have 2 models specifically, one is a complete humanoid structure with 66 degrees of freedom throughout all its joints and a more limited desk based something targeting a price point of $250-300 (tariff uncertainty seems a significant factor in price and release date uncertainty.) Currently, open source experimenting with 'embodied AI', that is AI systems that can perceive and interact with the world, is relatively limited due to the large up front investment and lack of standardized hardware. 3D printing faced a similar situation at one point, but the price points eventually declined as open and hackable hardware proliferated and price pressure increased. The scale of release and quality of the robots is so far unknown for this project, but it's worth keeping an eye on.

Transformer Based Rendering: RenderFormer

• https://microsoft.github.io/renderformer/
• https://arxiv.org/abs/2505.21925
• Machine learning techniques have been applied with varying degrees of success to traditional graphics pipelines, most successfully in AI upscaling and recently with 'neural shaders.' Microsoft research released some research that essentially brings the entire graphics pipeline into a transformer model; given 3d geometry in form of triangle locations and camera ray maps a pair of transformer components produce a rendered image without any traditional ray tracing or rasterization elements. Some previous attempts had focused on smaller portions, such as optimizing ray tracing or 'memorizing' specific scenes, or 'NeRFs', but the dual view dependent and view independent transformer modelling appears to have successfully generalized the entire graphics pipeline within an AI architecture.
• This isn't to say this method is superior or even practical; this example was limited to 4096 triangles, limited graphical effects, and isn't as accurate as raster/ray tracing. A particular obstacle is the reality that transformer performance is largely quadratic, and since resolution scaling is also quadratic (ie double dimensions from 1920 x 1080 to 3840 x 2160 is 4x the pixels) the scaling could be brutal. The applications are not clear, whether this could be applied to real time graphics or some sort of AI perception pipeline, but the generalization is noteworthy and something to keep tabs on; Microsoft probably didn't do this research for fun.

Interesting Datasets

SynLogic

- <https://huggingface.co/datasets/MiniMaxAI/SynLogic>
- <https://arxiv.org/abs/2505.19641>
- MiniMaxAI released a large set of diverse synthetic reasoning training data, with around 16,000 'easy' tasks and 33,000 'hard' tasks. They feature 35 different kinds of tasks, eg Sudoku, and can allegedly bump 32B and lower model performance by several percentage points with simple fine tune runs. A key detail is that these are synthetic and mechanically verifiable.

Open R1 Mixture of Thoughts

- <https://huggingface.co/datasets/open-r1/Mixture-of-Thoughts>
- Another reasoning dataset, this one is much larger but less focused, a product of Huggingface's Open R1 effort to reproduce the results and methodology of R1. Uniquely, the methodology is quite detailed, almost earning its own root article, so I'll point you there instead.

Operation Spiders Web

• https://www.nytimes.com/2025/06/01/world/europe/russia-ukraine-strikes.html
• https://www.theguardian.com/world/2025/jun/01/ukraine-launches-major-drone-attack-on-russian-bombers-security-official-says
• Most emerging technologies currently, while usually advertised as safe and consumer focused, have substantial dual use, that is civilian and military, potential. While the individual technologies aren't novel, their combination and application to an active war are noteworthy and precedent setting. June 1st 2025 Ukraine launched what it named 'Operation Spider's Web' where 117 commercial drones (specifically quad copter designs) carrying explosives emerged from shipping containers with hidden/false roofs to attack Russian air bases all over the country. Each was remotely controlled and are claimed to have destroyed or damaged 41 Russian strategic bombers (strategic bomber meaning capable of delivering nuclear payloads but more commonly launch cruise missiles), with at least 8 confirmed via open source intelligence.
• There are several aspects of this event that are unconfirmed or blatantly contradictory, such as some outlets claiming each drone had an individual remote pilot while others claim rudimentary AI targeting derived from 3d scans of museum pieces. I won't go into the military implications of this event, but the precedent for this use of technology is significant. The primary component is the incredible asymmetry this kind of strategy implies; it would be exceptionally difficult and expensive to comprehensively defend against hidden remote controlled, or even autonomous, weaponized commercial drones. While the components have existed for some time, there can hardly be more dramatic proof of concept than destroying a substantial portion (10-35%, confirmed vs claimed) of a country's airborne nuclear delivery capabilities (not including silo based ICBMs or submarine based SLBMs) to inspire state and non state actors alike.

Meta Privacy Policy via AI

• https://www.npr.org/2025/05/31/nx-s1-5407870/meta-ai-facebook-instagram-risks
• NPR obtained Meta/Facebook internal documents allegedly showing plans to automate up to 90% of their internal safety and privacy reviews. Following a 2012 settlement with the FTC, Meta/Facebook has been required to have a review process to evaluate privacy and safety implications of product changes or rollouts. Under this new system, the apparent goal is to allow product groups to get instant, authoritative approval for rollouts after filling out a form that is evaluated by an AI model. NPR's reporting suggests that human review will not be the default or mandatory, instead largely being up to the group working on that project/product. Responding to a request for comment, Meta/Facebook claimed they had spent billions of dollars to 'support' user privacy and that this new system would only be applied to 'low risk' decisions, whereas 'novel and complex' issues would receive some amount of human involvement. Who, or what, decides what is low risk or novel would seem to be up to the AI system and/or the product team.

Flock

• https://www.404media.co/ice-taps-into-nationwide-ai-enabled-camera-network-data-shows/
• https://www.404media.co/license-plate-reader-company-flock-is-building-a-massive-people-lookup-tool-leak-shows/
• Did you know that there is a network across over 6,600 federated networks/municipalities and over 77,000 cameras that continuously and automatically scan car license plates? Did you know civil servants can search that network to track individuals via car without a warrant? Did you know they're moving to tracking individuals directly? This network is a product called 'Flock', and is used nationwide by many municipalities and police departments. Now you know.