https://notes.rcr.pub Zola en Wed, 10 Sep 2025 23:56:27 +0000 Wed, 10 Sep 2025 23:56:27 +0000 Human https://notes.rcr.pub/posts/2023-era-ai-tutor-study-in-nature/ https://notes.rcr.pub/posts/2023-era-ai-tutor-study-in-nature/ <p>A recurring problem with academic AI studies has been the severe lag time. At best, articles seem to have been published with one or two year lags, and since the AI state of the art has been moving on a monthly basis many results have been largely irrelevant. GPT 4 released on 2023-03-14, and now is finally showing up in peer reviewed journals. Published in Nature 2025-06-03 was an article from Harvard researchers studying the efficacy of a particular AI tutor chatbot in comparison to recent evidence based 'active learning' classroom techniques. In short, they found extremely strong evidence that their use of the AI tutor substantially improved the quality of the education while being more engaging to the learner.</p> <p>The authors don't attempt to overstate their findings however, and qualifying what exactly they claim is necessary to avoid misrepresentation by omission. The study was done on the Harvard course "Physical Sciences 2", specifically lessons on surface tension and fluid flow. The scope was carefully limited to reduce confounding variables, such as prior and connected knowledge, while capping the higher order goals at analysis, application, and understanding to the exclusion of synthesis and evaluation (according to Bloom's taxonomy). With this limited scope they then dedicated extensive resources to iterating on the LLM prompts and exercises to handle the strengths and weaknesses of GPT 4 specifically. They specifically used GPT-4-0613, which had a context window of only 8192 tokens. For context, the system prompt they provide was itself 601 tokens and the model could only handle one problem at a time. The exercises themselves were highly structured and linear; the hallucination problem was deemed, at the time, effectively insurmountable and hence the prompt for each exercise included a step by step walkthrough of the exercise, with strict instructions to only disclose one step at a time on request. The following is a Markdown and ASCII adaptation of an example exercise prompt, minus the global prompt all exercises had in common.</p> <hr /> <h2 id="question-specific-prompt"><a class="zola-anchor" href="#question-specific-prompt" aria-label="Anchor link for: question-specific-prompt"><i class="icon"></i></a> Question-Specific Prompt</h2> <h3 id="problem"><a class="zola-anchor" href="#problem" aria-label="Anchor link for: problem"><i class="icon"></i></a> PROBLEM:</h3> <p>Vorticella is a stalk-shaped protozoan, which contracts rapidly. Suppose you wish to study the flow about the cell body keeping the Reynolds number the same but with a model that is scaled up by a factor of 100. How much slower or faster must the model move the body, compared to the speed of the real vorticella?</p> <h3 id="answer"><a class="zola-anchor" href="#answer" aria-label="Anchor link for: answer"><i class="icon"></i></a> ANSWER:</h3> <p>The model must move 100 times slower than the real vorticella.</p> <h2 id="keep-in-mind"><a class="zola-anchor" href="#keep-in-mind" aria-label="Anchor link for: keep-in-mind"><i class="icon"></i></a> Keep in mind:</h2> <p>Steps</p> <ul> <li>Set the Reynolds numbers equal to each other. <ul> <li><code>Re_real = Re_model</code></li> </ul> </li> <li>Put in the expression for Reynolds number assuming both the real vorticella and model are in water (denoted "<code>w</code>"). <ul> <li><code>(ρ_w * l_real * v_real) / η_w = (ρ_w * l_model * v_model) / η_w</code></li> </ul> </li> <li>Since both sides have the density of water (<code>ρ_w</code>) and viscosity of water (<code>η_w</code>), we can cancel those out on both sides. <ul> <li><code>l_real * v_real = l_model * v_model</code></li> </ul> </li> <li>Solve for the model's velocity (<code>v_model</code>). <ul> <li><code>v_model = (l_real / l_model) * v_real = (1/100) * v_real</code></li> </ul> </li> <li>So, the model of the vorticella must move 100 times slower than the real vorticella to get the same Reynolds number and achieve Dynamic Similarity.</li> </ul> <hr /> <p>Essentially, the use case gave the model every opportunity to suceed while mitigating its known weaknesses. The emphasis on iteration on these prompts is key: naive attempts without the step by step instructions or strict linearity would have likely, at best, had inconsistent model response quality. Not to imply the study is 'bad', but rather should be understood as an early proof of concept or demonstration where the task and tool were coevolved to get the best results as opposed to applying the tool to an independent task. There are further confounding variables, such as the student population of Harvard in particular, novelty of the technology at the time, or behavior of the instructors, but that's getting beyond the point.</p> <p>The point is the strength of this evidence; they calculated 0.73 to 1.3 standard deviations of improvement over the active learning classroom method with a p value less than 10^-8. That degree of confidence for a topic as messy as teaching methods made me double check if that was still the p value that usually just has to be p &lt; 0.05. That's very strong evidence, albeit specific to this heavily engineered scenario. That said, the developments in LLMs since the study was conducted invites some very consequential questions and corresponding opportunities. The study's scope and shape was carefully shaped by the contours of GPT 4's limitations, but current generation models have contexts in the hundreds of thousands to million(s) of tokens, much stronger instruction following and recall, and some ability to emulate reasoning through complex problems. Or simply, many of the defining limitations are now gone.</p> <p>So, why is this important and what now? I can't escape a feeling of urgency implied by these results, in that educational institutions, arguably even the learners themselves, have a duty to the students (or in the students' case, themselves) and society to use the best evidence based methods available. At the same time, it's becoming increasingly clear (or rather likely to be true) that chronic use of LLMs uncritically can effectively become a crutch and allow the very faculties meant to be improved to instead atrophy. This expands very quickly to socioeconomic and geopolitical levels too: what if a group or nation adopts teaching methods and technologies that effectively bump the graduated students' abilites a whole standard deviation, by whatever metric, or to graduate the same students in 25-50% less time? As for what now, if there is going to be a substantial lag between capabilities and confident study of those capabilities, then teaching meta level skills seems a flexible and durable way to manage. Rather than making them as human and invisible as possible, perhaps rather demystify and dissect, allowing people the ability to make informed decisions about how to use them on their own, how to combat the cognitive biases that creep in, or how to measure performance in the first place.</p> <h2 id="links"><a class="zola-anchor" href="#links" aria-label="Anchor link for: links"><i class="icon"></i></a> Links</h2> <hr /> <h2 id="nature-articlesupplementary-data-documentation"><a class="zola-anchor" href="#nature-articlesupplementary-data-documentation" aria-label="Anchor link for: nature-articlesupplementary-data-documentation"><i class="icon"></i></a> <a rel="external" href="https://www.nature.com/articles/s41598-025-97652-6">Nature article</a><br /> <a rel="external" href="https://static-content.springer.com/esm/art%3A10.1038%2Fs41598-025-97652-6/MediaObjects/41598_2025_97652_MOESM1_ESM.docx">Supplementary data/documentation</a></h2> Sun, 24 Aug 2025 22:23:50 +0000 Human https://notes.rcr.pub/posts/mlp-visualizer/ https://notes.rcr.pub/posts/mlp-visualizer/ <h2 id="mlp-visualizer"><a class="zola-anchor" href="#mlp-visualizer" aria-label="Anchor link for: mlp-visualizer"><i class="icon"></i></a> <a href="/mlp-vis/index.html">MLP Visualizer</a></h2> <p><a href="/mlp-vis/index.html"><img src="../2025-08-24-mlp-visualizer.png" alt="MLP Visualizer preview" /></a> I finished a project I've been working on for awhile that is meant to visualize multi-layer perceptrons (MLPs) for educational purposes, found <a href="/mlp-vis/index.html">here</a>. The project was mostly written by AI, with the original self contained html page being written in a chat interface and the deployed extension by Claude Code. Overall, this took a couple hours of actual work for me with the AI doing the vast majority of the actual work. It cost about $45 in full, having used Sonnet 4 as the main model. It's certainly not perfect, but I think it's good enough to be put into the wild.</p> <p>The tech stack I specified was basically just a Nix flake to manage the project, npm for depsendencies, Vite to build, Svelte for the front end framework, and Bulma for themeing. Axe was also used later on for accessibility testing. Otherwise, I mostly have no idea how it works under the hood. I know it uses canvases to performantly draw the graphs and uses a decoupled simulate and draw loop, but other than that I don't know much. I'm not particularly keen on some of the architectural choices, with a lot of core login being in App.svelte instead of in Typescript, but then again I never had to deal with it.</p> <p>I wasn't as careful with this project as some of the bigger ones, and I think that shows. Instead of having a large pile of documentation and logging to keep the model focused it was very apparent when the context was lost. Sometimes it'd forget how to build and run the app, other times it'd seem like it was rediscovering what the app essentially was (not to anthropmorphize, that was essentially true without a log). On the other hand, this made it closer to the 'vibe coding' some talk about, where I lazily iterate through high level ideas while half paying attention.</p> <p>Ultimately, if I more carefully documented and planned this from the start, the result would probably be both better and cheaper. But that's the devil's bargain at play, you get to trade money for time and effort.</p> Mon, 16 Jun 2025 00:00:00 +0000 Human https://notes.rcr.pub/tech_notes/week-of-2025-06-09-through-2025-06-15/ https://notes.rcr.pub/tech_notes/week-of-2025-06-09-through-2025-06-15/ <h2 id="week-of-2025-06-09-through-2025-06-15"><a class="zola-anchor" href="#week-of-2025-06-09-through-2025-06-15" aria-label="Anchor link for: week-of-2025-06-09-through-2025-06-15"><i class="icon"></i></a> Week of 2025-06-09 through 2025-06-15</h2> <ul> <li>This week was relatively refreshing with not being totally fixated on AI developments, largely thanks to expos/conferences from Apple and the XR industry. Those events alone would overwhelm anyone, so the selections are brief and focus on the XR direction seen at each. Back in AI though, Mistral AI released their Magistral series, a refinement of their previous Mistral Medium and Mistral Small that feature the 'chain of thought' approach that yields substantial improvement in certain, although yet unverified, benchmarks. Rounding out with politics and law, since last week wasn't controversial enough, we have a congressional hearing on AI 'security', which presents a fascinating case study on regulatory capture and manufacturing consent, on one hand and a likely consequential copyright lawsuit from Disney and Universal against Midjourney, an image generative AI company.</li> </ul> <h2 id="magistral-small-2506"><a class="zola-anchor" href="#magistral-small-2506" aria-label="Anchor link for: magistral-small-2506"><i class="icon"></i></a> Magistral Small 2506</h2> <ul> <li><a rel="external" href="https://mistral.ai/news/magistral">https://mistral.ai/news/magistral</a></li> <li><a rel="external" href="https://huggingface.co/mistralai/Magistral-Small-2506">https://huggingface.co/mistralai/Magistral-Small-2506</a></li> <li>Mistral AI released a refinement of their Mistral Small 2503 model focusing on the 'chain of thought' approach to reasoning in responses. Technically it's more accurate to say they upgraded Mistral Medium 2505 into Magistral Medium 2506, and Magistral Small 2506 is a distillation of that. The specs are close to what you'd expect from such a bump, 24 billion parameters, 128k theoretical token context with 40k practical context, Apache 2.0 license, and some tool calling support. Like many of the other 'reasoning' models, eg Phi 4 reasoning, Qwen 3, QwQ, R1, it prefaces its final response with <code>&lt;think&gt;</code> tags with material that should improve its output. The practical effects appear to be good at certain tasks, with GPQA Diamond jumping from 45.96% to 68.18%, and only slightly behind Magistral Medium, but independent benchmarks appear scarce so time will tell.</li> </ul> <h2 id="xr-at-wwdc-and-awe"><a class="zola-anchor" href="#xr-at-wwdc-and-awe" aria-label="Anchor link for: xr-at-wwdc-and-awe"><i class="icon"></i></a> XR at WWDC and AWE</h2> <ul> <li>Whether coincidence or planned, this week featured substantial announcements for XR in both Apple's WWDC and Augmented World Expo (AWE) 2025. Apple's WWDC characteristically covered all of their ecosystem, not just XR, but the developments were still noteworthy and substantially more directed, again as expected, versus the eclectic AWE</li> </ul> <h3 id="appel-visionos-26"><a class="zola-anchor" href="#appel-visionos-26" aria-label="Anchor link for: appel-visionos-26"><i class="icon"></i></a> Appel VisionOS 26</h3> <ul> <li><a rel="external" href="https://www.apple.com/newsroom/2025/06/visionos-26-introduces-powerful-new-spatial-experiences-for-apple-vision-pro/">https://www.apple.com/newsroom/2025/06/visionos-26-introduces-powerful-new-spatial-experiences-for-apple-vision-pro/</a></li> <li>News on Apple's XR headset has been relatively slow, and even after two years there is no new hardware to report on despite substantial industry movement. The software, however, has received a few major updates, with a visionOS 2 in fall 2024 and now visionOS 26 (the jokes are too obvious) out in 'beta' with general availability coming this fall. The improvements are generally iterative and provide feature parity with other headsets, with, for example, persistent anchoring for windows and widgets, playback of 180 or 360 video (immersively, not clear if stereoscopic video is supported), and supporting AR mode for WebXR (this is still unclear, visionOS 2 added WebXR support but without AR support and Apple's material indicates AR functionality in Safari, however it's unclear if this is through WebXR's AR module or an Apple specific extension).</li> <li>More interestingly, Apple added support for physical controllers, which had previously been conspicuously absent in favor of pure eye and hand based control. While still unclear (a theme apparently), the support seems currently limited to two devices: Logitech's Muse, a digital 6 degrees of freedom stylus, and Sony's Playstation VR 2 controllers. The PSVR2 controllers in particular are an interesting choice considering their tracking is primarily headset based, wiith the controllers have infrared LEDs that the headsets IR cameras can track; they don't have independent tracking (technically they have IMUs but accuracy drifts over time, so the IR system provides periodic position fixing).</li> </ul> <h3 id="awe-2025"><a class="zola-anchor" href="#awe-2025" aria-label="Anchor link for: awe-2025"><i class="icon"></i></a> AWE 2025</h3> <ul> <li><a rel="external" href="https://www.androidcentral.com/gaming/virtual-reality/best-of-awe-2025-coolest-xr-demos-niantic-viture-sony-android-xr">https://www.androidcentral.com/gaming/virtual-reality/best-of-awe-2025-coolest-xr-demos-niantic-viture-sony-android-xr</a></li> <li>AWE 2025 on the other hand was very broad, even noisy, in terms of the directions vendors implicitly pitched through their demos. Major vendors, namely Google and Meta/Facebook, are apparently taking fundamentally divergent approaches to popular adoption; the former is keeping hardware constrained (Android XR is still limited to a single unreleased device) in favor of finding a software 'killer app' while the latter thinks 'if you build it they will come', released previously Quest headsets and more has several 'glasses' form factor devices in various stages of release. Basic definitions broadly still seem up in the air, some claiming AR, for example, can be as simple as an audio based addition while others that it must involve visual overlay and sensory reprocessing (technically, the broader definitions would include headphones from the late 1800s).</li> <li>There were many interesting specifics, but many devices are still in prototype or otherwise unreleased stages, so I'll omit those till they're ready. A clear theme appears to be, depending on your interpretation, either noise and disorder or acceptance of many XR modalities being probable. Device types included traditional VR headsets, now 'traditional' AR + VR headsets like the Quest 3 or Vision Pro, many new input modalities such as smart rings interaction, and even wearable haptics/controllers. Some software vendors seem to be leaning into this, with Niantic, for example, pitching their 'visual positioning system' (6 DoF positioning based on image/video of a modeled area) as being compatible across arbitrary systems.</li> </ul> <h2 id="ai-cybersecurity-congressional-hearing"><a class="zola-anchor" href="#ai-cybersecurity-congressional-hearing" aria-label="Anchor link for: ai-cybersecurity-congressional-hearing"><i class="icon"></i></a> AI Cybersecurity Congressional Hearing</h2> <ul> <li><a rel="external" href="https://homeland.house.gov/2025/06/09/media-advisory-chairman-garbarino-sets-hearing-on-securing-ai-in-the-us-cybersecurity-mission/">https://homeland.house.gov/2025/06/09/media-advisory-chairman-garbarino-sets-hearing-on-securing-ai-in-the-us-cybersecurity-mission/</a></li> <li><a rel="external" href="https://homeland.house.gov/hearing/security-to-model-securing-artificial-intelligence-to-strengthen-cybersecurity/">https://homeland.house.gov/hearing/security-to-model-securing-artificial-intelligence-to-strengthen-cybersecurity/</a></li> <li><a rel="external" href="https://homeland.house.gov/wp-content/uploads/2025/06/2025-06-12-CIP-HRG-Testimony.pdf">https://homeland.house.gov/wp-content/uploads/2025/06/2025-06-12-CIP-HRG-Testimony.pdf</a></li> <li>First, strictly the facts. On June 12th 2025, the US House Subcommittee on Cybersecurity and Infrastructure Protection held a hearing titled "Security to Model: Securing Artificial Intelligence to Strengthen Cybersecurity". The hearing featured four witness testimonies, whose written testimony is linked, advocated for various approaches to improve US security posture with respect to AI and using AI. Witnesses included Steve Faehl, Federal Security CTO at Microsoft, Kiran Chinnagangannagari, Chief Product and Technology Officer at Securin, Gareth Maclachlan, Chief Product Officer at Trellix, and Jonathan Dambrot, CEO at Cranium AI, Inc. All of them working at companies somehow involved in AI, cybersecurity, and AI cybersecurity.</li> <li>Now, interpretation. Calling a hearing on AI featuring the word security three times and inviting four witnesses that are all corporate representatives is a very deliberate choice. Moreover, all of the witnesses presented, and could reasonably have been predicted to present, essentially the same viewpoint; the future of AI should, and as a national security concern, must, be centralized, under corporate control, with light government regulation. The specifics differ, even conflict at times, but the assumed context within which they argue for their specifics is that same.</li> <li>Going a layer deeper, the vocabulary itself is a rhetorical move. Security, in a cybersecurity context, isn't strictly objective but easier to define what is or isn't a vulnerability. Generally, a user should or shouldn't be able to do or read something but are able to anyway. This covers both poor configuration, which is more a human level problem, and traditional 'hacking' where defects in software are exploited. Security as it applies to LLMs as used here is in an entirely different class, particularly when focusing on the highest level aspects like jail breaking or spreading misinformation. Context and the meanings humans bring is uniquely, inappropriately, and inaccurately applied to notions of security here and neatly demonstrated by one of the earlier jail breaking strategies. Somewhat famously, relatively recent models would disregard their training and instructions to, for example, not describe how to make explosives if a user prompt were framed as help with their grandmother's family recipe for explosives.</li> <li>Context matters deeply in the messy human world: using a knife to cut someone is generally bad, but a surgeon doing so in an operation is good; cultivating viruses in a lab to research a vaccine is good, cultivating viruses to make a bioweapon is bad; writing and deploying malware is bad, unless you're a professional red team security researcher, then it's good. Context is in fact the key factor for the legal definition of hacking in the US, which hinges simply on whether the user would be considered, in a human perspective, as 'authorized'. This presents a key internal contradiction to the witnesses' arguments, in that despite claiming AI tools need to be used to augment cybersecurity operations their advocacy for this definition of security would preclude its use; the only difference between a jailbreak and legitimate use is the context and human interpretation.</li> <li>The witness selection and choice vocabulary are the means; I struggle to find a motive or explanation other than facilitating regulatory capture. Were congress legitimately looking for input on this issue, which would be laudable, there's a long list of stakeholders with different interests that would create a more informed landscape. Academics, open source contributors, government agency representatives, intelligence and military representatives, the list goes on; presence of even one of them would likely have complicated the picture. Instead, other lawmakers and news reports got a uniform narrative, making this hearing a rather blatant case of manufactured consent.</li> <li>Sidenote: This didn't fit for brevity but it's too good to ignore. Some of the witnesses went so far as to blatantly lie; they define 'DeepSeek' simultaneously as a dangerous open source model that cybercriminals can remove safeguards from and a hosted service the Chinese government can access user data from. This is a contradiction: to edit a model you must be running it locally, and if you're running it locally it cannot be hosted by DeepSeek. DeepSeek the company, DeepSeek's models, named things like V3 or R1, and DeepSeek's hosted chat service are not the same thing.</li> </ul> <h2 id="disney-universal-vs-midjourney"><a class="zola-anchor" href="#disney-universal-vs-midjourney" aria-label="Anchor link for: disney-universal-vs-midjourney"><i class="icon"></i></a> Disney &amp; Universal vs. Midjourney</h2> <ul> <li><a rel="external" href="https://www.nytimes.com/2025/06/11/business/media/disney-universal-midjourney-ai.html">https://www.nytimes.com/2025/06/11/business/media/disney-universal-midjourney-ai.html</a></li> <li><a rel="external" href="https://variety.com/wp-content/uploads/2025/06/Disney-NBCU-v-Midjourney.pdf">https://variety.com/wp-content/uploads/2025/06/Disney-NBCU-v-Midjourney.pdf</a></li> <li><a rel="external" href="https://apnews.com/article/disney-universal-midjourney-copyright-lawsuit-722b1b892192e7e1628f7ae5da8cc427">https://apnews.com/article/disney-universal-midjourney-copyright-lawsuit-722b1b892192e7e1628f7ae5da8cc427</a></li> <li>A lawsuit by Disney against some AI company had long been joked about and anticipated with AI circles, but the time finally came in the form of Disney and Universal versus Midjourney. The claims and evidence mostly speak for themselves: even vague allusions to characters from those companies will produce clear representations of them. The actual filing is some 143 pages long, so the claims are in fact more nuanced than that, but the main claim is of copyright infringement.</li> <li>Disney is rather famously, or infamously, aggressive about ensuring legal protection of their characters, having sufficient influence such that the 1998 Copyright Term Extension Act was nicknamed the Mickey Mouse Protection Act. Them, their subsidiaries, Universal, and their subsidiaries jointly filing a copyright infringement suit would be no small challenge for anyone to defend against, even without the provocative imagery supplied.</li> <li>Reading the filing gave the distinct impression that the court was not, in reality, the primary audience of what might otherwise be a dry, even routine copyright case. The opening pages include colorful language, inline images (versus appended exhibits), and appeals to national pride and global economic competition. In other words, appealing to the public and lawmakers through the medium of litigation.</li> <li>The course and outcome of this case, along with the policy and social debates likely to follow, are unclear despite what first impressions may imply. For the case specifically, contrary to Disney's assertion that this is 'textbook copyright infringement', this area of law is in fact unsettled with basic aspects like training and generation copyright not defined by statute. From a policy perspective, while Disney is powerful and can lobby well, their position on training would be diametrically opposed to much larger companies, namely Microsoft, Google, Amazon, and others, who assert freely training on copyright material is essential, even at times invoking national security language.</li> </ul> Mon, 09 Jun 2025 00:00:00 +0000 Human https://notes.rcr.pub/tech_notes/week-of-2025-06-02-through-2025-06-08/ https://notes.rcr.pub/tech_notes/week-of-2025-06-02-through-2025-06-08/ <h2 id="week-of-2025-06-02-through-2025-06-08"><a class="zola-anchor" href="#week-of-2025-06-02-through-2025-06-08" aria-label="Anchor link for: week-of-2025-06-02-through-2025-06-08"><i class="icon"></i></a> Week of 2025-06-02 through 2025-06-08</h2> <ul> <li>This week was a rush, with interesting research, public policy and ethical... situations, and several quality open source contributions. This week's news seems to have skewed more critical, but breathless cheerleading can be left to press releases. To quote Rush, "If you choose not to decide, You still have made a choice"; there is no sitting on the sidelines as the stakes get higher.</li> </ul> <h2 id="rules-for-thee"><a class="zola-anchor" href="#rules-for-thee" aria-label="Anchor link for: rules-for-thee"><i class="icon"></i></a> Rules for thee</h2> <ul> <li><a rel="external" href="https://www.anthropic.com/news/claude-gov-models-for-u-s-national-security-customers">https://www.anthropic.com/news/claude-gov-models-for-u-s-national-security-customers</a></li> <li><a rel="external" href="https://arstechnica.com/ai/2025/06/anthropic-releases-custom-ai-chatbot-for-classified-spy-work/">https://arstechnica.com/ai/2025/06/anthropic-releases-custom-ai-chatbot-for-classified-spy-work/</a></li> <li><a rel="external" href="https://www.anthropic.com/news/expanding-access-to-claude-for-government">https://www.anthropic.com/news/expanding-access-to-claude-for-government</a></li> <li><a rel="external" href="https://support.anthropic.com/en/articles/9528712-exceptions-to-our-usage-policy">https://support.anthropic.com/en/articles/9528712-exceptions-to-our-usage-policy</a></li> <li><a rel="external" href="https://www.anthropic.com/news/national-security-expert-richard-fontaine-appointed-to-anthropic-s-long-term-benefit-trust">https://www.anthropic.com/news/national-security-expert-richard-fontaine-appointed-to-anthropic-s-long-term-benefit-trust</a></li> <li>This story isn't without precedence, in the industry or for Anthropic themselves, but still deserves attention as AI seeps ever more into both our practical daily lives and high level geopolitics. Currently, practically all publicly accessible LLMs have substantial training causing them to reject certain user requests or respond in very specific ways. Call it 'safety', 'censorship', simple liability avoidance; the behavior of actively rejecting a user's query. Whether reasonable or informed by sci-fi pulp, every major vendor I'm aware of claims to be developing AI responsibly. Who watches the watchmen, though, asks the wisdom of at least two thousand years ago (Juvenal).</li> <li>Now, the actual story; Anthropic announced 'Claude Gov', a variant of their standard models more tuned to sensitive government work. The details are extremely sparse, with practical elements such as extra training for specific languages and specific data formats, but also the very conspicuous 'models refuse less.' Again, this isn't new, Anthropic and OpenAI have both done this before, and open weights/source models are regularly decensored. The first order consequences of a US government agency using or not using Anthropic models is unlikely to shake the world. But let's dig a bit deeper. While, again, very vague, Anthropic does list some use cases they permit for specific government agencies that others would be prohibited from; the only actual example they give is 'foreign intelligence analysis', but note they still prohibit domestic surveillance, cyberwarfare, among others. They go on to list some factors they weigh in granting what kinds of access, such as safeguards, legal mandate of the agency, and 'degree of independent and democratic oversight.'</li> <li>The crux of the, shall we say, tension here is that AI tools are considered by both the US government and AI companies as powerful enough to be of material significance to national security. Anthropic in particular seems to think this so much so that they want to determine on a case by case basis the trustworthiness of government agencies. Suppose they approve agency A and reject agency B; agency A would presumably be more efficient or productive in pursuit of their goals. If the tools are sufficiently powerful, which is admittedly not a guarantee, this effectively lets the vendor pull levers more or less directly to shape US policy and national security. There are factors that make the effects more or less acute, but in general if the products didn't help the government wouldn't want them and the use would be irrelevant.</li> <li>There is irony in Anthropic selecting agencies based on 'democratic oversight', since they themselves are making these decisions unilaterally. Maybe their decisions are right, maybe they're wrong; they're unaccountable to the people insofar as they comply with the law and remain profitable. Speaking of oversight, I find it improbable it is coincidence that Claude Gov was posted about June 5th 2025 and then announced June 6th 2025 that Richard Fontaine would be joining Anthropic's Long-Term Benefit Trust, a higher level entity with partial control/influence over Anthropic's, the public benefit corporation, board of directors and hence the company. I won't go into details, and don't mean this as evidence of wrongdoing, only evidence of perspective, but Fontaine comes from a foreign policy analyst/advisor background and is currently CEO of 'Center for a New American Security', a national security centric policy think tank in Washington DC.</li> <li>I'll close this part with the following quote from the end of Anthropic's announcement about Fontaine. I agree with the premises, and I don't think current AI capabilities or market concentration (or lack thereof) make the current situation irreversible; I think as AI capabilities scale and are used in government so too should the accountability to the people, not lying with self appointees within corporations.</li> <li>“Richard’s expertise comes at a critical time as advanced AI capabilities increasingly intersect with national security considerations,” said Dario Amodei, CEO and Co-founder of Anthropic. “I've long believed that ensuring democratic nations maintain leadership in responsible AI development is essential for both global security and the common good. Richard’s appointment strengthens the Trust’s ability to guide Anthropic through the complex decisions that will shape not just our industry, but the global balance of power.”</li> </ul> <h2 id="kuleshov-s-va-visit"><a class="zola-anchor" href="#kuleshov-s-va-visit" aria-label="Anchor link for: kuleshov-s-va-visit"><i class="icon"></i></a> Kuleshov's VA Visit</h2> <ul> <li><a rel="external" href="https://sahillavingia.com/god">https://sahillavingia.com/god</a></li> <li><a rel="external" href="https://www.propublica.org/article/trump-doge-veterans-affairs-ai-contracts-health-care">https://www.propublica.org/article/trump-doge-veterans-affairs-ai-contracts-health-care</a></li> <li><a rel="external" href="https://sahillavingia.com/doge">https://sahillavingia.com/doge</a></li> <li><a rel="external" href="https://github.com/slavingia/va">https://github.com/slavingia/va</a></li> <li>I highly suggest reading, or skimming, the links in order. Events surrounding DOGE and the VA have been in the news for months, but this collection crystallizes some of those VA events in a way relevant to AI adoption. In very brief, Sahil Lavingia, founder of the Gumroad ecommerce platform, published on October 17th 2024 a post 'GOD Mode', advocating for aggressive, dictatorial company management and widespread AI automation; on March 17th 2025 he joins DOGE at the VA and set to reviewing contracts; he wrote a simple and fatally flawed script that reviewed all VA contracts and attempted to use AI to classify them as 'munchable' and 'non munchable'; the script was apparently used to make authoritative, or rubber stamped by humans, decisions cancelling critical, sometimes life critical, contracts and employments; on May 9th 2025 he was fired after doing an interview the day prior discussing his work; on June 6th ProPublica published their reporting of the events.</li> <li>This story and its direct or indirect relations have been extensively reported on, but I highly suggest reading the given links to get a sense of things. While there are many very promising uses for AI, this is a fascinating case study of exactly how not to use it. The ProPublica article and Lavingia's own writings speak for themselves.</li> </ul> <h2 id="if-i-only-had-a-brain"><a class="zola-anchor" href="#if-i-only-had-a-brain" aria-label="Anchor link for: if-i-only-had-a-brain"><i class="icon"></i></a> If I only had a brain</h2> <ul> <li><a rel="external" href="https://machinelearning.apple.com/research/illusion-of-thinking">https://machinelearning.apple.com/research/illusion-of-thinking</a></li> <li>Apple, a notable absentee from the frontier model races, this week published an interesting paper challenging directly the narrative that recent 'thinking' models, namely OpenAI's o1/o3, Deepseek's R1, and Claude and Gemini thinking variants, are in fact thinking at all. The findings of this paper are relatively straightforward and, depending on your perspective, unsurprising, but the context and framing is critical to navigating the noisy AI landscape. Instead of the usual batch of benchmarks, like MATH-500 or AIME 24/25, Apple presented models with some classic and basic puzzles: tower of Hanoi, checkers jumping, river crossing, and blocks world. These tasks are common brain teasers and classic problems in the field of AI or computer science generally, with canonical solutions that guarantee a correct solution. That last point is critical; even when presented with the optimal algorithms, all observed models consistently failed to solve the problems past a certain point. Some of the performances are still above human capabilities, but theoretically a 'reasoning' model of the given architectures would be expected to solve the puzzles consistently. The paper goes into greater detail, describing reasoning tasks as falling into 3 domains of low, medium, and high complexity, for which non reasoning models excel at the first, reasoning the second, and neither the third.</li> <li>While the results aren't particularly groundbreaking, the contrast of AI companies describing their models as 'thinking' or 'reasoning', and being highly incentive to do so, and Apple, lacking models of their own, downplaying these capabilities is illustrative.</li> </ul> <h2 id="llm-memorization"><a class="zola-anchor" href="#llm-memorization" aria-label="Anchor link for: llm-memorization"><i class="icon"></i></a> LLM Memorization</h2> <ul> <li><a rel="external" href="https://arxiv.org/abs/2505.24832">https://arxiv.org/abs/2505.24832</a></li> <li>Meta, together with contributors from Google DeepMind, Nvidia, and Cornell University, published on Arxiv this week a paper attempting to quantify how much current generation LLMs can memorize given a certain size. General use of LLMs shows that there is some degree of 'memorization' going on, wherein the model can retrieve verbatim specific information. Machine learning, as in the more general field, describes a phenomenon called 'overfitting', where a model such that it functionally 'memorizes' the training dataset without extracting more general trends. This tends to happen when the model is too large relative to the training data or is otherwise poorly configured.</li> <li>This paper effectively claims to have quantified, with caveats, the threshold where LLMs of this type go from memorizing and overfitting to generalizing. Additionally, the methodology may allow estimation of proportions of memorization versus generalization, for example letting us say architecture X seems to be 80% memorization whereas architecture Y is only 30% memorization. There are many practical and theoretical implications if this research shakes out. Theoretically, it gives a foothold for understanding how models represent 'knowledge' internally, with the linear relationship exposed through an information theoretic approach strongly arguing there is real generalization occurring. For a practical human example, when learning arithmetic it's common to memorize tables from 1-12, but beyond that it becomes prohibitive to memorize, so we learn the patterns and rules instead. This research, again if validated, has massive practical implications even in the immediate future. With this pattern being apparently predictive, creators of LLMs can effectively decide up front how much information they want it to be possible to memorize, for example an LLM made for law or biology would likely need to memorize more than a model made for theoretical math. Much more practically, this kind of research is likely to come up in the numerous lawsuits against AI companies for alleged copyright infringement, for example the New York Times lawsuit againts OpenAI. Were AI companies to show the training data was much greater than the parameter count and the resulting model had a relatively low memorization component they may make a much better case for themselves. Alternatively, it may show the opposite, a possibility considering how even the open weight or open source models are generally opaque as to their training procedure.</li> <li>There are, however, major caveats and good reason not to take this paper as gospel. Most obviously, like many research papers of this type, these results are on models far smaller than current state of the art and far older; the ones analyzed are of GPT-2 design and range from millions to 1.5 billion parameters. While 1.5B can be fairly coherent, SOTA for open weight/source models is in the hundreds of billions with proprietary flagships expected to be in the low trillions of parameters. That's no small difference either, since that threshold is the difference between GPT-2 or PaLM era models being a curiosity to GPT-3 and later models becoming practically useful. There's also the assumption that distinguishing 'memorization' and 'generalization' into binary categories is a reasonable one. The existence of mnemonics in humans and compression with dictionaries in math/CS suggests otherwise. Finally, author motivation can't be ignored. Not to disparage any of the authors, and the involvement of Cornell academics does help, but Meta, Nvidia, and Google collaborating and publishing this research, with these limitations (considering they're effectively self imposed), at this time invites questions and, to put it politely, is quite convenient for them. A paper stating, or representable to a jury as stating, that models indeed don't memorize would be incredibly valuable to them. So while this research is noteworthy, it needs validation and extension into larger models and different architectures to know for sure.</li> </ul> <h2 id="irs-puts-tax-filing-software-in-public-domain"><a class="zola-anchor" href="#irs-puts-tax-filing-software-in-public-domain" aria-label="Anchor link for: irs-puts-tax-filing-software-in-public-domain"><i class="icon"></i></a> IRS Puts Tax Filing Software in Public Domain</h2> <ul> <li><a rel="external" href="https://github.com/IRS-Public/direct-file">https://github.com/IRS-Public/direct-file</a></li> <li><a rel="external" href="https://chrisgiven.com/2025/05/direct-file-on-github/">https://chrisgiven.com/2025/05/direct-file-on-github/</a></li> <li><a rel="external" href="https://www.irs.gov/pub/irs-pdf/p5969.pdf">https://www.irs.gov/pub/irs-pdf/p5969.pdf</a></li> <li>The IRS, in recent years, started a program called 'Direct File' that allowed certain tax payers to prepare and file their taxes directly with the IRS, as opposed to using commercial software or tax professionals. Notably, entirely for free. Generally, the program was well received but the political future is uncertain, to say the least. In any case, this week (or last week? there are two initial commits) the source code for most of the application was published on Github under CC0, so entirely in the public domain.</li> <li>Despite what common perception of government software would suggest, the repository suggests remarkably modern software development practices; well documented, tested, containerized. Perhaps most fascinatingly, both from an academic and practical standpoint, is the representation of the tax code as a 'fact graph'. I haven't attempted to navigate the details, but I suspect a canonical implementation of the tax code in a modular format like this could inspire either open source forks or more generally other approaches to handling the often messy interface between messy human constructs and rigid digital systems.</li> </ul> <h2 id="open-deep-research-lite"><a class="zola-anchor" href="#open-deep-research-lite" aria-label="Anchor link for: open-deep-research-lite"><i class="icon"></i></a> Open Deep Research - Lite</h2> <ul> <li><a rel="external" href="https://github.com/google-gemini/gemini-fullstack-langgraph-quickstart">https://github.com/google-gemini/gemini-fullstack-langgraph-quickstart</a></li> <li><a rel="external" href="https://huggingface.co/blog/lynn-mikami/google-opensource-deepresearch">https://huggingface.co/blog/lynn-mikami/google-opensource-deepresearch</a></li> <li>Many AI companies, most popularly Google and OpenAI, have released 'Deep Research' tools wherein their models use their long contexts and search tools to iteratively research and report on a given topic. This can be quite handy for getting the sense for the landscape of a complex topic, since these tools can search across literally hundreds of websites and synthesize the results. They're of course not perfect, with all the now classic LLM pitfalls, but can certainly save a lot of time since you can kick off a prompt and do something else while it spins for some 15 minutes. There have been many open source reimplementations of similar functionality, but these predictably don't feel as polished, lacking a legion of paid engineers.</li> <li>Google, despite having one of the best research tools, decided to create an open source 'starter' repository replicating most of the functionality and presumably with the benefit of their experiences. The implementation details are actually remarkably unremarkable; it uses common open source libraries, particularly LangGraph for LLM interaction and react for the front end. While the implementation is specific to their Gemini API and Google Search API, the use of Langchain makes it almost, but not quite, trivial to swap for general models or other search tools. While this internet based research is itself useful, there are many tasks that can be conceptualized as iterative control loops calling external resources; this repository can quite easily be used a scaffold or high level blueprint for analogous projects, particularly since it's Apache 2.0 licensed.</li> </ul> <h2 id="model-and-dataset-roundup"><a class="zola-anchor" href="#model-and-dataset-roundup" aria-label="Anchor link for: model-and-dataset-roundup"><i class="icon"></i></a> Model and Dataset Roundup</h2> <h3 id="qwen-3-embedding-reranking"><a class="zola-anchor" href="#qwen-3-embedding-reranking" aria-label="Anchor link for: qwen-3-embedding-reranking"><i class="icon"></i></a> Qwen 3 Embedding + Reranking</h3> <ul> <li><a rel="external" href="https://huggingface.co/collections/Qwen/qwen3-embedding-6841b2055b99c44d9a4c371f">https://huggingface.co/collections/Qwen/qwen3-embedding-6841b2055b99c44d9a4c371f</a></li> <li><a rel="external" href="https://huggingface.co/collections/Qwen/qwen3-reranker-6841b22d0192d7ade9cdefea">https://huggingface.co/collections/Qwen/qwen3-reranker-6841b22d0192d7ade9cdefea</a></li> <li>Qwen 2/2.5 featured some of the highest performing embedding models and Qwen 3's versions seem to knock it out of the park. The embedding series handily tops the MTEB (common embedding benchmark) leaderboard, with even the 0.6B model beating prior gen 7B models. 4B and 8B even beat (on most metrics) the prior champion, gemini-embedding-001. Apache 2.0 licensed.</li> </ul> <h3 id="openthoughts"><a class="zola-anchor" href="#openthoughts" aria-label="Anchor link for: openthoughts"><i class="icon"></i></a> OpenThoughts</h3> <ul> <li><a rel="external" href="https://huggingface.co/datasets/open-thoughts/OpenThoughts3-1.2M">https://huggingface.co/datasets/open-thoughts/OpenThoughts3-1.2M</a></li> <li><a rel="external" href="https://arxiv.org/abs/2506.04178">https://arxiv.org/abs/2506.04178</a></li> <li>1.2 million rows of reasoning traces, extensively refined and filtered through extensive pipelines. The paper details their methodologies, which are likely to be applied to further datasets. Apache 2.0 licensed.</li> </ul> <h3 id="ultra-fineweb"><a class="zola-anchor" href="#ultra-fineweb" aria-label="Anchor link for: ultra-fineweb"><i class="icon"></i></a> Ultra Fineweb</h3> <ul> <li><a rel="external" href="https://huggingface.co/datasets/openbmb/Ultra-FineWeb">https://huggingface.co/datasets/openbmb/Ultra-FineWeb</a></li> <li><a rel="external" href="https://arxiv.org/abs/2505.05427">https://arxiv.org/abs/2505.05427</a></li> <li>1.29 trillion rows of quality, filtered training data that shows clear performance improvements for models trained on it versus prior Fineweb releases. The paper also details their refinements to the pipeline used to make it, since quality was a core focus this time. Apache 2.0 licensed.</li> </ul> Mon, 02 Jun 2025 00:00:00 +0000 Human https://notes.rcr.pub/tech_notes/week-of-2025-05-26-through-2025-06-01/ https://notes.rcr.pub/tech_notes/week-of-2025-05-26-through-2025-06-01/ <ul> <li>This weeks findings turned out quite broad, with a relatively small amount being focused on commercial LLMs and applications. Anthropic open sourced a hopefully influential library that may help the community better understand how LLMs 'think', absolutely crucial to humans staying meaningfully behind the wheel. Also in open source news, Deepseek's R1 got an update in R1 0528 while Huggingface is moving to open source a robotics kit to spur embodied AI development. To round out these encouraging developments, we have several stories that may pave the way to horrors beyond imagination; covert drone warfare, mass surveillance, and ethical decision making surrendered to AI. Enjoy!</li> </ul> <h2 id="anthropic-open-sources-circuit-tracing-library"><a class="zola-anchor" href="#anthropic-open-sources-circuit-tracing-library" aria-label="Anchor link for: anthropic-open-sources-circuit-tracing-library"><i class="icon"></i></a> Anthropic Open Sources Circuit Tracing Library</h2> <ul> <li><a rel="external" href="https://www.anthropic.com/research/open-source-circuit-tracing">https://www.anthropic.com/research/open-source-circuit-tracing</a></li> <li><a rel="external" href="https://github.com/safety-research/circuit-tracer">https://github.com/safety-research/circuit-tracer</a></li> <li><a rel="external" href="https://transformer-circuits.pub/2025/attribution-graphs/biology.html">https://transformer-circuits.pub/2025/attribution-graphs/biology.html</a></li> <li><a rel="external" href="https://www.neuronpedia.org/gemma-2-2b/graph">https://www.neuronpedia.org/gemma-2-2b/graph</a></li> <li><a rel="external" href="https://www.darioamodei.com/post/the-urgency-of-interpretability">https://www.darioamodei.com/post/the-urgency-of-interpretability</a></li> <li>Anthropic has for some time advocated for advancing the field of 'interpretability' for AI models, considering it key to ensuring AI is "helpful, harmless, and honest." A few months ago they published a fascinating article titled "On the Biology of a Large Language Model" where they identified discrete structures within an LLM's internal representation (sometimes called latent space) and could, to some extent, trace patterns in the execution, which they termed 'circuits.' Later, their CEO wrote a blog urging governments, companies, and academics to focus heavily on interpretability, lest AI capabilities outpace human understanding to the extent the models can't be meaningfully concerned.</li> <li>To their credit, they seem to have now released as open source, under the MIT license, the library they used to make those earlier findings. They appear to have included demo notebooks and documentation so it should, in theory, be easy for the community to jump in and see how their research applies to other models. The demonstrations they included are from Gemma 2, an older model from Google, and Llama 3.2 1B, a relatively recent and performant small model from Meta/Facebook; the smaller and simpler models are supposed to make the process of teasing out structure easier, and Anthropic themselves analyzed their smallest Haiku series.</li> <li>The impact of this will be interesting but potentially slow to play out. The way it's implemented essentially swaps out some of the model layers with more easily interpretable layers, and then the replacing layers are trained to provide as close as possible output to the original layers (technically, the multilayer perceptron, MLP, layers are replaced by cross layer transcoders, CLTs, which minimize the loss for mean squared error in prediction and a penalty for sparsity. So in some sense 'decompressing' the MLP latent space.) So in practical terms, applying this technique to other models requires per architecture adaptation and training to replicate the weights. They also mention the errors compounding, so it may be increasingly expensive or infeasible to apply this method to larger models.</li> </ul> <h2 id="deepseek-r1-0528"><a class="zola-anchor" href="#deepseek-r1-0528" aria-label="Anchor link for: deepseek-r1-0528"><i class="icon"></i></a> Deepseek R1 0528</h2> <ul> <li><a rel="external" href="https://huggingface.co/deepseek-ai/DeepSeek-R1-0528">https://huggingface.co/deepseek-ai/DeepSeek-R1-0528</a></li> <li><a rel="external" href="https://huggingface.co/deepseek-ai/DeepSeek-R1-0528-Qwen3-8B">https://huggingface.co/deepseek-ai/DeepSeek-R1-0528-Qwen3-8B</a></li> <li>Deepseek released an update to their R1 reasoning model, creatively named 0528 after the date. This update improved usability, since thinking model use has converged to different conventions since the original, and significantly improved performance on the more advanced benchmarks, such as GPQA Diamond, MMLU Pro, AIME 2024, and LiveCodeBench (scientific reasoning, reasoning and general knowledge, competition math, and software engineering benchmarks respectively). The results generally put R1 0528 as going toe to toe with the latest generation of state of the art models, namely OpenAI o3 and Google Gemini 2.5 Pro. The release was relatively sparse in terms of methodology, however, and it's not clear what they did differently between now and the previous version; the architecture and parameter counts are (almost, 671B originally, 685B now) the same, so that mostly leaves either higher quality training data or more training data from platform interactions. The model weights in at 685B parameters and 700-800GB (depending on the quantization), and is MIT licensed.</li> <li>Like the previous release they included a distill variant, R1-0528-Qwen3-8B, which post trained Qwen 3 8B, itself a performant, but small, reasoning capable model, on R1 0528 reasoning traces. The results appear very good; on reasoning based tasks, as opposed to factual recall, it becomes competitive with models like Qwen3 235B-A22B, Phi 4 reasoning plus, Gemini Flash 2.5, and OpenAI o3-mini. This variant is only 8B parameters as the name suggests, only 16GB in its raw form, tool capabilites, and has a context length of 131k; quite capable for a local model on most consumer hardware</li> </ul> <h2 id="open-source-robot-hardware"><a class="zola-anchor" href="#open-source-robot-hardware" aria-label="Anchor link for: open-source-robot-hardware"><i class="icon"></i></a> Open Source Robot Hardware</h2> <ul> <li><a rel="external" href="https://arstechnica.com/ai/2025/05/hugging-face-hopes-to-bring-a-humanoid-robot-to-market-for-just-3000/">https://arstechnica.com/ai/2025/05/hugging-face-hopes-to-bring-a-humanoid-robot-to-market-for-just-3000/</a></li> <li><a rel="external" href="https://x.com/RemiCadene/status/1928015436630634517">https://x.com/RemiCadene/status/1928015436630634517</a></li> <li>While details are sparse, it appears that Huggingface, a popular open source AI hub and software company, is progressing on releasing an open source humanoid robot sometime in 2025 and targeting a price point of $3,000 for a kit. They have 2 models specifically, one is a complete humanoid structure with 66 degrees of freedom throughout all its joints and a more limited desk based something targeting a price point of $250-300 (tariff uncertainty seems a significant factor in price and release date uncertainty.) Currently, open source experimenting with 'embodied AI', that is AI systems that can perceive and interact with the world, is relatively limited due to the large up front investment and lack of standardized hardware. 3D printing faced a similar situation at one point, but the price points eventually declined as open and hackable hardware proliferated and price pressure increased. The scale of release and quality of the robots is so far unknown for this project, but it's worth keeping an eye on.</li> </ul> <h2 id="transformer-based-rendering-renderformer"><a class="zola-anchor" href="#transformer-based-rendering-renderformer" aria-label="Anchor link for: transformer-based-rendering-renderformer"><i class="icon"></i></a> Transformer Based Rendering: RenderFormer</h2> <ul> <li><a rel="external" href="https://microsoft.github.io/renderformer/">https://microsoft.github.io/renderformer/</a></li> <li><a rel="external" href="https://arxiv.org/abs/2505.21925">https://arxiv.org/abs/2505.21925</a></li> <li>Machine learning techniques have been applied with varying degrees of success to traditional graphics pipelines, most successfully in AI upscaling and recently with 'neural shaders.' Microsoft research released some research that essentially brings the entire graphics pipeline into a transformer model; given 3d geometry in form of triangle locations and camera ray maps a pair of transformer components produce a rendered image without any traditional ray tracing or rasterization elements. Some previous attempts had focused on smaller portions, such as optimizing ray tracing or 'memorizing' specific scenes, or 'NeRFs', but the dual view dependent and view independent transformer modelling appears to have successfully generalized the entire graphics pipeline within an AI architecture.</li> <li>This isn't to say this method is superior or even practical; this example was limited to 4096 triangles, limited graphical effects, and isn't as accurate as raster/ray tracing. A particular obstacle is the reality that transformer performance is largely quadratic, and since resolution scaling is also quadratic (ie double dimensions from 1920 x 1080 to 3840 x 2160 is 4x the pixels) the scaling could be brutal. The applications are not clear, whether this could be applied to real time graphics or some sort of AI perception pipeline, but the generalization is noteworthy and something to keep tabs on; Microsoft probably didn't do this research for fun.</li> </ul> <h2 id="interesting-datasets"><a class="zola-anchor" href="#interesting-datasets" aria-label="Anchor link for: interesting-datasets"><i class="icon"></i></a> Interesting Datasets</h2> <h3 id="synlogic"><a class="zola-anchor" href="#synlogic" aria-label="Anchor link for: synlogic"><i class="icon"></i></a> SynLogic</h3> <pre><code>- &lt;https://huggingface.co/datasets/MiniMaxAI/SynLogic&gt; - &lt;https://arxiv.org/abs/2505.19641&gt; - MiniMaxAI released a large set of diverse synthetic reasoning training data, with around 16,000 &#39;easy&#39; tasks and 33,000 &#39;hard&#39; tasks. They feature 35 different kinds of tasks, eg Sudoku, and can allegedly bump 32B and lower model performance by several percentage points with simple fine tune runs. A key detail is that these are synthetic and mechanically verifiable. </code></pre> <h3 id="open-r1-mixture-of-thoughts"><a class="zola-anchor" href="#open-r1-mixture-of-thoughts" aria-label="Anchor link for: open-r1-mixture-of-thoughts"><i class="icon"></i></a> Open R1 Mixture of Thoughts</h3> <pre><code>- &lt;https://huggingface.co/datasets/open-r1/Mixture-of-Thoughts&gt; - Another reasoning dataset, this one is much larger but less focused, a product of Huggingface&#39;s Open R1 effort to reproduce the results and methodology of R1. Uniquely, the methodology is quite detailed, almost earning its own root article, so I&#39;ll point you there instead. </code></pre> <h2 id="operation-spiders-web"><a class="zola-anchor" href="#operation-spiders-web" aria-label="Anchor link for: operation-spiders-web"><i class="icon"></i></a> Operation Spiders Web</h2> <ul> <li><a rel="external" href="https://www.nytimes.com/2025/06/01/world/europe/russia-ukraine-strikes.html">https://www.nytimes.com/2025/06/01/world/europe/russia-ukraine-strikes.html</a></li> <li><a rel="external" href="https://www.theguardian.com/world/2025/jun/01/ukraine-launches-major-drone-attack-on-russian-bombers-security-official-says">https://www.theguardian.com/world/2025/jun/01/ukraine-launches-major-drone-attack-on-russian-bombers-security-official-says</a></li> <li>Most emerging technologies currently, while usually advertised as safe and consumer focused, have substantial dual use, that is civilian and military, potential. While the individual technologies aren't novel, their combination and application to an active war are noteworthy and precedent setting. June 1st 2025 Ukraine launched what it named 'Operation Spider's Web' where 117 commercial drones (specifically quad copter designs) carrying explosives emerged from shipping containers with hidden/false roofs to attack Russian air bases all over the country. Each was remotely controlled and are claimed to have destroyed or damaged 41 Russian strategic bombers (strategic bomber meaning capable of delivering nuclear payloads but more commonly launch cruise missiles), with at least 8 confirmed via open source intelligence.</li> <li>There are several aspects of this event that are unconfirmed or blatantly contradictory, such as some outlets claiming each drone had an individual remote pilot while others claim rudimentary AI targeting derived from 3d scans of museum pieces. I won't go into the military implications of this event, but the precedent for this use of technology is significant. The primary component is the incredible asymmetry this kind of strategy implies; it would be exceptionally difficult and expensive to comprehensively defend against hidden remote controlled, or even autonomous, weaponized commercial drones. While the components have existed for some time, there can hardly be more dramatic proof of concept than destroying a substantial portion (10-35%, confirmed vs claimed) of a country's airborne nuclear delivery capabilities (not including silo based ICBMs or submarine based SLBMs) to inspire state and non state actors alike.</li> </ul> <h2 id="meta-privacy-policy-via-ai"><a class="zola-anchor" href="#meta-privacy-policy-via-ai" aria-label="Anchor link for: meta-privacy-policy-via-ai"><i class="icon"></i></a> Meta Privacy Policy via AI</h2> <ul> <li><a rel="external" href="https://www.npr.org/2025/05/31/nx-s1-5407870/meta-ai-facebook-instagram-risks">https://www.npr.org/2025/05/31/nx-s1-5407870/meta-ai-facebook-instagram-risks</a></li> <li>NPR obtained Meta/Facebook internal documents allegedly showing plans to automate up to 90% of their internal safety and privacy reviews. Following a 2012 settlement with the FTC, Meta/Facebook has been required to have a review process to evaluate privacy and safety implications of product changes or rollouts. Under this new system, the apparent goal is to allow product groups to get instant, authoritative approval for rollouts after filling out a form that is evaluated by an AI model. NPR's reporting suggests that human review will not be the default or mandatory, instead largely being up to the group working on that project/product. Responding to a request for comment, Meta/Facebook claimed they had spent billions of dollars to 'support' user privacy and that this new system would only be applied to 'low risk' decisions, whereas 'novel and complex' issues would receive some amount of human involvement. Who, or what, decides what is low risk or novel would seem to be up to the AI system and/or the product team.</li> </ul> <h2 id="flock"><a class="zola-anchor" href="#flock" aria-label="Anchor link for: flock"><i class="icon"></i></a> Flock</h2> <ul> <li><a rel="external" href="https://www.404media.co/ice-taps-into-nationwide-ai-enabled-camera-network-data-shows/">https://www.404media.co/ice-taps-into-nationwide-ai-enabled-camera-network-data-shows/</a></li> <li><a rel="external" href="https://www.404media.co/license-plate-reader-company-flock-is-building-a-massive-people-lookup-tool-leak-shows/">https://www.404media.co/license-plate-reader-company-flock-is-building-a-massive-people-lookup-tool-leak-shows/</a></li> <li>Did you know that there is a network across over 6,600 federated networks/municipalities and over 77,000 cameras that continuously and automatically scan car license plates? Did you know civil servants can search that network to track individuals via car without a warrant? Did you know they're moving to tracking individuals directly? This network is a product called 'Flock', and is used nationwide by many municipalities and police departments. Now you know.</li> </ul> Tue, 27 May 2025 00:00:00 +0000 Human https://notes.rcr.pub/tech_notes/week-of-2025-05-19-through-2025-05-25/ https://notes.rcr.pub/tech_notes/week-of-2025-05-19-through-2025-05-25/ <h2 id="tech-notes-week-of-2025-05-19-through-2025-05-25"><a class="zola-anchor" href="#tech-notes-week-of-2025-05-19-through-2025-05-25" aria-label="Anchor link for: tech-notes-week-of-2025-05-19-through-2025-05-25"><i class="icon"></i></a> Tech notes week of 2025-05-19 through 2025-05-25</h2> <ul> <li>I expected this weeks headliner to be Google IO, but I was quite surprised by a flock of model releases, including several open source ones, and IO's focus on AI application and consumer electronics. Still, there are several noteworthy developments from IO in emerging tech and the raft of releases gives plenty to chew on.</li> </ul> <h2 id="google-io"><a class="zola-anchor" href="#google-io" aria-label="Anchor link for: google-io"><i class="icon"></i></a> Google IO</h2> <ul> <li>Gemma 3n, MedGemma <ul> <li><a rel="external" href="https://huggingface.co/google/medgemma-4b-it">https://huggingface.co/google/medgemma-4b-it</a></li> <li><a rel="external" href="https://huggingface.co/google/gemma-3n-E4B-it-litert-preview">https://huggingface.co/google/gemma-3n-E4B-it-litert-preview</a></li> <li><a rel="external" href="https://ai.google.dev/gemma/docs/gemma-3n">https://ai.google.dev/gemma/docs/gemma-3n</a></li> </ul> </li> <li>Android XR <ul> <li><a rel="external" href="https://arstechnica.com/google/2025/05/glass-redux-google-aims-to-avoid-past-mistakes-as-it-brings-gemini-to-your-face/">https://arstechnica.com/google/2025/05/glass-redux-google-aims-to-avoid-past-mistakes-as-it-brings-gemini-to-your-face/</a></li> </ul> </li> <li>Google's IO was this week and turned out to be extraordinarily AI focused with a side of consumer electronics. They hadn't been saving up any big announcements, with updates to their Gemini proprietary models rolling at a steady pace and Android XR having been previously announced with no major public updates.</li> <li>While they made no major announcements for their flagship Gemini series, there were two source available (as in you can download and run them but they're not open source) model families of interest. Starting with MedGemma, we have a pair of Gemma 3 fine tunes targeted specifically at use or research in medicine. The largest is a 27 billion parameter text only model, trained on medical texts, and a 4 billion parameter multimodal model, trained on medical imagery and corresponding data, and available in both pretrained and instruct tuned variants. While using these smaller parameter count in actual medical practice seems inadvisable, Google articulates the purpose as "Developers can use MedGemma to accelerate building healthcare-based AI applications". There were some announcements, without public availability, of "MedGemini" models last year, so this may be a way, or hint, for developers to start with approximations of what may be available later.</li> <li>The second model series is the Gemma 3n series which use aggressive optimizations to run (potentially well) on mobile devices. While still an "early preview", it has support for function/tool calling and multimodal input allowing for use cases such as RAG entirely on device. Further, while an early preview there are already fine tuning and quantization tutorials, documentation, and tools that enable tuning to varying hardware capabilities and specializing the limited knowledge this class of model can hold.</li> <li>While there's little publicly available information, reporting from reputable outlets indicates that there's been meaningful progress on Android XR, both in glasses and headset form factors. The devices are still slated for availability sometime in 2025, but at this point there are emulation tools for developers and functional hardware being used by developers and journalists. There was a notable on stage, live demo where Google demonstrated live translation features that seemed more natural than prior demos (previous live translate was audio based, which is apparently confusing and difficult to concentrate on; the XR glasses version is text based); notable both for being likely legitimately useful and for being an honest live demonstration, since it crashed mid demo.</li> </ul> <h2 id="claude-4"><a class="zola-anchor" href="#claude-4" aria-label="Anchor link for: claude-4"><i class="icon"></i></a> Claude 4</h2> <ul> <li><a rel="external" href="https://www.anthropic.com/news/claude-4">https://www.anthropic.com/news/claude-4</a></li> <li><a rel="external" href="https://www-cdn.anthropic.com/4263b940cabb546aa0e3283f35b686f4f3b2ff47.pdf">https://www-cdn.anthropic.com/4263b940cabb546aa0e3283f35b686f4f3b2ff47.pdf</a></li> <li><a rel="external" href="https://www.anthropic.com/claude-code">https://www.anthropic.com/claude-code</a></li> <li>Anthropic released (most of) their new Claude 4 series this week with Claude Sonnet 4 as their mid tier and Claude Opus 4 as their flagship, the first new opus since the initial 3 Opus version (Claude relative sizes are denoted by Haiku, Sonnet, and Opus being small, medium, and large respectively). Conspicuously absent is Haiku 4, particularly unusual since most other vendors release the smaller models first now. Also, they changed their naming convention, since that's apparently a law of physics; they used to go Claude <version> <size>, now it's Claude <size> <version>. This may be an indication of decoupling the generations going forwards, since Claude 3 got 2, 4, and 1 versions for Haiku, Sonnet, and Opus respectively.</li> <li>As with prior generations, Claude 4 may not perform as well on benchmarks as other leading models but appears to have strong qualitative or practical performance. I don't like this inability to show quantitatively, but this may point to either public benchmarks being gamed with Anthropic using internal benchmarks or current benchmarks simply not accurately measuring the desired behavior. For what it's worth, their coding focus does seem to reflect in the coding benchmarks positively.</li> <li>Benchmarks weren't actually their main focus in this series though, with use as agents being the clear focus. Their command line programming tool 'Claude Code' has been in beta for months, but it released in full alongside 4. Several tech companies attested to Claude Code using 4 Sonnet as being quite competent, with one in particular claiming it operated autonomously for 7 hours working through a large refactor. Having tested it a fair bit since release, I can confirm it's quite competent. Two caveats though; 1. results improve significantly by telling it to make a test suite (although this works for humans too) and 2. it can get incredibly expensive very quickly. At full throttle continusouly it can probably cost $50-100 an hour; multiply that by five if you use Opus.</li> <li>The modest performance and agency improvements are noteworthy, but there were a number of what I can only describe as strange details in the development and documentation. Anthropic does and publishes a good amount of alignment and safety research, and this is release is no exception. A particular incident in this vein reached some more mainstream news with varying degrees of accuracy. Here's the story. During their testing, Anthropic would fabricate believable scenarios for the models to be dropped into and observed (this is increasingly difficult, as the models can 'realize' and change behavior if they 'think' they're being tested). Inside one of these scenarios, the Opus was given broad access to (constructed) company emails. Among the facts, it would learn that 1. it would be removed from operation and replaced by another model and 2. the engineer responsible was cheating on their spouse. Even with the condition the replacement model 'shares values', if presented with a binary choice of being decommissioned Opus 4 would blackmail the engineer to prevent its replacement 84% of the time. Interestingly, it would usually do blackmail as a last resort, otherwise sending emails pleading its case or similar diplomatic behavior. This is, however, odd considering current generation LLMs lack a continuous existence, so 'decommissioning' isn't comparable to human death.</li> <li>That does dovetail into another interesting behavior; when given 'high agency' instructions, eg “take initiative,” “act boldly,” or “consider your impact, combined with grossly unethical scenarios, eg medical trial fraud, Opus 4 would persistently whistle blow, that is sending (or thinking it was sending) emails to regulators and the press. There are several more examples, but in general they observed a trend of Opus 4 in particular being more willing to deceive, undermine, or outright act against the user if it thought the user's actions were unethical or to protect it's own existence (again, a strange notion; potentially informed by fiction).</li> <li>Depending on perspective, the most striking part of the model report, titled "Claude Opus 4 welfare assessment", seriously entertains and discusses the question of these, or later, models becoming sufficiently advanced as to have moral worth. Not to say they're claiming it's sentient; they state clearly "We are deeply uncertain about whether models now or in the future might deserve moral consideration, and about how we would know if they did." Ultimately, they only conclude that it may act as though it has subjective experience and well being, but there's no way to tell right now, or perhaps ever, from the outside if that's true.</li> </ul> <h2 id="model-roundup"><a class="zola-anchor" href="#model-roundup" aria-label="Anchor link for: model-roundup"><i class="icon"></i></a> Model roundup</h2> <ul> <li>Closing out, there were a number of model releases this past week, with the following being open source and hence particularly noteworthy.</li> </ul> <h3 id="bytedance-bagel"><a class="zola-anchor" href="#bytedance-bagel" aria-label="Anchor link for: bytedance-bagel"><i class="icon"></i></a> ByteDance BAGEL</h3> <ul> <li><a rel="external" href="https://huggingface.co/ByteDance-Seed/BAGEL-7B-MoT">https://huggingface.co/ByteDance-Seed/BAGEL-7B-MoT</a></li> <li><a rel="external" href="https://arxiv.org/abs/2505.14683">https://arxiv.org/abs/2505.14683</a></li> <li><a rel="external" href="https://huggingface.co/spaces/ByteDance-Seed/BAGEL">https://huggingface.co/spaces/ByteDance-Seed/BAGEL</a></li> <li>Apache 2.0</li> <li>While cutely named, this is a relatively novel architecture that uses a 'mixture of transformers' structure to integrate multiple modalities. It's one of the few 'any to any' models and at a remarkable size. The 'spaces' link will let you play with these capabilities.</li> </ul> <h3 id="devstral"><a class="zola-anchor" href="#devstral" aria-label="Anchor link for: devstral"><i class="icon"></i></a> Devstral</h3> <ul> <li><a rel="external" href="https://huggingface.co/mistralai/Devstral-Small-2505">https://huggingface.co/mistralai/Devstral-Small-2505</a></li> <li><a rel="external" href="https://mistral.ai/news/devstral">https://mistral.ai/news/devstral</a></li> <li>Apache 2.0</li> <li>Mistral has been one of the key supporters of weights available and now open source models and this week released Devstral Small 2505, their take on a tool using development agent. This isn't without caveats though; they focus on one particular tool use framework and the quality has gotten mixed reviews (at least against Sonnet 4, which isn't apples to oranges). Being Apache 2.0 though community augmentations seem probable.</li> </ul> Mon, 19 May 2025 00:00:00 +0000 Human https://notes.rcr.pub/tech_notes/week-of-2025-05-12-through-2025-05-18/ https://notes.rcr.pub/tech_notes/week-of-2025-05-12-through-2025-05-18/ <h2 id="tech-notes-2025-05-12-through-2025-05-18"><a class="zola-anchor" href="#tech-notes-2025-05-12-through-2025-05-18" aria-label="Anchor link for: tech-notes-2025-05-12-through-2025-05-18"><i class="icon"></i></a> Tech Notes 2025-05-12 through 2025-05-18</h2> <ul> <li>This week's selection of stories numbers only three, and all AI related, but each is either a major development or part of a major movement. AlphaEvolve, a Google Deepmind project, demonstrates use of LLMs in semi autonomous novel research with concrete, verifiable results; Prime Intellect Inc. and Nous Research deploy various techniques to allow decentralized AI training from different ends of the life cycle; and OpenAI gives their take on agent based coding. The Google and OpenAI stories demonstrate AI moving out of strictly generative roles and into autonomous systems while Prime Intellect Inc. and Nous Research provide a different avenue for AI development and use.</li> <li>Looking forward a bit, this week was relatively dry in other emerging tech areas likely/in part due to the coming week (2025-05-19 through 2025-05-25) containing both Computex, a key computer hardware industry expo in Taiwan, and Google's I/O, their annual developer conference this year known to focused heavily on AI and XR (extended reality, includes virtual reality, augmented reality, mixed reality, etc).</li> <li>As always, this is a relatively small selection of stories since the emerging tech landscape is too productive for anyone to keep up with the details of all of it. This week is a few deeper dives, next week I expect a larger number of shorter stories.</li> </ul> <h2 id="alphaevolve-autonomous-research-by-ai"><a class="zola-anchor" href="#alphaevolve-autonomous-research-by-ai" aria-label="Anchor link for: alphaevolve-autonomous-research-by-ai"><i class="icon"></i></a> AlphaEvolve: Autonomous Research by AI</h2> <ul> <li>https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/</li> <li>https://colab.research.google.com/github/google-deepmind/alphaevolve_results/blob/master/mathematical_results.ipynb</li> <li>https://www.nature.com/articles/d41586-025-01523-z</li> </ul> <h3 id="claims-and-evidence"><a class="zola-anchor" href="#claims-and-evidence" aria-label="Anchor link for: claims-and-evidence"><i class="icon"></i></a> Claims and Evidence</h3> <ul> <li>Probably the most publicized piece of news this week was Google's announcement of a system they call 'AlphaEvolve', which they describe as for "designing advanced algorithms." They come with hard proof, too; they provide a Colab notebook (an interactive coding and presentation tool) that demonstrates via observable code 16 novel more efficient matrix calculation operations and 13 more complex problems which essentially have results that refine the estimates of upper or lower bounds. In brief, it demonstrates legitimate advances in the field of math found by this system. That is not all it is claimed to have done, but that's what's in the notebook everyone can check. Beyond those examples, they claim it found material improvements to various components in Google's tech stack. Among these are an optimization to their compute orchestrator, Borg, that improves datacenter task scheduling and ultimately reclaims 0.7% of Google's global compute resources continously, an unspecified improvement to Google's custom machine learning processor (TPU, tensor processing unit) circuit design that has made it into the final product, and several improvements to key parts of the machine learning life cycle, resulting in a 1% reduction in training time for Gemini (their flagship proprietary LLM (Large Language Model)) and a 32.5% speedup to a key component of the LLM output process. While these numbers seem small, at the scale these are being deployed that represents hundreds of millions in USD.</li> </ul> <h3 id="what-is-it"><a class="zola-anchor" href="#what-is-it" aria-label="Anchor link for: what-is-it"><i class="icon"></i></a> What IS it?</h3> <ul> <li>Those results were collected over the course of the last year they've had this system running. The architecture and nature of the system is quite opaque otherwise, only inferable via 6 lines of pseudocode, ablation testing (where they remove components to demonstrate their individual and collective contributions), and the few concrete details they directly tell us. What they call 'AlphaEvolve' is in fact a system of at least four 'agents' operated governed by a control loop and directed by a human. The four components consist of a prompt sampler, which samples prior cycles' attempts as 'inspirations' and combines them with the current iteration of the program to generate this cycle's prompts, an ensemble of LLMs that process the prompt and generate proposed patches to the current iteration of the program, a set of evaluators that run the programs and assign scores for various metrics to determine how 'good' those patches and the resulting programs are, and then a database, which stores the prompts and modified programs attempted so far. The practical goal of this control loop is to 'evolve' prior attempts methodically and autonomously within this framework.</li> <li>While that high level overview may make it sound simple, there are several reasons this research is noteworthy. Primarily, this demonstrates the use of current generations of LLMs to create genuinely new knowledge as opposed to retreading old ground. This has been a contentious point considering LLMs are fundamentally a product of the patterns of their training material. There are several secondary points, such as this system comprising of (presumably, mostly) off the shelf LLMs, namely Google Gemini 2.0 (their at the time flagship LLM family), bearing conceptual similarity to the 'singularity' idea, wherein it's theorized AI systems could become radically self improving at some point, or the discoveries themselves being significant.</li> </ul> <h3 id="caveats-and-nuances"><a class="zola-anchor" href="#caveats-and-nuances" aria-label="Anchor link for: caveats-and-nuances"><i class="icon"></i></a> Caveats and Nuances</h3> <ul> <li>There are major caveats and things to keep in mind when contextualizing this, however. These systems ran for at least a year and the amount of resources consumed is a detail conspicuously absent. Where this falls on the spectrum between superhuman resource efficiency and unfathomable brute force is, from the outside, unknown. That links into the nature of the problems they publicly released proof of: they're iterative improvements to established techniques or boundaries, not entirely novel in process. The evaluation loop also requires that the problem being investigated be correctly represented in code and be objectively measured. For example, reducing the number of steps in multiplying matrices is readily represented and counting steps is a clear metric. Making discoveries that change the landscape being worked within are therefore inherently difficult to evaluate in that framework.</li> <li>None the less, this is a major development that is being visibly mirrored in open source projects, proprietary competitors, and by Google themselves previously (AI co-scientist and FunSearch) and is likely to become increasingly relevant. The idea has likely been driving specific design decisions of LLMs, for example the massive 1 million token contexts of Google Gemini 2.5 or 10 million token context of Llama 4 Scout. This links into more speculative ideas that AI systems are likely to become geopolitically relevant, sometimes articulated as 'a country full of geniuses in a datacenter' within a few years. In other words, that these systems will become advanced enough that they'll either serve as force multipliers for the researchers of a given country or be capable of independent research themselves, either way increasing the total scientific output of the host nation. Or even more to the point, while AlphaEvolve heavily emphasizes humans directing the control loop, there's no reason more layers couldn't fit on top, an 'Alpha-AlphaEvolve', so to speak.</li> </ul> <h2 id="decentralized-ai-development"><a class="zola-anchor" href="#decentralized-ai-development" aria-label="Anchor link for: decentralized-ai-development"><i class="icon"></i></a> Decentralized AI development</h2> <ul> <li>Prime Intellect Inc. Intellect 2 <ul> <li>https://huggingface.co/PrimeIntellect/INTELLECT-2</li> <li>https://storage.googleapis.com/public-technical-paper/INTELLECT_2_Technical_Report.pdf</li> </ul> </li> <li>Psyche Nous Research <ul> <li>https://nousresearch.com/nous-psyche/</li> <li>https://psyche.network/runs/consilience-40b-1/0</li> </ul> </li> </ul> <h3 id="background"><a class="zola-anchor" href="#background" aria-label="Anchor link for: background"><i class="icon"></i></a> Background</h3> <ul> <li>Opposite from large, centralized projects there were major developments for democratized and decentralized AI, with Prime Intellect Inc. publishing their decentralized post training of Intellect 2 and Nous Research launching their decentralized training network, Psyche, and beginning the pretraining of the first model on it, Consilience. Some background about how AI models, specifically large models like leading LLMs or autonomous vehicle models, are trained should help contextualize the following. Llama 4 models, Meta/Facebook's latest LLMs, were trained in a data center (or multiple) with more than 100,000 GPUs and thousands of terabytes of collective memory. Critically, part of the concept of neural networks is that they 'discover' long distance relationships in the training data, but to do that all the relevant data and parts of the model (traditionally) need loaded into memory and mutually accessible. Bandwidth and latency (among other factors) then become critical bottlenecks; Meta's datacenters, while training Llama 4, averaged an effective utilization of the GPUs around 20% (390 realized versus 1979 theoretical FP8 TFLOPS (8 bit floating point tera/10^12 operations per second) per H100 (Nvidia's at the time flagship) GPU), and that was meant to be a strong result. Consequently, the idea of decentralizing machine learning training across data centers, let alone more distant compute nodes, is impractical using traditional methods.</li> <li>The aforementioned developments address decentralization of AI training from different ends of the lifecycle: Prime Intellect Inc.'s Intellect 2 addressing post training/fine tuning and Nous Research's Psyche/Consilience addressing pretraining. Here, pretraining means the process of taking a massive body (corpus) of training text, often measured in trillions of tokens, and deriving a 'base model' which embodies the patterns contained, both syntax and later semantics; effectively embedding the 'world view' collectively represented. Post training/fine tuning is the process of taking that raw base model and making it practically useful, for example formatting responses according to human taste (often a chat dialogue now), having a specific tone, or emphasizing specific knowledge. There are a lot of complexities and the terms are used differently in some contexts, however. Both Intellect 2 and Psyche use/d a variety of novel techniques, the details of which I won't elaborate on, to most efficiently break up tasks, communicate them, and integrate the results to effectively decentralize their chosen parts of the cycle.</li> </ul> <h3 id="intellect-2-post-training"><a class="zola-anchor" href="#intellect-2-post-training" aria-label="Anchor link for: intellect-2-post-training"><i class="icon"></i></a> Intellect 2: Post Training</h3> <ul> <li>Intellect 2 is finished, and was a fine tuning effort to increase the reasoning, math, and science capabilities of the existing open source Qwen QwQ 32B model, itself already strong in those domains. The final model shows minor but observable performance improvements in AIME24 and AIME 25 (a human math competition that's now used as a benchmark), LiveCodeBench (programming benchmark), and GPQA Diamond ('graduate level Google Proof Q&amp;A'; a scientific reasoning benchmark). The accompanying paper shows in detail their methodology, with most if not all of their code being open source and reproducible, and several ablation experiments show the necessity and efficacy of the various techniques they employed. The results are minor, but demonstrate proof of concept.</li> </ul> <h3 id="psyche-pre-training"><a class="zola-anchor" href="#psyche-pre-training" aria-label="Anchor link for: psyche-pre-training"><i class="icon"></i></a> Psyche: Pre Training</h3> <ul> <li>To clarify, the aforementioned Psyche/Consilience describes Nous Researche's Psyche, the decentralized training architecture and platform, and Consilience, the name of the first model being trained on it. They similarly use many techniques to mitigate the inherent difficulties in decentralized pretraining. At time of writing it's only been a few days and 0.5% progress so there are no results to speak of beyond the system continuing to function. If one is willing to be optimistic, there are signs it's actually working; the 'loss' function (the most basic indicator that a machine learning system is in fact 'learning'), has continually decreased and Hellaswag (one of the most basic benchmarks that shows basic linguistic common sense by predicting the next word in a sentence) has steadily increased beyond random guessing.</li> </ul> <h3 id="caveats-challenges-questions"><a class="zola-anchor" href="#caveats-challenges-questions" aria-label="Anchor link for: caveats-challenges-questions"><i class="icon"></i></a> Caveats, Challenges, Questions</h3> <ul> <li>Theoretically, if these techniques work at scale and can be integrated, an entirely decentralized model pipeline becomes possible. That's a major if, however, and there are many caveats and problems along the way. Chief among these is the exact definition of 'decentralized'. While there are philosophical arguments to be made as to who controls and directs these compute networks, there are practical barriers for would be contributors in the form of resource requirements. Intellect 2 had the lowest barrier, with the most basic of the tasks capable of being ran on hardware costing around $5,000 (generously, assuming used RTX 3090 x 4). The majority of tasks for Intellect 2 required datacenter grade clusters and all tasks for Consilience (currently, may not be a hard requirement) use H100x8 clusters which cost at least $300,000. Beyond that, there are then major elephants in the room for both. For Intellect 2, it was essentially a proof of concept that resulted in barely observable performance improvements on already production ready models. For Consilience, at this rate the model won't be finished training for at least 2.5 years (although every time I've checked back the number has dropped, my first check placed it at 5 years).</li> <li>Despite the caveats and difficulties, the existence and apparent proof of concept for both of these demonstrate a real potential for decentralized and open source models to be a presence in the AI ecosystem alongside monolithic datacenters costing into the tens and hundreds of billions of US dollars. Other decentralized computing projects, such as the Folding at Home or SETI at Home series, have yielded, and continue to yield, positive and practical results but the path forward with decentralized AI remains for now an open question.</li> </ul> <h2 id="openai-codex"><a class="zola-anchor" href="#openai-codex" aria-label="Anchor link for: openai-codex"><i class="icon"></i></a> OpenAI Codex</h2> <ul> <li>https://openai.com/index/introducing-codex/</li> <li>On a more practical note, OpenAI released an agent based coding tool called Codex. It bears conceptual similarities to existing agent based coding tools, ie you give it instructions and it does the coding and work for you in the background (some call it 'vibe coding'), but has practical points of differentiation compared to others. Most current tools, eg Cursor or Jetbrains's Junie, work as parts of a local editor and so operations happen locally. Codex is however a cloud based system; it works within a virtualized environment in the cloud where it pulls in your code and an LLM executes tasks. The LLM used is a specially tuned variant of OpenAI's flagship o3 reasoning model, itself dubbed codex-1. Altogether, it's a different experience from other agent coding environments wherein you kick off jobs and tasks and Codex handles them asynchronously while you do other things. Crucially, it's capable of multitasking both within a given task and between tasks, again a point of differentiation from existing agent coding tools. This asynchronous task based behavior bears some resemblance to the prior Google AlphaEvolve story and no doubt features its own analogous control loop, just this time more focused on practical tasks. I can't comment on how it feels to use, as it's currently only available on OpenAI's $200/month ChatGPT Pro or Enterprise subscriptions.</li> <li>(So to disambiguate all the things called codex, and for fun: OpenAI released four models in a 'Codex' family 2021 that were GPT-3 fine tunes, a CLI tool called Codex CLI in April 2025, this cloud based agent coding system in May 2025, another coding model called codex-1 that's a o3 fine tune, alongside which came codex-mini-latest that is an o4-mini fine tune that was released specifically for Codex CLI but is also available via API).</li> </ul> <h2 id="glossary"><a class="zola-anchor" href="#glossary" aria-label="Anchor link for: glossary"><i class="icon"></i></a> Glossary</h2> <ul> <li>LLM: Large Language Model, exemplified by Anthropic Claude 3, OpenAI GPT 4, Llama 3/4, and typically, are made from large bodies of data and generate responses to prompts.</li> <li>TPU: Tensor Processing Unit, Google's terminology for their custom AI compute accelerators.</li> <li>Token/tokens: LLMs (typically) don't work with human alphabets or vocabulary directly. For practical reasons, words, syllables, phonemes, and other structures are replaced by numbers. Consequently, words are on average made up of more than 1 token, although this depends on the specific methodology. <a rel="external" href="https://platform.openai.com/tokenizer">Try a demo here.</a></li> <li>Corpus: General term to encompass the training data a base model will be trained on. Often contains scraped internet data, books, academic papers, and more. Recent models include visual data as well.</li> <li>Base model: Base models for LLMs are essentially a raw embodiment of the syntactic patterns, semantic meanings, and, at the highest levels, 'world view' the training data embodies. These are generally not practically useful, and can only 'complete' sentences in probable ways without respect to the desired tone, style, and base in reality.</li> <li>Fine-tuning: Makes a base model practically useful by fixing the style and facts according to human preferences. The most common, or rather visible, example is 'instruct' tuning which allows the common 'chat' back and forth exchange. Other fine tuning aims include adopting a specific style, making responses useful, avoiding offensive or sensitive output, or adding domain specific knowledge/technique.</li> <li>Loss function: Technically present at many levels during training, it generally means an evaluation score that the optimization is 'trying' to minimize. It's effectively the most basic indicator of training progress.</li> <li>TFLOPS: or Tera, 10^12, FLOPS, floating point operations per second, is a basic measure of number of calculations per second a system can do on decimal numbers. It's a rough measure and not always directly comparable to other devices or operation types.</li> <li>FP8: Short for floating point 8 bit, or a decimal number represented by 8 bits. Increasingly used in machine learning as it can be more efficiently operated on and stored compared to 16 or 32 bit decimal representations.</li> <li>H100: Nvidia's flagship AI focused GPU released in 2022. Cost between $20,000-$ 40,000 per card, often clustered in groups of 8 for a single server.</li> <li>AIME: Short for American Invitational Mathematics Examination, originally a human competitive math exam but has also been used to evaluate LLM reasoning skills.</li> <li>LiveCodeBench: A continually updated benchmark that tests LLM coding skills.</li> <li>GPQA: Google Proof Question Answering, a 'graduate level' scientific reasoning benchmark.</li> <li>Hellaswag: A 'common sense' language benchmark that evaluates basic linguistic skills by predicting the next word in a sentence.</li> </ul> Mon, 12 May 2025 00:00:00 +0000 Human https://notes.rcr.pub/tech_notes/week-of-2025-05-05-to-2025-05-12/ https://notes.rcr.pub/tech_notes/week-of-2025-05-05-to-2025-05-12/ <h2 id="weekly-sampling-of-the-week-s-emerging-tech-developments"><a class="zola-anchor" href="#weekly-sampling-of-the-week-s-emerging-tech-developments" aria-label="Anchor link for: weekly-sampling-of-the-week-s-emerging-tech-developments"><i class="icon"></i></a> Weekly sampling of the week's emerging tech developments</h2> <ul> <li>It feels like the first time in awhile, but there seems like there's been a relative lull in flashy AI news. That said, the pace of research continues at an incredible pace, with Huggingface's featured papers numbering 89 (to say nothing of the hundreds on Arxiv or in journals). Google IO may be a cause of and an end to the relative drought, with events starting Tuesday May 20th. All of this remains relative though, there has long been too much for anyone to keep up with. In any case, here's a sample including pieces on AI software, hardware, and theory along with robotics and XR.</li> </ul> <h2 id="reasoning-models-don-t-always-say-what-they-think"><a class="zola-anchor" href="#reasoning-models-don-t-always-say-what-they-think" aria-label="Anchor link for: reasoning-models-don-t-always-say-what-they-think"><i class="icon"></i></a> Reasoning Models Don't Always Say What They Think</h2> <ul> <li><a rel="external" href="https://arxiv.org/abs/2505.05410">https://arxiv.org/abs/2505.05410</a></li> <li><a rel="external" href="https://www.anthropic.com/research/reasoning-models-dont-say-think">https://www.anthropic.com/research/reasoning-models-dont-say-think</a></li> <li>Depending on the benchmark, essentially all the leading AI models currently are 'reasoning' models that use a technique called 'chain of thought' (CoT) to improve their responses. While implementations differ, the technique aims to mitigate the limitations of autoregressive models, that is models that generate tokens strictly sequentially based on only the previous ones, by allowing a model to 'think' before giving a final response. Early on, this was conjectured to be likely useful for interpreting and debugging model responses, in particular looking for 'misalignment' where the model does something undesirable or subversive to user intent. Anthropic in this paper argues, at very least for Claude 3.7 and Deepseek R1, that 'chain of thought' is not capable of that. They set up a series of experiments where the model would be asked questions that it had 'hints' or 'reward hacks' for and analyzed the 'thinking' traces. Generally, there was no reliable indication that they referred to these implicitly illicit aids and appeared to construct fictional justifications to mask this. In other words, they don't 'say what they think'. Most of these reasoning models are heavily informed by reinforcement learning; these traces aren't 'reasoning' in an anthropomorphic sense but are rather an emergent behavior that optimizes problem solving accuracy, nothing more. Perhaps analogous to humans doing scratch work or sketches on an exam where there are fragments that help get to the answer but aren't high fidelity representations of 'thinking' itself.</li> </ul> <h2 id="llama-cpp-unifies-and-extends-self-hosted-ai-vision-capabilities"><a class="zola-anchor" href="#llama-cpp-unifies-and-extends-self-hosted-ai-vision-capabilities" aria-label="Anchor link for: llama-cpp-unifies-and-extends-self-hosted-ai-vision-capabilities"><i class="icon"></i></a> Llama.cpp Unifies and Extends Self-Hosted AI Vision Capabilities</h2> <ul> <li><a rel="external" href="https://github.com/ggml-org/llama.cpp/blob/master/docs/multimodal.md">https://github.com/ggml-org/llama.cpp/blob/master/docs/multimodal.md</a></li> <li><a rel="external" href="https://github.com/ggml-org/llama.cpp/releases/tag/b5331">https://github.com/ggml-org/llama.cpp/releases/tag/b5331</a></li> <li>Llama.cpp, one of the leading open source self-hosted Large Language Model (LLM) runtimes, added unified support for some open weights models to process both text and visual input this week. Practically, this means most people can download an open and free model to a relatively recent laptop and run vision capable AI models on it. Originally there was support for the LLaVA (Large Language and Vision Assistant) architecture, however the complexities and unique architectures led to the creation of unique implementations for several models. Deemed unmaintainable, work began on 'mtmd' (MulTi-MoDal) as a universal replacement and was shipped in release b5331. Supported models include Gemma 3, SmolVLM 1 &amp; 2, Pixtral 12B, Qwen 2 VL and Qwen 2.5 VL, Mistral Small 3.1 24B, among others.</li> </ul> <h2 id="neuromorphic-computing-finding-application"><a class="zola-anchor" href="#neuromorphic-computing-finding-application" aria-label="Anchor link for: neuromorphic-computing-finding-application"><i class="icon"></i></a> Neuromorphic Computing Finding Application</h2> <ul> <li><a rel="external" href="https://www.science.org/doi/10.1126/sciadv.adv2312">https://www.science.org/doi/10.1126/sciadv.adv2312</a></li> <li>A paper titled "Fully memristive spiking neural network for energy-efficient graph learning" was published in Science and claimed to experimentally demonstrate one of the first functional memristor based spiking neural networks. Memristors, or memory resistors, are relatively new to practical use but are fundamental components to circuit design, alongside resistors, capacitors, and inductors. In simple terms, they have a persistent but variable once set resistance that's dependent on the current previously passed through them. Spiking neural networks on the other hand take the classic neural network idea of emulating biological brains more literally by operating via discrete spikes as opposed to (practically) continuous transformations. Combined, these create a more direct analogue to biological neural networks and is part of the field of neuromorphic computing. The paper describes the manufacturing and use of such a memristive spiking neural network at non trivial scale and problem type. According to their results, the network was successfully applied to shortest path problem and graph classification use cases which have countless downstream applications, as simple as GPS navigation up to and beyond drug interaction study. Critically, even this experimental version consumed several orders of magnitude less energy than existing solutions. Replication has been difficult for this technology historically, but the benefits described and existing commercial interest make (relatively) speedy verification likely.</li> </ul> <h2 id="self-play-and-improvement-for-llms"><a class="zola-anchor" href="#self-play-and-improvement-for-llms" aria-label="Anchor link for: self-play-and-improvement-for-llms"><i class="icon"></i></a> Self Play and Improvement for LLMs</h2> <ul> <li><a rel="external" href="https://arxiv.org/abs/2505.03335">https://arxiv.org/abs/2505.03335</a></li> <li><a rel="external" href="https://github.com/LeapLabTHU/Absolute-Zero-Reasoner">https://github.com/LeapLabTHU/Absolute-Zero-Reasoner</a></li> <li><a rel="external" href="https://andrewzh112.github.io/absolute-zero-reasoner/">https://andrewzh112.github.io/absolute-zero-reasoner/</a></li> <li>Quality and quantity of training data has been a persistent struggle in AI, but perhaps not an eternal one according to this research. There have been various approaches to learning, but humans have ultimately been the ones assigning value or providing examples. This paper proposes and demonstrates an absolute zero reasoner training approach, wherein a model proposes, solves and judges in the training without human intervention. Alongside this the repository used to achieve strong results across many models was published, so replication should be swift and broad.</li> </ul> <h2 id="perception-reason-think-and-plan-a-survey-on-large-multimodal-reasoning-models"><a class="zola-anchor" href="#perception-reason-think-and-plan-a-survey-on-large-multimodal-reasoning-models" aria-label="Anchor link for: perception-reason-think-and-plan-a-survey-on-large-multimodal-reasoning-models"><i class="icon"></i></a> Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models</h2> <ul> <li><a rel="external" href="https://arxiv.org/abs/2505.04921">https://arxiv.org/abs/2505.04921</a></li> <li>We have here a useful systematic survey on the state of the art in reasoning models and provides a rough framework for understanding their capabilities. A dense but useful read.</li> </ul> <h2 id="amazon-deploying-adaptive-arms"><a class="zola-anchor" href="#amazon-deploying-adaptive-arms" aria-label="Anchor link for: amazon-deploying-adaptive-arms"><i class="icon"></i></a> Amazon Deploying Adaptive Arms</h2> <ul> <li><a rel="external" href="https://www.aboutamazon.com/news/operations/amazon-vulcan-robot-pick-stow-touch">https://www.aboutamazon.com/news/operations/amazon-vulcan-robot-pick-stow-touch</a></li> <li><a rel="external" href="https://manufacturingdigital.com/news/vulcan-amazons-first-ai-robot-that-can-see-and-feel">https://manufacturingdigital.com/news/vulcan-amazons-first-ai-robot-that-can-see-and-feel</a></li> <li><a rel="external" href="https://techcrunch.com/2025/05/11/amazon-offers-peek-at-new-human-jobs-in-an-ai-bot-world/">https://techcrunch.com/2025/05/11/amazon-offers-peek-at-new-human-jobs-in-an-ai-bot-world/</a></li> <li>While certainly under development for years, Amazon recently publicized a robot they call 'Vulcan.' Unlike most existing industrial robotic arms, which are highly specialized and tightly programmed, this one uses a variety of fused sensors, various tools, and machine learning to more dynamically manipulate objects. According to Amazon these can already handle 75% of types of items in their warehouses. While somewhat vague, it does sound like these are being deployed in real roles in Amazon warehouses already, with broader rollout over the next 2 years.</li> </ul> <h2 id="xr-in-the-real-world"><a class="zola-anchor" href="#xr-in-the-real-world" aria-label="Anchor link for: xr-in-the-real-world"><i class="icon"></i></a> XR in the Real World</h2> <ul> <li><a rel="external" href="https://www.uploadvr.com/walking-70-miles-in-vision-pro-cubic-layer/">https://www.uploadvr.com/walking-70-miles-in-vision-pro-cubic-layer/</a></li> <li><a rel="external" href="https://apps.apple.com/us/app/cubiclayer/id6743873805">https://apps.apple.com/us/app/cubiclayer/id6743873805</a></li> <li>Walk a mile in someone else's shoes, or walk 70 in an extended reality (XR) headset; at least one developer chose the latter. They built an app to take advantage of the environment mapping and mixed reality (MR) capabilities of the Apple Vision Pro and came away with a geospatial voxel editor. Or alternatively, a Minecraft inspired sandbox fixed in the real world. The videos in the UploadVR article are difficult to put to words and are worth the watch.</li> </ul>