Week of 2025-06-02 through 2025-06-08

Table of Contents

1. Week of 2025-06-02 through 2025-06-08
2. Rules for thee
3. Kuleshov's VA Visit
4. If I only had a brain
5. LLM Memorization
6. IRS Puts Tax Filing Software in Public Domain
7. Open Deep Research - Lite
8. Model and Dataset Roundup
   1. Qwen 3 Embedding + Reranking
   2. OpenThoughts
   3. Ultra Fineweb

Week of 2025-06-02 through 2025-06-08

• This week was a rush, with interesting research, public policy and ethical... situations, and several quality open source contributions. This week's news seems to have skewed more critical, but breathless cheerleading can be left to press releases. To quote Rush, "If you choose not to decide, You still have made a choice"; there is no sitting on the sidelines as the stakes get higher.

Rules for thee

• https://www.anthropic.com/news/claude-gov-models-for-u-s-national-security-customers
• https://arstechnica.com/ai/2025/06/anthropic-releases-custom-ai-chatbot-for-classified-spy-work/
• https://www.anthropic.com/news/expanding-access-to-claude-for-government
• https://support.anthropic.com/en/articles/9528712-exceptions-to-our-usage-policy
• https://www.anthropic.com/news/national-security-expert-richard-fontaine-appointed-to-anthropic-s-long-term-benefit-trust
• This story isn't without precedence, in the industry or for Anthropic themselves, but still deserves attention as AI seeps ever more into both our practical daily lives and high level geopolitics. Currently, practically all publicly accessible LLMs have substantial training causing them to reject certain user requests or respond in very specific ways. Call it 'safety', 'censorship', simple liability avoidance; the behavior of actively rejecting a user's query. Whether reasonable or informed by sci-fi pulp, every major vendor I'm aware of claims to be developing AI responsibly. Who watches the watchmen, though, asks the wisdom of at least two thousand years ago (Juvenal).
• Now, the actual story; Anthropic announced 'Claude Gov', a variant of their standard models more tuned to sensitive government work. The details are extremely sparse, with practical elements such as extra training for specific languages and specific data formats, but also the very conspicuous 'models refuse less.' Again, this isn't new, Anthropic and OpenAI have both done this before, and open weights/source models are regularly decensored. The first order consequences of a US government agency using or not using Anthropic models is unlikely to shake the world. But let's dig a bit deeper. While, again, very vague, Anthropic does list some use cases they permit for specific government agencies that others would be prohibited from; the only actual example they give is 'foreign intelligence analysis', but note they still prohibit domestic surveillance, cyberwarfare, among others. They go on to list some factors they weigh in granting what kinds of access, such as safeguards, legal mandate of the agency, and 'degree of independent and democratic oversight.'
• The crux of the, shall we say, tension here is that AI tools are considered by both the US government and AI companies as powerful enough to be of material significance to national security. Anthropic in particular seems to think this so much so that they want to determine on a case by case basis the trustworthiness of government agencies. Suppose they approve agency A and reject agency B; agency A would presumably be more efficient or productive in pursuit of their goals. If the tools are sufficiently powerful, which is admittedly not a guarantee, this effectively lets the vendor pull levers more or less directly to shape US policy and national security. There are factors that make the effects more or less acute, but in general if the products didn't help the government wouldn't want them and the use would be irrelevant.
• There is irony in Anthropic selecting agencies based on 'democratic oversight', since they themselves are making these decisions unilaterally. Maybe their decisions are right, maybe they're wrong; they're unaccountable to the people insofar as they comply with the law and remain profitable. Speaking of oversight, I find it improbable it is coincidence that Claude Gov was posted about June 5th 2025 and then announced June 6th 2025 that Richard Fontaine would be joining Anthropic's Long-Term Benefit Trust, a higher level entity with partial control/influence over Anthropic's, the public benefit corporation, board of directors and hence the company. I won't go into details, and don't mean this as evidence of wrongdoing, only evidence of perspective, but Fontaine comes from a foreign policy analyst/advisor background and is currently CEO of 'Center for a New American Security', a national security centric policy think tank in Washington DC.
• I'll close this part with the following quote from the end of Anthropic's announcement about Fontaine. I agree with the premises, and I don't think current AI capabilities or market concentration (or lack thereof) make the current situation irreversible; I think as AI capabilities scale and are used in government so too should the accountability to the people, not lying with self appointees within corporations.
• “Richard’s expertise comes at a critical time as advanced AI capabilities increasingly intersect with national security considerations,” said Dario Amodei, CEO and Co-founder of Anthropic. “I've long believed that ensuring democratic nations maintain leadership in responsible AI development is essential for both global security and the common good. Richard’s appointment strengthens the Trust’s ability to guide Anthropic through the complex decisions that will shape not just our industry, but the global balance of power.”

Kuleshov's VA Visit

• https://sahillavingia.com/god
• https://www.propublica.org/article/trump-doge-veterans-affairs-ai-contracts-health-care
• https://sahillavingia.com/doge
• https://github.com/slavingia/va
• I highly suggest reading, or skimming, the links in order. Events surrounding DOGE and the VA have been in the news for months, but this collection crystallizes some of those VA events in a way relevant to AI adoption. In very brief, Sahil Lavingia, founder of the Gumroad ecommerce platform, published on October 17th 2024 a post 'GOD Mode', advocating for aggressive, dictatorial company management and widespread AI automation; on March 17th 2025 he joins DOGE at the VA and set to reviewing contracts; he wrote a simple and fatally flawed script that reviewed all VA contracts and attempted to use AI to classify them as 'munchable' and 'non munchable'; the script was apparently used to make authoritative, or rubber stamped by humans, decisions cancelling critical, sometimes life critical, contracts and employments; on May 9th 2025 he was fired after doing an interview the day prior discussing his work; on June 6th ProPublica published their reporting of the events.
• This story and its direct or indirect relations have been extensively reported on, but I highly suggest reading the given links to get a sense of things. While there are many very promising uses for AI, this is a fascinating case study of exactly how not to use it. The ProPublica article and Lavingia's own writings speak for themselves.

If I only had a brain

• https://machinelearning.apple.com/research/illusion-of-thinking
• Apple, a notable absentee from the frontier model races, this week published an interesting paper challenging directly the narrative that recent 'thinking' models, namely OpenAI's o1/o3, Deepseek's R1, and Claude and Gemini thinking variants, are in fact thinking at all. The findings of this paper are relatively straightforward and, depending on your perspective, unsurprising, but the context and framing is critical to navigating the noisy AI landscape. Instead of the usual batch of benchmarks, like MATH-500 or AIME 24/25, Apple presented models with some classic and basic puzzles: tower of Hanoi, checkers jumping, river crossing, and blocks world. These tasks are common brain teasers and classic problems in the field of AI or computer science generally, with canonical solutions that guarantee a correct solution. That last point is critical; even when presented with the optimal algorithms, all observed models consistently failed to solve the problems past a certain point. Some of the performances are still above human capabilities, but theoretically a 'reasoning' model of the given architectures would be expected to solve the puzzles consistently. The paper goes into greater detail, describing reasoning tasks as falling into 3 domains of low, medium, and high complexity, for which non reasoning models excel at the first, reasoning the second, and neither the third.
• While the results aren't particularly groundbreaking, the contrast of AI companies describing their models as 'thinking' or 'reasoning', and being highly incentive to do so, and Apple, lacking models of their own, downplaying these capabilities is illustrative.

LLM Memorization

• https://arxiv.org/abs/2505.24832
• Meta, together with contributors from Google DeepMind, Nvidia, and Cornell University, published on Arxiv this week a paper attempting to quantify how much current generation LLMs can memorize given a certain size. General use of LLMs shows that there is some degree of 'memorization' going on, wherein the model can retrieve verbatim specific information. Machine learning, as in the more general field, describes a phenomenon called 'overfitting', where a model such that it functionally 'memorizes' the training dataset without extracting more general trends. This tends to happen when the model is too large relative to the training data or is otherwise poorly configured.
• This paper effectively claims to have quantified, with caveats, the threshold where LLMs of this type go from memorizing and overfitting to generalizing. Additionally, the methodology may allow estimation of proportions of memorization versus generalization, for example letting us say architecture X seems to be 80% memorization whereas architecture Y is only 30% memorization. There are many practical and theoretical implications if this research shakes out. Theoretically, it gives a foothold for understanding how models represent 'knowledge' internally, with the linear relationship exposed through an information theoretic approach strongly arguing there is real generalization occurring. For a practical human example, when learning arithmetic it's common to memorize tables from 1-12, but beyond that it becomes prohibitive to memorize, so we learn the patterns and rules instead. This research, again if validated, has massive practical implications even in the immediate future. With this pattern being apparently predictive, creators of LLMs can effectively decide up front how much information they want it to be possible to memorize, for example an LLM made for law or biology would likely need to memorize more than a model made for theoretical math. Much more practically, this kind of research is likely to come up in the numerous lawsuits against AI companies for alleged copyright infringement, for example the New York Times lawsuit againts OpenAI. Were AI companies to show the training data was much greater than the parameter count and the resulting model had a relatively low memorization component they may make a much better case for themselves. Alternatively, it may show the opposite, a possibility considering how even the open weight or open source models are generally opaque as to their training procedure.
• There are, however, major caveats and good reason not to take this paper as gospel. Most obviously, like many research papers of this type, these results are on models far smaller than current state of the art and far older; the ones analyzed are of GPT-2 design and range from millions to 1.5 billion parameters. While 1.5B can be fairly coherent, SOTA for open weight/source models is in the hundreds of billions with proprietary flagships expected to be in the low trillions of parameters. That's no small difference either, since that threshold is the difference between GPT-2 or PaLM era models being a curiosity to GPT-3 and later models becoming practically useful. There's also the assumption that distinguishing 'memorization' and 'generalization' into binary categories is a reasonable one. The existence of mnemonics in humans and compression with dictionaries in math/CS suggests otherwise. Finally, author motivation can't be ignored. Not to disparage any of the authors, and the involvement of Cornell academics does help, but Meta, Nvidia, and Google collaborating and publishing this research, with these limitations (considering they're effectively self imposed), at this time invites questions and, to put it politely, is quite convenient for them. A paper stating, or representable to a jury as stating, that models indeed don't memorize would be incredibly valuable to them. So while this research is noteworthy, it needs validation and extension into larger models and different architectures to know for sure.

IRS Puts Tax Filing Software in Public Domain

• https://github.com/IRS-Public/direct-file
• https://chrisgiven.com/2025/05/direct-file-on-github/
• https://www.irs.gov/pub/irs-pdf/p5969.pdf
• The IRS, in recent years, started a program called 'Direct File' that allowed certain tax payers to prepare and file their taxes directly with the IRS, as opposed to using commercial software or tax professionals. Notably, entirely for free. Generally, the program was well received but the political future is uncertain, to say the least. In any case, this week (or last week? there are two initial commits) the source code for most of the application was published on Github under CC0, so entirely in the public domain.
• Despite what common perception of government software would suggest, the repository suggests remarkably modern software development practices; well documented, tested, containerized. Perhaps most fascinatingly, both from an academic and practical standpoint, is the representation of the tax code as a 'fact graph'. I haven't attempted to navigate the details, but I suspect a canonical implementation of the tax code in a modular format like this could inspire either open source forks or more generally other approaches to handling the often messy interface between messy human constructs and rigid digital systems.

Open Deep Research - Lite

• https://github.com/google-gemini/gemini-fullstack-langgraph-quickstart
• https://huggingface.co/blog/lynn-mikami/google-opensource-deepresearch
• Many AI companies, most popularly Google and OpenAI, have released 'Deep Research' tools wherein their models use their long contexts and search tools to iteratively research and report on a given topic. This can be quite handy for getting the sense for the landscape of a complex topic, since these tools can search across literally hundreds of websites and synthesize the results. They're of course not perfect, with all the now classic LLM pitfalls, but can certainly save a lot of time since you can kick off a prompt and do something else while it spins for some 15 minutes. There have been many open source reimplementations of similar functionality, but these predictably don't feel as polished, lacking a legion of paid engineers.
• Google, despite having one of the best research tools, decided to create an open source 'starter' repository replicating most of the functionality and presumably with the benefit of their experiences. The implementation details are actually remarkably unremarkable; it uses common open source libraries, particularly LangGraph for LLM interaction and react for the front end. While the implementation is specific to their Gemini API and Google Search API, the use of Langchain makes it almost, but not quite, trivial to swap for general models or other search tools. While this internet based research is itself useful, there are many tasks that can be conceptualized as iterative control loops calling external resources; this repository can quite easily be used a scaffold or high level blueprint for analogous projects, particularly since it's Apache 2.0 licensed.

Model and Dataset Roundup

Qwen 3 Embedding + Reranking

• https://huggingface.co/collections/Qwen/qwen3-embedding-6841b2055b99c44d9a4c371f
• https://huggingface.co/collections/Qwen/qwen3-reranker-6841b22d0192d7ade9cdefea
• Qwen 2/2.5 featured some of the highest performing embedding models and Qwen 3's versions seem to knock it out of the park. The embedding series handily tops the MTEB (common embedding benchmark) leaderboard, with even the 0.6B model beating prior gen 7B models. 4B and 8B even beat (on most metrics) the prior champion, gemini-embedding-001. Apache 2.0 licensed.

OpenThoughts

• https://huggingface.co/datasets/open-thoughts/OpenThoughts3-1.2M
• https://arxiv.org/abs/2506.04178
• 1.2 million rows of reasoning traces, extensively refined and filtered through extensive pipelines. The paper details their methodologies, which are likely to be applied to further datasets. Apache 2.0 licensed.

Ultra Fineweb

• https://huggingface.co/datasets/openbmb/Ultra-FineWeb
• https://arxiv.org/abs/2505.05427
• 1.29 trillion rows of quality, filtered training data that shows clear performance improvements for models trained on it versus prior Fineweb releases. The paper also details their refinements to the pipeline used to make it, since quality was a core focus this time. Apache 2.0 licensed.