NO/FOMO

Independent AI signal, once a day

The AI briefing worth opening.

ISSUE DATE2026-01-20ENGLISH EDITION
This issue
—
All time
—

Hacker News

6 stories
01

Show HN: Mastra 1.0, open-source JavaScript agent framework from the Gatsby devs

Mastra 1.0, an open-source TypeScript agent framework developed by the creators of Gatsby, has officially launched in stable release. Following its initial debut on Hacker News nearly a year ago, the project has experienced substantial growth, now boasting over 300,000 weekly npm downloads and 19,400 GitHub stars. Licensed under Apache 2.0, Mastra is already deployed in production environments at companies like Replit, PayPal, and Sanity. The framework provides a comprehensive toolkit for agent development, including features for creating multi-agent workflows, running evaluations, conducting inspections in a local studio, and emitting observability data. This stable release underscores its maturity and capacity to support the rapidly advancing landscape of AI agent technologies, offering developers a robust platform to build, manage, and monitor complex agent systems.

MastraTypeScriptAgent FrameworkOpen-sourceMulti-agent WorkflowsAI AgentArtificial IntelligenceMachine Learning
02

Running Claude Code dangerously (safely)

This article critically examines the challenges and solutions involved in executing code generated by large language models, particularly focusing on Anthropic's Claude, in practical environments. The central theme revolves around reconciling the inherent risks of running untrusted, AI-produced code with the desire to leverage its potential for automation and development. It delves into strategies for creating secure execution environments, such as sandboxing, containerization, or virtual machines, to isolate and control the impact of potentially malicious or flawed code snippets. The discussion likely emphasizes best practices for code validation, threat modeling, and implementing stringent access controls. By outlining methods to safely operationalize AI-generated code, the piece provides crucial guidance for developers and organizations looking to integrate advanced AI agents into their workflows without compromising system integrity or data security, thereby turning a seemingly "dangerous" capability into a controlled and "safe" asset for innovation.

ClaudeLarge Language ModelCode ExecutionAI SecuritySandboxingArtificial IntelligenceLarge Language ModelAI Agent
03

Scaling long-running autonomous coding

The article titled "Scaling long-running autonomous coding" explores the significant challenges and prospective solutions involved in developing and deploying AI systems capable of generating and managing code autonomously over extended operational periods. This emerging domain of autonomous coding agents demands robust architectural designs to handle continuous tasks, adapt to evolving project requirements, and maintain code quality at scale. Critical considerations include efficient management of computational resources, ensuring consistent performance, and devising effective error handling and self-correction mechanisms for these intelligent systems. The discussion likely covers strategies for orchestrating multiple agents, optimizing their collaboration, and designing feedback loops that facilitate continuous improvement. Overcoming these scaling hurdles is essential for realizing the full potential of AI in software development, paving the way for more efficient and self-sufficient coding environments where AI assumes an increasingly integral role throughout the software lifecycle.

Autonomous CodingAI AgentsScalabilityCode GenerationSoftware EngineeringArtificial IntelligenceAI AgentMachine Learning
04

How Hightouch built their long-running agent harness

Hightouch has engineered a sophisticated long-running agent harness, a critical infrastructure designed to manage and orchestrate persistent software agents over extended operational periods. This development likely addresses fundamental challenges in distributed systems, including ensuring high availability, fault tolerance, and efficient resource allocation for autonomous tasks. The article is expected to delve into the core architectural principles, design patterns, and engineering decisions that underpin this robust and scalable system. Key areas of focus would include state management, resilient inter-process communication, comprehensive error handling mechanisms, and streamlined deployment strategies, all essential for maintaining operational integrity in complex, dynamic environments. The harness aims to significantly simplify the development and deployment lifecycle of agents by offering a standardized framework that abstracts away inherent complexities associated with lifecycle management and operational reliability. This technical deep-dive provides invaluable insights for engineers and architects tasked with designing and implementing similar distributed agent-based systems, highlighting best practices for creating reliable and performant agent infrastructures.

Agent SystemsDistributed SystemsSoftware ArchitectureSystem DesignFault ToleranceAI AgentArtificial IntelligenceOthers
05

Giving university exams in the age of chatbots

The emergence of advanced AI chatbots presents a significant paradigm shift for higher education, compelling universities to critically re-evaluate traditional examination structures and assessment methodologies. With sophisticated AI tools capable of generating human-like text, the integrity of written assignments and exams is under considerable scrutiny. This scenario necessitates innovative approaches to academic evaluation, moving beyond rote memorization towards fostering critical thinking, problem-solving, and analytical skills that are less susceptible to automated generation. Educational institutions are exploring adaptive strategies, including the redesign of curricula, implementation of project-based assessments, oral examinations, and integrating AI tools responsibly within learning environments. The core challenge lies in developing assessment frameworks that effectively measure genuine student comprehension and intellectual capabilities in an era where AI can readily assist in content creation, thereby ensuring academic rigor and fairness.

Artificial IntelligenceChatbotsAcademic IntegrityEducational TechnologyAssessment MethodsArtificial IntelligenceLarge Language ModelNatural Language Processing
06

OpenAI is rolling out age prediction

OpenAI has announced the implementation of an age prediction system, signaling a strategic move towards enhancing user safety and ensuring responsible AI deployment. This initiative, highlighted by a dedicated page on their website, aims to allow for tailored content experiences, enforce age-appropriate access controls, and align with global regulatory standards designed to protect minors online. The integration of such technology underscores the company's commitment to ethical AI practices and addressing the complex societal implications of advanced artificial intelligence. While specific technical details are not fully disclosed in this brief announcement, age prediction systems often leverage machine learning models to analyze various data points, including behavioral patterns and contextual information, all while maintaining a strong emphasis on user privacy and data security. This development aligns with broader industry trends focusing on more robust content governance and protective measures for digital users.

OpenAIAge PredictionAI SafetyResponsible AIUser ProtectionArtificial IntelligenceMachine LearningNatural Language Processing

huggingface

6 stories
01

ABC-Bench: Benchmarking Agentic Backend Coding in Real-World Development

The evolution of Large Language Models (LLMs) into autonomous agents has expanded the scope of AI coding from localized code generation to complex, repository-level, and execution-driven problem solving. However, current benchmarks predominantly evaluate code logic in static contexts, neglecting the dynamic, full-process requirements of real-world engineering, particularly in backend development which demands rigorous environment configuration and service deployment. To address this gap, we introduce ABC-Bench, a benchmark explicitly designed to evaluate agentic backend coding within a realistic, executable workflow. Using a scalable automated pipeline, we curated 224 practical tasks spanning 8 languages and 19 frameworks from open-source repositories. Distinct from previous evaluations, ABC-Bench require the agents to manage the entire development lifecycle from repository exploration to instantiating containerized services and pass the external end-to-end API tests. Our extensive evaluation reveals that even state-of-the-art models struggle to deliver reliable performance on these holistic tasks, highlighting a substantial disparity between current model capabilities and the demands of practical backend engineering. Our code is available at https://github.com/OpenMOSS/ABC-Bench.

Agentic Backend CodingLarge Language ModelsAutonomous AgentsBenchmarkingReal-World DevelopmentLarge Language ModelAI AgentArtificial Intelligence
02

Multiplex Thinking: Reasoning via Token-wise Branch-and-Merge

Large language models often solve complex reasoning tasks more effectively with Chain-of-Thought (CoT), but at the cost of long, low-bandwidth token sequences. Humans, by contrast, often reason softly by maintaining a distribution over plausible next steps. Motivated by this, we propose Multiplex Thinking, a stochastic soft reasoning mechanism that, at each thinking step, samples K candidate tokens and aggregates their embeddings into a single continuous multiplex token. This preserves the vocabulary embedding prior and the sampling dynamics of standard discrete generation, while inducing a tractable probability distribution over multiplex rollouts. Consequently, multiplex trajectories can be directly optimized with on-policy reinforcement learning (RL). Importantly, Multiplex Thinking is self-adaptive: when the model is confident, the multiplex token is nearly discrete and behaves like standard CoT; when it is uncertain, it compactly represents multiple plausible next steps without increasing sequence length. Across challenging math reasoning benchmarks, Multiplex Thinking consistently outperforms strong discrete CoT and RL baselines from Pass@1 through Pass@1024, while producing shorter sequences. The code and checkpoints are available at https://github.com/GMLR-Penn/Multiplex-Thinking.

Multiplex ThinkingLarge Language ModelsChain-of-ThoughtReasoningReinforcement LearningLarge Language ModelMachine LearningDeep Learning
03

Spurious Rewards Paradox: Mechanistically Understanding How RLVR Activates Memorization Shortcuts in LLMs

Reinforcement Learning with Verifiable Rewards (RLVR) is highly effective for enhancing LLM reasoning, yet recent evidence shows models like Qwen 2.5 achieve significant gains even with spurious or incorrect rewards. We investigate this phenomenon and identify a "Perplexity Paradox": spurious RLVR triggers a divergence where answer-token perplexity drops while prompt-side coherence degrades, suggesting the model is bypassing reasoning in favor of memorization. Using Path Patching, Logit Lens, JSD analysis, and Neural Differential Equations, we uncover a hidden Anchor-Adapter circuit that facilitates this shortcut. We localize a Functional Anchor in the middle layers (L18-20) that triggers the retrieval of memorized solutions, followed by Structural Adapters in later layers (L21+) that transform representations to accommodate the shortcut signal. Finally, we demonstrate that scaling specific MLP keys within this circuit allows for bidirectional causal steering-artificially amplifying or suppressing contamination-driven performance. Our results provide a mechanistic roadmap for identifying and mitigating data contamination in RLVR-tuned models. Code is available at https://github.com/idwts/How-RLVR-Activates-Memorization-Shortcuts.

Reinforcement Learning with Verifiable RewardsLarge Language ModelsMemorization ShortcutsMechanistic InterpretabilityData ContaminationLarge Language ModelNatural Language ProcessingDeep Learning
04

The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models

Large language models can represent a variety of personas but typically default to a helpful Assistant identity cultivated during post-training. We investigate the structure of the space of model personas by extracting activation directions corresponding to diverse character archetypes. Across several different models, we find that the leading component of this persona space is an "Assistant Axis," which captures the extent to which a model is operating in its default Assistant mode. Steering towards the Assistant direction reinforces helpful and harmless behavior; steering away increases the model's tendency to identify as other entities. Moreover, steering away with more extreme values often induces a mystical, theatrical speaking style. We find this axis is also present in pre-trained models, where it primarily promotes helpful human archetypes like consultants and coaches and inhibits spiritual ones. Measuring deviations along the Assistant Axis predicts "persona drift," a phenomenon where models slip into exhibiting harmful or bizarre behaviors that are uncharacteristic of their typical persona. We find that persona drift is often driven by conversations demanding meta-reflection on the model's processes or featuring emotionally vulnerable users. We show that restricting activations to a fixed region along the Assistant Axis can stabilize model behavior in these scenarios -- and also in the face of adversarial persona-based jailbreaks. Our results suggest that post-training steers models toward a particular region of persona space but only loosely tethers them to it, motivating work on training and steering strategies that more deeply anchor models to a coherent persona.

Large language modelsModel personasAssistant AxisPersona driftModel behavior stabilizationLarge Language ModelNatural Language ProcessingAI Agent
05

NAACL: Noise-AwAre Verbal Confidence Calibration for LLMs in RAG Systems

Accurately assessing model confidence is essential for deploying large language models (LLMs) in mission-critical factual domains. While retrieval-augmented generation (RAG) is widely adopted to improve grounding, confidence calibration in RAG settings remains poorly understood. We conduct a systematic study across four benchmarks, revealing that LLMs exhibit poor calibration performance due to noisy retrieved contexts. Specifically, contradictory or irrelevant evidence tends to inflate the model's false certainty, leading to severe overconfidence. To address this, we propose NAACL Rules (Noise-AwAre Confidence CaLibration Rules) to provide a principled foundation for resolving overconfidence under noise. We further design NAACL, a noise-aware calibration framework that synthesizes supervision from about 2K HotpotQA examples guided by these rules. By performing supervised fine-tuning (SFT) with this data, NAACL equips models with intrinsic noise awareness without relying on stronger teacher models. Empirical results show that NAACL yields substantial gains, improving ECE scores by 10.9% in-domain and 8.0% out-of-domain. By bridging the gap between retrieval noise and verbal calibration, NAACL paves the way for both accurate and epistemically reliable LLMs.

Large Language ModelsRetrieval-Augmented GenerationConfidence CalibrationNoise AwarenessSupervised Fine-tuningNatural Language ProcessingLarge Language ModelMachine Learning
06

SIN-Bench: Tracing Native Evidence Chains in Long-Context Multimodal Scientific Interleaved Literature

Evaluating whether multimodal large language models truly understand long-form scientific papers remains challenging: answer-only metrics and synthetic "Needle-In-A-Haystack" tests often reward answer matching without requiring a causal, evidence-linked reasoning trace in the document. We propose the "Fish-in-the-Ocean" (FITO) paradigm, which requires models to construct explicit cross-modal evidence chains within native scientific documents. To operationalize FITO, we build SIN-Data, a scientific interleaved corpus that preserves the native interleaving of text and figures. On top of it, we construct SIN-Bench with four progressive tasks covering evidence discovery (SIN-Find), hypothesis verification (SIN-Verify), grounded QA (SIN-QA), and evidence-anchored synthesis (SIN-Summary). We further introduce "No Evidence, No Score", scoring predictions when grounded to verifiable anchors and diagnosing evidence quality via matching, relevance, and logic. Experiments on eight MLLMs show that grounding is the primary bottleneck: Gemini-3-pro achieves the best average overall score (0.573), while GPT-5 attains the highest SIN-QA answer accuracy (0.767) but underperforms on evidence-aligned overall scores, exposing a gap between correctness and traceable support.

Multimodal Large Language ModelsLong-Context ReasoningScientific LiteratureEvidence ChainsBenchmarkingMultimodalLarge Language ModelArtificial Intelligence