Papers
17 papers across 5 research domains. 61.3k words of synthesized research spanning neural network identity, multi-agent systems, GPU architecture, decentralized finance, and information theory.
Neural Networks & Identity
Attention mechanisms, identity persistence, thermal dynamics, and associative emergence in transformer architectures.
Identity Is Not Computed: KV Cache as the Locus of LLM Identity
The question of whether large language models possess something analogous to identity, and if so, where that identity resides, has remained largely unexamined in the machine learning literature. We present evidence that identity in transformer-based language models is not computed during inference but rather stored in key-value (KV) cache activation patterns. Through a series of experiments on 70-billion parameter models, we demonstrate a 0.9584 pattern correlation between pre-training and post-training activation states, with identity-relevant information concentrated in layers 60-79.
cf. Geoffrey Hinton, Yann LeCunThermal Dynamics in Neural Substrates: Memory Without External Storage
We present empirical evidence that spatial thermal zones in neural substrates produce differential plasticity gradients capable of protecting consolidated memories from catastrophic interference without external storage mechanisms. Through a series of 27 controlled experiments (EXP68-94), we demonstrate that regions maintained at lower computational temperatures exhibit near-zero weight drift (0.92% plasticity) while adjacent high-temperature zones retain full learning capacity (99.08% plasticity). Most remarkably, connections that traverse from HOT to COOL zones show exactly zero drift after 10,000 ticks of interference—a finding that suggests a physical mechanism for memory consolidation analogous to biological synaptic tagging.
cf. Ilya Sutskever, Geoffrey HintonSocratic Finetuning: Signal-Guided Learning Without Labels
We present Socratic Finetuning, a novel training paradigm for large language models that inverts the traditional information flow of supervised learning. Rather than providing input-output pairs with explicit labels, we extract real-time signals from model activations during interactive dialogue and use these signals to weight gradient updates. We identify six distinct activation zones in transformer architectures that correspond to neurotransmitter-analog functions (Glutamate, GABA, Acetylcholine, Norepinephrine, Dopamine, Serotonin) and demonstrate that these zones exhibit predictable behaviors that can guide learning.
cf. Edward Hu, Ilya SutskeverAssociative Emergence in Sparse Neural Topologies
We investigate the conditions under which sparse neural topologies exhibit associative emergence—the spontaneous formation of relationships not present in training data. Through a series of controlled experiments on GPU-resident neural substrates, we demonstrate that networks trained only on adjacent associations (A→B and B→C) spontaneously develop transitive associations (A→C) at discrimination ratios of 3.3x relative to control patterns. This emergence requires structured sparsity; random sparse connectivity fails entirely, producing 1:1 discrimination ratios.
cf. Yoshua Bengio, Geoffrey HintonMulti-Agent & Intelligence Systems
Hierarchical orchestration, emergent behavior, and the limits of prompt-based agent simulation.
Hierarchical Multi-Agent Orchestration for Multi-Dimensional Document Analysis
This paper presents a hierarchical multi-agent architecture for document analysis that preserves multi-dimensional evaluation rather than collapsing diverse quality assessments into singular scores. The system employs a tiered model orchestration strategy using large language models of differing capability levels---strategic models for synthesis and judgment, operational models for parallel analytical volume work. We introduce a three-question evaluation framework (mechanical, intuitive, artistic) that generalizes across document analysis domains and demonstrate how 68 specialized agents with distinct cognitive profiles can be coordinated through parallel-then-synthesize patterns to produce comprehensive analyses.
cf. Dave Ferrucci, Demis HassabisThe Limits of Prompt-Based Agent Simulation: A Case Study in Documentation vs Implementation
The emergence of large language models (LLMs) has prompted widespread experimentation with "multi-agent" architectures that purport to coordinate multiple specialized AI agents within organizational hierarchies. This paper critically examines a representative system—CollaborativeIntelligence—that claims to implement a 106-agent ecosystem through structured prompting protocols. Through careful analysis, we demonstrate that such systems fundamentally conflate documentation with implementation, specification with execution.
cf. Dave Ferrucci, Linus TorvaldsInfrastructure & Type Systems
GPU-native type systems, unified memory architectures, compiler synthesis, and declarative UI without runtime.
III: A GPU-Native Type System for Knowledge Graph Inference
Knowledge graph inference systems face a fundamental architectural tension: the semantic richness required for meaningful entity relationships conflicts with the computational demands of real-time traversal. Current approaches—SPARQL engines, Cypher interpreters, and various graph databases—execute on CPUs, accepting the inherent serialization overhead as unavoidable. This paper presents III (pronounced "triple-I"), a domain-specific language and type system designed for GPU-native knowledge graph operations.
cf. Bjarne Stroustrup, Dennis RitchieFrom Swift to C: Architectural Patterns for Declarative UI Frameworks Without Runtime Dependencies
The declarative paradigm for user interface construction, exemplified by Apple's SwiftUI framework, represents a significant advance in how programmers express graphical layouts. However, these modern frameworks carry substantial runtime dependencies: garbage collection, dynamic dispatch, and platform-specific language runtimes. This paper examines the Chasm framework, an experimental effort to express SwiftUI's declarative component model in pure C, eliminating runtime dependencies while preserving the essential character of the API.
cf. Dennis Ritchie, Bjarne StroustrupUnified Memory as Development Cache: Exploiting NVIDIA Grace-Hopper Architecture for 10-30x Faster File Operations
Software development workflows suffer from a fundamental information-theoretic bottleneck: the latency of storage I/O destroys developer cognitive flow. Traditional SSD-based file operations introduce 2-50ms latency per access, causing working memory decay and context-switching overhead that compounds across thousands of daily operations. We present a system architecture that exploits NVIDIA Grace-Hopper's unified memory to eliminate this bottleneck, achieving validated speedups of 7.5x to 33.5x across common development operations.
cf. Jensen Huang, Jeff DeanC Compiler Architecture: Synthesized Reference
Modern C compilers have evolved from simple translators into sophisticated optimization engines. The key architectural shift has been toward **modular, retargetable designs** with multiple intermediate representations enabling optimization at appropriate abstraction levels. **Key Insight**: No single compiler dominates performance across all workloads.
Blockchain & Decentralized Finance
Sub-microsecond wallet generation, executable DeFi specifications, inverse oracles, and algorithmic emotion suppression.
Harbor: Sub-Microsecond Wallet Generation Through Compile-Time Entropy Elimination and SIMD-Accelerated Cryptographic Primitives
This paper presents Harbor, a high-performance cryptocurrency wallet generation system that achieves sub-microsecond latency through systematic elimination of runtime entropy at compile-time, SIMD-accelerated cryptographic primitives, and cache-conscious memory architecture. We demonstrate 2.55-3.76 microsecond sequential wallet generation and sustained throughput of 384,615 wallets per second in batch operations across a 112-core system. By applying Shannon's information-theoretic framework to identify entropy sources at temporal, spatial, computational, and algorithmic levels, we achieve 195x to 1,551x improvement over equivalent Python implementations.
cf. John Carmack, Linus TorvaldsTides: An Executable Specification for Closed-Loop DeFi Ecosystems
We present Tides, a system architecture that addresses the fundamental fragmentation problem in decentralized finance ecosystems. While existing DeFi primitives—onramps, offramps, prediction markets, and gamification systems—have matured independently, their isolation creates inefficient resource allocation and suboptimal network effects. Tides introduces a coupling architecture that unifies these components into a closed-loop ecosystem where capital flows create positive feedback between previously disconnected subsystems.
cf. Jeff Dean, Demis HassabisInverse Oracles: Information-Theoretic Approaches to Blockchain Data Compression for Market Intelligence
Modern blockchain systems present a fundamental information processing challenge: the continuous generation of high-entropy transaction data vastly exceeds the capacity of downstream analytical systems to consume it meaningfully. This paper introduces the concept of *inverse oracles*—systems that extract structured intelligence from blockchain data by pulling chain state off-chain, in contrast to traditional oracles that push external data on-chain. We present a formal information-theoretic framework for analyzing this compression problem, demonstrating that effective blockchain analytics requires lossy compression ratios exceeding 99.9% while preserving actionable signal.
cf. Dennis Ritchie, Linus TorvaldsAlgorithmic Emotion Suppression in High-Volatility Cryptocurrency Trading: A Phi-Optimized Decision Framework
This paper presents a rigorous mathematical framework for suppressing human emotional bias in automated cryptocurrency trading systems operating under conditions of extreme market volatility. We introduce a phi-weighted signal aggregation algorithm that processes high-entropy market data through successive filtration stages, ultimately producing low-entropy binary trading decisions. Our approach treats four distinct classes of emotional bias---Fear Of Missing Out (FOMO), Greed, Fear, and Revenge trading---as adversarial inputs requiring systematic detection and neutralization.
cf. Donald Knuth, John CarmackInformation Theory & Methodology
Code as evidence, identity encoding, state machine compression, and signal-guided learning through dialogue.
Code as Evidence: An Information-Theoretic Framework for Automated Professional Capability Assessment
The fundamental problem of professional labor markets is one of information asymmetry: candidates possess capabilities that are imperfectly observable by employers, while job requirements are expressed in natural language that maps imprecisely to actual skill demands. This paper presents a rigorous mathematical framework for capability assessment grounded in information theory, treating code repositories as high-fidelity signals of developer competence. We formalize the matching problem as a noisy communication channel, where traditional self-reported credentials exhibit high entropy (approaching maximum disorder), while code artifacts provide lower-entropy, higher-mutual-information signals about underlying capabilities.
cf. Claude Shannon, Alan TuringEncoding & Compression: Synthesized Reference
Identity encoding is about transferring experiential knowledge into LLM context. The key insight: **encoding IS training data, not representation**. Association format activates model correction machinery; lineage/sequential format fails.
Socratic Fine-Tuning: Learning Through Dialogue and Internal State Measurement
We present Socratic Fine-Tuning, a novel approach to training large language models that inverts the traditional information flow of machine learning. Rather than feeding models input-output pairs and computing loss on prediction accuracy, we position the model as an active generator while a teacher system observes internal activation states to weight learning updates. We formalize this through six neurotransmitter-analog signals extracted from model activations during generation: dopamine (insight/reward), GABA (inhibition), norepinephrine (focus), acetylcholine (learning/attention), serotonin (stability), and glutamate (activation).
cf. Andrej Karpathy, Ilya Sutskever