The Three-Legged Stool, A Co-Design Framework for AI Agent Architecture
Why Cognition, Memory, and Agency Must Be Built as One
Greg Cooper Founder & CTO, Gr4Ig, LLC April 2026 · greg@gr4ig.com
Abstract
This position paper proposes a unified design framework for AI agent systems — the Three-Legged Stool — and argues that structural instability in production deployments is a direct consequence of architectural isolation between Cognition, Memory, and Agency. The dominant paradigm treats these as modular components assembled post-hoc, producing “undesigned joints” that cannot sustain the high-bandwidth feedback loops required for autonomous reliability. We further identify a compounding problem: the three components are not only unjointed but unequal in development maturity, with Cognition receiving a decade of disproportionate investment. This imbalance makes stable joint design impossible until it is corrected. We propose that the field must pivot toward co-designed architectures in which all three components — and critically, the interfaces between them — are specified as a coherent system from inception. This paper presents the framework, documents its three critical interface protocols, and draws on the authors’ production experience with a six-agent orchestration system to illustrate the costs of the current approach.
1. Introduction
The AI agent field has reached a persistent performance ceiling: systems that perform impressively in constrained environments degrade under the weight of long-horizon, multi-step tasks [27,28]. Context accumulates. Reasoning quality falters. Execution latency climbs. The prevailing response has been to scale model parameters — yet each successive generation of larger models delivers diminishing returns in real-world agentic task completion.
This paper advances a different diagnosis. The bottleneck is not in the inference engine. It is in the system architecture surrounding it.
We observe what we call the Context Paradox: agent performance improves as context accumulates — up to a threshold. Beyond that threshold, it degrades sharply [4,5]. Larger context windows defer but do not resolve this degradation [5,6,7]. Better retrieval mechanisms reduce but cannot eliminate it. This is not a retrieval problem or a context-length problem. It is a structural problem. An agent operating in a poorly integrated architecture is condemned to accumulate noise alongside signal, because it lacks the co-designed feedback mechanisms required to distinguish between them.
An autonomous agent is a tripartite system comprising Cognition (inference and reasoning), Memory (state and knowledge), and Agency (the execution loop) [2,3]. In current implementations, these components are designed in silos and integrated after the fact. This produces undesigned joints — we define an undesigned joint as an inter-component interface whose specification, bandwidth, and failure modes were not considered during the design of either component it connects. Such interfaces lack the bidirectional feedback necessary for the system to function as a coherent whole.
This paper proposes that resolving the context paradox — and achieving genuine agentic reliability — requires a deliberate architectural shift: from modular assembly to co-design.
2. The Limits of Modular Orthodoxy
Agent development has historically prioritized one component of the architecture at the expense of systemic coherence. Three schools of orthodoxy have emerged, each reaching a point of diminishing returns:
Cognition-First: Prioritizes LLM reasoning capability while treating memory and agency as external wrappers [8,9,10,11,12]. This is the dominant paradigm in academic research. It produces models of increasing sophistication that remain unreliable in autonomous deployment.
Agency-First: Focuses on orchestration frameworks — task decomposition, tool routing, multi-agent coordination — while treating the model as a swappable API call and memory as a plugin [13,14,15,16,17]. This is the dominant paradigm in commercial agent frameworks.
Memory-First: Recent efforts — notably MemGPT/Letta [1] — have elevated stateful infrastructure to first-class concern. This work represents meaningful progress, but attempts to retrofit memory onto rigid, pre-existing agent loops. It produces interfaces that are neither stable nor properly bidirectional [18,19,20].
Each orthodoxy has produced genuine advances. None has resolved the reliability problem. We argue this is because each treats the joint as an afterthought — a connection point to be wired up after the component is built rather than a design constraint that should shape the component from the start [24,25,32].
The information loss occurring at these undesigned joints is the primary bottleneck preventing linear translation of component capability improvements into system-level task completion gains.
3. The Unequal Legs Problem
The joint design problem is compounded by a second structural issue: the three components are not merely unjointed — they are unequal in development maturity.
Cognition — longest leg: Model capability has received the dominant share of research investment and engineering attention since the transformer era began in 2017 [8]. This is a self-reinforcing dynamic: the longest leg attracts the most attention, accelerating its growth relative to the others, which tilts the system further, concentrating more attention on it still [27,28].
Memory — shortest leg: Treated as infrastructure plumbing for most of the field’s history [18]. Only recently elevated to a first-class design concern [1,2,20], it remains dramatically underdeveloped relative to cognition in both architectural sophistication and research attention.
Agency — medium, but hollow: The orchestration frameworks exist and the tooling is functional [13,14,15,16,17]. But the internal architecture of the agent loop — how it is paced, how state is managed across steps, how it interfaces with both cognition and memory — has received almost no serious architectural attention. The leg appears functional from the outside but lacks internal structural integrity.
The critical consequence of this imbalance is that stable joint design becomes impossible while the legs remain at such different levels of maturity. An interface designed today between a mature Cognition component and an immature Memory component will be wrong the moment Memory advances. This explains the persistent failure of memory retrofit efforts: the mismatch is not merely one of design philosophy — it is one of structural readiness.
Resolving the joint problem therefore requires simultaneous work on two fronts: leveling the legs through deliberate, disproportionate investment in Memory and Agency, and designing the joints concurrently as those legs develop. These cannot be sequential efforts. Leveling without joint design produces taller but still unstable architecture. Joint design without leveling produces interfaces that must be redesigned as the shorter legs grow.
Figure 1: From Structural Imbalance to Co-Designed Coherence
[Panel A] Three-legged stool with unequal legs (Cognition longest, Agency medium, Memory shortest) and absent joints, illustrating instability.
[→ Arrow labeled “Co-Design”] Transition arrow indicating the required architectural shift.
[Panel B] Three-legged stool with equal legs and labeled joints: Cognition↔Memory (Reciprocal Feedback), Memory↔Agency (Predictive Prefetching), Agency↔Cognition (Execution Transparency), illustrating a stable, level system.
Figure 1. The Co-Design transition: from unequal, unjointed components to a stable, coherent system. Note: Figure to be rendered as vector illustration for final publication.
4. Functional Definitions of the Three Legs
To achieve the architectural coherence proposed here, the three components must be defined as co-equal peers rather than primary system and supporting infrastructure:
Cognition (Reasoning Layer): The mechanism that transforms inputs into logical pathways, decisions, and actions. It defines what the agent can think. In current systems this is the LLM inference engine [8,11,12].
Memory (Information Layer): The hierarchy of storage, retrieval, consolidation, enrichment, and decay. It defines what the agent knows — and critically, how that knowledge is organized, maintained, and surfaced [18,19,20]. Memory is not a lookup table. It is an active participant in the agent’s cognitive process.
Agency (Activity Layer): The execution loop and tool orchestration [13,14]. It defines what the agent does. This includes not only task decomposition and tool routing, but the pacing of the loop, intermediate state management, and the structured surfacing of execution context back to the reasoning layer.
5. Critical Joints and Interface Protocols
We propose that the primary focus of near-term agent architecture R&D must be the design of three critical interfaces. These joints are not merely connection points — they are the mechanisms through which the system achieves coherence [26].
5.1 The Cognition–Memory Joint (Reciprocal Feedback)
Current implementations treat memory retrieval as a passive, one-directional operation: the agent queries, memory returns [18]. This joint must evolve into a bidirectional signal [19,21,22,23,33]. Reasoning traces should inform the memory system about the utility, relevance, and freshness of retrieved material, enabling dynamic re-indexing, confidence scoring, and automated pruning [20]. Simultaneously, memory architecture should actively shape what the Cognition layer is asked to process — surfacing precisely what current reasoning requires rather than returning accumulated context indiscriminately [4,5].
5.2 The Memory–Agency Joint (Predictive Prefetching)
Memory retrieval must be elevated from an optional tool call to a loop primitive. Rather than retrieving context reactively — after the agent has already committed to an execution path — this joint should enable the memory system to prefetch context based on anticipated execution patterns within the Agency layer [21,22,34]. The agent loop provides the signal; the memory system responds predictively. This requires that Agency expose its internal execution trajectory as a structured signal that Memory can act on.
5.3 The Agency–Cognition Joint (Execution Transparency)
The Agency layer must surface its internal execution state — telemetry, tool failure modes, intermediate outputs, and execution confidence — as first-class reasoning context available to the Cognition layer in real time [27,35]. Currently, the model initiates the agent loop and receives final outputs; it has no visibility into the loop’s internal state. This opacity prevents the Cognition layer from reasoning through the execution process rather than simply waiting for it to complete.
6. Case Study: Architectural Bottlenecks in Gr4Ig Brain
The necessity of co-design is concretely evidenced by the authors’ production experience with Gr4Ig Brain — a persistent, self-hosted semantic memory system deployed as shared context infrastructure across a six-agent orchestration team operating under a shared multi-agent collaboration framework.
Gr4Ig Brain was built on PostgreSQL with pgvector for semantic retrieval, exposed via a Model Context Protocol (MCP) server, and consumed by all six agents as a shared memory substrate. The system performed reliably within its designed scope: flat semantic retrieval over a growing corpus of operational knowledge.
In March 2026, we extended Gr4Ig Brain with a knowledge graph layer — Cognee v0.5.5, using KùzuDB as the graph engine — to provide relational reasoning across the stored corpus [29,30]. The individual components were technically functional in isolation: the knowledge graph ingested the corpus and completed its build pipeline successfully. However, the production deployment failed within days and was removed on April 2, 2026. The failure modes were specific and instructive:
Repeated SQLite corruption in Cognee’s internal database, manifesting as
file is not a databaseerrors under normal operational load.KùzuDB lock contention between the REST API server and the cognify pipeline, causing the graph build process to conflict with live queries.
Dataset state becoming stuck in
DATASET_PROCESSING_ERROREDrepeatedly, with no automated recovery path.The
DELETE /api/v1/datasetsadministrative endpoint returning Internal Server Error, making clean recovery impossible without manual intervention.
Each failure mode traces to the same root cause: the knowledge graph layer was integrated post-hoc into an agent architecture that had not been designed with relational memory as a first-class participant [24,25]. The KùzuDB graph and the pgvector semantic store operated in separate semantic spaces with no co-designed synchronization protocol [29]. The agent loop had no mechanism to signal which retrieval modality was appropriate for a given execution context. The Cognition layer had no feedback path to the memory system to indicate whether retrieved graph context was useful or noise [5].
The result was not a component failure. It was a joint failure. The individual legs were functional; the stool collapsed because the joints were absent [27]. This experience motivates the research agenda described in the following section.
7. Future Work: A Co-Design Research Agenda
Gr4Ig, LLC is currently rearchitecting the Gr4Ig Brain framework with the explicit objective of elevating Memory from a subservient retrieval service to a cooperative participant in the agent’s reasoning and execution cycle [26]. This reframing — from subservient to cooperative — is the central design challenge. It requires that Memory not merely respond to queries, but anticipate them, evaluate its own outputs, and actively participate in the agent loop.
The research agenda is organized around three focus areas:
Knowledge Graph Integration: Designing a knowledge graph layer that shares a co-designed semantic space with the vector retrieval layer and the agent loop, rather than operating as a parallel system bolted on after the fact [29,30,31]. The graph and the vector store must be synchronized at the architectural level, with a designed protocol governing when each retrieval modality is engaged.
Predictive Semantic Retrieval: Developing an intelligent prefetching layer — informed by the Agency layer’s execution trajectory — that proactively surfaces relevant context before the agent explicitly requests it [21,22,36]. This transforms retrieval from a reactive operation into an anticipatory one, reducing the latency and context-bloat costs of reactive retrieval [4,5] while improving the signal quality available to the Cognition layer.
Memory Hygiene and Enrichment: Establishing automated mechanisms for knowledge base consolidation, semantic clustering, confidence decay, and entry enrichment — driven by feedback signals from the Cognition layer [2,20]. Memory must not only grow; it must maintain structural integrity as it grows. This includes designing the feedback protocol by which reasoning traces inform memory organization, and the policies governing automated pruning and archival.
Full specifications for this rearchitected system — including interface protocols, implementation results, and evaluation methodology — will be shared as a follow-on publication once the system has reached production-grade stability.
8. Limitations
This paper makes several simplifying assumptions that future work should address. First, the Three-Legged Stool framework is a three-way partition; hybrid or pipeline architectures that blur the boundaries between Cognition, Memory, and Agency (e.g., differentiable retrieval heads embedded within transformer layers) may require a finer-grained joint taxonomy. Second, the case study in Section 6 draws on a single organization’s production deployment; while the failure modes are specific and reproducible, broader empirical validation across diverse agent architectures is required to establish their generality. Third, the co-design principle advanced here is prescriptive rather than demonstrated: we argue that jointly designing the three legs and their interfaces will produce more reliable systems, but we do not yet present a controlled comparison. The follow-on publication described in Section 7 will attempt to provide such evidence.
9. Conclusion
The AI agent field has spent the better part of a decade strengthening individual components. We have achieved remarkable models [8,11,12], functional orchestration frameworks [13,14,15,16], and increasingly capable vector retrieval systems [18,20]. Yet agentic reliability — the ability to complete complex, long-horizon tasks autonomously and consistently — remains elusive [27,28].
This paper has argued that the gap between impressive capability and reliable production performance is architectural. The dominant modular paradigm produces systems with undesigned joints and unequal legs — components assembled after the fact rather than co-designed from inception [24,25,32]. These structural deficiencies cannot be closed by improving any individual component in isolation. A stool with three strong legs and absent joints still collapses.
The path forward requires a deliberate pivot: from component-centric development to systemic co-design [26], in which Cognition, Memory, and Agency — and critically, the interfaces between them — are specified as a coherent whole from the first design decision. This is not a call to abandon existing work. It is a call to recontextualize it: to recognize that the investments made in individual legs have brought us to the threshold of a new design problem, and that crossing it requires the field to treat the joint as a primary object of research and engineering attention.
The framework proposed here — the Three-Legged Stool — is intended as a foundation for that work.
AI Assistance Transparency
This paper was produced with the assistance of the Gr4Ig AI agent team, a multi-agent system leveraging a variety of large language models. All ideas, arguments, frameworks, and conclusions represent the author’s own intellectual work. AI assistance was employed for tasks including research synthesis, structural refinement, and prose editing. The author takes full responsibility for the accuracy, integrity, and originality of this work.
References
Packer, C., Wooders, S., Lin, K., Fang, V., Patil, S. G., Stoica, I., & Gonzalez, J. E. (2023). MemGPT: Towards LLMs as Operating Systems. arXiv:2310.08560.
Zhang, G., Du, Y., Lam, K., Shi, W., & Li, J. (2025). Memory in the Age of AI Agents: A Survey. arXiv:2512.13564.
Hua, W., Wan, M., Vadrevu, S., Nadel, R., Zhang, Y., & Wang, C. (2025). Interactive Speculative Planning: Enhance Agent Efficiency through Co-design of System and User Interface. International Conference on Learning Representations (ICLR) 2025. arXiv:2410.00079.
Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the ACL, 12, 157–175. arXiv:2307.03172.
Wu, J., et al. (2025). Context Length Alone Hurts LLM Performance Despite Perfect Retrieval. Findings of EMNLP 2025. arXiv:2510.05381.
Wang, W., Min, J., & Zou, W. (2026). Intelligence Degradation in Long-Context LLMs. arXiv:2601.15300.
Ahilan (2026). Context Discipline and Performance Correlation: Analyzing LLM Performance and Quality Degradation Under Varying Context Lengths. arXiv:2601.11564.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems (NeurIPS) 30, 5998–6008.
Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving Language Understanding by Generative Pre-Training. OpenAI Technical Report.
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language Models are Unsupervised Multitask Learners. OpenAI Technical Report.
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., et al. (2020). Language Models are Few-Shot Learners. Advances in Neural Information Processing Systems (NeurIPS) 33, 1877–1901.
OpenAI (2023). GPT-4 Technical Report. arXiv:2303.08774.
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing Reasoning and Acting in Language Models. International Conference on Learning Representations (ICLR) 2023. arXiv:2210.03629.
Hong, S., et al. (2023). MetaGPT: Meta Programming for Multi-Agent Collaborative Framework. arXiv:2308.00833.
Zhang, Z., et al. (2025). AgentOrchestra: A Hierarchical Multi-Agent Framework for General-Purpose Task Solving. arXiv:2506.12508.
Adimulam, A., Gupta, R., & Kumar, S. (2026). The Orchestration of Multi-Agent Systems: Architectures, Protocols, and Enterprise Adoption. arXiv:2601.13671.
Li, Y., et al. (2025). Agent-Oriented Planning in Multi-Agent Systems. International Conference on Learning Representations (ICLR) 2025. arXiv:2410.02189.
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems (NeurIPS) 33, 9459–9474. arXiv:2005.11401.
Wang, C., et al. (2024). Episodic Memory Retrieval from LLMs: A Neuromorphic Mechanism. Findings of ACL 2024, 2490–2502.
Chen, Y., et al. (2025). EVOLVE-MEM: A Self-Adaptive Hierarchical Memory Architecture. Advances in Neural Information Processing Systems (NeurIPS) 2025.
Schroff, F., et al. (2024). SRMT: Shared Recurrent Memory Transformer for Multi-Agent Pathfinding. NeurIPS 2024. arXiv:2501.13200.
Liu, J., et al. (2026). Scaling Teams or Scaling Time? Memory Enabled Lifelong Learning in Multi-Agent Systems (LLMA-Mem). arXiv:2604.03295.
Zhao, R., et al. (2024). Memory Sharing for Large Language Model based Agents. arXiv:2404.09982.
Lin, W., Zhen, H.-L., Yang, S., Wang, X., et al. (2025). Towards Efficient Agents: A Co-Design of Inference Architecture and System. arXiv:2512.18337 [cs.CL].
Lee, Y.-A., Yi, G.-T., Liu, M.-Y., Lu, J.-C., Yang, G.-B., & Chen, Y.-N. (2025). Compound AI Systems Optimization: A Survey of Methods, Challenges, and Future Directions. EMNLP 2025 (Main Conference). arXiv:2506.08234 [cs.CL].
Qiu, Z., Sun, J., Xia, C., Zheng, J., & Peng, X. (2026). CI4A: Semantic Component Interfaces for Agents Empowering Web Automation. arXiv:2601.14790 [cs.AI].
Rabanser, S., Kapoor, S., Kirgis, P., Liu, K., Utpala, S., & Narayanan, A. (2026). Towards a Science of AI Agent Reliability. arXiv:2602.16666.
METR (2025). Measuring AI Ability to Complete Long Tasks. METR Technical Report.
Lee, J., Kim, H., Park, S., Choi, J., & Kim, Y. (2025). HybGRAG: Hybrid Retrieval-Augmented Generation on Textual and Relational Knowledge Bases. ACL 2025. arXiv:2408.04948.
Jiang, R., Wang, J., Zhao, G., Luo, C., Wang, K., & Zhang, W. (2026). Advancing Multimodal Agent Reasoning with Long-Term Neuro-Symbolic Memory (NS-Mem). arXiv:2603.15280.
Zhang, H., et al. (2025). Knowledge Graph and Large Language Model Co-learning. PMC12998269.
Xu, B. (2026). AI Agent Systems: Architectures, Applications, and Evaluation. arXiv:2601.01743 [cs.AI].
Zhang, Y., Shu, J., Ma, Y., Lin, X., Wu, S., & Sang, J. (2025). Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Tasks. arXiv:2510.12635 [cs.CL].
Mao, W., Liu, H., Liu, Z., Tan, H., Shi, Y., Wu, J., Zhang, A., & Wang, X. (2026). Collaborative Multi-Agent Optimization for Personalized Memory System. arXiv:2603.12631 [cs.AI].
Bansal, R., Zhang, A., Tiwari, R., Madaan, L., Duvvuri, V. S. S., Brandfonbrener, D., Alvarez-Melis, D., Bhargava, P., Kale, M., & Jelassi, S. (2026). Let’s (not) just put things in Context: Test-time Training for Long-context LLMs. International Conference on Learning Representations (ICLR) 2026.
Li, B., Du, S., & Guo, Y. (2026). Joint Optimization of Reasoning and Dual-Memory for Self-Learning Diagnostic Agent. arXiv:2604.07269 [cs.AI].