Home : News : Display
Aug. 28, 2026

From Reliability to Fit: Trust, Adaptation, and Distributed Cognition in AI-Enabled Mission Command

C. Anthony Pfaff
©2026 C. Anthony Pfaff

ABSTRACT: This special commentary argues that military artificial intelligence integration must pivot from model reliability to a “cognitive fit” within human-machine teams to ensure decision advantage under uncertainty. It advances beyond traditional artificial intelligence metrics by focusing on dynamic, operational adaptation and systemic resilience. The research employs an eco-cognitive framework, synthesizing John Boyd’s evolutionary theories with modern cognitive science and risk management principles. This analysis offers policy and military practitioners a crucial blueprint for designing command structures that preserve human judgment and mitigate risk.

Keywords: artificial intelligence (AI), cognitive fit, decision advantage, distributed cognition, human-machine integration, mission command

 

The military’s efforts to integrate artificial intelligence (AI) into its planning process, such as the Next Generation Command and Control (NGC2), are generally—and justifiably—aimed at decision advantage over an enemy. Such systems achieve that advantage by creating an ecosystem of previously siloed systems and using AI to analyze the data quickly, providing a continuous, integrated operating picture that, in principle, should enable more efficient staff processes and, consequently, better decision making.1 As such systems evolve, they are likely to be increasingly capable of supporting decision making through better situational awareness and using that situational awareness to make—or at least propose—the decisions themselves.2

Many current approaches to military AI focus on whether AI systems are reliable enough to trust. That focus works reasonably well when AI classifies objects, describes current conditions, or summarizes information. It works less well when AI generates plans, predicts enemy behaviors, or recommends actions under uncertainty. In these cases, the critical question is not whether an AI model is reliable in isolation but whether the larger human-machine system can understand and adapt effectively to a changing environment.3 That adaptation is a function of how the human-machine system distributes cognition and how that distribution fits with the operational environment to enable representation, inference, and adaptation through feedback. Thus, understanding and managing cognitive fit is essential to achieving decision advantage.

Decision advantage, in this context, is not just a function of deciding faster than an enemy, but also better. Faster and better, however, are often competing factors that require balancing machine speed with human control.4 Faster is easy to measure; however, better is not. Currently, trust (which is at stake in this process) is understood largely in terms of computational reliability—whether the system has been validated and shown to produce accurate, robust, and repeated results under its expected conditions of use.5 This standard of trust is fine for classifier models designed, for example, to identify enemy vehicles. While operators may not be certain in the moment whether a machine identification is accurate, they can, in principle at least, verify it and, if found to be faulty, retrain the model to make it more reliable.

The challenge is different for generative AI and large language model systems engaged in intelligence analysis and military operations planning. For example, with decision support systems that describe the current state of affairs in areas such as targeting, maintenance, or logistics, trust can be grounded in factors like data provenance, quality, curation, model validation, verification, robustness, calibration, performance across contexts, and expert oversight.6 These indicators, however, are least helpful when AI output is future-oriented—when systems generate plans, predictions, or assessments of enemy intent whose truth cannot be known in advance and may remain contestable even after the fact. For example, a failed plan may still be the best one available under the circumstances, whereas a successful plan may rest on spurious correlations or opaque reasoning. Here, the challenge is epistemic: How can a commander know whether it is prudent to act on future-oriented AI-model output?

The trouble with using such a reliabilist approach is twofold. First, as John Zerilli et al. point out, problems of control can worsen as AI improves. Human limitations in capability, attention, currency, and attitude create risks when humans use AI tools. Because AI systems are too fast and complex, human operators may be unable to control them effectively.7 Moreover, if AI-enabled operations reduce human interaction to monitoring displays, attention may wane, increasing the risk of error. Reliance on AI can also erode human skills, especially cognitive ones, which may decay over time and become too difficult to regain when needed.8 Finally, numerous examples of automation bias suggest that humans often trust AI output from generally reliable models, even when those models are known to fail occasionally.9 Thus, Zerilli et al. argue that AI can often be the most dangerous when it is the most reliable. Counterintuitively, systems that are more prone to failure are typically safer because they prompt more human attention and oversight.10 This point suggests a paradox. Artificial intelligence (AI) integration adds value by off-loading human cognitive burdens, but in doing so, it also makes humans less cognitively competent, potentially degrading the overall effectiveness of the system.

This last point suggests that effective AI integration requires understanding cognition from a system point of view, where the system includes humans, AI models, and other tools that provide the “cognitive scaffolding,” which are the external structures that enable cognitive off-loading and burden sharing.11 A second concern is that reliabilist approaches neglect the larger distributed cognitive system through which raw data is transformed into information, and information into knowledge, and knowledge into understanding. In distributed systems, such as combat units, cognition is not located within individual minds but is distributed across people, tools, and environments in which the systems operate.12

The fact that cognition is a system property and not just the sum of individual knowledge is an important point. AI–enabled mission-command systems, like NGC2, do not simply speed up existing military decision-making processes; they fundamentally redistribute cognition across a human-machine network. In AI-enabled systems, machines assume a larger share of the cognitive labor associated with sorting data, detecting anomalies, correlating signals, and generating candidate explanations. As a result, humans could play less of a role in assembling the environmental picture and focus on interpreting its meaning, assessing risks, and deciding how to act. This shift in cognitive load across people, tools, data, and processes means the design challenge is not merely to optimize model performance but to optimize the fit of the entire human-machine system to the operational environment.

Thinking in terms of fit will force soldiers to rethink how to fight. Conventional military decision-making approaches emphasize action over adaptation because they are designed to move information toward a decision and coordinate its execution. Understanding, in this context, functions primarily as a prerequisite for action where the staff seeks to establish a stable picture, translate it into a plan or order, synchronize resources, and execute decisions.13 Hierarchical roles, sequential staff processes, and prescriptive rules make the system effective at producing action, but less effective at continuously revising how it perceives and frames a changing environment. This point does not mean current systems fail to adapt but that adaptation is action-focused and may change how they achieve goals based on the success or failure of prior attempts. A system focused on adaptation would instead look at changing the mental models through which it makes sense of the environment and might question whether it had the right goals in the first place.

This legacy model of adaptation is suitable for situations where there is a right answer. As John Kay and Mervyn King observe, such “puzzles” have a right answer that can often be improved with more data and processing power. They contrast puzzles with “mysteries,” which have no objectively correct solution. In these cases, uncertainty can never be resolved, so other metrics are required to assess the quality of any proposed response.14 So, in situations where machine output cannot, even in principle, be verified relative to a particular goal, the central design question shifts from whether AI can produce accurate outputs quickly to whether the entire distributed system of humans, tools, data, and processes can produce knowledge and understanding sufficiently trustworthy to support decision advantage in a particular operating environment.

In his essay, “Destruction and Creation,” John Boyd saw the design problem for military systems as inherently epistemic, though he did not use the word. Rather, he analogized that militaries were much like living things, goal-oriented organisms that desire to survive and grow in environments characterized by scarce resources. In such environments, survival and growth depend on the ability to adapt by constantly dismantling old mental models and creating new ones, with mental models serving as how one makes sense of one’s environment. Successful adaptation depended on whether the new mental model fit the environment better than the old one did. Assessing that fit requires the organism to have adequate cognitive capabilities to produce and hold knowledge about the environment. For humans, at least, that ability emerges from an evolutionary process in which the mind adapts to reality by iteratively matching its internal framework to its external experiences.15 Scholars such as Lorenzo Magnani, who introduced the idea, might describe Boyd’s epistemology as “eco-cognitive,” as in knowledge arises from internal cognitive processes and external environmental factors and is shaped by the tools or technologies humans use to interact with the world.16

Interestingly, Boyd described establishing new mental models in destructive and creative terms in which deductive logic broke down the old system and an inductive synthesis of prior concepts and new information about the world produced a new model.17 Deductive and inductive logics, however, tend to be truth-preserving and prescriptive. They are truth-preserving because they establish the truth of a matter, and prescriptive because they prescribe inferential processes to guarantee truth.18 In cases where there may be no truth to preserve, and thus nothing to describe, one needs to find another path to knowledge and understanding.

This concern is especially germane to AI-enabled systems. AI-generated representations of the world are statistical estimates or inferences that depend on the quality, coverage, diversity, and representativeness of the data used to train AI models.19 Boyd’s ecological account of organizational adaptation suggests a change in focus from reliability to fit. Rather than asking whether a system is reliably accurate, this framework asks whether the system fits the operational environment well enough to help commanders generate useful understanding and adapt effectively. In this view, decision advantage comes from the entire system’s ability to adapt faster than an enemy, not simply to decide faster. Because success depends on iterative hypothesizing, whether a system fits the operational environment depends on three broad functions—representation, inference, and adaptation.

Representational fit assesses whether the system detects, organizes, and presents relevant information accurately, intelligibly, and without imposing excessive cognitive or operational costs. Inferential fit evaluates whether the human-machine system generates plausible, coherent, context-sensitive, and action-guiding explanations rather than relying on static or spurious correlations. Output and adaptive fit assess whether those explanations support appropriate, timely decisions and whether feedback enables the system to revise its representations, reconsider alternatives, and improve over repeated decision cycles. Fit is therefore assessed iteratively: Commanders and staffs evaluate these dimensions during each cycle, examine how they change over time, and use the results to identify when the distributed cognitive system’s understanding, reasoning, or behavior has begun to drift from the operational environment. In doing so, they frequently must reason abductively—that is, reason to best explanation—where the successful hypothesis is the most plausible one. Unlike deductive and inductive logic, abductive reasoning is not truth preserving, so suitable for problems, as stated above, where there is no truth to preserve. In this context, uncertainty is not a fault in the system, but a demand signal for adaptation.

It should now be clear how different “fighting with data” is from more conventional war fighting. Rather than “find, fix, and finish,” soldiers will spend more time interpreting, challenging, and adapting machine-generated output. Rather than solving puzzles about enemy location and relative combat capabilities—the machine will eventually handle most, if not all, of that—soldiers will focus on generating hypotheses about enemy intent and friendly courses of action while further assessing how the constantly changing operational environment affects the quality of those hypotheses. To optimize that process, commanders and staffs will have to manage how cognition is distributed across people, models, data, networks, and processes, deciding which functions to delegate, which judgments to retain, and how to maintain feedback loops that keep machine output aligned with mission intent and changing battlefield conditions. In this context, trust manifests as trust in one’s senses emerges—continuous interaction with the environment that enables successful-enough goal-oriented behavior—rather than passive confidence in a model’s reliability. Predators do not have to catch all the prey they chase, just enough to survive.

Despite this change, in many ways, an eco-cognitive approach is more intuitive because it evaluates AI-enabled systems much the way soldiers already evaluate combat organizations—not by simply assessing the functions of particular components but whether the entire arrangement of combat, combat support, and combat service support systems works in the environment for which it is designed. Commanders already know effective performance depends on the interaction of people, doctrine, and war-fighting functions. This approach extends that understanding to AI-enabled systems. Instead of treating an AI model as a staff officer, in which the concern is whether its estimate is accurate, it asks whether the larger human-machine system perceives relevant conditions, makes the environment intelligible, and generates hypotheses about the environment that enable timely action consistent with mission intent.

This approach is also more intuitive because it matches how soldiers reason under battlefield uncertainty. Current AI methodologies imply that trust should depend on things like data hygiene, validation scores, past reliability, or agreement with human experience. Those standards work reasonably well for puzzles, such as classifying enemy vehicles, but not for mysteries, such as predicting enemy intent or future courses of action. Measures of fit do not eliminate uncertainty as much as they manage it by letting commanders know whether a particular system is good enough for a specific mission, in a specific environment, at a specific time. In some ways, this model sees uncertainty as an asset—when properly perceived, it motivates more effective adaptation.

Finally, the approach treats adaptation as normal rather than as evidence of failure. The expression “no plan survives contact with the enemy,” often used to account for shortcomings, reflects what soldiers already know. Even when plans are based on perfect information and the flawless application of principle, first contact with the enemy changes the environment and thus the information on which the plan was based. This approach better accounts for what is already happening on the human side of the system—continuous sensing, judging, acting, assessing, and reframing—rather than building the technical knowledge necessary to employ AI-enabled systems as technically sophisticated alternatives to human staff.

Success in AI-enabled mission command will depend less on computational speed than on system designs that preserve human judgment, make uncertainty visible, sustain feedback loops, and maintain cognitive resonance between human users and machine outputs. In short, the future of war fighting lies in building human-machine cognitive ecologies that can learn, adapt, and fight effectively using data, not in replacing commanders with machines.

 

Acknowledgments
The author would like to thank Dr. Ava Loer, Army Research Institute; Dr. Glenn R. Downing, School of Advanced Military Studies; Blair Wilcox, LTC (US Army retired); and Dr. Jesse Kirkpatrick, codirector, George Mason University’s Autonomy and Robotics Center, for their support in developing this commentary.

Author’s Note
This note introduces concepts and findings covered in more detail in the monograph: Fighting with Data: Design Implications for AI-Enabled Mission-Command Systems (US Army War College Press, August 2026).

 
 

C. Anthony Pfaff
Dr. C. Anthony Pfaff is currently the director of the Strategic Research and Assessment Department at the US Army War College, where he was formerly the research professor for strategy, the military profession, and ethic. He has written widely on the integration of artificial intelligence and other disruptive technologies, focusing on strategic and ethical implications.

 
 

Endnotes

  1. 1. “Army Announces Next Generation Command and Control (NGC2) Prototype Award,” US Army, July 18, 2025, https://www.army.mil/article/287180/army_announces_next_generation_command_and_control_ngc2_prototype_award; “Next Generation Command and Control,” Lockheed Martin, July 20, 2026, https://www.lockheedmartin.com/en-us/products/next-generation-command-and-control.html#wingcacorngc2; and Waylon D. Petty, “What Is Next Generation Command and Control?,” NCO Journal (April 2026), https://www.armyupress.army.mil/Journals/NCO-Journal/Muddy-Boots/What-Is-Next-Generation-Command-and-Control/.
  2. 2. Courtney Crosby, “Operationalizing Artificial Intelligence for Algorithmic Warfare,” Military Review (July-August 2020), https://www.armyupress.army.mil/Journals/Military-Review/English-Edition-Archives/July-August-2020/Crosby-Operationalizing-AI/.
  3. 3. It is worth distinguishing between trust and trustworthiness. The former is a psychological disposition concerning individual attitudes toward how an agent will account for their goals or interests. Trustworthiness is a function of the properties that lead to that state. See Linda Onnasch et al., “Trust(worthiness) Issues with Trust in Human-Robot Interactions,” ACM Transactions on Human-Robot Interaction 15, no. 3 (February 2026), https://doi.org/10.1145/3778865. The author owes this point to Dr. Jesse Kirkpatrick, codirector of George Mason University’s Autonomy and Robotics Center, e-mail message to author July 31, 2026.
  4. 4. Army Futures Command (AFC), Army Futures Command Concept for Command and Control 2028: Pursuing Decision Dominance, AFC Pamphlet 71-20-9 (AFC, July 2021), 16; and Jen Sovada, “AI in Battlefield Intelligence: Expanding the Speed of Decision-Making,” Federal News Network, July 10, 2026, https://federalnewsnetwork.com/commentary/2026/07/ai-in-battlefield-intelligence-expanding-the-speed-of-decision-making/.
  5. 5. “3 AI Risks and Trustworthiness,” National Institute of Standards and Technology, July 22, 2026, https://airc.nist.gov/airmf-resources/airmf/3-sec-characteristics/; and Juan M. Durán and Nico Formanek, “Grounds for Trust: Essential Epistemic Opacity and Computational Reliabilism,” arXiv, March 2019, https://arxiv.org/abs/1904.01052.
  6. 6. National Institute of Standards and Technology (NIST), Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1 (NIST, 2023), 12–17, https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.100-1.pdf.
  7. 7. John Zerilli et al., A Citizen’s Guide to Artificial Intelligence (MIT Press, 2021), 83–87.
  8. 8. How monitoring displays affect attention is nuanced. Recent studies show that rapidly switching tasks can degrade performance; however, individuals’ capabilities to concentrate are not affected. See David Adam, “Are Attention Spans Really Shrinking? What the Science Says,” nature, May 6, 2026, https://www.nature.com/articles/d41586-026-01407-w. Other studies indicate that when using AI, the brain’s cognitive capacity decreases: Natallya Kos’myna, “Your Brain on ChatGPT: Accumulation of Cognitive Debt When Using an AI Assistant for Essay Writing Task,” MIT Media Lab, June 10, 2025, https://www.media.mit.edu/publications/your-brain-on-chatgpt/.
  9. 9. It is worth noting that models that rely on large data sets and have high statistical reliability may be the worst predictive models because they will likely not recognize the probabilities hidden in the margins of the relevant bell curve. The author owes this point to Dr. Glen R. Downing, School of Advanced Military Studies, Fort Leavenworth, KS, e-mail message to author July 31, 2026.
  10. 10. Zerilli et al., Citizen’s Guide, 89–90.
  11. 11. “Cognitive Scaffolding,” Fiveable, updated July 2026, https://fiveable.me/introduction-cognitive-science/key-terms/cognitive-scaffolding. Cognitive scaffolding generally includes any artificial aid that helps individuals think, remember, reason, or make decisions. An operational map, for example, would count as cognitive scaffolding as it off-loads memory by organizing information about friendly and enemy forces that enables a common understanding.
  12. 12. Edwin Hutchins, Cognition in the Wild (MIT Press, 1995), 360.
  13. 13. Information, 1–2.
  14. 14. John Kay and Mervyn King, Radical Uncertainty: Decision-Making Beyond the Numbers (W. W. Norton & Company, 2021), 20–21.
  15. 15. John Boyd, “Destruction and Creation,” in A Discourse on Winning and Losing, Colonel John Boyd Archive, September 3, 1976, https://www.coljohnboyd.com/#pdf-destruction-and-creation.
  16. 16. Lorenzo Magnani, “Abductive Cognition: The Epistemological and Eco-Cognitive Dimensions of Hypothetical Reasoning,” in Cognitive Systems Monographs, Vol. 3, ed. Rüdiger Dillmann et al., (Springer, 2009), 401; and Tommaso Bertolotti, Patterns of Rationality: Recurring Inferences in Science, Social Cognition and Religious Thinking (Springer Nature Link, 2015), 5. Bertolotti’s work further develops the concept of “ecological logic,” introduced by Magnani.
  17. 17. Boyd, “Destruction and Creation”; and Grant T. Hammond, The Mind of War: John Boyd and American Security (Smithsonian Books, 2004), 1118–20.
  18. 18. Magnani, Abductive Cognition, 68.
  19. 19. NIST, Artificial Intelligence Risk Management, 38–39; and Chuan Guo et al., “On Calibration of Modern Neural Networks,” in Proceedings of the 34th International Conference on Machine Learning (2017), https://arxiv.org/pdf/1706.04599.
 
 

Disclaimer: Articles, reviews and replies, review essays, and book reviews published in Parameters are unofficial expressions of opinion. The views and opinions expressed in Parameters are those of the authors and are not necessarily those of the Department of War, the Department of the Army, the US Army War College, or any other agency of the US government. The appearance of external hyperlinks does not constitute endorsement by the Department of War of the linked websites or the information, products, or services contained therein. The Department of War does not exercise any editorial, security, or other control over the information readers may find at these locations.