AI agents have evolved from Large Language Models (LLMs) that primarily generate responses to systems capable of reasoning and dynamic decision-making. By interpreting goals, inferring what is needed, and integrating external capabilities such as APIs and databases, the LLM serves as the decision-making core of an agent. Agentic AI refers to systems built around these intelligent agents.

AI Agents

An AI agent extends beyond a simple prompt by combining an LLM’s reasoning capability with functional layers such as tools, memory and orchestration. Tools provide additional functionality and programmatic access, memory maintains relevant context, and planning and orchestration coordinate execution. These functional capabilities that assist an LLM to be an AI agent can be visualised as layers as shown in the figure below [1]. The complexity of an agent can be built progressively by adding the layers required by the task. With addition of more layers, the agency of the system over tasks increases.

The building blocks of the agent stack come together through a workflow, where tools, deterministic components and intelligent components interact to achieve a goal. In a conventional workflow, the sequence of steps is predefined; in an agentic workflow, some of those steps can be determined dynamically based on the situation. Within this framework, we can define an agent.

In my opinion, an agent is an LLM-driven component that integrates at least two layers of the agent stack shown above, distinguishing it from a simple LLM call with a prompt. True agency emerges when these functional layers such as tools, memory, or planning enable the component to navigate parts of the workflow dynamically. The LLM provides the reasoning capability, while these additional layers enable it to act beyond generating a response.

From Agents to Applications

With the distinction between workflows and agents established, the important question for any project is not “How do I build an agent?” but “Does this problem need an agent at all?” This is where the right mental model becomes important. Start with the business application and its objectives, and decompose what the system needs to accomplish. Is it retrieving information, making a recommendation, scheduling something, taking an action, or combining several of these? For each capability, determine whether it can be implemented deterministically or requires the system to reason and dynamically decide what to do next. Some tasks may need conventional software, some an LLM within a predefined workflow and only some may warrant an agent. This naturally leads to a deterministic-first approach: use conventional software where the behaviour is known, and introduce intelligence where it adds value.

The next step is not necessarily to introduce an agent. An LLM can first be used as a capability within the application, for example, to summarise information, extract entities, classify a request, or generate response to a question. Multiple such capabilities can then be composed into a workflow, where a predefined sequence of steps is used to accomplish a task. The steps may involve LLM calls, deterministic code, tools, or other application components, and may execute sequentially or in parallel, but the application determines the overall flow. For example, one LLM call may extract information, two calls may independently analyse different aspects, and a second one may combine their outputs. This remains a workflow because the sequence and coordination are predetermined.

Scoping the Agentic Application

Agentic behaviour enters when the system needs to dynamically determine a part of that workflow. To make this decision, we first need to scope the application. Scoping can be approached from two complementary perspectives, the consumer and the source. From the consumer perspective, identify what the application needs to provide; what features users need, what decisions it must support and what actions it should perform. From the source perspective, identify the data and functional sources required to support those features, such as databases, APIs, documents, search systems, or other applications. Together, these perspectives map what the application needs to do to what it needs to know or access.

Once the scope is established, the first question is whether the identified capabilities can be brought together within a single agent. An agent can combine the required tools, memory, and planning capabilities and coordinate the workflow to achieve the application objective. This is often the simplest starting point, particularly when the capabilities are closely related and can share the same context. The agent can reason over the task, select the appropriate tools, and determine the sequence of actions required, keeping the overall architecture relatively simple. For example, an agent tasked with generating a business report may query a database to retrieve relevant data, call an API to obtain additional information, analyse the results, and then produce the report, coordinating these steps toward a single capability.

Pivot to Multi-Agent System

As the scope of the application becomes broader, different capabilities may require different forms of reasoning, tools, knowledge, or context. At this point, combining everything into a single agent can make its responsibilities increasingly difficult to define and maintain. The question then changes from “What can this agent do?” to “Should these responsibilities belong to the same agent?” This provides a natural point to consider a multi-agent system, where specialised agents are assigned distinct capabilities and collaborate through a workflow. The objective is not to increase the number of agents, but to establish meaningful boundaries between responsibilities.

The same consumer and source perspectives used during scoping can help determine these boundaries. From the consumer perspective, capabilities that have distinct objectives or require different reasoning approaches may be separated into specialised agents. From the source perspective, different data or functional sources may require specialised capabilities. For example, a SQL agent for database access when the number of tables increases and table-schema becomes complex, an API agent for application services, or a RAG-based component for documents. These agents can then be coordinated through a sequential, hierarchical or dynamic workflow, allowing each agent to focus on a well-defined responsibility while the overall system works toward the application objective.

The Agentic AI Mental Model

This mental models aids us with a useful progression for designing agentic systems. We begin with the business objective, decompose it into capabilities, determine whether each capability requires deterministic logic, an LLM, a workflow, or an agent, and then decide whether those agents should operate independently or collaborate as a multi-agent system. A single agent should therefore be the starting point rather than an assumption that the system must eventually become multi-agent. As the problem demands greater separation of capabilities, specialised knowledge, or distinct decision-making responsibilities, multiple agents may provide a more appropriate architecture. Thus, the complexity of the agentic system should emerge from the complexity of the problem, rather than the other way around.

The importance of this mental model lies in recognising that greater intelligence does not necessarily mean a better system. Introducing an LLM, a workflow, or an agent adds computational cost, latency, and architectural complexity, and these should be justified by the value they bring to the application. A deterministic component may be more reliable and efficient for a well-defined task, while an LLM or an agent becomes valuable when the problem requires interpretation, reasoning, or dynamic decision-making. Thus, the progression from deterministic components to LLM capabilities, workflows, and agents provides a way to introduce intelligence deliberately, avoiding unnecessary complexity and ensuring that the use of agentic AI is driven by the problem it solves rather than by the technology trend itself.

References

[1] Victor Dibia, “Multi-Agent Systems with AutoGen”.

[2] Anand, Oshin and Vijayan, Karthika, A Framework for Building Robust AI Agents (July 01, 2026). Available at SSRN: https://ssrn.com/abstract=7261000 or http://dx.doi.org/10.2139/ssrn.7261000