What are the main problems developers face when building systems where multiple AI agents work together?
I’ve been doing it for a long time. I’ve had zero issues. Have you been having problems or are you attempting to anticipate problems and create solutions for them?
I implemented what I’ve called a Cortex module. It essentially makes all the decisions and decides what bot does what. Meaning the model I talk to tells the other models what to do via tool calls. They all share 1 source of truth in memory.
Hey @Pimpcat-AU and @Uzer-namo-2024 am I right in assuming that you guys work together (same name for the “primary” module, referencing an image in the other’s post) or is this just a massive coincidence. I’m building a personal solution extremely similar to what you’ve each laid out above with similar functional requirements (consumer-grade hardware) indeed the screenshot in @Pimpcat-AU 's post could have come from my own system ;-). I’m just asking because I’m curious what NovBase is - a commercial entity, an open-source library etc and I’d be interested in finding out more (to compare approaches)?
No. He’s just copy pasting stuff to and from a chat model.
From what I’ve seen, the biggest challenge in multi-agent AI systems is not getting agents to “talk” to each other — it’s getting them to collaborate reliably and predictably in production.
Some major problems developers usually face are:
-
Coordination complexity — agents may conflict, repeat work, or lose track of responsibilities
-
Context sharing — maintaining consistent memory and state across agents is difficult
-
Error propagation — one weak agent can create cascading failures through the workflow
-
Hallucinations between agents — agents can reinforce incorrect assumptions from each other
-
Latency and cost — multi-agent systems can become very expensive and slow quickly
-
Debugging difficulty — tracing why a workflow failed becomes much harder compared to single-agent systems
-
Tool orchestration — managing permissions, actions, retries, and dependencies across agents is complex
-
Evaluation — measuring whether the collaboration is actually improving outcomes is still an unsolved problem for many teams
In practice, many production systems end up using a “controlled multi-agent” setup instead of fully autonomous agents. Usually there’s:
-
one orchestrator/planner agent
-
specialized worker agents
-
deterministic business logic around them
-
strong guardrails and validation layers
I think the industry is still early here. Multi-agent demos look impressive, but making them stable, cost-efficient, and production-ready is a very different challenge altogether.
I think one of the biggest issues is that chunking + embeddings break the original document structure.
RAG can retrieve relevant slices pretty well, but it often misses the overall context and relationships between parts of the data.
Feels like semantic similarity alone is not enough for complex reasoning.
Curious what direction people think makes more sense:
-
graph-structured documents?
-
compressed representations?
-
late chunking https://medium.com/accredian/traditional-vs-late-chunking-ed1a70022ff4
Or something else entirely?