Challenges and Experiences in Using AI Agents: What You Need to Know

Sergey Kozlov, director of the developer of the CRM system "Megaplan" and the knowledge base with AI assistant Collabis, together with Igor Sukhoverkhov, head of Collabis development, share their experience in implementing AI agents. They note that the quality of the agent's work strongly depends on how well-structured the Knowledge Base is: if it's messy, the AI responses will be inaccurate. Context is also important—simply writing "make it cool" is not enough; you need to deeply immerse the agent in the context of the task. The modern approach to working with AI is loop engineering, where the key factor is how the agent performs the task and uses tools. The authors warn that if the AI goes in the wrong direction, it wastes time and tokens. They give an example: in a tester's task, instead of pointing out a broken button, the agent delved into the code and tried to fix it. Most models handle simple and well-defined tasks well, but complex multi-step tasks require more expensive models—they can forget the beginning, just like humans. Manual control is mandatory: an agent today is like a junior, whose work must be checked by a senior. The risks of blocking access to foreign models are also mentioned, so it is recommended to store data in a personal Knowledge Base. The company "Megaplan" has developed an MCP server for data transfer, and there is also a Pet project based on GPT 5.6 running locally.