Six years of developing a Telegram bot: from tokenizer to LLM, RAG and vector databases
OpenAI
Google/DeepMind
The author tells the story of creating Io, a Telegram chat bot that evolved from a primitive tokenizer-based bot to a full-fledged LLM system using RAG and vector databases. The article covers the challenges of implementing memory, image and voice recognition, monetization, and infrastructure evolution over six years.
The author developed Io, an LLM-based Telegram bot capable of responding to messages, recognizing images and voice messages, and maintaining context. The first version in 2020 used a custom tokenizer with stemming for Russian, integrating weather from OpenWeather, city detection via DaData, and translation and voice recognition via Yandex services. In 2023, the author experimented with GPT-3.5 Turbo but later rewrote the bot to use GPT-4o mini with tool calling, discarding the tokenizer. In 2025, memory was first implemented using JSON metadata, then switched to a vector-based approach with Qdrant, finally moving to pgvector on PostgreSQL. The bot uses Gemini 3.6 Flash as the primary model and Nex-N2-mini for facts extraction and summarization. Additional features include guest mode, Wikipedia search, and explicit memory commands. Monetization was introduced with daily limits and subscriptions. The infrastructure evolved from Docker Compose to Coolify and then to k3s, with GitHub Actions for deployment. The author plans to improve monitoring, fix guest mode bugs, and implement web search.
- Abbreviations
- LLM = Large Language Model — большая языковая модель
- RAG = Retrieval-Augmented Generation — генерация с дополнением извлечением
- JSON = JavaScript Object Notation — текстовый формат обмена данными
- API = Application Programming Interface — программный интерфейс приложения
- S3 = Simple Storage Service — сервис хранения данных
Source: Habr — хаб ИИ —
original