Building a local RAG for complex semantic texts: when cloud AI fails and MiniLM breaks on philosophy
Sentence-Transformers (UKPLab)
LM Studio
A developer recounts building a local RAG pipeline for philosophical texts by Jane Roberts, using multilingual-e5-large embeddings and Qwen2.5 via LM Studio. The project includes data preparation, mark-up, chunking, and retrieval. The author seeks community feedback on Chroma index architecture, dialogue state management, and reranker usage.
A developer built a local RAG system for philosophical texts by Jane Roberts after cloud AI models hallucinated and failed. Raw scans were cleaned, marked up in XML with speaker tags, and chunked by paragraphs. For embeddings, multilingual-e5-large outperformed all-MiniLM-L6-v2 which showed high semantic drift on abstract terminology. The pipeline uses ChromaDB for vector storage and Qwen2.5-14B-coder via LM Studio for generation. Key engineering decisions include dual enrichment (passage: prefix for E5, Quotes from my book prompt for LLM), filtering out stage direction type delivery, and multi-language support. The author highlights three open questions: whether to split the Chroma index into multiple collections as data grows, how to implement dialogue memory without exploding token budget, and whether a cross-encoder reranker would improve retrieval accuracy on the current 911 chunks. The project is published as a research log on GitHub.
- Abbreviations
- RAG = Retrieval-Augmented Generation — генерация с дополнением извлечением
- XML = eXtensible Markup Language — расширяемый язык разметки
- OOD = Out-Of-Distribution — выход за пределы распределения
- LLM = Large Language Model — большая языковая модель
Source: Habr — хаб ИИ —
original