Local RAG for CAD API: Overcoming Knowledge Gaps in Proprietary Interfaces

An engineer faced the need to automate tasks in Smart3D, but knowledge of the API was a bottleneck. He built a local RAG pipeline: decompiled libraries, extracted facts, enriched them via LLM, created embedding indices, and implemented an agent with verification — so the model generates code based solely on real API entities.

Hexagon

Writing a Decoder-Only Transformer for LLM from Scratch in Python

The author of a series of articles describes in detail the implementation of a transformer block for a small decoder-only LLM using the PyTorch framework. Components covered include: Multi-Head Attention with masking, Feed Forward Network with GELU, normalization layer, and residual connections. The article explains the difference from the original architecture: pre-normalization (Pre-LN) is used instead of post-normalization for training stability.

Google/DeepMind OpenAI Meta