Technology Innovation Institute

En son yapay zeka haberleri, modeller ve sürümler Technology Innovation Institute. ['ZAYA1-8B']

Araştırma 🇺🇸

Latest innovations in LLM architectures: shared KV caches, per-layer embeddings, and compressed attention

New open LLMs increasingly focus on efficient long-context processing. Gemma 4 (Google) uses KV tensor sharing between layers to reduce cache size, as well as per-layer embeddings to improve parametric efficiency. Laguna XS.2 (Poolside) employs per-layer attention budgeting, and DeepSeek V4 introduces mHC and compressed attention mechanisms.

Google/DeepMind

Poolside

DeepSeek

Technology Innovation Institute

Sebastian Raschka27.07 · 13:06

Araştırma 🇺🇸

Survey of LLM Research Papers for the First Half of 2026

Sebastian Raschka published a curated list of key scientific papers on large language models (LLMs) from January to May 2026. The selection highlights trends: hybrid architectures (alternating attention and state-space layers), efficient inference, agentic systems, and diffusion language models. Special attention is given to Nvidia's Nemotron 3 with hybrid design and Qwen3.6 with Gated DeltaNet.

NVIDIA

Alibaba/Qwen

Mistral

Baidu

Cohere

Technology Innovation Institute

MiniMax

Sebastian Raschka27.07 · 12:03