AI Infrastructure Faces New Challenges: Models Grow, Heterogeneous Computing and Agent Complexity Reshape the Field
Huawei
4Paradigm (Fourth Paradigm)
DeepSeek
Alibaba/Qwen
Cosmos (NVIDIA)
Google/DeepMind
Meta
Anthropic
As AI model architectures become increasingly complex and diverse, AI infrastructure must adapt to heterogeneous computing, multi-model pipelines, and rising agent-driven workloads. Experts from Huawei, Inferact vLLM, and 4Paradigm discuss key challenges: managing diverse AI accelerators (NVIDIA, Ascend, Cambricon, etc.), optimizing multi-modal inference, and how AI may eventually automate much of infrastructure management, shifting engineers from implementation to planning and SLA definition.
In a recent InfoQ live discussion, Huawei AI open-source ecology director Huang Zhipeng hosted Inferact vLLM committer Mo Zifeng and 4Paradigm (Fourth Paradigm) system R&D expert Yang Shouren to examine AI infrastructure evolution. Mo noted that model architectures are growing far faster than toolchains, with examples like DeepSeek V4's complex MoE and multi-modal models evolving from LLaVA to full-modal Cosmos V. Yang highlighted the core challenge of heterogeneous compute management as domestic AI accelerators proliferate (NVIDIA, Ascend, Cambricon, Kunlun, Tianshu, Muxi), causing consistency issues where a tensor output may fail on certain shapes on a specific GPU. Yang introduced HAMi, an open-source project for GPU virtualization and sharing across devices, and discussed the limitations of Kubernetes DRA (Dynamic Resource Allocation) for advanced device management. The group also explored the impact of AI agents on open-source communities: agents now help with rote PR tasks but also flood repositories with low-quality submissions that committers must manually triage. Mo described adopting a 'human-only' filtering strategy via work/school email verification. Looking ahead, Yang predicted AI will gradually take over infrastructure operations, freeing engineers to focus on higher-level SLA/SLO/cost design, but noted that open-source projects must maintain quality trust through comprehensive testing. Mo added that platform-type software will remain essential because even powerful agents benefit from the ecosystem's collective optimizations, and the 'Matthew effect' will concentrate usage around well-maintained projects like vLLM. The conversation closed with practical advice on handling unsupported hardware operators: vendor support, CPU offload, or iterative profiling using agents, though corner-case bugs remain difficult to catch.
- Сокращения
- MLP = Multilayer Perceptron — многослойный перцептрон
- MoE = Mixture of Experts — смесь экспертов
- VLM = Vision-Language Model — модель зрения-языка
- AR = Autoregressive — авторегрессионный
- DiT = Diffusion Transformer — диффузионный трансформер
- K8s = Kubernetes — Кубернетес
- DRA = Dynamic Resource Allocation — динамическое распределение ресурсов
- CRD = Custom Resource Definition — пользовательское определение ресурса
- GPU = Graphics Processing Unit — графический процессор
- NPU = Neural Processing Unit — нейронный процессор
- NUMA = Non-Uniform Memory Access — неоднородный доступ к памяти
- PCI = Peripheral Component Interconnect — периферийный компонентный интерконнект
- PD = Prefill-Decode — префилл-декод
- TP = Tensor Parallelism — тензорный параллелизм
- PP = Pipeline Parallelism — конвейерный параллелизм
- DP = Data Parallelism — параллелизм данных
- KV = Key-Value — ключ-значение
- SLA = Service Level Agreement — соглашение об уровне обслуживания
- SLO = Service Level Objective — целевой показатель уровня обслуживания
- SRE = Site Reliability Engineering — инженерия надежности сайтов
- CI/CD = Continuous Integration / Continuous Deployment — непрерывная интеграция / непрерывное развертывание
- OOM = Out of Memory — нехватка памяти
- RL = Reinforcement Learning — обучение с подкреплением
Source: InfoQ 中国 —
original