AMD LLM Extension 26.06 release notes#
2 min read time
This is the sixth release of the AMD LLM Extension toolkit, an open-source software toolkit built on the ROCm platform for large language model (LLM) extensions, integrations, and performance enablement on AMD GPUs. The domain brings together training, post-training, inference, and orchestration components to make modern LLM stacks practical and reproducible on AMD hardware.
Release highlights#
Note
AMD LLM Extension 26.06 introduces updates to two components (verl and Ray) as part of the toolkit; other components remain unchanged (ComfyUI, FlashInfer, llama.cpp, ROCm-RAG, and Triton Inference Server).
This release updates the following component with support for ROCm 7.0.2:
verl (Volcano Engine Reinforcement Learning for LLMs) is an open-source framework for reinforcement-learning post-training of large language models and is the reference implementation of HybridFlow.
This release updates the following component with support for ROCm 7.2.1:
Ray is an open-source framework for scaling Python and AI workloads, providing the distributed compute and orchestration layer used by verl for multi-GPU and multi-node training.
System requirements#
ROCm‑LLMExt components span a range of ROCm version requirements depending on the specific extension. Ensure you follow the installation instructions for each component, which list the exact ROCm dependencies, or refer to the Compatibility matrix to verify the supported ROCm versions.
AMD LLM Extension components#
The following table lists AMD LLM Extension component versions for the 26.06 release. Click to go to the component’s source on GitHub.
| Name | Version | Source |
|---|---|---|
| verl | 0.7.1 | |
| Ray | 2.55.1 | |
| llama.cpp | b6652 | |
| FlashInfer | 0.5.3 | |
| Triton Inference Server | 25.12 | |
| ComfyUI | 0.18.2 | |
| ROCm-RAG | 1.0.0 |
Detailed component changelogs#
verl 0.7.1#
This release updates verl in AMD LLM Extension from 0.6.0 to 0.7.1, and adds support for ROCm 7.0.2 on AMD Instinct MI300X, MI325X, and MI355X GPUs with PyTorch 2.9.1 and Python 3.12 on Ubuntu 22.04.
Ray 2.55.1#
This release updates Ray in AMD LLM Extension from 2.51.1 to 2.55.1, and adds support for ROCm 7.2.1 on AMD Instinct MI300X, MI325X, and MI355X GPUs with PyTorch 2.10.0 and Python 3.12 on Ubuntu 22.04.