Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Intel Posts Important Update on Project Battlematrix, Releases LLM Scaler 1.0
#1
Intel Posts Important Update on Project Battlematrix, Releases LLM Scaler 1.0

Intel today released an update on Project Battlematrix, its scalable and accessible inference workstations initiative announced in May, along the sidelines of Computex 2025. The initiative is designed to expedite Intel's AI strategy by streamlining the integration of Intel Arc Pro B-series "Battlemage" GPUs through a specialized, inference-optimized software stack. This new stack prioritizes end-user intuitiveness and adherence to industry standards, and features a containerized solution tailored for Linux environments. It promises exceptional inference performance, supported by multi-GPU scaling. Additionally, Battlematrix incorporates enterprise-grade reliability features, including ECC, single-root input/output virtualization (SR-IOV), telemetry monitoring, and remote firmware updates. These enhancements aim to make high-performance AI workloads more accessible and efficient for developers and enterprises.

The latest milestone sees Intel roll out version 1.0 of the LLM Scaler container, a pivotal release enabling early adoption and testing by business users. This update builds on the vLLM framework with notable performance boosts, such as up to 1.8x improvement in token-per-output-per-second (TPOP) for long input sequences exceeding 4K on 32B KPI models, and a remarkable 4.2x gain for 70B KPI models at 40K sequence lengths. Optimization delivers around 10% higher output throughput for 8B-32B KPI models compared to prior versions, alongside features like by-layer online quantization to minimize GPU memory usage, experimental pipeline parallelism (PP) support, torch.compile integration, and speculative decoding. Read full story


https://www.techpowerup.com/339834/intel...scaler-1-0
  


Forum Jump:


Users browsing this thread:
1 Guest(s)