TensorRT-LLMs/examples/wide_ep/slurm_scripts

TensorRT LLM Wide-EP Benchmark Scripts

This directory contains scripts for benchmarking TensorRT LLM wide-ep performance using SLURM job scheduler.

⚠️ DISCLAIMER

These scripts are currently not QA'ed and are provided for demonstration purposes only.

Please note that:

• These scripts have not undergone formal quality assurance testing
• They are intended for demonstration and educational purposes
• Use at your own risk in production environments
• Always review and test scripts thoroughly before running in your specific environment

Scripts Overview

Core Scripts

Note that, core implementation of the slurm scripts are included in examples/disaggregated/slurm/benchmark.

1. submit_e2e.sh - Main entry point for submitting E2E benchmark jobs
2. submit_gen_only.sh - Main entry point for submitting gen-only benchmark jobs
3. process_gen_iterlog.py - Processes benchmark results and generates reports

Usage

Prerequisites

Before running the scripts, ensure you have:

• Access to a SLURM cluster
• Container image with TensorRT LLM installed
• Model files accessible on the cluster
• Required environment variables set

Run E2E Benchmarks

# Refer to `examples/disaggregated/slurm/benchmark/`
# Please find the `disaggr_torch.slurm` script in the `examples/disaggregated/slurm/benchmark/` directory.
# Make sure that SLURM parameters are correctly set in `disaggr_torch.slurm` before executing this script.
./submit_e2e.sh

Run gen-only Benchmarks and post-processes the results using process_gen_iterlog.py

./submit_gen_only.sh

python3 process_gen_iterlog.py --dir_prefix <path>

process_gen_iterlog.py will be responsible for:

• Parses iteration logs from workers
• Calculates throughput metrics
• Generates CSV reports
• Supports MTP (Multi-Token Prediction) analysis