TensorRT-LLMs

mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00

History

Zhenhuan Chen 9212e9a740 [TRTLLM-4911] feat(scaffolding): make sampling_params only setable by controller (#4151 ) feat(scaffolding): make sampling_params only setable by controller Signed-off-by: Zhenhuan Chen <chenzhh3671@gmail.com>		2025-05-12 15:29:09 +08:00
..
README.md	feat(part 2): Enhance the integrated robustness of scaffolding with __init__.py #3305 (#3731 )	2025-04-24 18:47:03 +08:00
stream_generation_controller.py	feat(part 2): Enhance the integrated robustness of scaffolding with __init__.py #3305 (#3731 )	2025-04-24 18:47:03 +08:00
stream_generation_run.py	[TRTLLM-4911] feat(scaffolding): make sampling_params only setable by controller (#4151 )	2025-05-12 15:29:09 +08:00

This example shows how to use the StreamGenerationTask and stream_generation_handler to enable efficient streaming-based generation workflows.

How to run the example?

python stream_generation_run.py