kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
2b27810198
TensorRT-LLMs/examples/scaffolding/contrib/AsyncGeneration
History
Cao Dong 62cea877b1
[None][feat] Move StreamGeneration to scaffolding main directory (#8347) 
Signed-off-by: Dong Cao <docao@nvidia.com>
2025-10-14 17:16:04 +08:00
..
README.md feat(part 2): Enhance the integrated robustness of scaffolding with __init__.py #3305 (#3731) 2025-04-24 18:47:03 +08:00
stream_generation_controller.py [None][feat] Move StreamGeneration to scaffolding main directory (#8347) 2025-10-14 17:16:04 +08:00
stream_generation_run.py [TRTLLM-4911] feat(scaffolding): make sampling_params only setable by controller (#4151) 2025-05-12 15:29:09 +08:00

README.md

This example shows how to use the StreamGenerationTask and stream_generation_handler to enable efficient streaming-based generation workflows.

How to run the example?

python stream_generation_run.py

See more detail on tensorrt_llm/scaffolding/contrib/AsyncGeneration/README.md.