kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-02-04 10:11:47 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
0acdecb2c3
TensorRT-LLMs/examples/scaffolding/contrib/AsyncGeneration
History
Zhenhuan Chen dee6644ed9
feat(scaffolding): add streaming scaffolding_llm.generate_async support (#5345) 
Signed-off-by: Zhenhuan Chen <chenzhh3671@gmail.com>
2025-07-08 15:08:40 +09:00
..
README.md
stream_generation_controller.py feat(scaffolding): add streaming scaffolding_llm.generate_async support (#5345) 2025-07-08 15:08:40 +09:00
stream_generation_run.py

README.md

This example shows how to use the StreamGenerationTask and stream_generation_handler to enable efficient streaming-based generation workflows.

How to run the example?

python stream_generation_run.py

See more detail on tensorrt_llm/scaffolding/contrib/AsyncGeneration/README.md.