kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-25 21:22:57 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
9c03a7ab74
TensorRT-LLMs/examples/scaffolding/contrib/AsyncGeneration
History
WeiHaocheng 0f01826dde
feat: support task collection for to collect information (#3328) (#3824) 
Signed-off-by: fredw (generated by with_the_same_user script) <20514172+WeiHaocheng@users.noreply.github.com>
2025-05-09 17:09:01 +08:00
..
README.md feat(part 2): Enhance the integrated robustness of scaffolding with __init__.py #3305 (#3731) 2025-04-24 18:47:03 +08:00
stream_generation_controller.py feat(part 2): Enhance the integrated robustness of scaffolding with __init__.py #3305 (#3731) 2025-04-24 18:47:03 +08:00
stream_generation_run.py feat: support task collection for to collect information (#3328) (#3824) 2025-05-09 17:09:01 +08:00

README.md

This example shows how to use the StreamGenerationTask and stream_generation_handler to enable efficient streaming-based generation workflows.

How to run the example?

python stream_generation_run.py

See more detail on tensorrt_llm/scaffolding/contrib/AsyncGeneration/README.md.