kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
0253036a4e
TensorRT-LLMs/examples/scaffolding/contrib/AsyncGeneration
History
Zhenhuan Chen dee6644ed9
feat(scaffolding): add streaming scaffolding_llm.generate_async support (#5345) 
Signed-off-by: Zhenhuan Chen <chenzhh3671@gmail.com>
2025-07-08 15:08:40 +09:00
..
README.md feat(part 2): Enhance the integrated robustness of scaffolding with __init__.py #3305 (#3731) 2025-04-24 18:47:03 +08:00
stream_generation_controller.py feat(scaffolding): add streaming scaffolding_llm.generate_async support (#5345) 2025-07-08 15:08:40 +09:00
stream_generation_run.py [TRTLLM-4911] feat(scaffolding): make sampling_params only setable by controller (#4151) 2025-05-12 15:29:09 +08:00

README.md

This example shows how to use the StreamGenerationTask and stream_generation_handler to enable efficient streaming-based generation workflows.

How to run the example?

python stream_generation_run.py

See more detail on tensorrt_llm/scaffolding/contrib/AsyncGeneration/README.md.