TensorRT-LLMs/cpp/tensorrt_llm/pybind
Robin Kobus 3ee4332fb1
refactor: Replace DecoderFinishedEvent with CudaEvent in decoder classes (#3078)
- Updated the `forwardAsync` method in `GptDecoderBatched` and `iGptDecoderBatched` to return `CudaEvent` instead of `DecoderFinishedEventPtr`, simplifying event handling.
- Removed the `DecoderFinishedEvent` class and its associated usage across various files, streamlining the codebase.
- Adjusted related methods and Python bindings to accommodate the new event structure, ensuring compatibility and maintaining functionality.

These changes enhance the clarity and efficiency of the decoding process in the batch manager.

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
2025-03-28 14:50:52 +08:00
..
batch_manager fix: disable kv cache reuse when minimum window size is reached, instead of maximum window size (#2983) 2025-03-24 22:49:52 +08:00
common Update TensorRT-LLM (#2820) 2025-02-25 21:21:49 +08:00
executor feat: Add BW measurement (#3070) 2025-03-28 10:53:00 +08:00
runtime refactor: Replace DecoderFinishedEvent with CudaEvent in decoder classes (#3078) 2025-03-28 14:50:52 +08:00
userbuffers Update TensorRT-LLM (#2783) 2025-02-13 18:40:22 +08:00
bindings.cpp v1.2 (#3082) 2025-03-26 23:31:29 +08:00
CMakeLists.txt Update TensorRT-LLM (#2849) 2025-03-04 18:44:00 +08:00