kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
c75d7cd684
TensorRT-LLMs/cpp/include/tensorrt_llm/runtime
History
Robin Kobus 3ee4332fb1
refactor: Replace DecoderFinishedEvent with CudaEvent in decoder classes (#3078) 
- Updated the `forwardAsync` method in `GptDecoderBatched` and `iGptDecoderBatched` to return `CudaEvent` instead of `DecoderFinishedEventPtr`, simplifying event handling.
- Removed the `DecoderFinishedEvent` class and its associated usage across various files, streamlining the codebase.
- Adjusted related methods and Python bindings to accommodate the new event structure, ensuring compatibility and maintaining functionality.

These changes enhance the clarity and efficiency of the decoding process in the batch manager.

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
2025-03-28 14:50:52 +08:00
..
utils Update TensorRT-LLM (#2820) 2025-02-25 21:21:49 +08:00
bufferManager.h Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
common.h Update TensorRT-LLM (#2502) 2024-11-26 16:51:34 +08:00
cudaEvent.h Update TensorRT-LLM (#787) 2024-01-02 17:54:32 +08:00
cudaStream.h Update TensorRT-LLM (#2532) 2024-12-04 21:16:56 +08:00
decoderState.h Update (#2978) 2025-03-23 16:39:35 +08:00
decodingInput.h Update TensorRT-LLM (#2820) 2025-02-25 21:21:49 +08:00
decodingOutput.h Update (#2978) 2025-03-23 16:39:35 +08:00
eagleBuffers.h Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00
eagleModule.h Update TensorRT-LLM (#2792) 2025-02-18 21:27:39 +08:00
explicitDraftTokensBuffers.h Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00
generationInput.h Update TensorRT-LLM (#1598) 2024-05-14 16:43:41 +08:00
generationOutput.h Update TensorRT-LLM (#1598) 2024-05-14 16:43:41 +08:00
gptDecoder.h Update TensorRT-LLM (#2582) 2024-12-16 21:50:47 -08:00
gptDecoderBatched.h refactor: Replace DecoderFinishedEvent with CudaEvent in decoder classes (#3078) 2025-03-28 14:50:52 +08:00
gptJsonConfig.h Update TensorRT-LLM (#2532) 2024-12-04 21:16:56 +08:00
gptSession.h Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
iBuffer.h Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
iGptDecoderBatched.h refactor: Replace DecoderFinishedEvent with CudaEvent in decoder classes (#3078) 2025-03-28 14:50:52 +08:00
ipcNvlsMemory.h Update TensorRT-LLM (#2792) 2025-02-18 21:27:39 +08:00
ipcUtils.h Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00
iStatefulGptDecoder.h refactor: Remove speculative decoding parameters from stateful decoders (#3024) 2025-03-26 20:16:26 +08:00
iTensor.h Update TensorRT-LLM (#2532) 2024-12-04 21:16:56 +08:00
lookaheadBuffers.h Update TensorRT-LLM (#2582) 2024-12-16 21:50:47 -08:00
lookaheadModule.h Update TensorRT-LLM (#2582) 2024-12-16 21:50:47 -08:00
loraCache.h Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
loraCachePageManagerConfig.h Update TensorRT-LLM (#1598) 2024-05-14 16:43:41 +08:00
loraModule.h Update TensorRT-LLM (#2755) 2025-02-11 03:01:00 +00:00
medusaModule.h Update TensorRT-LLM (#2436) 2024-11-12 15:27:49 +08:00
memoryCounters.h Update TensorRT-LLM (#2110) 2024-08-13 22:34:33 +08:00
modelConfig.h Update TensorRT-LLM (#2873) 2025-03-11 21:13:42 +08:00
promptTuningParams.h Update TensorRT-LLM (#1598) 2024-05-14 16:43:41 +08:00
rawEngine.h Update TensorRT-LLM (#2215) 2024-09-10 18:21:22 +08:00
request.h Update TensorRT-LLM (#2413) 2024-11-05 16:27:06 +08:00
runtimeDefaults.h Update TensorRT-LLM (#2436) 2024-11-12 15:27:49 +08:00
samplingConfig.h v1.2 (#3082) 2025-03-26 23:31:29 +08:00
speculativeDecodingMode.h Update TensorRT-LLM (#2436) 2024-11-12 15:27:49 +08:00
speculativeDecodingModule.h Update TensorRT-LLM (#1763) 2024-06-11 16:59:02 +08:00
statefulGptDecoderBatched.h refactor: Replace DecoderFinishedEvent with CudaEvent in decoder classes (#3078) 2025-03-28 14:50:52 +08:00
tllmLogger.h Update TensorRT-LLM (#787) 2024-01-02 17:54:32 +08:00
worldConfig.h Update TensorRT-LLM (#2849) 2025-03-04 18:44:00 +08:00