TensorRT-LLMs/cpp
Robin Kobus e12e7a753d
refactor: Expose DecoderState via bindings and integrate in TRTLLMDecoder (#3139)
* refactor: Expose DecoderState via bindings and integrate in TRTLLMDecoder

- Introduced a new `DecoderState` class in the C++ bindings, encapsulating key functionalities for managing decoding state.
- Adjusted the Python `TRTLLMDecoder` to access properties from `decoder_state`, ensuring consistency and clarity in the decoding process.

These changes streamline the decoder's architecture and enhance maintainability.

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>

* chore: Remove unused new_tokens from DecoderState bindings

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>

---------

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
2025-04-05 07:42:35 +08:00
..
cmake fix #3109: early exit cmake if find_library() does not find any lib (#3113) 2025-03-29 19:59:03 +08:00
include/tensorrt_llm Reapply "refactor: Replace DecoderFinishedEvent with CudaEvent in decoder clas…" (#3183) (#3195) 2025-04-04 15:56:28 +02:00
micro_benchmarks perf: Add optimizations for deepseek in min latency mode (#3093) 2025-04-02 09:05:24 +08:00
tensorrt_llm refactor: Expose DecoderState via bindings and integrate in TRTLLMDecoder (#3139) 2025-04-05 07:42:35 +08:00
tests update FP4 quantize layout (#3045) 2025-04-03 13:13:54 -04:00
CMakeLists.txt fix: upgrade cmake minimum from 3.18 to 3.27 (#3208) 2025-04-02 15:14:36 +08:00