TensorRT-LLMs/cpp
Robin Kobus b7a38feb14
chore: Clean up cpp runtime (#3537)
* add space in test output

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>

* perf: reduce executor lock scope

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>

* refactor: Move TokenRangeRetentionConfig implementation to cpp file

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>

* fix: Improve finished steps handling for external draft tokens

- Fixed a bug where the whole finished steps tensor was being zeroes instead of the slices.
- Replaced the creation of a temporary tensor for finished steps with a direct slice from the input tensor, improving efficiency and readability.
- Updated the tensor management logic to streamline the process of setting zero values for finished steps during batch processing.

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>

* chore: Clean up includes

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>

---------

Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>
2025-04-15 16:06:14 +08:00
..
cmake fix #3109: early exit cmake if find_library() does not find any lib (#3113) 2025-03-29 19:59:03 +08:00
include/tensorrt_llm chore: Clean up cpp runtime (#3537) 2025-04-15 16:06:14 +08:00
micro_benchmarks feat: Add FP8 support for SM 120 (#3248) 2025-04-14 16:05:41 -07:00
tensorrt_llm chore: Clean up cpp runtime (#3537) 2025-04-15 16:06:14 +08:00
tests chore: Clean up cpp runtime (#3537) 2025-04-15 16:06:14 +08:00
CMakeLists.txt infra: move nvrtc_wrapper to conan (#3282) 2025-04-15 05:31:01 +08:00
conandata.yml infra: move nvrtc_wrapper to conan (#3282) 2025-04-15 05:31:01 +08:00
conanfile.py infra: move nvrtc_wrapper to conan (#3282) 2025-04-15 05:31:01 +08:00