Robin Kobus b7a38feb14 chore: Clean up cpp runtime (#3537) ... * add space in test output Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com> * perf: reduce executor lock scope Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com> * refactor: Move TokenRangeRetentionConfig implementation to cpp file Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com> * fix: Improve finished steps handling for external draft tokens - Fixed a bug where the whole finished steps tensor was being zeroes instead of the slices. - Replaced the creation of a temporary tensor for finished steps with a direct slice from the input tensor, improving efficiency and readability. - Updated the tensor management logic to streamline the process of setting zero values for finished steps during batch processing. Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com> * chore: Clean up includes Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com> --------- Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com>		2025-04-15 16:06:14 +08:00
..
utils	feat: Add numNodes to ParallelConfig (#3346)	2025-04-13 13:55:04 +02:00
bufferManager.cpp	Update TensorRT-LLM (#2873)	2025-03-11 21:13:42 +08:00
bufferView.h	Update TensorRT-LLM (#2783)	2025-02-13 18:40:22 +08:00
CMakeLists.txt	Update (#2978)	2025-03-23 16:39:35 +08:00
cudaMemPool.cpp	Update TensorRT-LLM (#2755)	2025-02-11 03:01:00 +00:00
cudaMemPool.h	Update TensorRT-LLM (#2755)	2025-02-11 03:01:00 +00:00
decoderState.cpp	refactor: batch slot management in decoder classes (#3300)	2025-04-13 05:05:13 +08:00
decodingLayerWorkspace.cpp	Update TensorRT-LLM (#2184)	2024-09-03 12:14:23 +02:00
decodingLayerWorkspace.h	Update TensorRT-LLM (#2436)	2024-11-12 15:27:49 +08:00
decodingOutput.cpp	Update (#2978)	2025-03-23 16:39:35 +08:00
eagleBuffers.cpp	Update TensorRT-LLM (#2873)	2025-03-11 21:13:42 +08:00
explicitDraftTokensBuffers.cpp	Update TensorRT-LLM (#2873)	2025-03-11 21:13:42 +08:00
explicitDraftTokensModule.h	Update TensorRT-LLM (#1763)	2024-06-11 16:59:02 +08:00
generationConfig.cpp	Update TensorRT-LLM (#2110)	2024-08-13 22:34:33 +08:00
generationConfig.h	Update TensorRT-LLM (#2110)	2024-08-13 22:34:33 +08:00
gptDecoder.cpp	Feat: Variable-Beam-Width-Search (VBWS) part3 (#3338)	2025-04-08 23:51:27 +08:00
gptDecoderBatched.cpp	chore: Clean up cpp runtime (#3537)	2025-04-15 16:06:14 +08:00
gptJsonConfig.cpp	Feat: Variable-Beam-Width-Search (VBWS) part3 (#3338)	2025-04-08 23:51:27 +08:00
gptSession.cpp	refactor: Remove speculative decoding parameters from stateful decoders (#3024)	2025-03-26 20:16:26 +08:00
iBuffer.cpp	Update TensorRT-LLM (#2755)	2025-02-11 03:01:00 +00:00
ipcNvlsMemory.cpp	chore: Stabilize ABI boundary for internal kernel library (#3117)	2025-04-11 15:07:50 +08:00
ipcSocket.cpp	Update TensorRT-LLM (#2755)	2025-02-11 03:01:00 +00:00
ipcSocket.h	Update TensorRT-LLM (#2755)	2025-02-11 03:01:00 +00:00
ipcUtils.cpp	chore: remove usernames from comments (#3291)	2025-04-05 13:44:28 +08:00
iTensor.cpp	Update TensorRT-LLM (#2849)	2025-03-04 18:44:00 +08:00
jsonSerialization.h	Update TensorRT-LLM (#2436)	2024-11-12 15:27:49 +08:00
layerProfiler.cpp	Update TensorRT-LLM (#1554)	2024-05-07 23:34:28 +08:00
layerProfiler.h	Update TensorRT-LLM (#1554)	2024-05-07 23:34:28 +08:00
lookaheadBuffers.cpp	Feat: Variable-Beam-Width-Search (VBWS) part3 (#3338)	2025-04-08 23:51:27 +08:00
loraCache.cpp	Update TensorRT-LLM (#2755)	2025-02-11 03:01:00 +00:00
loraManager.cpp	Update TensorRT-LLM (#2755)	2025-02-11 03:01:00 +00:00
loraManager.h	Update TensorRT-LLM (#2413)	2024-11-05 16:27:06 +08:00
loraModule.cpp	Update TensorRT-LLM (#2755)	2025-02-11 03:01:00 +00:00
loraUtils.cpp	Update TensorRT-LLM (#2820)	2025-02-25 21:21:49 +08:00
loraUtils.h	Update TensorRT-LLM (#2820)	2025-02-25 21:21:49 +08:00
memoryCounters.cpp	Update TensorRT-LLM (#2110)	2024-08-13 22:34:33 +08:00
ncclCommunicator.cpp	Update TensorRT-LLM (#2755)	2025-02-11 03:01:00 +00:00
ncclCommunicator.h	Update TensorRT-LLM (#2792)	2025-02-18 21:27:39 +08:00
promptTuningParams.cpp	Update TensorRT-LLM (#1598)	2024-05-14 16:43:41 +08:00
rnnStateBuffers.cpp	open source 7f370deb0090d885d7518c2b146399ba3933c004 (#2273)	2024-09-30 13:51:19 +02:00
rnnStateBuffers.h	Update TensorRT-LLM (#2755)	2025-02-11 03:01:00 +00:00
runtimeBuffers.cpp	Update TensorRT-LLM (#2755)	2025-02-11 03:01:00 +00:00
runtimeBuffers.h	Update TensorRT-LLM (#2755)	2025-02-11 03:01:00 +00:00
runtimeKernels.cu	Update TensorRT-LLM (#2849)	2025-03-04 18:44:00 +08:00
runtimeKernels.h	Update TensorRT-LLM (#2849)	2025-03-04 18:44:00 +08:00
statefulGptDecoder.cpp	chore: remove usernames from comments (#3291)	2025-04-05 13:44:28 +08:00
statefulGptDecoder.h	refactor: Remove speculative decoding parameters from stateful decoders (#3024)	2025-03-26 20:16:26 +08:00
statefulGptDecoderBatched.cpp	refactor: batch slot management in decoder classes (#3300)	2025-04-13 05:05:13 +08:00
tensorView.h	Update TensorRT-LLM (#1793)	2024-06-18 18:18:23 +08:00
tllmBuffers.cpp	Update TensorRT-LLM (#2792)	2025-02-18 21:27:39 +08:00
tllmBuffers.h	chore: Stabilize ABI boundary for internal kernel library (#3117)	2025-04-11 15:07:50 +08:00
tllmLogger.cpp	Update TensorRT-LLM (#787)	2024-01-02 17:54:32 +08:00
tllmRuntime.cpp	feat: Introduce UB allocator for pytorch flow (#3257)	2025-04-08 18:39:49 +08:00
tllmRuntime.h	Update TensorRT-LLM (#2783)	2025-02-13 18:40:22 +08:00
torch.h	Update TensorRT-LLM (#2110)	2024-08-13 22:34:33 +08:00
torchUtils.h	Update TensorRT-LLM (#2755)	2025-02-11 03:01:00 +00:00
torchView.h	Update TensorRT-LLM (#1168)	2024-02-27 17:37:34 +08:00
transformerBuffers.cpp	chore: remove usernames from comments (#3291)	2025-04-05 13:44:28 +08:00
transformerBuffers.h	Update TensorRT-LLM (#2792)	2025-02-18 21:27:39 +08:00
workerPool.cpp	Update TensorRT-LLM (#2156)	2024-08-27 18:20:59 +08:00
workerPool.h	Update TensorRT-LLM (#2156)	2024-08-27 18:20:59 +08:00
worldConfig.cpp	Update TensorRT-LLM (#2849)	2025-03-04 18:44:00 +08:00