Netanel Haber 6ee94c7ac8 Reintroduce with perf fixes: feature: unify new_tokens format sample state to trtllm samper tokens format (#5513) ... 58a8a8f - these changes were previously merged to main here. 6aef149 - the changes were temporarily reverted in main, due to a significant perf regression in models using the TorchSampler (observed by @byshiue). This PR is meant to re-merge these changes along with a fix to prevent the regression. The first commit of this PR is actually just the reverted revert - filter it out of the changes to see previously unmerged changes. Signed-off-by: Netanel Haber <nhaber@nvidia.com>		2025-06-30 11:58:59 -07:00
..
__init__.py	Update TensorRT-LLM (#2755)	2025-02-11 03:01:00 +00:00
_util.py	Reintroduce with perf fixes: feature: unify new_tokens format sample state to trtllm samper tokens format (#5513)	2025-06-30 11:58:59 -07:00
config_utils.py	feat: support kv cache reuse for MLA (#3571)	2025-05-15 15:22:21 +08:00
config.py	Chore: remove unused variables (#5314)	2025-06-24 22:27:32 +08:00
cuda_graph_runner.py	[TRTLLM-4983] feat: enable overlap scheduler between draft forwards (#4802)	2025-06-15 23:09:16 +08:00
finish_reason.py	[TRTLLM-5974][feat] Support disaggregated serving in TRTLLM Sampler (#5328)	2025-06-25 17:41:36 +02:00
grammar_matcher.py	test: Add LLGuidance test and refine guided decoding (#5348)	2025-06-25 14:12:56 +08:00
guided_decoder.py	Reintroduce with perf fixes: feature: unify new_tokens format sample state to trtllm samper tokens format (#5513)	2025-06-30 11:58:59 -07:00
handle_context_logits.py	refactor: Update decoder buffer and logits management (#4450)	2025-06-18 08:10:32 +08:00
handle_generation_logits.py	refactor: Update decoder buffer and logits management (#4450)	2025-06-18 08:10:32 +08:00
kv_cache_transceiver.py	Solve underallocation in VSWA+/VGQA (#4667)	2025-06-12 12:12:46 +08:00
layerwise_nvtx_marker.py	Update TensorRT-LLM (#2849)	2025-03-04 18:44:00 +08:00
llm_request.py	Reintroduce with perf fixes: feature: unify new_tokens format sample state to trtllm samper tokens format (#5513)	2025-06-30 11:58:59 -07:00
make_decoding_batch_input_output.py	refactor: Speculative decoding buffers part 2 (#5316)	2025-06-27 17:41:48 +02:00
model_engine.py	Reintroduce with perf fixes: feature: unify new_tokens format sample state to trtllm samper tokens format (#5513)	2025-06-30 11:58:59 -07:00
py_executor_creator.py	[TRTLLM-5000][feat] NGrams V2 (#4569)	2025-06-27 23:00:17 +08:00
py_executor.py	Reintroduce with perf fixes: feature: unify new_tokens format sample state to trtllm samper tokens format (#5513)	2025-06-30 11:58:59 -07:00
resource_manager.py	[TRTLLM-5000][feat] NGrams V2 (#4569)	2025-06-27 23:00:17 +08:00
sampler.py	Reintroduce with perf fixes: feature: unify new_tokens format sample state to trtllm samper tokens format (#5513)	2025-06-30 11:58:59 -07:00
scheduler.py	Reintroduce with perf fixes: feature: unify new_tokens format sample state to trtllm samper tokens format (#5513)	2025-06-30 11:58:59 -07:00
seq_slot_manager.py	Reintroduce with perf fixes: feature: unify new_tokens format sample state to trtllm samper tokens format (#5513)	2025-06-30 11:58:59 -07:00