kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

append group_entries[0]

Browse Source 
Signed-off-by: Balaram Buddharaju <169953907+brb-nv@users.noreply.github.com>
This commit is contained in:
Balaram Buddharaju 2026-01-09 13:25:22 -08:00
parent 2b746f0ad4
commit 0452a58a32
1 changed files with 1 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
tensorrt_llm/_torch/pyexecutor/executor_request_queue.py
View File

@ -384,8 +384,7 @@ class ExecutorRequestQueue:
f"CP ranks within DP group {dp_group_idx} have mismatched request counts: " \
f"{[entry[0] for entry in group_entries]}"
# Use token count from cp_rank0.
aggregated_responses.append(
[group_entries[0][0], group_entries[0][1]])
aggregated_responses.append(group_entries[0])
responses_list = aggregated_responses
else:
responses_list = self.dist.tp_allgather(