kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

[TRTQA-2802][fix]: add --host for mgmn serve examples script (#4175)

Browse Source 
remove prepare data

Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com>
This commit is contained in:
xinhe-nv 2025-05-12 13:28:42 +08:00 committed by GitHub
parent 1333f4f5d5
commit 186e2b8c38
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
examples/llm-api/llm_mgmn_trtllm_serve.sh
View File

@ -33,7 +33,6 @@
# not supported in Slurm mode, you need to download the model and put it in
# the LOCAL_MODEL directory.
echo "Starting trtllm-serve..."
# Just launch trtllm-serve job with trtllm-llmapi-launch command.
srun -l \
@ -51,5 +50,6 @@ srun -l \
trtllm-serve $LOCAL_MODEL \
--tp_size 16 \
--backend pytorch \
--host 0.0.0.0 \
${ADDITIONAL_OPTIONS}
"