kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
2,610 Commits 25 Branches 59 Tags 2.1 GiB
feat/spark-single-gpu-dev
Commit Graph

31 Commits

Author SHA1 Message Date
Zhongdongming Dai
9bb0c9500e
[None][docs] Update Dynasor paper info (#7137) 
Signed-off-by: Zhongdongming Dai <zhongdongmin@nvidia.com>
 2025-08-29 18:47:47 -07:00
Zhenhuan Chen
992b273045
[https://nvbugs/5387375] fix(scaffolding): fix scaffolding aime test in test_e2e (#6140) 
Signed-off-by: Zhenhuan Chen <chenzhh3671@gmail.com>
 2025-07-18 10:34:37 +08:00
Zhenhuan Chen
dee6644ed9
feat(scaffolding): add streaming scaffolding_llm.generate_async support (#5345) 
Signed-off-by: Zhenhuan Chen <chenzhh3671@gmail.com>
 2025-07-08 15:08:40 +09:00
Yan Chunwei
dfce61f4b9
[TRTLLM-5530][BREAKING CHANGE] refactor: LLM arglist rename mixed_sampler to enable_mixed_sampler (#5751) 
Signed-off-by: Superjomn <328693+Superjomn@users.noreply.github.com>
 2025-07-07 17:05:14 +08:00
Clay
7a319524da
feat: support more parameters in openai worker of scaffolding (#5115) 
Signed-off-by: Clay <ccs96307@gmail.com>
 2025-07-04 09:35:34 +08:00
Zhenhuan Chen
528ff52ef4
[https://nvbugs/5365714] fix(scaffolding): use default LLM rather than trt backend LLM (#5705) 
Signed-off-by: Zhenhuan Chen <chenzhh3671@gmail.com>
 2025-07-03 23:54:20 +09:00
Yan Chunwei
9bd42ecf9b
[TRTLLM-5208][BREAKING CHANGE] chore: make pytorch LLM the default (#5312) 
Signed-off-by: Superjomn <328693+Superjomn@users.noreply.github.com>
 2025-06-20 03:01:10 +08:00
Yan Chunwei
5506f60037
chore [BREAKING CHANGE]: Flatten PyTorchConfig knobs into TorchLlmArgs (#4603) 
Signed-off-by: Superjomn <328693+Superjomn@users.noreply.github.com>
 2025-05-28 18:43:04 +08:00
Anton
5dff0bff8f
[#4633][doc] Fixed typo in scaffolding README.md (#4634) 
* Fixed typos in the scaffolding README.MD

Signed-off-by: Anton <44649959+amemov@users.noreply.github.com>

* Fixed links for 'More examples' and 'Contribute Guide'

Signed-off-by: Anton <44649959+amemov@users.noreply.github.com>

---------

Signed-off-by: Anton <44649959+amemov@users.noreply.github.com>
 2025-05-25 09:04:12 +08:00
Kunyao Wu
60a6c20174
Scaffoldingllm supports MCP (#4410) 
* support mcp

# Conflicts:
#	tensorrt_llm/scaffolding/worker.py

Signed-off-by: wu1du2 <wu1du2@gmail.com>

* move all into contrib/mcp

# Conflicts:
#	examples/scaffolding/contrib/mcp/mcptest.py
#	tensorrt_llm/scaffolding/__init__.py
#	tensorrt_llm/scaffolding/contrib/__init__.py
#	tensorrt_llm/scaffolding/contrib/mcp/__init__.py
#	tensorrt_llm/scaffolding/contrib/mcp/mcp_controller.py
#	tensorrt_llm/scaffolding/task.py
#	tensorrt_llm/scaffolding/worker.py

Signed-off-by: wu1du2 <wu1du2@gmail.com>

* support sandbox, websearch

# Conflicts:
#	examples/scaffolding/contrib/mcp/mcptest.py
#	examples/scaffolding/contrib/mcp/weather/weather.py
#	tensorrt_llm/scaffolding/contrib/mcp/mcp_controller.py
#	tensorrt_llm/scaffolding/contrib/mcp/mcp_utils.py
#	tensorrt_llm/scaffolding/contrib/mcp/mcp_worker.py
#	tensorrt_llm/scaffolding/worker.py

Signed-off-by: wu1du2 <wu1du2@gmail.com>

* remove pics

Signed-off-by: wu1du2 <wu1du2@gmail.com>

* pre-commit fix

# Conflicts:
#	tensorrt_llm/scaffolding/contrib/mcp/__init__.py
#	tensorrt_llm/scaffolding/contrib/mcp/mcp_utils.py
#	tensorrt_llm/scaffolding/contrib/mcp/mcp_worker.py

Signed-off-by: wu1du2 <wu1du2@gmail.com>

* fix spell

Signed-off-by: wu1du2 <wu1du2@gmail.com>

* rebase

Signed-off-by: wu1du2 <wu1du2@gmail.com>

---------

Signed-off-by: wu1du2 <wu1du2@gmail.com>
 2025-05-23 01:54:49 +00:00
WeiHaocheng
a201ce9d53
docs: update the introduction for scaffolding (#4360) 
Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
 2025-05-21 14:54:01 +08:00
Zhenhuan Chen
e70a205dab
[TRTLLM-4638] feat(scaffolding): update Reward Controller to PRM specific controller with step split (#4337) 
Signed-off-by: Zhenhuan Chen <chenzhh3671@gmail.com>
 2025-05-19 17:53:41 +08:00
Netanel Haber
9cd8148f28
API Breaking Change + Readability: "decoder"->"sampler" (#4121) 
* *decoder*->*sampler*; new_tensors_device: dict[str, torch.Tensor] -> device: SampleStateTensors

* **Breaking Change**, as it changes public interfaces, main changes:
* PyTorchConfig [consumed via LLM(pytorch_backend_config)]: Configuration parameters mixed_decoder and enable_trtllm_decoder -> sampler.
* Command-line argument --enable_trtllm_decoder becomes --enable_trtllm_sampler in examples/pytorch/quickstart_advanced.py.

---------

Signed-off-by: Netanel Haber <58652339+netanel-haber@users.noreply.github.com>
 2025-05-16 23:52:25 +08:00
WeiHaocheng
54d28718c7
feat: support benchmark on scaffolding (#3328) (#4286) 
Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
 2025-05-16 12:28:49 +08:00
yuxianq
0e87fcc228
refactor: use x is None instead of x == None. (#4244) 
Signed-off-by: Yuxian Qiu <142763828+yuxianq@users.noreply.github.com>
 2025-05-15 20:00:04 +08:00
Kaiyu Xie
b4e5df0ee0
Breaking change: perf: Enable scheduling overlap by default (#4174) 
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
 2025-05-15 14:27:36 +08:00
Zhenhuan Chen
9212e9a740
[TRTLLM-4911] feat(scaffolding): make sampling_params only setable by controller (#4151) 
feat(scaffolding): make sampling_params only setable by controller

Signed-off-by: Zhenhuan Chen <chenzhh3671@gmail.com>
 2025-05-12 15:29:09 +08:00
WeiHaocheng
0f01826dde
feat: support task collection for to collect information (#3328) (#3824) 
Signed-off-by: fredw (generated by with_the_same_user script) <20514172+WeiHaocheng@users.noreply.github.com>
 2025-05-09 17:09:01 +08:00
WeiHaocheng
8a994d879f
feat: fix erros on scaffolding README (#3899) 
Signed-off-by: fredw (generated by with_the_same_user script) <20514172+WeiHaocheng@users.noreply.github.com>
 2025-04-29 10:15:06 +08:00
Zhenhuan Chen
ad15e45f07
[TRTLLM-4638 ][feat] add best of n support with reward model in scaffolding (#3807) 
Signed-off-by: Zhenhuan Chen <chenzhh3671@gmail.com>
 2025-04-28 17:15:33 +08:00
WeiHaocheng
3fc2a16920
feat(part 2): Enhance the integrated robustness of scaffolding with __init__.py #3305 (#3731) 
Signed-off-by: fredw (generated by with_the_same_user script) <20514172+WeiHaocheng@users.noreply.github.com>
 2025-04-24 18:47:03 +08:00
WeiHaocheng
c6081abb0e
feat: Make scaffolding Controller more generic #3408 (#3416) 
Signed-off-by: fredw (generated by with_the_same_user script) <20514172+WeiHaocheng@users.noreply.github.com>
 2025-04-12 21:35:38 +08:00
WeiHaocheng
6eee15900e
feat: Enhance the integrated robustness of scaffolding with __init__.py #3305 (#3312) 
Signed-off-by: fredw (generated by with_the_same_user script) <20514172+WeiHaocheng@users.noreply.github.com>
 2025-04-09 21:13:47 +08:00
WeiHaocheng
ff35af77ea
feat: refactor scaffolding worker and support openai api worker (#3166) 
Signed-off-by: Fred Wei <20514172+WeiHaocheng@users.noreply.github.com>
Signed-off-by: fredw <20514172+WeiHaocheng@users.noreply.github.com>
 2025-04-01 18:31:52 +08:00
WeiHaocheng
f665f83256
feat: improve scaffolding shutdown process (#3084) 2025-03-31 20:39:20 +08:00
Erin
c75d7cd684
move BuildConfig functional args to llmargs (#3036) 
Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>
 2025-03-29 02:20:18 +08:00
WeiHaocheng
7ac04ada2a
doc: Add README.md for scaffolding (#3048) 
* Add README.md for scaffolding

Signed-off-by: fredw <20514172+WeiHaocheng@users.noreply.github.com>

* Update tensorrt_llm/scaffolding/README.md

Co-authored-by: dongxuy04 <78518666+dongxuy04@users.noreply.github.com>
Signed-off-by: WeiHaocheng <20514172+WeiHaocheng@users.noreply.github.com>

---------

Signed-off-by: fredw <20514172+WeiHaocheng@users.noreply.github.com>
Signed-off-by: WeiHaocheng <20514172+WeiHaocheng@users.noreply.github.com>
Co-authored-by: dongxuy04 <78518666+dongxuy04@users.noreply.github.com>
 2025-03-25 13:58:01 +08:00
Kaiyu Xie
2631f21089
Update (#2978) 
Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com>
 2025-03-23 16:39:35 +08:00
Kaiyu Xie
3aa6b11d13
Update TensorRT-LLM (#2936) 
* Update TensorRT-LLM

---------

Co-authored-by: changcui <cuichang147@gmail.com>
 2025-03-18 21:25:19 +08:00
Kaiyu Xie
ab5b19e027
Update TensorRT-LLM (#2820) 2025-02-25 21:21:49 +08:00
Kaiyu Xie
2ea17cdad2
Update TensorRT-LLM (#2792) 
* Update TensorRT-LLM

---------

Co-authored-by: jlee <jungmoolee@clika.io>
 2025-02-18 21:27:39 +08:00