TensorRT-LLMs/tests/integration/defs
Michal Guzek 24153c068e
[TRTLLM-4932] Add QA accuracy tests for NIM-prioritized models (#4242)
* Add tests

Signed-off-by: moraxu <mguzek@nvidia.com>

* Add tests v2

Signed-off-by: moraxu <mguzek@nvidia.com>

* Add fixes

Signed-off-by: moraxu <mguzek@nvidia.com>

* Skip fp8 test for Ultra

Signed-off-by: moraxu <mguzek@nvidia.com>

* Add tests for Phi

Signed-off-by: moraxu <mguzek@nvidia.com>

* Skip tests for Phi

Signed-off-by: moraxu <mguzek@nvidia.com>

* Skip tests for Phi - fix

Signed-off-by: moraxu <mguzek@nvidia.com>

* Skip tests for Phi - comment out acc refs

Signed-off-by: moraxu <mguzek@nvidia.com>

* Add more test granularity

Signed-off-by: moraxu <mguzek@nvidia.com>

* Fix examples_test_list.txt

Signed-off-by: moraxu <mguzek@nvidia.com>

* Update test list file

Signed-off-by: moraxu <mguzek@nvidia.com>

* Update yaml files

Signed-off-by: moraxu <mguzek@nvidia.com>

* Address review comments

Signed-off-by: moraxu <mguzek@nvidia.com>

* Remove MMLU tests

Signed-off-by: moraxu <mguzek@nvidia.com>

* Add remaining models

Signed-off-by: moraxu <mguzek@nvidia.com>

---------

Signed-off-by: moraxu <mguzek@nvidia.com>
2025-05-24 19:17:21 +08:00
..
accuracy [TRTLLM-4932] Add QA accuracy tests for NIM-prioritized models (#4242) 2025-05-24 19:17:21 +08:00
cpp fix: temp disable the problem test (#4445) 2025-05-19 21:54:32 +08:00
deterministic chore: Cleanup deprecated APIs from LLM-API (part 1/2) (#3732) 2025-05-07 13:20:25 +08:00
disaggregated fix: cleanup process tree for disaggregated test (#4116) 2025-05-21 11:01:14 +08:00
examples test: FIX test_ptp_quickstart_advanced_deepseek_v3_2nodes_8gpus (#4283) 2025-05-15 15:57:44 +08:00
llmapi chore: refactor llmapi e2e tests (#3803) 2025-05-05 07:37:24 +08:00
perf test: update test filter in perf test yml file to select cases by gpu name and add cases for RTX 6000 pro (#4282) 2025-05-20 10:58:05 +08:00
stress_test Breaking change: perf: Enable scheduling overlap by default (#4174) 2025-05-15 14:27:36 +08:00
sysinfo Update (#2978) 2025-03-23 16:39:35 +08:00
triton_server Add missing rcca folder (#4591) 2025-05-24 03:28:10 +08:00
__init__.py Update (#2978) 2025-03-23 16:39:35 +08:00
.test_durations [https://nvbugs/5123103][fix] Fix torch compile for DeepSeekV3 (#3952) 2025-05-19 22:12:25 +08:00
agg_unit_mem_df.csv test: reorganize tests folder hierarchy (#2996) 2025-03-27 12:07:53 +08:00
ci_profiler.py Update (#2978) 2025-03-23 16:39:35 +08:00
common.py fix: cleanup process tree for disaggregated test (#4116) 2025-05-21 11:01:14 +08:00
conftest.py fix: cleanup process tree for disaggregated test (#4116) 2025-05-21 11:01:14 +08:00
local_venv.py tests: https://nvbugs/5219534 remove failed tests from test list (#4113) 2025-05-12 14:13:40 +08:00
pytest.ini Move Triton backend to TRT-LLM main (#3549) 2025-05-16 07:15:23 +08:00
runner_interface.py Update (#2978) 2025-03-23 16:39:35 +08:00
test_cache.py chore: clean some ci of qa test (#3083) 2025-03-31 14:30:41 +08:00
test_cases.yml Update (#2978) 2025-03-23 16:39:35 +08:00
test_e2e.py [TR[TLLM-4618][feat] Add remaining NVFP4 Nemotron Super 49B test on RTX6000 Pro (SM120) (#4548) 2025-05-23 10:42:32 -07:00
test_list_parser.py infra: Add test list name check (#3097) 2025-04-20 23:02:16 +08:00
test_list_validation.py Update (#2978) 2025-03-23 16:39:35 +08:00
test_mlpf_results.py Update (#2978) 2025-03-23 16:39:35 +08:00
test_sanity.py Update (#2978) 2025-03-23 16:39:35 +08:00
test_unittests.py [TRTLLM-4886][infra]Try another timeout opt to exit test thread directly instead of gracefully (#4341) 2025-05-16 17:56:40 +08:00
trt_test_alternative.py fix: cleanup process tree for disaggregated test (#4116) 2025-05-21 11:01:14 +08:00