Enwei Zhu b2f69db507 test: Accuracy test improvement (Part 3.1): Extend accuracy test suite with LLM API and initial implementation of trtllm-eval (#3167) ... * add eval_llmapi Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> tmp commit port to CLI tool Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> move Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> setup llmapi Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> fix spec_dec_algo Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> _update_from_hf_quant_config Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> fix Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> migrate test_pytorch.py Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> fix fp8 block scales Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> fix fp8 rowwise Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> adj alpha Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> move test_pytorch.py cases Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> move Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> rename test_accuracy.py to test_cli.py Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> clean Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> * fix cnn_dailymail Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> * renaming to cli flow Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> * rename MMLU Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> * rename Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> * add error Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> * fix Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> --------- Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>		2025-04-01 22:20:29 +08:00
..
_llmapi_perf_evaluator	Update (#2978)	2025-03-23 16:39:35 +08:00
accuracy	test: Accuracy test improvement (Part 3.1): Extend accuracy test suite with LLM API and initial implementation of trtllm-eval (#3167)	2025-04-01 22:20:29 +08:00
deterministic	Update (#2978)	2025-03-23 16:39:35 +08:00
disaggregated	fix: disable cuda graph and MTP for overlap tests (#3155)	2025-03-31 11:35:35 -07:00
examples	test: Accuracy test improvement (Part 3.1): Extend accuracy test suite with LLM API and initial implementation of trtllm-eval (#3167)	2025-04-01 22:20:29 +08:00
perf	Update (#2978)	2025-03-23 16:39:35 +08:00
sysinfo	Update (#2978)	2025-03-23 16:39:35 +08:00
__init__.py	Update (#2978)	2025-03-23 16:39:35 +08:00
_run_llmapi_llm.py	Update (#2978)	2025-03-23 16:39:35 +08:00
.test_durations	chore: clean some ci of qa test (#3083)	2025-03-31 14:30:41 +08:00
agg_unit_mem_df.csv	test: reorganize tests folder hierarchy (#2996)	2025-03-27 12:07:53 +08:00
ci_profiler.py	Update (#2978)	2025-03-23 16:39:35 +08:00
common.py	test: Add Eagle tests with untrained heads (#2991)	2025-04-01 11:41:59 +08:00
conftest.py	chore: clean some ci of qa test (#3083)	2025-03-31 14:30:41 +08:00
cpp_common.py	Port multi GPU changes to GitHub (#3027)	2025-03-27 05:55:03 +08:00
local_venv.py	Update (#2978)	2025-03-23 16:39:35 +08:00
pytest.ini	Update (#2978)	2025-03-23 16:39:35 +08:00
runner_interface.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_cache.py	chore: clean some ci of qa test (#3083)	2025-03-31 14:30:41 +08:00
test_cases.yml	Update (#2978)	2025-03-23 16:39:35 +08:00
test_cpp.py	Port multi GPU changes to GitHub (#3027)	2025-03-27 05:55:03 +08:00
test_e2e.py	perf: Readd iteration logging for trtllm-bench. (#3039)	2025-04-01 08:13:09 +08:00
test_list_parser.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_list_validation.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_mlpf_results.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_sanity.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_unittests.py	test: reorganize tests folder hierarchy (#2996)	2025-03-27 12:07:53 +08:00
trt_test_alternative.py	Update (#2978)	2025-03-23 16:39:35 +08:00
turtle_defs.json	Update (#2978)	2025-03-23 16:39:35 +08:00