Enwei Zhu b2f69db507 test: Accuracy test improvement (Part 3.1): Extend accuracy test suite with LLM API and initial implementation of trtllm-eval (#3167) ... * add eval_llmapi Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> tmp commit port to CLI tool Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> move Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> setup llmapi Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> fix spec_dec_algo Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> _update_from_hf_quant_config Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> fix Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> migrate test_pytorch.py Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> fix fp8 block scales Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> fix fp8 rowwise Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> adj alpha Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> move test_pytorch.py cases Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> move Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> rename test_accuracy.py to test_cli.py Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> clean Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> * fix cnn_dailymail Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> * renaming to cli flow Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> * rename MMLU Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> * rename Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> * add error Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> * fix Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> --------- Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com>		2025-04-01 22:20:29 +08:00
..
run_llm_fp8_quant_llama_70b.py	Update (#2978)	2025-03-23 16:39:35 +08:00
run_llm_lad_mtbench.py	Update (#2978)	2025-03-23 16:39:35 +08:00
run_llm_quickstart_atexit.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_bert.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_bindings.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_chatglm.py	fix bug of glm-4-9b ci (#3184) bug nvbug_5196515	2025-04-01 16:58:42 +08:00
test_commandr.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_deepseek.py	test: Add gpqa tests for DeepSeek models (#3063)	2025-03-27 19:47:06 +08:00
test_draft_target_model.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_eagle.py	test: Add Eagle tests with untrained heads (#2991)	2025-04-01 11:41:59 +08:00
test_enc_dec.py	test: Accuracy test improvement (Part 2): Incorporate mmlu to accuracy test suite (#2982)	2025-03-25 07:34:10 +08:00
test_exaone.py	Add EXAONE-Deep (#3054)	2025-03-26 14:24:04 +08:00
test_gemma.py	test: Accuracy test improvement (Part 2): Incorporate mmlu to accuracy test suite (#2982)	2025-03-25 07:34:10 +08:00
test_gpt.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_gptj.py	test:remove opt/mpt/gptj/gptneox/bloom/falcon/baichuan/internlm/deep_… (#2987)	2025-03-24 14:18:06 +08:00
test_granite.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_internlm.py	test:remove opt/mpt/gptj/gptneox/bloom/falcon/baichuan/internlm/deep_… (#2987)	2025-03-24 14:18:06 +08:00
test_llama.py	test: Accuracy test improvement (Part 2): Incorporate mmlu to accuracy test suite (#2982)	2025-03-25 07:34:10 +08:00
test_llm_api_with_mpi.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_mamba.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_medusa.py	test: Add Eagle tests with untrained heads (#2991)	2025-04-01 11:41:59 +08:00
test_mistral.py	test: Accuracy test improvement (Part 2): Incorporate mmlu to accuracy test suite (#2982)	2025-03-25 07:34:10 +08:00
test_mixtral.py	test: Accuracy test improvement (Part 2): Incorporate mmlu to accuracy test suite (#2982)	2025-03-25 07:34:10 +08:00
test_multimodal.py	test: add random image test for llama-3.2-11b-vision (#3055)	2025-03-26 15:38:16 +08:00
test_nemotron_nas.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_nemotron.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_openai.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_phi.py	fix bug of test_phi (#3050)	2025-03-25 13:12:06 +08:00
test_prompt_lookup.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_qwen2audio.py	chore: Handle qwen2audio inputs ids expansion during processing (#3080)	2025-03-26 15:00:27 +08:00
test_qwen.py	test: Accuracy test improvement (Part 2): Incorporate mmlu to accuracy test suite (#2982)	2025-03-25 07:34:10 +08:00
test_qwenvl.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_recurrentgemma.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_redrafter.py	Update (#2978)	2025-03-23 16:39:35 +08:00
test_whisper.py	Update (#2978)	2025-03-23 16:39:35 +08:00