TensorRT-LLMs/tests/unittest/api_stability/references/completion_output.yaml
Erin 83f37614ef
feat: Support Top-K logprobs and prompt_logprobs in LLMAPI (#3388)
* support return logprob in llmapi

Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>

update and add test

Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>

stability test

Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>

* revert removal of old flag

Signed-off-by: Erin Ho <erinh@nvidia.com>
Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>

---------

Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>
Signed-off-by: Erin Ho <erinh@nvidia.com>
2025-05-01 12:47:14 -04:00

24 lines
559 B
YAML

methods:
__init__:
parameters:
cumulative_logprob:
annotation: Optional[float]
default: null
disaggregated_params:
annotation: Optional[tensorrt_llm.disaggregated_params.DisaggregatedParams]
default: null
return_annotation: None
properties:
length:
annotation: int
default: inspect._empty
logprobs_diff:
annotation: List[float]
default: inspect._empty
text_diff:
annotation: str
default: inspect._empty
token_ids_diff:
annotation: List[int]
default: inspect._empty