TensorRT-LLMs/tests/unittest/api_stability/references_committed/completion_output.yaml
Erin 83f37614ef
feat: Support Top-K logprobs and prompt_logprobs in LLMAPI (#3388)
* support return logprob in llmapi

Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>

update and add test

Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>

stability test

Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>

* revert removal of old flag

Signed-off-by: Erin Ho <erinh@nvidia.com>
Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>

---------

Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>
Signed-off-by: Erin Ho <erinh@nvidia.com>
2025-05-01 12:47:14 -04:00

30 lines
833 B
YAML

methods:
__init__:
parameters:
index:
annotation: int
default: inspect._empty
text:
annotation: str
default: ''
token_ids:
annotation: Optional[List[int]]
default: []
finish_reason:
annotation: Optional[Literal['stop', 'length', 'timeout', 'cancelled']]
default: null
stop_reason:
annotation: Union[int, str, NoneType]
default: null
generation_logits:
annotation: Optional[torch.Tensor]
default: null
logprobs:
annotation: Optional[list[dict[int, tensorrt_llm.executor.result.Logprob]]]
default: null
prompt_logprobs:
annotation: Optional[list[dict[int, tensorrt_llm.executor.result.Logprob]]]
default: null
return_annotation: None
properties: {}