TensorRT-LLMs/tests/unittest/api_stability/references_committed/request_output.yaml
Erin 83f37614ef
feat: Support Top-K logprobs and prompt_logprobs in LLMAPI (#3388)
* support return logprob in llmapi

Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>

update and add test

Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>

stability test

Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>

* revert removal of old flag

Signed-off-by: Erin Ho <erinh@nvidia.com>
Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>

---------

Signed-off-by: Erin Ho <14718778+hchings@users.noreply.github.com>
Signed-off-by: Erin Ho <erinh@nvidia.com>
2025-05-01 12:47:14 -04:00

30 lines
745 B
YAML

methods:
aresult:
parameters: {}
return_annotation: tensorrt_llm.executor.result.GenerationResult
result:
parameters:
timeout:
annotation: Optional[float]
default: None
return_annotation: tensorrt_llm.executor.result.GenerationResult
properties:
request_id:
annotation: int
default: inspect._empty
prompt:
annotation: Optional[str]
default: inspect._empty
prompt_token_ids:
annotation: List[int]
default: inspect._empty
outputs:
annotation: List[tensorrt_llm.executor.result.CompletionOutput]
default: inspect._empty
context_logits:
annotation: Optional[torch.Tensor]
default: inspect._empty
finished:
annotation: bool
default: inspect._empty