[Perf] Skip detokenization in online beam search (#46422)

Signed-off-by: Guy Stone <[email protected]> Signed-off-by: Guy Stone <[email protected]> Co-authored-by: Claude <[email protected]>
2026-08-10 07:48:09 +00:00 · 2026-06-23 15:46:09 -07:00
parent 0a3e2dbc09
commit 3cc871aaf1
1 changed files with 1 additions and 0 deletions
@@ -61,6 +61,7 @@ class BeamSearchOnlineMixin(ABC):
            logprobs=logprobs_num,
            max_tokens=1,
            temperature=temperature,
+            detokenize=False,
        )
        all_beams = [
            BeamSearchSequence(