EAGLE-3(Two Model Engine)
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
No
No
Yes
Yes
Yes
Yes
Yes
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -425,6 +428,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -409,6 +411,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -425,6 +428,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -409,6 +411,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -425,6 +428,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -409,6 +411,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -425,6 +428,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers The following examples demonstrate how to use TensorRT LLM’s multimodal support in various scenarios, including quick run examples, serving endpoints, and performance benchmarking.
Quick start
-Quickly try out TensorRT LLM’s multimodal support using our LLM-API example :
+Quickly try out TensorRT LLM’s multimodal support using our LLM-API example :
quickstart_multimodal.py --model_dir Efficient-Large-Model/NVILA-8B --modality image
Qwen/Qwen2-VL-7B-Instruct --backend pytorch
You can then send OpenAI-compatible requests, such as via curl or API clients, to the server endpoint. See curl chat client for multimodal script as an example.
+You can then send OpenAI-compatible requests, such as via curl or API clients, to the server endpoint. See curl chat client for multimodal script as an example.
@@ -703,9 +707,9 @@
diff --git a/latest/features/overlap-scheduler.html b/latest/features/overlap-scheduler.html
index b77d807f09..7b6649aa80 100644
--- a/latest/features/overlap-scheduler.html
+++ b/latest/features/overlap-scheduler.html
@@ -59,7 +59,7 @@
@@ -71,7 +71,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -409,6 +411,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -425,6 +428,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -409,6 +411,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -425,6 +428,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers contiguous and pagedThe paged KV cache decomposes the KV cache into blocks that are distributed to
the different requests by a cache manager during processing. That cache manager
keeps track of the sequences, allocates new blocks from a pool, and recycles those blocks when required. See the simplified implementation of
-tensorrt_llm.runtime.KVCacheManager tensorrt_llm.runtime.KVCacheManager Batch Manager .
+Batch Manager .
@@ -781,9 +785,9 @@ A more efficient C++ implementation is included in the
diff --git a/latest/features/parallel-strategy.html b/latest/features/parallel-strategy.html
index 8ae0cc1071..d04b12c523 100644
--- a/latest/features/parallel-strategy.html
+++ b/latest/features/parallel-strategy.html
@@ -59,7 +59,7 @@
@@ -71,7 +71,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -409,6 +411,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -425,6 +428,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -409,6 +411,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -425,6 +428,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Ray Orchestrator (Prototype) — TensorRT LLM
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+ Ray Orchestrator (Prototype)
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Ray Orchestrator (Prototype)
+
+
Note
+
This project is under active development and currently in a prototype stage. The current focus is on core functionality, with performance optimization coming soon. While we strive for correctness, there are currently no guarantees regarding functionality, stability, or reliability.
+
+Motivation
+The Ray orchestrator uses Ray instead of MPI to manage workers for single- and multi-node inference. It’s a first step toward making TensorRT-LLM a better fit for Reinforcement Learning from Human Feedback (RLHF) workflows. For RLHF, Ray can dynamically spawn and reconnect distributed inference actors, each with its own parallelism strategy. This feature is a prototype and under active development. MPI remains the default in TensorRT-LLM.
+
+
+Basic Usage
+To use Ray orchestrator, you need to first install Ray.
+cd examples/ray_orchestrator
+pip install -r requirements.txt
+
+
To run a simple TP=2
+ llm_inference_distributed_ray.py
+
+
This example is the same as in /examples/llm-api orchestrator_type="ray" LLM()
+
+
+Features
+Currently available:
+
+Initial testing has been focused on LLaMA and DeepSeek variants. Please open an Issue if you encounter problems with other models so we can prioritize support.
+
+
+
+Architecture
+This feature introduces new classes such as RayExecutor RayGPUWorker torch.distributed without MPI. We welcome contributions to improve and extend this support.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -409,6 +411,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -425,6 +428,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers To use the feature:
Enable the enable_trtllm_sampler LLM
Pass a SamplingParams generate()
Pass a SamplingParams generate()
The following example prepares two identical prompts which will give different results due to the sampling parameters chosen:
from tensorrt_llm import LLM , SamplingParams
@@ -572,7 +576,7 @@
To enable guided decoding, you must:
Set the guided_decoding_backend 'xgrammar' 'llguidance' LLM
Create a GuidedDecodingParams
+Create a GuidedDecodingParams
Note: Depending on the type of format, a different parameter needs to be chosen to construct the object (json regex grammar structural_tag
@@ -593,14 +597,14 @@
llm . generate ( "Generate a JSON response" , sampling_params )
You can find a more detailed example on guided decoding here .
+You can find a more detailed example on guided decoding here .
Logits processor
Logits processors allow you to modify the logits produced by the network before sampling, enabling custom generation behavior and constraints.
To use a custom logits processor:
-Create a custom class that inherits from LogitsProcessor __call__
Create a custom class that inherits from LogitsProcessor __call__
Pass an instance of this class to the logits_processor SamplingParams
The following example demonstrates logits processing:
@@ -628,7 +632,7 @@
llm . generate ([ "Hello, my name is" ], sampling_params )
-You can find a more detailed example on logits processors here .
+You can find a more detailed example on logits processors here .
@@ -779,9 +783,9 @@
diff --git a/latest/features/speculative-decoding.html b/latest/features/speculative-decoding.html
index 3fcb877f17..1d045ecf06 100644
--- a/latest/features/speculative-decoding.html
+++ b/latest/features/speculative-decoding.html
@@ -59,7 +59,7 @@
@@ -71,7 +71,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -409,6 +411,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -425,6 +428,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -402,6 +404,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
+
+ --chat_template_kwargs
+
+
@@ -649,6 +670,13 @@
+
+ --enable_chunked_prefill
+
+
@@ -880,6 +908,8 @@
trtllm-eval-mmmu command line option
-
-
-
-
conv_transpose2d() (in module tensorrt_llm.functional)
+convert_enable_disable() (tensorrt_llm.plugin.PluginConfig class method)
convert_load_format() (tensorrt_llm.llmapi.TorchLlmArgs class method)
(tensorrt_llm.llmapi.MTPDecodingConfig method)
(tensorrt_llm.llmapi.NGramDecodingConfig method)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig method)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig method)
(tensorrt_llm.llmapi.SchedulerConfig method)
(tensorrt_llm.llmapi.MTPDecodingConfig attribute)
(tensorrt_llm.llmapi.NGramDecodingConfig attribute)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig attribute)
(tensorrt_llm.llmapi.UserProvidedDecodingConfig attribute)
(tensorrt_llm.llmapi.MTPDecodingConfig method)
(tensorrt_llm.llmapi.NGramDecodingConfig method)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig method)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig method)
(tensorrt_llm.llmapi.SchedulerConfig method)
disable_overlap_scheduler (tensorrt_llm.llmapi.TorchLlmArgs attribute)
disaggregated_params (tensorrt_llm.llmapi.CompletionOutput attribute)
+
+ DisaggregatedParams (class in tensorrt_llm.llmapi)
DiT (class in tensorrt_llm.models)
div() (in module tensorrt_llm.functional)
+dora_plugin (tensorrt_llm.plugin.PluginConfig attribute)
dora_plugin() (in module tensorrt_llm.functional)
(tensorrt_llm.llmapi.TorchLlmArgs attribute)
(tensorrt_llm.llmapi.TrtLlmArgs attribute)
+(tensorrt_llm.plugin.PluginConfig attribute)
(tensorrt_llm.runtime.GenerationSession property)
EAGLE (tensorrt_llm.models.SpeculativeDecodingMode attribute)
eagle3_layers_to_capture (tensorrt_llm.llmapi.EagleDecodingConfig attribute)
+
+ eagle3_one_model (tensorrt_llm.llmapi.EagleDecodingConfig attribute)
eagle_choices (tensorrt_llm.llmapi.EagleDecodingConfig attribute)
+
+ EagleDecodingConfig (class in tensorrt_llm.llmapi)
EagleDecodingConfig.Config (class in tensorrt_llm.llmapi)
@@ -2338,6 +2456,8 @@
enable_min_latency (tensorrt_llm.llmapi.TorchLlmArgs attribute)
enable_padding (tensorrt_llm.llmapi.CudaGraphConfig attribute)
+enable_paged_kv_cache() (tensorrt_llm.plugin.PluginConfig method)
enable_partial_reuse (tensorrt_llm.llmapi.KvCacheConfig attribute)
(tensorrt_llm.llmapi.MTPDecodingConfig.Config attribute)
(tensorrt_llm.llmapi.NGramDecodingConfig.Config attribute)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig.Config attribute)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig.Config attribute)
(tensorrt_llm.llmapi.SchedulerConfig.Config attribute)
(tensorrt_llm.llmapi.TrtLlmArgs attribute) , [1] , [2]
file_prefix (tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig attribute)
+fill_attention_const_params_for_long_rope() (tensorrt_llm.layers.attention.AttentionParams method)
fill_attention_const_params_for_rope() (tensorrt_llm.layers.attention.AttentionParams method)
@@ -2724,6 +2850,8 @@
FP8_BLOCK_SCALES (tensorrt_llm.llmapi.QuantAlgo attribute)
FP8_PER_CHANNEL_PER_TOKEN (tensorrt_llm.llmapi.QuantAlgo attribute)
+fp8_rowwise_gemm_plugin (tensorrt_llm.plugin.PluginConfig attribute)
free_gpu_memory_fraction (tensorrt_llm.llmapi.KvCacheConfig attribute)
from_arguments() (tensorrt_llm.models.SpeculativeDecodingMode static method)
+
+ from_checkpoint() (tensorrt_llm.models.PretrainedConfig class method)
from_model_config_cpp() (tensorrt_llm.runtime.ModelConfig class method)
+from_nemo() (tensorrt_llm.models.GPTConfig class method)
fuse_fp4_quant (tensorrt_llm.plugin.PluginConfig attribute)
+fuse_qkv_projections() (tensorrt_llm.models.SD3Transformer2DModel method)
FusedGatedMLP (class in tensorrt_llm.layers.mlp)
@@ -2991,13 +3135,19 @@
gemm_allreduce() (in module tensorrt_llm.functional)
gemm_allreduce_plugin (tensorrt_llm.runtime.GenerationSession property)
+ gemm_allreduce_plugin (tensorrt_llm.plugin.PluginConfig attribute)
gemm_plugin (tensorrt_llm.plugin.PluginConfig attribute)
+gemm_swiglu() (in module tensorrt_llm.functional)
+gemm_swiglu_plugin (tensorrt_llm.plugin.PluginConfig attribute)
GEMMA2_ADDED_FIELDS (tensorrt_llm.models.GemmaConfig attribute)
get_context_phase_params() (tensorrt_llm.llmapi.DisaggregatedParams method)
get_executor_config() (tensorrt_llm.llmapi.TorchLlmArgs method)
-get_first_past_key_value() (tensorrt_llm.layers.attention.KeyValueCacheParams method)
+ get_first_past_key_value() (tensorrt_llm.layers.attention.KeyValueCacheParams method)
+get_hf_config() (tensorrt_llm.models.GemmaConfig static method)
get_kv_cache_events() (tensorrt_llm.llmapi.LLM method)
@@ -3157,8 +3307,12 @@
gpt_attention() (in module tensorrt_llm.functional)
gpt_attention_plugin (tensorrt_llm.runtime.ModelConfig attribute)
+ gpt_attention_plugin (tensorrt_llm.plugin.PluginConfig attribute)
+
+ GPTConfig (class in tensorrt_llm.models)
GPTForCausalLM (class in tensorrt_llm.models)
@@ -3268,6 +3422,8 @@
identity() (in module tensorrt_llm.functional)
+identity_plugin (tensorrt_llm.plugin.PluginConfig attribute)
ignore_eos (tensorrt_llm.llmapi.SamplingParams attribute)
is_alibi() (tensorrt_llm.functional.PositionEmbeddingType method)
is_comm_session() (tensorrt_llm.llmapi.MpiCommSession method)
+is_context_fmha_enabled() (tensorrt_llm.plugin.PluginConfig method)
is_deferred() (tensorrt_llm.functional.PositionEmbeddingType method)
(tensorrt_llm.llmapi.MTPDecodingConfig method)
(tensorrt_llm.llmapi.NGramDecodingConfig method)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig method)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig method)
(tensorrt_llm.llmapi.SchedulerConfig method)
K
+ layernorm_quantization_plugin (tensorrt_llm.plugin.PluginConfig attribute)
+LayerNormPositionType (class in tensorrt_llm.functional)
LayerNormType (class in tensorrt_llm.functional)
@@ -3713,6 +3879,8 @@
(tensorrt_llm.llmapi.MTPDecodingConfig attribute)
(tensorrt_llm.llmapi.NGramDecodingConfig attribute)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig attribute)
(tensorrt_llm.llmapi.TorchLlmArgs attribute)
(tensorrt_llm.functional.Tensor method)
log_field_changes() (tensorrt_llm.plugin.PluginConfig class method)
+log_softmax() (in module tensorrt_llm.functional)
logits_processor (tensorrt_llm.llmapi.SamplingParams attribute)
@@ -3785,8 +3955,12 @@
lora_path (tensorrt_llm.llmapi.LoRARequest attribute)
lora_plugin (tensorrt_llm.runtime.ModelConfig attribute)
+ lora_plugin (tensorrt_llm.plugin.PluginConfig attribute)
+
+ lora_plugin() (in module tensorrt_llm.functional)
lora_target_modules (tensorrt_llm.runtime.ModelConfig attribute)
@@ -3794,8 +3968,12 @@
LoRARequest (class in tensorrt_llm.llmapi)
low_latency_gemm() (in module tensorrt_llm.functional)
+low_latency_gemm_plugin (tensorrt_llm.plugin.PluginConfig attribute)
low_latency_gemm_swiglu() (in module tensorrt_llm.functional)
+low_latency_gemm_swiglu_plugin (tensorrt_llm.plugin.PluginConfig attribute)
lower() (tensorrt_llm.llmapi.BatchingType method)
@@ -3841,8 +4019,12 @@
mamba_conv1d() (in module tensorrt_llm.functional)
mamba_conv1d_plugin (tensorrt_llm.runtime.ModelConfig attribute)
+ mamba_conv1d_plugin (tensorrt_llm.plugin.PluginConfig attribute)
+
+ mamba_ssm_cache_dtype (tensorrt_llm.llmapi.KvCacheConfig attribute)
MambaForCausalLM (class in tensorrt_llm.models)
+manage_weights (tensorrt_llm.plugin.PluginConfig attribute)
mapping (tensorrt_llm.runtime.GenerationSession attribute)
@@ -3917,6 +4101,8 @@
(tensorrt_llm.llmapi.MTPDecodingConfig attribute)
(tensorrt_llm.llmapi.NGramDecodingConfig attribute)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig attribute)
(tensorrt_llm.llmapi.UserProvidedDecodingConfig attribute)
(tensorrt_llm.llmapi.MTPDecodingConfig attribute)
(tensorrt_llm.llmapi.NGramDecodingConfig attribute)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig attribute)
(tensorrt_llm.llmapi.UserProvidedDecodingConfig attribute)
max_tokens_in_buffer (tensorrt_llm.llmapi.CacheTransceiverConfig attribute)
max_total_draft_tokens (tensorrt_llm.llmapi.EagleDecodingConfig attribute)
+
+ MAX_UTILIZATION (tensorrt_llm.llmapi.CapacitySchedulerPolicy attribute)
max_verification_set_size (tensorrt_llm.llmapi.LookaheadDecodingConfig attribute)
@@ -4190,6 +4382,10 @@
(tensorrt_llm.llmapi.MTPDecodingConfig attribute)
(tensorrt_llm.llmapi.NGramDecodingConfig attribute)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig attribute)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig attribute)
(tensorrt_llm.llmapi.SchedulerConfig attribute)
(tensorrt_llm.llmapi.MTPDecodingConfig attribute)
(tensorrt_llm.llmapi.NGramDecodingConfig attribute)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig attribute)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig attribute)
(tensorrt_llm.llmapi.SchedulerConfig attribute)
(tensorrt_llm.llmapi.MTPDecodingConfig class method)
(tensorrt_llm.llmapi.NGramDecodingConfig class method)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig class method)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig class method)
(tensorrt_llm.llmapi.SchedulerConfig class method)
(tensorrt_llm.llmapi.MTPDecodingConfig method)
(tensorrt_llm.llmapi.NGramDecodingConfig method)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig method)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig method)
(tensorrt_llm.llmapi.SchedulerConfig method)
(tensorrt_llm.llmapi.MTPDecodingConfig method)
(tensorrt_llm.llmapi.NGramDecodingConfig method)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig method)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig method)
(tensorrt_llm.llmapi.SchedulerConfig method)
(tensorrt_llm.llmapi.MTPDecodingConfig method)
(tensorrt_llm.llmapi.NGramDecodingConfig method)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig method)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig method)
(tensorrt_llm.llmapi.SchedulerConfig method)
(tensorrt_llm.llmapi.MTPDecodingConfig property)
(tensorrt_llm.llmapi.NGramDecodingConfig property)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig property)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig property)
(tensorrt_llm.llmapi.SchedulerConfig property)
(tensorrt_llm.llmapi.MTPDecodingConfig attribute)
(tensorrt_llm.llmapi.NGramDecodingConfig attribute)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig attribute)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig attribute)
(tensorrt_llm.llmapi.SchedulerConfig attribute)
(tensorrt_llm.llmapi.MTPDecodingConfig property)
(tensorrt_llm.llmapi.NGramDecodingConfig property)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig property)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig property)
(tensorrt_llm.llmapi.SchedulerConfig property)
(tensorrt_llm.llmapi.MTPDecodingConfig class method)
(tensorrt_llm.llmapi.NGramDecodingConfig class method)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig class method)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig class method)
(tensorrt_llm.llmapi.SchedulerConfig class method)
(tensorrt_llm.llmapi.MTPDecodingConfig class method)
(tensorrt_llm.llmapi.NGramDecodingConfig class method)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig class method)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig class method)
(tensorrt_llm.llmapi.SchedulerConfig class method)
(tensorrt_llm.llmapi.MTPDecodingConfig method)
(tensorrt_llm.llmapi.NGramDecodingConfig method)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig method)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig method)
(tensorrt_llm.llmapi.SchedulerConfig method)
(tensorrt_llm.llmapi.TorchCompileConfig method)
(tensorrt_llm.llmapi.UserProvidedDecodingConfig method)
+(tensorrt_llm.plugin.PluginConfig method)
model_rebuild() (tensorrt_llm.llmapi.AttentionDpConfig class method)
@@ -4656,6 +4898,10 @@
(tensorrt_llm.llmapi.MTPDecodingConfig class method)
(tensorrt_llm.llmapi.NGramDecodingConfig class method)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig class method)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig class method)
(tensorrt_llm.llmapi.SchedulerConfig class method)
(tensorrt_llm.llmapi.MTPDecodingConfig class method)
(tensorrt_llm.llmapi.NGramDecodingConfig class method)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig class method)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig class method)
(tensorrt_llm.llmapi.SchedulerConfig class method)
(tensorrt_llm.llmapi.MTPDecodingConfig class method)
(tensorrt_llm.llmapi.NGramDecodingConfig class method)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig class method)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig class method)
(tensorrt_llm.llmapi.SchedulerConfig class method)
(tensorrt_llm.llmapi.MTPDecodingConfig class method)
(tensorrt_llm.llmapi.NGramDecodingConfig class method)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig class method)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig class method)
(tensorrt_llm.llmapi.SchedulerConfig class method)
MOE_FINALIZE_ALLREDUCE_RESIDUAL_RMS_NORM (tensorrt_llm.functional.AllReduceFusionOp attribute)
+moe_plugin (tensorrt_llm.plugin.PluginConfig attribute)
moe_tensor_parallel_size (tensorrt_llm.llmapi.TorchLlmArgs attribute)
@@ -4896,6 +5156,8 @@
MultimodalEncoder (class in tensorrt_llm.llmapi)
MultimodalModelRunner (class in tensorrt_llm.runtime)
+multiple_profiles (tensorrt_llm.plugin.PluginConfig attribute)
multiply_and_lora() (tensorrt_llm.layers.linear.LinearBase method)
NATIVE_QUANT_FLOW (tensorrt_llm.models.GemmaForCausalLM attribute)
NCCL (tensorrt_llm.functional.AllReduceStrategy attribute)
+nccl_plugin (tensorrt_llm.plugin.PluginConfig attribute)
NCCL_SYMMETRIC (tensorrt_llm.functional.AllReduceStrategy attribute)
NONE (tensorrt_llm.models.SpeculativeDecodingMode attribute)
nonzero() (in module tensorrt_llm.functional)
+norm_quant_fusion (tensorrt_llm.plugin.PluginConfig attribute)
normalize_log_probs (tensorrt_llm.llmapi.TrtLlmArgs attribute)
num_beams (tensorrt_llm.runtime.SamplingConfig attribute)
+
padding (tensorrt_llm.functional.AttentionMaskType attribute)
paged_kv_cache (tensorrt_llm.runtime.GenerationSession property)
+ page_size (tensorrt_llm.llmapi.RocketSparseAttentionConfig attribute)
paged_state (tensorrt_llm.runtime.GenerationSession property)
+ paged_kv_cache (tensorrt_llm.plugin.PluginConfig attribute)
paged_state (tensorrt_llm.plugin.PluginConfig attribute)
+
+ (tensorrt_llm.llmapi.TrtLlmArgs property)
params_imply_greedy_decoding() (tensorrt_llm.llmapi.SamplingParams static method)
+parse_file() (tensorrt_llm.llmapi.AttentionDpConfig class method)
path (tensorrt_llm.llmapi.LoRARequest property)
+
+ prompt_budget (tensorrt_llm.llmapi.RocketSparseAttentionConfig attribute)
+prompt_logprobs (tensorrt_llm.llmapi.CompletionOutput attribute)
@@ -5413,6 +5713,8 @@
Q
quantize_and_export() (in module tensorrt_llm.quantization)
+quantize_per_token_plugin (tensorrt_llm.plugin.PluginConfig attribute)
+quantize_tensor_plugin (tensorrt_llm.plugin.PluginConfig attribute)
QuantMode (class in tensorrt_llm.quantization)
ReDrafterForQWenLM (class in tensorrt_llm.models)
reduce() (in module tensorrt_llm.functional)
+reduce_fusion (tensorrt_llm.plugin.PluginConfig attribute)
reduce_scatter() (in module tensorrt_llm.functional)
relu() (in module tensorrt_llm.functional)
remove_input_padding (tensorrt_llm.runtime.GenerationSession property)
+ remove_input_padding (tensorrt_llm.plugin.PluginConfig attribute)
request_perf_metrics (tensorrt_llm.llmapi.CompletionOutput attribute)
+
+ request_stats_max_iterations (tensorrt_llm.llmapi.TorchLlmArgs attribute)
rmsnorm_quantization_plugin (tensorrt_llm.plugin.PluginConfig attribute)
+rnn_conv_dim_size (tensorrt_llm.runtime.GenerationSession property)
SAVE_HIDDEN_STATES (tensorrt_llm.models.SpeculativeDecodingMode attribute)
+SaveHiddenStatesDecodingConfig (class in tensorrt_llm.llmapi)
+SaveHiddenStatesDecodingConfig.Config (class in tensorrt_llm.llmapi)
+scatter() (in module tensorrt_llm.functional)
scatter_nd() (in module tensorrt_llm.functional)
@@ -5837,6 +6163,10 @@
(tensorrt_llm.llmapi.MTPDecodingConfig class method)
(tensorrt_llm.llmapi.NGramDecodingConfig class method)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig class method)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig class method)
(tensorrt_llm.llmapi.SchedulerConfig class method)
(tensorrt_llm.llmapi.MTPDecodingConfig class method)
(tensorrt_llm.llmapi.NGramDecodingConfig class method)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig class method)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig class method)
(tensorrt_llm.llmapi.SchedulerConfig class method)
Session (class in tensorrt_llm.runtime)
set_attn_processor() (tensorrt_llm.models.SD3Transformer2DModel method)
+set_context_fmha() (tensorrt_llm.plugin.PluginConfig method)
set_default_max_input_len() (tensorrt_llm.llmapi.TorchLlmArgs method)
@@ -5915,9 +6251,19 @@
(tensorrt_llm.llmapi.TrtLlmArgs method)
set_dora_plugin() (tensorrt_llm.plugin.PluginConfig method)
+set_fp8_rowwise_quant_plugins() (tensorrt_llm.plugin.PluginConfig method)
+SET_FROM_OPTIONAL (C macro)
set_if_not_exist() (tensorrt_llm.models.PretrainedConfig method)
+set_lora_plugin() (tensorrt_llm.plugin.PluginConfig method)
+set_nccl_plugin() (tensorrt_llm.plugin.PluginConfig method)
+set_qserve_plugins() (tensorrt_llm.plugin.PluginConfig method)
set_rank() (tensorrt_llm.models.PretrainedConfig method)
set_shapes() (tensorrt_llm.runtime.Session method)
+set_smooth_quant_plugins() (tensorrt_llm.plugin.PluginConfig method)
setup() (tensorrt_llm.runtime.GenerationSession method)
skip_special_tokens (tensorrt_llm.llmapi.SamplingParams attribute)
+
slice() (in module tensorrt_llm.functional)
SliceInputType (class in tensorrt_llm.functional)
sliding_window_causal (tensorrt_llm.functional.AttentionMaskType attribute)
+smooth_quant_gemm_plugin (tensorrt_llm.plugin.PluginConfig attribute)
+smooth_quant_plugins (tensorrt_llm.plugin.PluginConfig attribute)
smoothquant_val (tensorrt_llm.llmapi.QuantConfig attribute)
spaces_between_special_tokens (tensorrt_llm.llmapi.SamplingParams attribute)
sparse_attention_config (tensorrt_llm.llmapi.TorchLlmArgs attribute)
+
+ spec_dec_mode (tensorrt_llm.llmapi.AutoDecodingConfig property)
@@ -6021,6 +6379,8 @@
(tensorrt_llm.llmapi.MTPDecodingConfig property)
(tensorrt_llm.llmapi.NGramDecodingConfig property)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig property)
(tensorrt_llm.llmapi.UserProvidedDecodingConfig property)
(tensorrt_llm.llmapi.MTPDecodingConfig attribute)
(tensorrt_llm.llmapi.NGramDecodingConfig attribute)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig attribute)
(tensorrt_llm.llmapi.TorchLlmArgs property)
stream_interval (tensorrt_llm.llmapi.TorchLlmArgs attribute)
streaming (tensorrt_llm.llmapi.RequestOutput.PostprocWorker.Input attribute)
+streamingllm (tensorrt_llm.plugin.PluginConfig attribute)
stride (tensorrt_llm.functional.SliceInputType attribute)
(tensorrt_llm.llmapi.MTPDecodingConfig method)
(tensorrt_llm.llmapi.NGramDecodingConfig method)
+(tensorrt_llm.llmapi.RocketSparseAttentionConfig method)
+(tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig method)
(tensorrt_llm.llmapi.UserProvidedDecodingConfig method)
tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kFLOAT>::name (C++ member)
validate_and_init_tokenizer() (tensorrt_llm.llmapi.TorchLlmArgs method)
@@ -12731,10 +13137,12 @@
(tensorrt_llm.llmapi.TrtLlmArgs class method)
validate_enable_build_cache() (tensorrt_llm.llmapi.TrtLlmArgs method)
-
+ weight_only_groupwise_quant_matmul_plugin (tensorrt_llm.plugin.PluginConfig attribute)
+weight_only_quant_matmul_plugin (tensorrt_llm.plugin.PluginConfig attribute)
+weight_sparsity (tensorrt_llm.llmapi.BuildConfig attribute)
weight_streaming (tensorrt_llm.llmapi.BuildConfig attribute)
@@ -12890,6 +13302,8 @@
where() (in module tensorrt_llm.functional)
WhisperEncoder (class in tensorrt_llm.models)
+window_size (tensorrt_llm.llmapi.RocketSparseAttentionConfig attribute)
with_traceback() (tensorrt_llm.llmapi.RequestError method)
(tensorrt_llm.llmapi.TrtLlmArgs attribute) , [1] , [2]
write_interval (tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig attribute)
+
@@ -13043,9 +13459,9 @@
diff --git a/latest/index.html b/latest/index.html
index b0de668361..e7be96463b 100644
--- a/latest/index.html
+++ b/latest/index.html
@@ -61,7 +61,7 @@
@@ -72,7 +72,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -623,6 +628,8 @@
AutoDecodingConfig AttentionDpConfig LoRARequest SaveHiddenStatesDecodingConfig RocketSparseAttentionConfig Roadmap Ray Orchestrator (Prototype)
+
@@ -826,6 +841,16 @@
Future Work
+
Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary)
+Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs
Table of Contents Background Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -409,6 +411,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
To use the C++ benchmark scripts under benchmark/cpp , for example gptManagerBenchmark.cpp --benchmarks
+To use the C++ benchmark scripts under benchmark/cpp , for example gptManagerBenchmark.cpp --benchmarks
./scripts/build_wheel.py --benchmarks
install ./build/tensorrt_llm*.
diff --git a/latest/installation/containers.html b/latest/installation/containers.html
index c675b36863..2f25e61962 100644
--- a/latest/installation/containers.html
+++ b/latest/installation/containers.html
@@ -59,7 +59,7 @@
@@ -71,7 +71,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -421,6 +424,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Container image tags
In the example shell commands, x.y.z IMAGE_TAG tensorrt_llm.__version__ 1.2.0rc0 1.2.0rc1
diff --git a/latest/installation/index.html b/latest/installation/index.html
index b4e22155ee..ad11170be6 100644
--- a/latest/installation/index.html
+++ b/latest/installation/index.html
@@ -59,7 +59,7 @@
@@ -71,7 +71,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -405,6 +407,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -421,6 +424,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -405,6 +407,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -421,6 +424,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Install CUDA Toolkit following the CUDA Installation Guide for Linux and
make sure CUDA_HOME
+
+
Tip
+
TensorRT LLM 1.1 supports both CUDA 12.9 and 13.0. The wheel package release only supports CUDA 12.9, while CUDA 13.0 is only supported through NGC container release.
+
# Optional step: Only required for NVIDIA Blackwell GPUs and SBSA platform
pip3 install torch == 2 .7.1 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128
sudo apt-get -y install libopenmpi-dev
+
+# Optional step: Only required for disagg-serving
+sudo apt-get -y install libzmq3-dev
PyTorch CUDA 12.8 package is required for supporting NVIDIA Blackwell GPUs and SBSA platform. On prior GPUs or Linux x86_64 platform, this extra installation is not required.
@@ -710,9 +721,9 @@ The setup methods depends on your slurm configuration, pls check with your admin
diff --git a/latest/legacy/advanced/disaggregated-service.html b/latest/legacy/advanced/disaggregated-service.html
index 5b6a9e9c9d..ab36911c34 100644
--- a/latest/legacy/advanced/disaggregated-service.html
+++ b/latest/legacy/advanced/disaggregated-service.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers TensorRT-LLM includes a high-level C++ API called the Executor API which allows you to execute requests
asynchronously, with in-flight batching, and without the need to define callbacks.
A software component (referred to as “the client” in the text that follows) can interact
-with the executor using the API defined in the executor.h executor.h _cpp_gen/executor.rst .
The following sections provide an overview of the main classes defined in the Executor API.
@@ -570,7 +574,7 @@ This allows the runtime to reconfigure itself for a new beam width when no reque
stop_token_ids = [ tokenizer . eos_token_id ]
-Refer to tensorrt_llm/llmapi/tokenizer.py
+Refer to tensorrt_llm/llmapi/tokenizer.py
Each request can be optionally specified with a GuidedDecodingParams
C++ Executor API Example
-Two C++ examples are provided that shows how to use the Executor API and can be found in the examples/cpp/executor
+Two C++ examples are provided that shows how to use the Executor API and can be found in the examples/cpp/executor
Python Bindings for the Executor API
-Python bindings for the Executor API are also available to use the Executor API from Python. The Python bindings are defined in bindings.cpp and once built, are available in package tensorrt_llm.bindings.executor 'help('tensorrt_llm.bindings.executor')
-In addition, three Python examples are provided to demonstrate how to use the Python bindings to the Executor API for single and multi-GPU models. They can be found in examples/bindings
+Python bindings for the Executor API are also available to use the Executor API from Python. The Python bindings are defined in bindings.cpp and once built, are available in package tensorrt_llm.bindings.executor 'help('tensorrt_llm.bindings.executor')
+In addition, three Python examples are provided to demonstrate how to use the Python bindings to the Executor API for single and multi-GPU models. They can be found in examples/bindings
In-flight Batching with the Triton Inference Server
@@ -782,9 +786,9 @@ the TensorRT-LLM C++ Executor API.
diff --git a/latest/legacy/advanced/expert-parallelism.html b/latest/legacy/advanced/expert-parallelism.html
index a9cc76b9ef..27be4bfa52 100644
--- a/latest/legacy/advanced/expert-parallelism.html
+++ b/latest/legacy/advanced/expert-parallelism.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers tensorrt_llm.runtime.KVCacheManager tensorrt_llm.runtime.KVCacheManager Batch Manager .
+Batch Manager .
@@ -965,9 +969,9 @@ is computed as:
diff --git a/latest/legacy/advanced/gpt-runtime.html b/latest/legacy/advanced/gpt-runtime.html
index 861968002c..0e73b0339d 100644
--- a/latest/legacy/advanced/gpt-runtime.html
+++ b/latest/legacy/advanced/gpt-runtime.html
@@ -61,7 +61,7 @@
@@ -71,7 +71,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -409,6 +411,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -425,6 +428,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers GptDecoder
diff --git a/latest/legacy/advanced/graph-rewriting.html b/latest/legacy/advanced/graph-rewriting.html
index abd5491e52..abf639e773 100644
--- a/latest/legacy/advanced/graph-rewriting.html
+++ b/latest/legacy/advanced/graph-rewriting.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers LoraWe
diff --git a/latest/legacy/advanced/lowprecision-pcie-allreduce.html b/latest/legacy/advanced/lowprecision-pcie-allreduce.html
index aed7922235..59902a2021 100644
--- a/latest/legacy/advanced/lowprecision-pcie-allreduce.html
+++ b/latest/legacy/advanced/lowprecision-pcie-allreduce.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers ../summarize.py --engine_di
diff --git a/latest/legacy/architecture/checkpoint.html b/latest/legacy/architecture/checkpoint.html
index ba0895d16d..b6dddf4336 100644
--- a/latest/legacy/architecture/checkpoint.html
+++ b/latest/legacy/architecture/checkpoint.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers --checkpoint_dir ./op
diff --git a/latest/legacy/architecture/core-concepts.html b/latest/legacy/architecture/core-concepts.html
index a2bc22fdcb..0f8c19c40e 100644
--- a/latest/legacy/architecture/core-concepts.html
+++ b/latest/legacy/architecture/core-concepts.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Core Concepts
section of the TensorRT documentation before proceeding further.
-In TensorRT-LLM, the tensorrt_llm.Builder
In TensorRT-LLM, the tensorrt_llm.Builder tensorrt.Builder tensorrt_llm.Builder.create_network tensorrt.INetworkDefinition INetworkDefinition tensorrt_llm.functional
+tensorrt_llm.functional A simple example of such a free function is tensorrt_llm.activation tensorrt.IActivationLayer
@@ -649,14 +653,14 @@ limitation, TensorRT offers a powerful mechanism known as
plugins .
The plugins are nodes inserted in the network graph definition that map to user-defined
GPU kernels. TensorRT-LLM uses a number of such plugins. They can be found in
-the cpp/tensorrt_llm/plugins
+the cpp/tensorrt_llm/plugins Plugins are written in C++ and follow a well-defined interface described in the
Extending TensorRT with Custom Layers
section of the TensorRT
Developer Guide .
When executed within a TensorRT engine, plugins trigger the execution of
their encapsulated GPU kernels. A fairly simple example of plugins is the
-QuantizeTensorPlugin QuantizeTensorPlugin QuantizeTensorPlugin::enqueue
// In cpp/tensorrt_llm/plugins/quantizeTensorPlugin/quantizeTensorPlugin.cpp:
@@ -700,7 +704,7 @@ using TensorRT plugins that wrap communication primitives from the
plugin that optimize the All-Reduce primitive in the presence of All-to-all
connections between GPUs (through NVSwitch in DGX systems).
The communication plugins can be found in
-cpp/tensorrt_llm/plugins/ncclPlugin
+cpp/tensorrt_llm/plugins/ncclPlugin
and the multi-GPU functions are exposed in the TensorRT-LLM Model Definition API
as:
# In tensorrt_llm/functional.py:
@@ -990,9 +994,9 @@ srun \
diff --git a/latest/legacy/architecture/model-weights-loader.html b/latest/legacy/architecture/model-weights-loader.html
index e62787d434..76035e168f 100644
--- a/latest/legacy/architecture/model-weights-loader.html
+++ b/latest/legacy/architecture/model-weights-loader.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters
+
Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading
@@ -365,6 +366,7 @@
Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware
+
Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
diff --git a/latest/legacy/architecture/workflow.html b/latest/legacy/architecture/workflow.html
index cba3f3873f..53c45ef561 100644
--- a/latest/legacy/architecture/workflow.html
+++ b/latest/legacy/architecture/workflow.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
-
+
@@ -335,6 +335,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters
+
Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading
@@ -365,6 +366,7 @@
Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware
+
Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers push <your_dockerhub_use
diff --git a/latest/legacy/dev-on-cloud/dev-on-runpod.html b/latest/legacy/dev-on-cloud/dev-on-runpod.html
index f58988e1a3..2d59348d4b 100644
--- a/latest/legacy/dev-on-cloud/dev-on-runpod.html
+++ b/latest/legacy/dev-on-cloud/dev-on-runpod.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -403,6 +405,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -419,6 +422,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers benchmarks/cpp/prepare_dataset.py
diff --git a/latest/legacy/performance/perf-benchmarking.html b/latest/legacy/performance/perf-benchmarking.html
index 044d2aa322..1fdbff0e31 100644
--- a/latest/legacy/performance/perf-benchmarking.html
+++ b/latest/legacy/performance/perf-benchmarking.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters
+
Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading
@@ -365,6 +366,7 @@
Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware
+
Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers 1 .00
diff --git a/latest/legacy/performance/performance-tuning-guide/deciding-model-sharding-strategy.html b/latest/legacy/performance/performance-tuning-guide/deciding-model-sharding-strategy.html
index 128f7e8afa..4da1918411 100644
--- a/latest/legacy/performance/performance-tuning-guide/deciding-model-sharding-strategy.html
+++ b/latest/legacy/performance/performance-tuning-guide/deciding-model-sharding-strategy.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
diff --git a/latest/legacy/python-api/tensorrt_llm.functional.html b/latest/legacy/python-api/tensorrt_llm.functional.html
index 806908d9de..d5d2dac050 100644
--- a/latest/legacy/python-api/tensorrt_llm.functional.html
+++ b/latest/legacy/python-api/tensorrt_llm.functional.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
-
+
@@ -335,6 +335,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters
+
Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading
@@ -365,6 +366,7 @@
Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware
+
Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers elementwise_affine: bool = True is_causal: bool = False attn_forward_funcname: str = 'joint_attn_forward' mapping=<tensorrt_llm.mapping.Mapping object> mapping=<tensorrt_llm.mapping.MpiTopology object> dtype=None num_classes: int hidden_size: int dropout_prob: float = 0.0 mapping=<tensorrt_llm.mapping.Mapping object> mapping=<tensorrt_llm.mapping.MpiTopology object> dtype=None
embedding_dim pooled_projection_dim mapping=<tensorrt_llm.mapping.Mapping object> mapping=<tensorrt_llm.mapping.MpiTopology object> dtype=None
@@ -1498,7 +1502,7 @@ the number of tokens used for each task, should be equal to prompt_embedding_tab
num_classes embedding_dim class_dropout_prob=0.0 mapping=<tensorrt_llm.mapping.Mapping object> mapping=<tensorrt_llm.mapping.MpiTopology object> dtype=None dim_out: int approximate: str = 'tanh' bias: bool = True mapping=<tensorrt_llm.mapping.Mapping object> mapping=<tensorrt_llm.mapping.MpiTopology object> dtype=None dim_out: int approximate: str = 'tanh' bias: bool = True mapping=<tensorrt_llm.mapping.Mapping object> mapping=<tensorrt_llm.mapping.MpiTopology object> dtype=None dim_in: int dim_out: int bias: bool = True mapping=<tensorrt_llm.mapping.Mapping object> mapping=<tensorrt_llm.mapping.MpiTopology object> dtype=None dim_in: int dim_out: int bias: bool = True mapping=<tensorrt_llm.mapping.Mapping object> mapping=<tensorrt_llm.mapping.MpiTopology object> dtype=None dim_out: int bias: bool = True activation: str = 'silu' mapping=<tensorrt_llm.mapping.Mapping object> mapping=<tensorrt_llm.mapping.MpiTopology object> dtype=None norm_elementwise_affine: bool = False norm_eps: float = 1e-05 chunk_dim: int = 0 mapping=<tensorrt_llm.mapping.Mapping object> mapping=<tensorrt_llm.mapping.MpiTopology object> dtype=None num_embeddings: int | None = None norm_type: str = 'layer_norm' bias: bool = True mapping=<tensorrt_llm.mapping.Mapping object> mapping=<tensorrt_llm.mapping.MpiTopology object> dtype=None embedding_dim: int norm_type: str = 'layer_norm' bias: bool = True mapping=<tensorrt_llm.mapping.Mapping object> mapping=<tensorrt_llm.mapping.MpiTopology object> dtype=None eps: float = 1e-05 bias: bool = True norm_type: str = 'layer_norm' mapping=<tensorrt_llm.mapping.Mapping object> mapping=<tensorrt_llm.mapping.MpiTopology object> dtype=None embedding_dim: int norm_type: str = 'layer_norm' bias: bool = True mapping=<tensorrt_llm.mapping.Mapping object> mapping=<tensorrt_llm.mapping.MpiTopology object> dtype=None Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers
pretrained_model_name_or_path: str dtype='float16' mapping=<tensorrt_llm.mapping.Mapping object> mapping=<tensorrt_llm.mapping.MpiTopology object> **kwargs
@@ -3014,9 +3018,14 @@ rand_data_validation: [bs, max_draft_len]
USER_PROVIDED = 128
+
+SAVE_HIDDEN_STATES = 256
+
+
-AUTO = 256
+AUTO = 512
@@ -3515,6 +3524,7 @@ ranges of the dimensions of when using TRT dynamic shapes.
EAGLE NGRAM USER_PROVIDED SAVE_HIDDEN_STATES AUTO from_arguments()
@@ -3626,9 +3636,9 @@ ranges of the dimensions of when using TRT dynamic shapes.
diff --git a/latest/legacy/python-api/tensorrt_llm.plugin.html b/latest/legacy/python-api/tensorrt_llm.plugin.html
index 5b240df220..d487f2da69 100644
--- a/latest/legacy/python-api/tensorrt_llm.plugin.html
+++ b/latest/legacy/python-api/tensorrt_llm.plugin.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
-
+
@@ -335,6 +335,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters
+
Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading
@@ -365,6 +366,7 @@
Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware
+
Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
-
Note: All the fields should use a prefix “_”; PluginConfigMeta will wrap each field as a property.
-This ensures the fields can only be assigned with allowed values.
+
+Show JSON schema {
+ "title" : "PluginConfig" ,
+ "description" : "The config that manages plugin-related options.\n\nThere are two option categories:\n* Plugin options (typically with xxx_plugin naming). These options can be assigned with:\n * \"float16\"/\"bfloat16\"/\"float32\"/\"int32\", which means the plugin is enabled with the specified precision; (Some plugins only support limited dtype, i.e., gemm_swiglu_plugin and low_latency_gemm_swiglu_plugin only supports fp8 now)\n * \"auto\", which means the plugin is enabled with the precision of `dtype` field (the `dtype` field must be same to model dtype, i.e., the one in PretrainedConfig);\n * None, which means the plugin is disabled.\n* Other features. These options can be assigned with boolean:\n * True, which means the plugin is enabled;\n * False, which means the plugin is disabled." ,
+ "type" : "object" ,
+ "properties" : {
+ "dtype" : {
+ "default" : "float16" ,
+ "description" : "Base dtype for the model and plugins" ,
+ "title" : "Dtype" ,
+ "type" : "string"
+ },
+ "bert_attention_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "auto" ,
+ "float16" ,
+ "float32" ,
+ "bfloat16" ,
+ "int32" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : "auto" ,
+ "description" : "The plugin that uses efficient kernels and enables an in-place update of the KV cache for attention layer of BERT-like encoder models." ,
+ "title" : "Bert Attention Plugin"
+ },
+ "gpt_attention_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "auto" ,
+ "float16" ,
+ "float32" ,
+ "bfloat16" ,
+ "int32" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : "auto" ,
+ "description" : "The plugin that uses efficient kernels and enables an in-place update of the KV cache for attention layer of GPT-like decoder models." ,
+ "title" : "Gpt Attention Plugin"
+ },
+ "gemm_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "auto" ,
+ "float16" ,
+ "float32" ,
+ "bfloat16" ,
+ "int32" ,
+ "fp8" ,
+ "nvfp4" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : null ,
+ "description" : "The GEMM plugin that utilizes NVIDIA cuBLASLt to perform GEMM operations. Note: it's only affective for non-quantized gemm operations (except FP8).Note: For FP8, it also requires same calibration in checkpoint." ,
+ "title" : "Gemm Plugin"
+ },
+ "gemm_swiglu_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "fp8" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : null ,
+ "description" : "The GEMM + SwiGLU fusion in Gated-MLP combines two Matmul operations and one SwiGLU operation into a single kernel. Currently this is only supported for FP8 precision on Hopper." ,
+ "title" : "Gemm Swiglu Plugin"
+ },
+ "fp8_rowwise_gemm_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "auto" ,
+ "float16" ,
+ "float32" ,
+ "bfloat16" ,
+ "int32" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : null ,
+ "description" : "The quantized GEMM for fp8, which uses per token dynamic scales for activation and per channel static scales for weights.Note: It also requires same calibration in checkpoint." ,
+ "title" : "Fp8 Rowwise Gemm Plugin"
+ },
+ "qserve_gemm_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "auto" ,
+ "float16" ,
+ "float32" ,
+ "bfloat16" ,
+ "int32" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : null ,
+ "description" : "The quantized GEMM from [QServe](https://arxiv.org/abs/2405.04532), which employs 4-bit quantization for weights and 8-bit quantization for activations." ,
+ "title" : "Qserve Gemm Plugin"
+ },
+ "identity_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "auto" ,
+ "float16" ,
+ "float32" ,
+ "bfloat16" ,
+ "int32" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : null ,
+ "description" : "The identity plugin simply copies inputs to outputs, it's used mostly for debugging purpose." ,
+ "title" : "Identity Plugin"
+ },
+ "nccl_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "auto" ,
+ "float16" ,
+ "float32" ,
+ "bfloat16" ,
+ "int32" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : "auto" ,
+ "description" : "The NCCL plugin wraps NCCL operators to support multi-GPU and even multi-nodes." ,
+ "title" : "Nccl Plugin"
+ },
+ "lora_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "auto" ,
+ "float16" ,
+ "float32" ,
+ "bfloat16" ,
+ "int32" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : null ,
+ "description" : "Enable LoRA." ,
+ "title" : "Lora Plugin"
+ },
+ "dora_plugin" : {
+ "default" : false ,
+ "description" : "Enable DoRA." ,
+ "title" : "Dora Plugin" ,
+ "type" : "boolean"
+ },
+ "weight_only_groupwise_quant_matmul_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "auto" ,
+ "float16" ,
+ "float32" ,
+ "bfloat16" ,
+ "int32" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : null ,
+ "description" : "Enable weight-only groupwise quantization matmul operators." ,
+ "title" : "Weight Only Groupwise Quant Matmul Plugin"
+ },
+ "weight_only_quant_matmul_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "auto" ,
+ "float16" ,
+ "float32" ,
+ "bfloat16" ,
+ "int32" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : null ,
+ "description" : "Enable weight-only quantization matmul operators." ,
+ "title" : "Weight Only Quant Matmul Plugin"
+ },
+ "smooth_quant_plugins" : {
+ "default" : true ,
+ "description" : "Enable a group of plugins to support smooth quantization." ,
+ "title" : "Smooth Quant Plugins" ,
+ "type" : "boolean"
+ },
+ "smooth_quant_gemm_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "auto" ,
+ "float16" ,
+ "float32" ,
+ "bfloat16" ,
+ "int32" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : null ,
+ "description" : "Enable plugin that supports smooth quantization gemm kernels." ,
+ "title" : "Smooth Quant Gemm Plugin"
+ },
+ "layernorm_quantization_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "auto" ,
+ "float16" ,
+ "float32" ,
+ "bfloat16" ,
+ "int32" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : null ,
+ "description" : "Enable plugin that supports layernorm quantization kernels." ,
+ "title" : "Layernorm Quantization Plugin"
+ },
+ "rmsnorm_quantization_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "auto" ,
+ "float16" ,
+ "float32" ,
+ "bfloat16" ,
+ "int32" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : null ,
+ "description" : "Enable plugin that supports rmsnorm quantization kernels." ,
+ "title" : "Rmsnorm Quantization Plugin"
+ },
+ "quantize_per_token_plugin" : {
+ "default" : false ,
+ "description" : "Enable plugin that supports per-token quantization." ,
+ "title" : "Quantize Per Token Plugin" ,
+ "type" : "boolean"
+ },
+ "quantize_tensor_plugin" : {
+ "default" : false ,
+ "description" : "Enable plugin that supports per-tensor quantization." ,
+ "title" : "Quantize Tensor Plugin" ,
+ "type" : "boolean"
+ },
+ "moe_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "auto" ,
+ "float16" ,
+ "float32" ,
+ "bfloat16" ,
+ "int32" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : "auto" ,
+ "description" : "Enable some customized kernels to speed up the MoE layer of MoE models." ,
+ "title" : "Moe Plugin"
+ },
+ "mamba_conv1d_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "auto" ,
+ "float16" ,
+ "float32" ,
+ "bfloat16" ,
+ "int32" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : "auto" ,
+ "description" : "Enable customized kernels to speed up conv1d operator for Mamba." ,
+ "title" : "Mamba Conv1D Plugin"
+ },
+ "low_latency_gemm_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "fp8" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : null ,
+ "description" : "The GEMM plugin that optimized specially for low latency scenarios." ,
+ "title" : "Low Latency Gemm Plugin"
+ },
+ "low_latency_gemm_swiglu_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "fp8" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : null ,
+ "description" : "The GEMM + SwiGLU fusion plugin that optimized specially for low latency scenarios." ,
+ "title" : "Low Latency Gemm Swiglu Plugin"
+ },
+ "gemm_allreduce_plugin" : {
+ "anyOf" : [
+ {
+ "enum" : [
+ "float16" ,
+ "bfloat16" ,
+ null
+ ]
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : null ,
+ "description" : "The GEMM + AllReduce kernel fusion plugin." ,
+ "title" : "Gemm Allreduce Plugin"
+ },
+ "context_fmha" : {
+ "default" : true ,
+ "description" : "Enable the fused multi-head attention during the context phase, will trigger a kernel that performs the MHA/MQA/GQA block using a single kernel." ,
+ "title" : "Context Fmha" ,
+ "type" : "boolean"
+ },
+ "bert_context_fmha_fp32_acc" : {
+ "default" : false ,
+ "description" : "Enable the FP32 accumulator for context FMHA in the bert_attention_plugin. If disabled, FP16 is used, better performance but potentially worse accuracy is expected." ,
+ "title" : "Bert Context Fmha Fp32 Acc" ,
+ "type" : "boolean"
+ },
+ "paged_kv_cache" : {
+ "anyOf" : [
+ {
+ "type" : "boolean"
+ },
+ {
+ "type" : "null"
+ }
+ ],
+ "default" : null ,
+ "description" : "Enable paged KV cache, which helps manage memory for the KV cache more efficiently, and usually leads to an increase in the batch size and an improved efficiency." ,
+ "title" : "Paged Kv Cache"
+ },
+ "remove_input_padding" : {
+ "default" : true ,
+ "description" : "Pack different tokens together, which reduces both the amount of computations and memory consumption." ,
+ "title" : "Remove Input Padding" ,
+ "type" : "boolean"
+ },
+ "norm_quant_fusion" : {
+ "default" : false ,
+ "description" : "Fuse the LayerNorm and quantization kernels into a single kernel, resulting in improved end-to-end performance." ,
+ "title" : "Norm Quant Fusion" ,
+ "type" : "boolean"
+ },
+ "reduce_fusion" : {
+ "default" : false ,
+ "description" : "Fuse the ResidualAdd and LayerNorm kernels after AllReduce into a single kernel, resulting in improved end-to-end performance." ,
+ "title" : "Reduce Fusion" ,
+ "type" : "boolean"
+ },
+ "user_buffer" : {
+ "default" : false ,
+ "description" : "Eliminate extra copies from the local buffer to the shared buffer in the communication kernel, leading to improved end-to-end performance. This feature must be enabled with `--reduce_fusion enable` and is currently only supported for the FP8 LLAMA model." ,
+ "title" : "User Buffer" ,
+ "type" : "boolean"
+ },
+ "tokens_per_block" : {
+ "default" : 32 ,
+ "description" : "Define how many tokens are contained in each paged kv cache block." ,
+ "title" : "Tokens Per Block" ,
+ "type" : "integer"
+ },
+ "use_paged_context_fmha" : {
+ "default" : true ,
+ "description" : "Allow advanced features like KV cache reuse and chunked context." ,
+ "title" : "Use Paged Context Fmha" ,
+ "type" : "boolean"
+ },
+ "use_fp8_context_fmha" : {
+ "default" : true ,
+ "description" : "When FP8 quantization is activated, the attention can be further accelerated by enabling FP8 Context FMHA" ,
+ "title" : "Use Fp8 Context Fmha" ,
+ "type" : "boolean"
+ },
+ "fuse_fp4_quant" : {
+ "default" : false ,
+ "description" : "Whether to fuse FP4 quantization into attention kernel." ,
+ "title" : "Fuse Fp4 Quant" ,
+ "type" : "boolean"
+ },
+ "multiple_profiles" : {
+ "default" : false ,
+ "description" : "Enables multiple TensorRT optimization profiles in the built engines, will benefits the performance especially when GEMM plugin is disabled, because more optimization profiles help TensorRT have more chances to select better kernels. Note: This feature increases engine build time but no other adverse effects are expected." ,
+ "title" : "Multiple Profiles" ,
+ "type" : "boolean"
+ },
+ "paged_state" : {
+ "default" : true ,
+ "description" : "Enable paged state, which helps manage memory for the RNN state more efficiently." ,
+ "title" : "Paged State" ,
+ "type" : "boolean"
+ },
+ "streamingllm" : {
+ "default" : false ,
+ "description" : "Enable [StreamingLLM](https://arxiv.org/abs/2309.17453), which uses a window attention to perform efficient and stable LLM on long texts." ,
+ "title" : "Streamingllm" ,
+ "type" : "boolean"
+ },
+ "manage_weights" : {
+ "default" : false ,
+ "description" : "Enable TensorRT LLM managed weights to speed up engine building process." ,
+ "title" : "Manage Weights" ,
+ "type" : "boolean"
+ },
+ "use_fused_mlp" : {
+ "default" : true ,
+ "description" : "Enable horizontal fusion in Gated-MLP that combines two Matmul operations into a single one followed by a separate SwiGLU kernel." ,
+ "title" : "Use Fused Mlp" ,
+ "type" : "boolean"
+ },
+ "pp_reduce_scatter" : {
+ "default" : false ,
+ "description" : "Enable a pipeline parallelism optimization with ReduceScatter + AllGather targeting large MoE models." ,
+ "title" : "Pp Reduce Scatter" ,
+ "type" : "boolean"
+ }
+ }
+}
+
+
+Config:
+
+
+Fields:
+
+bert_attention_plugin (Literal['auto', 'float16', 'float32', 'bfloat16', 'int32', None] | None)
bert_context_fmha_fp32_acc (bool)
context_fmha (bool)
dora_plugin (bool)
dtype (str)
fp8_rowwise_gemm_plugin (Literal['auto', 'float16', 'float32', 'bfloat16', 'int32', None] | None)
fuse_fp4_quant (bool)
gemm_allreduce_plugin (Literal['float16', 'bfloat16', None] | None)
gemm_plugin (Literal['auto', 'float16', 'float32', 'bfloat16', 'int32', 'fp8', 'nvfp4', None] | None)
gemm_swiglu_plugin (Literal['fp8', None] | None)
gpt_attention_plugin (Literal['auto', 'float16', 'float32', 'bfloat16', 'int32', None] | None)
identity_plugin (Literal['auto', 'float16', 'float32', 'bfloat16', 'int32', None] | None)
layernorm_quantization_plugin (Literal['auto', 'float16', 'float32', 'bfloat16', 'int32', None] | None)
lora_plugin (Literal['auto', 'float16', 'float32', 'bfloat16', 'int32', None] | None)
low_latency_gemm_plugin (Literal['fp8', None] | None)
low_latency_gemm_swiglu_plugin (Literal['fp8', None] | None)
mamba_conv1d_plugin (Literal['auto', 'float16', 'float32', 'bfloat16', 'int32', None] | None)
manage_weights (bool)
moe_plugin (Literal['auto', 'float16', 'float32', 'bfloat16', 'int32', None] | None)
multiple_profiles (bool)
nccl_plugin (Literal['auto', 'float16', 'float32', 'bfloat16', 'int32', None] | None)
norm_quant_fusion (bool)
paged_kv_cache (bool | None)
paged_state (bool)
pp_reduce_scatter (bool)
qserve_gemm_plugin (Literal['auto', 'float16', 'float32', 'bfloat16', 'int32', None] | None)
quantize_per_token_plugin (bool)
quantize_tensor_plugin (bool)
reduce_fusion (bool)
remove_input_padding (bool)
rmsnorm_quantization_plugin (Literal['auto', 'float16', 'float32', 'bfloat16', 'int32', None] | None)
smooth_quant_gemm_plugin (Literal['auto', 'float16', 'float32', 'bfloat16', 'int32', None] | None)
smooth_quant_plugins (bool)
streamingllm (bool)
tokens_per_block (int)
use_fp8_context_fmha (bool)
use_fused_mlp (bool)
use_paged_context_fmha (bool)
user_buffer (bool)
weight_only_groupwise_quant_matmul_plugin (Literal['auto', 'float16', 'float32', 'bfloat16', 'int32', None] | None)
weight_only_quant_matmul_plugin (Literal['auto', 'float16', 'float32', 'bfloat16', 'int32', None] | None)
+Validators:
+
+convert_enable_disable all fields
log_field_changes all fields
validate_dtype_not_auto dtype
+
+
+
+field bert_attention_plugin : Literal [ 'auto' , 'float16' , 'float32' , 'bfloat16' , 'int32' , None ] | None = 'auto'
+The plugin that uses efficient kernels and enables an in-place update of the KV cache for attention layer of BERT-like encoder models.
+
+
+
+
+field bert_context_fmha_fp32_acc : bool = False
+Enable the FP32 accumulator for context FMHA in the bert_attention_plugin. If disabled, FP16 is used, better performance but potentially worse accuracy is expected.
+
+
+
+
+field context_fmha : bool = True
+Enable the fused multi-head attention during the context phase, will trigger a kernel that performs the MHA/MQA/GQA block using a single kernel.
+
+
+
+
+field dora_plugin : bool = False
+Enable DoRA.
+
+
+
+
+field dtype : str = 'float16'
+Base dtype for the model and plugins
+
+
+
+
+field fp8_rowwise_gemm_plugin : Literal [ 'auto' , 'float16' , 'float32' , 'bfloat16' , 'int32' , None ] | None = None
+The quantized GEMM for fp8, which uses per token dynamic scales for activation and per channel static scales for weights.Note: It also requires same calibration in checkpoint.
+
+
+
+
+field fuse_fp4_quant : bool = False
+Whether to fuse FP4 quantization into attention kernel.
+
+
+
+
+field gemm_allreduce_plugin : Literal [ 'float16' , 'bfloat16' , None ] | None = None
+The GEMM + AllReduce kernel fusion plugin.
+
+
+
+
+field gemm_plugin : Literal [ 'auto' , 'float16' , 'float32' , 'bfloat16' , 'int32' , 'fp8' , 'nvfp4' , None ] | None = None
+The GEMM plugin that utilizes NVIDIA cuBLASLt to perform GEMM operations. Note: it’s only affective for non-quantized gemm operations (except FP8).Note: For FP8, it also requires same calibration in checkpoint.
+
+
+
+
+field gemm_swiglu_plugin : Literal [ 'fp8' , None ] | None = None
+The GEMM + SwiGLU fusion in Gated-MLP combines two Matmul operations and one SwiGLU operation into a single kernel. Currently this is only supported for FP8 precision on Hopper.
+
+
+
+
+field gpt_attention_plugin : Literal [ 'auto' , 'float16' , 'float32' , 'bfloat16' , 'int32' , None ] | None = 'auto'
+The plugin that uses efficient kernels and enables an in-place update of the KV cache for attention layer of GPT-like decoder models.
+
+
+
+
+field identity_plugin : Literal [ 'auto' , 'float16' , 'float32' , 'bfloat16' , 'int32' , None ] | None = None
+The identity plugin simply copies inputs to outputs, it’s used mostly for debugging purpose.
+
+
+
+
+field layernorm_quantization_plugin : Literal [ 'auto' , 'float16' , 'float32' , 'bfloat16' , 'int32' , None ] | None = None
+Enable plugin that supports layernorm quantization kernels.
+
+
+
+
+field lora_plugin : Literal [ 'auto' , 'float16' , 'float32' , 'bfloat16' , 'int32' , None ] | None = None
+Enable LoRA.
+
+
+
+
+field low_latency_gemm_plugin : Literal [ 'fp8' , None ] | None = None
+The GEMM plugin that optimized specially for low latency scenarios.
+
+
+
+
+field low_latency_gemm_swiglu_plugin : Literal [ 'fp8' , None ] | None = None
+The GEMM + SwiGLU fusion plugin that optimized specially for low latency scenarios.
+
+
+
+
+field mamba_conv1d_plugin : Literal [ 'auto' , 'float16' , 'float32' , 'bfloat16' , 'int32' , None ] | None = 'auto'
+Enable customized kernels to speed up conv1d operator for Mamba.
+
+
+
+
+field manage_weights : bool = False
+Enable TensorRT LLM managed weights to speed up engine building process.
+
+
+
+
+field moe_plugin : Literal [ 'auto' , 'float16' , 'float32' , 'bfloat16' , 'int32' , None ] | None = 'auto'
+Enable some customized kernels to speed up the MoE layer of MoE models.
+
+
+
+
+field multiple_profiles : bool = False
+Enables multiple TensorRT optimization profiles in the built engines, will benefits the performance especially when GEMM plugin is disabled, because more optimization profiles help TensorRT have more chances to select better kernels. Note: This feature increases engine build time but no other adverse effects are expected.
+
+
+
+
+field nccl_plugin : Literal [ 'auto' , 'float16' , 'float32' , 'bfloat16' , 'int32' , None ] | None = 'auto'
+The NCCL plugin wraps NCCL operators to support multi-GPU and even multi-nodes.
+
+
+
+
+field norm_quant_fusion : bool = False
+Fuse the LayerNorm and quantization kernels into a single kernel, resulting in improved end-to-end performance.
+
+
+
+
+field paged_kv_cache : bool | None = None
+Enable paged KV cache, which helps manage memory for the KV cache more efficiently, and usually leads to an increase in the batch size and an improved efficiency.
+
+
+
+
+field paged_state : bool = True
+Enable paged state, which helps manage memory for the RNN state more efficiently.
+
+
+
+
+field pp_reduce_scatter : bool = False
+Enable a pipeline parallelism optimization with ReduceScatter + AllGather targeting large MoE models.
+
+
+
+
+field qserve_gemm_plugin : Literal [ 'auto' , 'float16' , 'float32' , 'bfloat16' , 'int32' , None ] | None = None
+The quantized GEMM from [QServe](https://arxiv.org/abs/2405.04532 ), which employs 4-bit quantization for weights and 8-bit quantization for activations.
+
+
+
+
+field quantize_per_token_plugin : bool = False
+Enable plugin that supports per-token quantization.
+
+
+
+
+field quantize_tensor_plugin : bool = False
+Enable plugin that supports per-tensor quantization.
+
+
+
+
+field reduce_fusion : bool = False
+Fuse the ResidualAdd and LayerNorm kernels after AllReduce into a single kernel, resulting in improved end-to-end performance.
+
+
+
+
+field remove_input_padding : bool = True
+Pack different tokens together, which reduces both the amount of computations and memory consumption.
+
+
+
+
+field rmsnorm_quantization_plugin : Literal [ 'auto' , 'float16' , 'float32' , 'bfloat16' , 'int32' , None ] | None = None
+Enable plugin that supports rmsnorm quantization kernels.
+
+
+
+
+field smooth_quant_gemm_plugin : Literal [ 'auto' , 'float16' , 'float32' , 'bfloat16' , 'int32' , None ] | None = None
+Enable plugin that supports smooth quantization gemm kernels.
+
+
+
+
+field smooth_quant_plugins : bool = True
+Enable a group of plugins to support smooth quantization.
+
+
+
+
+field streamingllm : bool = False
+Enable [StreamingLLM](https://arxiv.org/abs/2309.17453 ), which uses a window attention to perform efficient and stable LLM on long texts.
+
+
+
+
+field tokens_per_block : int = 32
+Define how many tokens are contained in each paged kv cache block.
+
+
+
+
+field use_fp8_context_fmha : bool = True
+When FP8 quantization is activated, the attention can be further accelerated by enabling FP8 Context FMHA
+
+
+
+
+field use_fused_mlp : bool = True
+Enable horizontal fusion in Gated-MLP that combines two Matmul operations into a single one followed by a separate SwiGLU kernel.
+
+
+
+
+field use_paged_context_fmha : bool = True
+Allow advanced features like KV cache reuse and chunked context.
+
+
+
+
+field user_buffer : bool = False
+Eliminate extra copies from the local buffer to the shared buffer in the communication kernel, leading to improved end-to-end performance. This feature must be enabled with –reduce_fusion enable and is currently only supported for the FP8 LLAMA model.
+
+
+
+
+field weight_only_groupwise_quant_matmul_plugin : Literal [ 'auto' , 'float16' , 'float32' , 'bfloat16' , 'int32' , None ] | None = None
+Enable weight-only groupwise quantization matmul operators.
+
+
+
+
+field weight_only_quant_matmul_plugin : Literal [ 'auto' , 'float16' , 'float32' , 'bfloat16' , 'int32' , None ] | None = None
+Enable weight-only quantization matmul operators.
+
+
+
+
+validator convert_enable_disable » all fields [source]
+Allow passing enable/disable strings which map to boolean/None values.
+
+
+
+
+enable_paged_kv_cache ( tokens_per_block : int = 32 ) [source]
+
+
+
+
+classmethod from_arguments ( args : Namespace ) [source]
+Create a PluginConfig from argparse arguments.
+
+
+
+
+is_context_fmha_enabled ( ) [source]
+
+
+
+
+validator log_field_changes » all fields [source]
+Log all field changes for debugging.
+
+
+
+
+model_post_init ( context : Any / ) → None
+This function is meant to behave like a BaseModel method to initialise private attributes.
+It takes context as an argument since that’s what pydantic-core passes when calling it.
+
+Parameters:
+
+
+
+
+
+
+
+set_context_fmha (
+
+
+context_fmha_type = ContextFMHAType.enabled
+
+) [source]
+
+
+
+set_dora_plugin ( enable : bool = False ) [source]
+
+
+
+
+set_fp8_rowwise_quant_plugins ( dtype : str = 'auto' ) [source]
+
+
+
+
+set_lora_plugin ( dtype : str = None ) [source]
+
+
+
+
+set_nccl_plugin ( dtype : str = 'auto' ) [source]
+
+
+
+
+set_qserve_plugins ( dtype : str = 'auto' ) [source]
+
+
+
+
+set_smooth_quant_plugins ( dtype : str = 'auto' ) [source]
+
+
to_legacy_setting ( ) [source]
@@ -534,6 +1442,21 @@ migrated to the centralized building script tensorrt_llm/commands/build.py
After the migration is done, this function may or may not be deleted.
+
+
+validate ( ) [source]
+
+
+
+
+validator validate_dtype_not_auto » dtype [source]
+
+
+
+
+property context_fmha_type
+
+
@@ -570,7 +1493,64 @@ migrated to the centralized building script
tensorrt_llm/commands/build.py
@@ -667,9 +1647,9 @@ migrated to the centralized building script tensorrt_llm/commands/build.py
diff --git a/latest/legacy/python-api/tensorrt_llm.quantization.html b/latest/legacy/python-api/tensorrt_llm.quantization.html
index f6d6fafa93..755e9f108f 100644
--- a/latest/legacy/python-api/tensorrt_llm.quantization.html
+++ b/latest/legacy/python-api/tensorrt_llm.quantization.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
GenerationSession
@@ -3252,9 +3271,9 @@ mrope_position_deltas (torch.Tensor of shape (batch_size)
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers
Memory pool
-TensorRT-LLM C++ runtime is using stream-ordered memory allocator to allocate and free buffers, see BufferManager::initMemoryPool , which uses the default memory pool managed by the CUDA driver. When a TrtGptModel TrtGptModel
-However, nvidia-smi BufferManager::memoryPoolReserved()) and BufferManager::memoryPoolFree()) , respectively.
+TensorRT-LLM C++ runtime is using stream-ordered memory allocator to allocate and free buffers, see BufferManager::initMemoryPool , which uses the default memory pool managed by the CUDA driver. When a TrtGptModel TrtGptModel
+However, nvidia-smi BufferManager::memoryPoolReserved()) and BufferManager::memoryPoolFree()) , respectively.
Known Issues
@@ -754,9 +758,9 @@ Here some explanations on how these values affect the memory:
diff --git a/latest/legacy/reference/multimodal-feature-support-matrix.html b/latest/legacy/reference/multimodal-feature-support-matrix.html
index c95a29fdb8..51da69fb7e 100644
--- a/latest/legacy/reference/multimodal-feature-support-matrix.html
+++ b/latest/legacy/reference/multimodal-feature-support-matrix.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -403,6 +405,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -419,6 +422,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Examples of how to enable SmoothQuant for GPT, GPT-J and LLaMA can be found in
-the examples/quantization folder of that release.
+the examples/quantization folder of that release.
INT4 and INT8 Weight-Only (W4A16 and W8A16)
@@ -571,8 +575,8 @@ a model and dequantizing those weights on-the-fly in linear layers (Matmuls).
The activations are encoded using floating-point values (FP16 or BF16).
To use INT4/INT8 Weight-Only methods, the user must determine the scaling
factors to use to quantize and dequantize the weights of the model.
-This release includes examples for GPT and
-LLaMA .
+This release includes examples for GPT and
+LLaMA .
NVFP4 (Blackwell)
@@ -1096,7 +1100,7 @@ The language component decides which quantization methods are supported by a giv
Technical Detail: The QuantMode
The quantization method is controlled by the
-QuantMode QuantMode
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -404,6 +406,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -420,6 +423,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -409,6 +411,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -425,6 +428,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -409,6 +411,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -425,6 +428,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers peft_cache_config (Optional [ tensorrt_llm.llmapi.llm_args.PeftCacheConfig ] ) – prototype
scheduler_config (tensorrt_llm.llmapi.llm_args.SchedulerConfig prototype
cache_transceiver_config (Optional [ tensorrt_llm.llmapi.llm_args.CacheTransceiverConfig ] ) – prototype
speculative_config (Union [ tensorrt_llm.llmapi.llm_args.DraftTargetDecodingConfig , tensorrt_llm.llmapi.llm_args.EagleDecodingConfig , tensorrt_llm.llmapi.llm_args.LookaheadDecodingConfig , tensorrt_llm.llmapi.llm_args.MedusaDecodingConfig , tensorrt_llm.llmapi.llm_args.MTPDecodingConfig , tensorrt_llm.llmapi.llm_args.NGramDecodingConfig , tensorrt_llm.llmapi.llm_args.UserProvidedDecodingConfig , tensorrt_llm.llmapi.llm_args.AutoDecodingConfig , NoneType ] ) – stable
sparse_attention_config (Optional [ tensorrt_llm.llmapi.llm_args.RocketSparseAttentionConfig ] ) – prototype
speculative_config (Union [ tensorrt_llm.llmapi.llm_args.DraftTargetDecodingConfig , tensorrt_llm.llmapi.llm_args.EagleDecodingConfig , tensorrt_llm.llmapi.llm_args.LookaheadDecodingConfig , tensorrt_llm.llmapi.llm_args.MedusaDecodingConfig , tensorrt_llm.llmapi.llm_args.MTPDecodingConfig , tensorrt_llm.llmapi.llm_args.NGramDecodingConfig , tensorrt_llm.llmapi.llm_args.UserProvidedDecodingConfig , tensorrt_llm.llmapi.llm_args.SaveHiddenStatesDecodingConfig , tensorrt_llm.llmapi.llm_args.AutoDecodingConfig , NoneType ] ) – stable
max_batch_size (Optional [ int ] ) – stable
max_input_len (Optional [ int ] ) – stable
max_seq_len (Optional [ int ] ) – stable
max_beam_width (Optional [ int ] ) – stable
max_num_tokens (Optional [ int ] ) – stable
max_num_tokens (Optional [ int ] ) – stable
gather_generation_logits (bool ) – prototype
num_postprocess_workers (int ) – prototype
postprocess_tokenizer_dir (Optional [ str ] ) – prototype
reasoning_parser (Optional [ str ] ) – prototype
return_perf_metrics (bool ) – prototype
orchestrator_type (Optional [ Literal [ 'rpc' , 'ray' ] ] ) – prototype
garbage_collection_gen0_threshold (int ) – beta
cuda_graph_config (Optional [ tensorrt_llm.llmapi.llm_args.CudaGraphConfig ] ) – beta only (the reason is that it’s hard to capture a single graph with prefill requests since the input shapes are a function of the sequence lengths). Note that each CUDA graph can use up to 200 MB of extra memory. Defaults to None.
attention_dp_config (Optional [ tensorrt_llm.llmapi.llm_args.AttentionDpConfig ] ) – beta
kv_cache_retention_config : KvCacheRetentionConfig | Sequence [ KvCacheRetentionConfig ] | None = None disaggregated_params : DisaggregatedParams | Sequence [ DisaggregatedParams ] | None = None scheduling_params : SchedulingParams | List [ SchedulingParams ] | None = None cache_salt : str | Sequence [ str ] | None = None ) → RequestOutput | List [ RequestOutput ] kv_cache_retention_config (tensorrt_llm.bindings.executor.KvCacheRetentionConfig , Sequence [ tensorrt_llm.bindings.executor.KvCacheRetentionConfig ] , optional ) – Configuration for the request’s retention in the KV Cache. Defaults to None.
disaggregated_params (tensorrt_llm.disaggregated_params.DisaggregatedParams , Sequence [ tensorrt_llm.disaggregated_params.DisaggregatedParams ] , optional ) – Disaggregated parameters. Defaults to None.
scheduling_params (tensorrt_llm.scheduling_params.SchedulingParams , List [ tensorrt_llm.scheduling_params.SchedulingParams ] , optional ) – Scheduling parameters. Defaults to None.
cache_salt (str , Sequence [ str ] , optional ) – If specified, KV cache will be salted with the provided string to limit the kv cache reuse to the requests with the same string. Defaults to None.
Returns:
@@ -1113,6 +1121,10 @@ after prompts have been submitted.
'cancelled'] | None = None stop_reason: int | str | None = None generation_logits: ~torch.Tensor | None = None additional_context_outputs: ~typing.Dict[str ~torch.Tensor] | None = None additional_generation_outputs: ~typing.Dict[str ~torch.Tensor] | None = None disaggregated_params: ~tensorrt_llm.disaggregated_params.DisaggregatedParams | None = None request_perf_metrics: ~tensorrt_llm.bindings.executor.RequestPerfMetrics | None = None _postprocess_result: ~typing.Any = None finish_reason (Literal [ 'stop' , 'length' , 'timeout' , 'cancelled' ] , optional ) – The reason why the sequence is finished. Defaults to None.
stop_reason (int , str , optional ) – The stop string or token id that caused the completion to stop, None if the completion finished for some other reason. Defaults to None.
generation_logits (torch.Tensor , optional ) – The logits on the generated output token ids. Defaults to None.
additional_context_outputs (Dict [ str , torch.Tensor ] , optional ) – The additional context outputs. Defaults to None.
additional_generation_outputs (Dict [ str , torch.Tensor ] , optional ) – The additional generation outputs. Defaults to None.
disaggregated_params (tensorrt_llm.disaggregated_params.DisaggregatedParams , optional ) – Parameters needed for disaggregated serving. Includes the type of request, the first generated tokens, the context request id and the any additional state needing to be transferred from context and generation instances. Defaults to None.
request_perf_metrics (tensorrt_llm.bindings.executor.RequestPerfMetrics , optional ) – Performance metrics for the request. Defaults to None.
'cancelled'] | None = None stop_reason: int | str | None = None generation_logits: ~torch.Tensor | None = None additional_context_outputs: ~typing.Dict[str ~torch.Tensor] | None = None additional_generation_outputs: ~typing.Dict[str ~torch.Tensor] | None = None disaggregated_params: ~tensorrt_llm.disaggregated_params.DisaggregatedParams | None = None request_perf_metrics: ~tensorrt_llm.bindings.executor.RequestPerfMetrics | None = None _postprocess_result: ~typing.Any = None ) → None
+
+additional_context_outputs : Dict [ str , Tensor ] | None
+
+
+
+
+additional_generation_outputs : Dict [ str , Tensor ] | None
+
+
cumulative_logprob : float | None
@@ -1441,6 +1469,8 @@ after prompts have been submitted.
is_final error metrics request_perf_metrics disaggregated_params
)
@@ -1473,6 +1503,12 @@ after prompts have been submitted.
Alias for field number 0
+
+disaggregated_params : Any
+Alias for field number 6
+
+
error : str
@@ -1491,6 +1527,12 @@ after prompts have been submitted.
Alias for field number 4
+
+
+request_perf_metrics : Any
+Alias for field number 5
+
+
res : Any
@@ -1783,7 +1825,7 @@ after prompts have been submitted.
exclude_input_from_output : bool = True return_encoder_output : bool = False return_perf_metrics : bool = False additional_model_outputs : List [ AdditionalModelOutput ] | None = None additional_model_outputs : List [ str ] | None = None _context_logits_auto_enabled : bool = False _generation_logits_auto_enabled : bool = False _return_log_probs : bool = False n (int ) – Number of sequences to generate. Defaults to 1.
best_of (int , optional ) – Number of sequences to consider for best output. Defaults to None.
use_beam_search (bool ) – Whether to use beam search. Defaults to False.
top_k (int , optional ) – Controls number of logits to sample from. None means using C++ runtime default 0, i.e., all logits. Defaults to None.
top_p (float , optional ) – Controls the top-P probability to sample from. None means using C++ runtime default 0.f. Defaults to None.
top_k (int , optional ) – Controls number of logits to sample from. Can assume non-negative values, where 0 means ‘all logits’. Defaults to None.
+The value None is treated as “not specified” in the following.
+If neither temperature, top_p, nor top_k are specified, sampling is greedy.
+If temperature > 0 and/or top_p < 1 are specified, sampling will proceed accordingly and top_k will default to top_k = 0.
+Setting top_k = 1 results in greedy sampling.
top_p (float , optional ) – Controls the top-P probability to sample from. Can have values between 0 and 1. Defaults to None.
+The value None is treated as “not specified” in the following.
+If neither temperature, top_p, nor top_k are specified, sampling is greedy.
+If temperature > 0 and/or top_k > 1 are specified, sampling will proceed accordingly and top_p will default to top_p = 1.
+Setting top_p = 0 should result in greedy sampling, but is currently disallowed in the backend.
top_p_min (float , optional ) – Controls decay in the top-P algorithm. topPMin is lower-bound. None means using C++ runtime default 1.e-6. Defaults to None.
top_p_reset_ids (int , optional ) – Controls decay in the top-P algorithm. Indicates where to reset the decay. None means using C++ runtime default 1. Defaults to None.
top_p_decay (float , optional ) – Controls decay in the top-P algorithm. The decay value. None means using C++ runtime default 1.f. Defaults to None.
seed (int , optional ) – Controls the random seed used by the random number generator in sampling. None means using C++ runtime default 0. Defaults to None.
temperature (float , optional ) – Controls the modulation of logits when sampling new tokens. It can have values > 0.f. None means using C++ runtime default 1.0f. Defaults to None.
temperature (float , optional ) – Controls the modulation of logits when sampling new tokens. It can have values >= 0.f. Defaults to None.
+The value None is treated as “not specified” in the following.
+If neither temperature, top_p, nor top_k are specified, sampling is greedy.
+If top_p < 1 and/or top_k > 1 are specified, sampling will proceed accordingly and temperature will default to temperature = 1.
+Setting temperature = 0 results in greedy sampling.
min_tokens (int , optional ) – Lower bound on the number of tokens to generate. Values < 1 have no effect. None means using C++ runtime default 1. Defaults to None.
beam_search_diversity_rate (float , optional ) – Used to penalize tokens based on how often they appear in the sequence. It can have any value > 0.f. Values < 1.f encourages repetition, values > 1.f discourages it. None means using C++ runtime default 1.f. Defaults to None.
repetition_penalty (float , optional ) – Used to penalize tokens based on how often they appear in the sequence. It can have any value > 0.f. Values < 1.f encourages repetition, values > 1.f discourages it. None means using C++ runtime default 1.f. Defaults to None.
exclude_input_from_output (bool ) – Controls if output tokens in Result should include the input tokens. Defaults to True.
return_encoder_output (bool ) – Controls if Result should contain encoder output hidden states (for encoder-only and encoder-decoder models). Defaults to False.
return_perf_metrics (bool ) – Controls if Result should contain the performance metrics for this request. Defaults to False.
additional_model_outputs (List [ tensorrt_llm.sampling_params.AdditionalModelOutput ] , optional ) – The additional outputs to gather from the model. Defaults to None.
additional_model_outputs (List [ str ] , optional ) – The additional outputs to gather from the model. Defaults to None.
lookahead_config (tensorrt_llm.bindings.executor.LookaheadDecodingConfig , optional ) – Lookahead decoding config. Defaults to None.
guided_decoding (tensorrt_llm.sampling_params.GuidedDecodingParams , optional ) – Guided decoding params. Defaults to None.
ignore_eos (bool ) – Whether to ignore the EOS token and continue generating tokens after the EOS token is generated. Defaults to False.
exclude_input_from_output : bool = True return_encoder_output : bool = False return_perf_metrics : bool = False additional_model_outputs : List [ AdditionalModelOutput ] | None = None additional_model_outputs : List [ str ] | None = None _context_logits_auto_enabled : bool = False _generation_logits_auto_enabled : bool = False _return_log_probs : bool = False ) → None
+
+static params_imply_greedy_decoding (
+
+
+* temperature : float | None top_p : float | None top_k : int | None
+
+) [source]
+
add_special_tokens : bool
@@ -1938,7 +2006,7 @@ The BatchedLogitsProcessor class is recommended for callback creation. The callb
-additional_model_outputs : List [ AdditionalModelOutput ] | None
+additional_model_outputs : List [ str ] | None
@@ -4480,6 +4548,8 @@ If rebuilding _was_ required, returns True if rebuilding was succes
speculative_model_dir : str | Path | None = None max_concurrency : int | None = None load_format : str | None = None acceptance_window : int | None = None acceptance_length_threshold : float | None = None max_window_size : int = 4 max_ngram_size : int = 3 max_verification_set_size : int = 4 True if rebuilding was succes
) [source] Bases: DecodingBaseConfig PybindMirror
Configuration for lookahead speculative decoding.
+
+
+field acceptance_length_threshold : float | None = None
+
+
+
+
+field acceptance_window : int | None = None
+
+
field load_format : str | None = None
@@ -5198,7 +5278,7 @@ a subset of the possible backends.
-validator validate_positive_values » max_ngram_size , max_verification_set_size , max_window_size [source]
+validator validate_positive_values » max_window_size , max_verification_set_size , max_ngram_size [source]
@@ -5230,7 +5310,7 @@ a subset of the possible backends.
-model_fields = {'load_format': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'max_concurrency': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_draft_len': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_ngram_size': FieldInfo(annotation=int, required=False, default=3, description='Number of tokens per NGram.'), 'max_verification_set_size': FieldInfo(annotation=int, required=False, default=4, description='Number of NGrams in verification branch per step.'), 'max_window_size': FieldInfo(annotation=int, required=False, default=4, description='Number of NGrams in lookahead branch per step.'), 'speculative_model_dir': FieldInfo(annotation=Union[str, Path, NoneType], required=False, default=None)}
+model_fields = {'acceptance_length_threshold': FieldInfo(annotation=Union[float, NoneType], required=False, default=None), 'acceptance_window': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'load_format': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'max_concurrency': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_draft_len': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_ngram_size': FieldInfo(annotation=int, required=False, default=3, description='Number of tokens per NGram.'), 'max_verification_set_size': FieldInfo(annotation=int, required=False, default=4, description='Number of NGrams in verification branch per step.'), 'max_window_size': FieldInfo(annotation=int, required=False, default=4, description='Number of NGrams in lookahead branch per step.'), 'speculative_model_dir': FieldInfo(annotation=Union[str, Path, NoneType], required=False, default=None)}
@@ -5265,12 +5345,24 @@ a subset of the possible backends.
speculative_model_dir : str | Path | None = None max_concurrency : int | None = None load_format : str | None = None acceptance_window : int | None = None acceptance_length_threshold : float | None = None medusa_choices : List [ List [ int ] ] | None = None num_medusa_heads : int | None = None
) [source] Bases: DecodingBaseConfig
+
+
+field acceptance_length_threshold : float | None = None
+
+
+
+
+field acceptance_window : int | None = None
+
+
field load_format : str | None = None
@@ -5919,7 +6011,7 @@ a subset of the possible backends.
-model_fields = {'load_format': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'max_concurrency': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_draft_len': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'medusa_choices': FieldInfo(annotation=Union[List[List[int]], NoneType], required=False, default=None), 'num_medusa_heads': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'speculative_model_dir': FieldInfo(annotation=Union[str, Path, NoneType], required=False, default=None)}
+model_fields = {'acceptance_length_threshold': FieldInfo(annotation=Union[float, NoneType], required=False, default=None), 'acceptance_window': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'load_format': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'max_concurrency': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_draft_len': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'medusa_choices': FieldInfo(annotation=Union[List[List[int]], NoneType], required=False, default=None), 'num_medusa_heads': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'speculative_model_dir': FieldInfo(annotation=Union[str, Path, NoneType], required=False, default=None)}
@@ -5954,6 +6046,8 @@ a subset of the possible backends.
speculative_model_dir : str | Path | None = None max_concurrency : int | None = None load_format : str | None = None acceptance_window : int | None = None acceptance_length_threshold : float | None = None eagle_choices : List [ List [ int ] ] | None = None greedy_sampling : bool | None = True posterior_threshold : float | None = None ) [source] Bases: DecodingBaseConfig
+
+
+field acceptance_length_threshold : float | None = None
+
+
+
+
+field acceptance_window : int | None = None
+
+
field dynamic_tree_max_topK : int | None = None
@@ -6661,7 +6765,7 @@ a subset of the possible backends.
-model_fields = {'dynamic_tree_max_topK': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'eagle3_layers_to_capture': FieldInfo(annotation=Union[Set[int], NoneType], required=False, default=None), 'eagle3_one_model': FieldInfo(annotation=Union[bool, NoneType], required=False, default=True), 'eagle_choices': FieldInfo(annotation=Union[List[List[int]], NoneType], required=False, default=None), 'greedy_sampling': FieldInfo(annotation=Union[bool, NoneType], required=False, default=True), 'load_format': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'max_concurrency': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_draft_len': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_non_leaves_per_layer': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_total_draft_tokens': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'num_eagle_layers': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'posterior_threshold': FieldInfo(annotation=Union[float, NoneType], required=False, default=None), 'speculative_model_dir': FieldInfo(annotation=Union[str, Path, NoneType], required=False, default=None), 'use_dynamic_tree': FieldInfo(annotation=Union[bool, NoneType], required=False, default=False)}
+model_fields = {'acceptance_length_threshold': FieldInfo(annotation=Union[float, NoneType], required=False, default=None), 'acceptance_window': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'dynamic_tree_max_topK': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'eagle3_layers_to_capture': FieldInfo(annotation=Union[Set[int], NoneType], required=False, default=None), 'eagle3_one_model': FieldInfo(annotation=Union[bool, NoneType], required=False, default=True), 'eagle_choices': FieldInfo(annotation=Union[List[List[int]], NoneType], required=False, default=None), 'greedy_sampling': FieldInfo(annotation=Union[bool, NoneType], required=False, default=True), 'load_format': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'max_concurrency': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_draft_len': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_non_leaves_per_layer': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_total_draft_tokens': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'num_eagle_layers': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'posterior_threshold': FieldInfo(annotation=Union[float, NoneType], required=False, default=None), 'speculative_model_dir': FieldInfo(annotation=Union[str, Path, NoneType], required=False, default=None), 'use_dynamic_tree': FieldInfo(annotation=Union[bool, NoneType], required=False, default=False)}
@@ -6704,6 +6808,8 @@ Otherwise, assume Eagle3 base set and return 3.
speculative_model_dir : str | Path | None = None max_concurrency : int | None = None load_format : str | None = None acceptance_window : int | None = None acceptance_length_threshold : float | None = None num_nextn_predict_layers : int = 1 use_relaxed_acceptance_for_thinking : bool = False relaxed_topk : int = 1 field END_THINKING_PHASE_TOKEN : int = 128799
+
+
+field acceptance_length_threshold : float | None = None
+
+
+
+
+field acceptance_window : int | None = None
+
+
field load_format : str | None = None
@@ -7394,7 +7510,7 @@ a subset of the possible backends.
-model_fields = {'BEGIN_THINKING_PHASE_TOKEN': FieldInfo(annotation=int, required=False, default=128798), 'END_THINKING_PHASE_TOKEN': FieldInfo(annotation=int, required=False, default=128799), 'load_format': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'max_concurrency': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_draft_len': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'mtp_eagle_one_model': FieldInfo(annotation=bool, required=False, default=True), 'num_nextn_predict_layers': FieldInfo(annotation=int, required=False, default=1), 'num_nextn_predict_layers_from_model_config': FieldInfo(annotation=int, required=False, default=1), 'relaxed_delta': FieldInfo(annotation=float, required=False, default=0.0), 'relaxed_topk': FieldInfo(annotation=int, required=False, default=1), 'speculative_model_dir': FieldInfo(annotation=Union[str, Path, NoneType], required=False, default=None), 'use_mtp_vanilla': FieldInfo(annotation=bool, required=False, default=False), 'use_relaxed_acceptance_for_thinking': FieldInfo(annotation=bool, required=False, default=False)}
+model_fields = {'BEGIN_THINKING_PHASE_TOKEN': FieldInfo(annotation=int, required=False, default=128798), 'END_THINKING_PHASE_TOKEN': FieldInfo(annotation=int, required=False, default=128799), 'acceptance_length_threshold': FieldInfo(annotation=Union[float, NoneType], required=False, default=None), 'acceptance_window': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'load_format': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'max_concurrency': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_draft_len': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'mtp_eagle_one_model': FieldInfo(annotation=bool, required=False, default=True), 'num_nextn_predict_layers': FieldInfo(annotation=int, required=False, default=1), 'num_nextn_predict_layers_from_model_config': FieldInfo(annotation=int, required=False, default=1), 'relaxed_delta': FieldInfo(annotation=float, required=False, default=0.0), 'relaxed_topk': FieldInfo(annotation=int, required=False, default=1), 'speculative_model_dir': FieldInfo(annotation=Union[str, Path, NoneType], required=False, default=None), 'use_mtp_vanilla': FieldInfo(annotation=bool, required=False, default=False), 'use_relaxed_acceptance_for_thinking': FieldInfo(annotation=bool, required=False, default=False)}
@@ -13658,6 +13774,8 @@ If rebuilding _was_ required, returns True if rebuilding was succes
speculative_model_dir : str | Path | None = None max_concurrency : int | None = None load_format : str | None = None acceptance_window : int | None = None acceptance_length_threshold : float | None = None max_matching_ngram_size : int = 0 is_keep_all : bool = True is_use_oldest : bool = True
+
+
+field acceptance_length_threshold : float | None = None
+
+
+
+
+field acceptance_window : int | None = None
+
+
field is_keep_all : bool = True
@@ -14341,7 +14469,7 @@ a subset of the possible backends.
-model_fields = {'is_keep_all': FieldInfo(annotation=bool, required=False, default=True), 'is_public_pool': FieldInfo(annotation=bool, required=False, default=True), 'is_use_oldest': FieldInfo(annotation=bool, required=False, default=True), 'load_format': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'max_concurrency': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_draft_len': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_matching_ngram_size': FieldInfo(annotation=int, required=False, default=0), 'speculative_model_dir': FieldInfo(annotation=Union[str, Path, NoneType], required=False, default=None)}
+model_fields = {'acceptance_length_threshold': FieldInfo(annotation=Union[float, NoneType], required=False, default=None), 'acceptance_window': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'is_keep_all': FieldInfo(annotation=bool, required=False, default=True), 'is_public_pool': FieldInfo(annotation=bool, required=False, default=True), 'is_use_oldest': FieldInfo(annotation=bool, required=False, default=True), 'load_format': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'max_concurrency': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_draft_len': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_matching_ngram_size': FieldInfo(annotation=int, required=False, default=0), 'speculative_model_dir': FieldInfo(annotation=Union[str, Path, NoneType], required=False, default=None)}
@@ -14376,12 +14504,24 @@ a subset of the possible backends.
speculative_model_dir : str | Path | None = None max_concurrency : int | None = None load_format : str | None = None acceptance_window : int | None = None acceptance_length_threshold : float | None = None drafter : object resource_manager : object = None
) [source] Bases: DecodingBaseConfig
+
+
+field acceptance_length_threshold : float | None = None
+
+
+
+
+field acceptance_window : int | None = None
+
+
field drafter : object [Required]
@@ -15030,7 +15170,7 @@ a subset of the possible backends.
-model_fields = {'drafter': FieldInfo(annotation=object, required=True), 'load_format': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'max_concurrency': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_draft_len': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'resource_manager': FieldInfo(annotation=object, required=False, default=None), 'speculative_model_dir': FieldInfo(annotation=Union[str, Path, NoneType], required=False, default=None)}
+model_fields = {'acceptance_length_threshold': FieldInfo(annotation=Union[float, NoneType], required=False, default=None), 'acceptance_window': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'drafter': FieldInfo(annotation=object, required=True), 'load_format': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'max_concurrency': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_draft_len': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'resource_manager': FieldInfo(annotation=object, required=False, default=None), 'speculative_model_dir': FieldInfo(annotation=Union[str, Path, NoneType], required=False, default=None)}
@@ -15735,10 +15875,22 @@ If rebuilding _was_ required, returns True if rebuilding was succes
speculative_model_dir : str | Path | None = None max_concurrency : int | None = None load_format : str | None = None acceptance_window : int | None = None acceptance_length_threshold : float | None = None
) [source] Bases: DecodingBaseConfig
+
+
+field acceptance_length_threshold : float | None = None
+
+
+
+
+field acceptance_window : int | None = None
+
+
field load_format : str | None = None
@@ -16377,7 +16529,7 @@ a subset of the possible backends.
-model_fields = {'load_format': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'max_concurrency': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_draft_len': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'speculative_model_dir': FieldInfo(annotation=Union[str, Path, NoneType], required=False, default=None)}
+model_fields = {'acceptance_length_threshold': FieldInfo(annotation=Union[float, NoneType], required=False, default=None), 'acceptance_window': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'load_format': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'max_concurrency': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_draft_len': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'speculative_model_dir': FieldInfo(annotation=Union[str, Path, NoneType], required=False, default=None)}
@@ -16447,12 +16599,13 @@ a subset of the possible backends.
peft_cache_config: ~tensorrt_llm.llmapi.llm_args.PeftCacheConfig | None = None scheduler_config: ~tensorrt_llm.llmapi.llm_args.SchedulerConfig = <factory> cache_transceiver_config: ~tensorrt_llm.llmapi.llm_args.CacheTransceiverConfig | None = None speculative_config: ~tensorrt_llm.llmapi.llm_args.DraftTargetDecodingConfig | ~tensorrt_llm.llmapi.llm_args.EagleDecodingConfig | ~tensorrt_llm.llmapi.llm_args.LookaheadDecodingConfig | ~tensorrt_llm.llmapi.llm_args.MedusaDecodingConfig | ~tensorrt_llm.llmapi.llm_args.MTPDecodingConfig | ~tensorrt_llm.llmapi.llm_args.NGramDecodingConfig | ~tensorrt_llm.llmapi.llm_args.UserProvidedDecodingConfig | ~tensorrt_llm.llmapi.llm_args.AutoDecodingConfig | None = None sparse_attention_config: ~tensorrt_llm.llmapi.llm_args.RocketSparseAttentionConfig | None = None speculative_config: ~tensorrt_llm.llmapi.llm_args.DraftTargetDecodingConfig | ~tensorrt_llm.llmapi.llm_args.EagleDecodingConfig | ~tensorrt_llm.llmapi.llm_args.LookaheadDecodingConfig | ~tensorrt_llm.llmapi.llm_args.MedusaDecodingConfig | ~tensorrt_llm.llmapi.llm_args.MTPDecodingConfig | ~tensorrt_llm.llmapi.llm_args.NGramDecodingConfig | ~tensorrt_llm.llmapi.llm_args.UserProvidedDecodingConfig | ~tensorrt_llm.llmapi.llm_args.SaveHiddenStatesDecodingConfig | ~tensorrt_llm.llmapi.llm_args.AutoDecodingConfig | None = None max_batch_size: int | None = None max_input_len: int | None = None max_seq_len: int | None = None max_beam_width: int | None = None max_num_tokens: int | None = None max_num_tokens: int | None = 8192 gather_generation_logits: bool = False num_postprocess_workers: int = 0 postprocess_tokenizer_dir: str | None = None _mpi_session: object | None = None backend: str | None = None return_perf_metrics: bool = False orchestrator_type: ~typing.Literal['rpc' 'ray'] | None = None build_config: object | None = None garbage_collection_gen0_threshold: int = 20000 cuda_graph_config: ~tensorrt_llm.llmapi.llm_args.CudaGraphConfig | None = <factory>
-field max_num_tokens : int | None = None
+field max_num_tokens : int | None = 8192 stable
@@ -16799,6 +16954,12 @@ a subset of the possible backends.
prototype
+
+
+field orchestrator_type : Literal [ 'rpc' , 'ray' ] | None = None
+prototype
+
+
field peft_cache_config : PeftCacheConfig | None = None
@@ -16871,6 +17032,12 @@ a subset of the possible backends.
stable
+
+
+field sparse_attention_config : SparseAttentionConfig | None = None
+prototype
+
+
field speculative_config : SpeculativeConfig = None
@@ -17247,12 +17414,13 @@ validated to form a valid model.
peft_cache_config: ~tensorrt_llm.llmapi.llm_args.PeftCacheConfig | None = None scheduler_config: ~tensorrt_llm.llmapi.llm_args.SchedulerConfig = <factory> cache_transceiver_config: ~tensorrt_llm.llmapi.llm_args.CacheTransceiverConfig | None = None speculative_config: ~tensorrt_llm.llmapi.llm_args.DraftTargetDecodingConfig | ~tensorrt_llm.llmapi.llm_args.EagleDecodingConfig | ~tensorrt_llm.llmapi.llm_args.LookaheadDecodingConfig | ~tensorrt_llm.llmapi.llm_args.MedusaDecodingConfig | ~tensorrt_llm.llmapi.llm_args.MTPDecodingConfig | ~tensorrt_llm.llmapi.llm_args.NGramDecodingConfig | ~tensorrt_llm.llmapi.llm_args.UserProvidedDecodingConfig | ~tensorrt_llm.llmapi.llm_args.AutoDecodingConfig | None = None sparse_attention_config: ~tensorrt_llm.llmapi.llm_args.RocketSparseAttentionConfig | None = None speculative_config: ~tensorrt_llm.llmapi.llm_args.DraftTargetDecodingConfig | ~tensorrt_llm.llmapi.llm_args.EagleDecodingConfig | ~tensorrt_llm.llmapi.llm_args.LookaheadDecodingConfig | ~tensorrt_llm.llmapi.llm_args.MedusaDecodingConfig | ~tensorrt_llm.llmapi.llm_args.MTPDecodingConfig | ~tensorrt_llm.llmapi.llm_args.NGramDecodingConfig | ~tensorrt_llm.llmapi.llm_args.UserProvidedDecodingConfig | ~tensorrt_llm.llmapi.llm_args.SaveHiddenStatesDecodingConfig | ~tensorrt_llm.llmapi.llm_args.AutoDecodingConfig | None = None max_batch_size: int | None = None max_input_len: int | None = None max_seq_len: int | None = None max_beam_width: int | None = None max_num_tokens: int | None = None max_num_tokens: int | None = 8192 gather_generation_logits: bool = False num_postprocess_workers: int = 0 postprocess_tokenizer_dir: str | None = None _mpi_session: object | None = None backend: str | None = None return_perf_metrics: bool = False orchestrator_type: ~typing.Literal['rpc' 'ray'] | None = None auto_parallel: bool = False auto_parallel_world_size: int | None = None enable_tqdm: bool = False
-field max_num_tokens : int | None = None
+field max_num_tokens : int | None = 8192 The maximum number of tokens.
@@ -17520,6 +17690,12 @@ validated to form a valid model.
The number of processes used for postprocessing the generated tokens, including detokenization.
+
+
+field orchestrator_type : Literal [ 'rpc' , 'ray' ] | None = None
+The orchestrator type to use. Defaults to None, which uses MPI.
+
+
field peft_cache_config : PeftCacheConfig | None = None
@@ -17580,6 +17756,12 @@ validated to form a valid model.
Whether to skip the tokenizer initialization.
+
+
+field sparse_attention_config : SparseAttentionConfig | None = None
+Sparse attention config.
+
+
field speculative_config : SpeculativeConfig = None
@@ -17905,6 +18087,8 @@ validated to form a valid model.
speculative_model_dir : str | Path | None = None max_concurrency : int | None = None load_format : str | None = None acceptance_window : int | None = None acceptance_length_threshold : float | None = None
) [source] This config will automatically select a good, draft-model free
speculation algorithm with some heuristic.
Attributes that are inherited from the base class are ignored.
+
+
+field acceptance_length_threshold : float | None = None
+
+
+
+
+field acceptance_window : int | None = None
+
+
field load_format : str | None = None
@@ -18545,7 +18739,7 @@ a subset of the possible backends.
-model_fields = {'load_format': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'max_concurrency': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_draft_len': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'speculative_model_dir': FieldInfo(annotation=Union[str, Path, NoneType], required=False, default=None)}
+model_fields = {'acceptance_length_threshold': FieldInfo(annotation=Union[float, NoneType], required=False, default=None), 'acceptance_window': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'load_format': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'max_concurrency': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_draft_len': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'speculative_model_dir': FieldInfo(annotation=Union[str, Path, NoneType], required=False, default=None)}
@@ -19283,6 +19477,1460 @@ If rebuilding _was_ required, returns True if rebuilding was succes
+
+
+class tensorrt_llm.llmapi. SaveHiddenStatesDecodingConfig (
+
+
+* max_draft_len : int | None = None speculative_model_dir : str | Path | None = None max_concurrency : int | None = None load_format : str | None = None acceptance_window : int | None = None acceptance_length_threshold : float | None = None output_directory : str write_interval : int = 20 file_prefix : str = 'data' eagle3_layers_to_capture : Set [ int ] | None = None max_total_draft_tokens : int | None = 1 eagle_choices : List [ List [ int ] ] | None = None
+
+) [source] Bases: DecodingBaseConfig
+
+
+field acceptance_length_threshold : float | None = None
+
+
+
+
+field acceptance_window : int | None = None
+
+
+
+
+field eagle3_layers_to_capture : Set [ int ] | None = None
+
+
+
+
+field eagle_choices : List [ List [ int ] ] | None = None
+
+
+
+
+field file_prefix : str = 'data'
+
+
+
+
+field load_format : str | None = None
+
+
+
+
+field max_concurrency : int | None = None
+
+
+
+
+field max_draft_len : int | None = None
+
+
+
+
+field max_total_draft_tokens : int | None = 1
+
+
+
+
+field output_directory : str [Required]
+
+
+
+
+field speculative_model_dir : str | Path | None = None
+
+
+
+
+field write_interval : int = 20
+
+
+
+
+class Config
+Bases: object
+
+
+
+
+
+
+
+
+__init__ ( ** data : Any ) → None
+Create a new model by parsing and validating input data from keyword arguments.
+Raises [ValidationError ][pydantic_core.ValidationError] if the input data cannot be
+validated to form a valid model.
+self is explicitly positional-only to allow self as a field name.
+
+
+
+
+classmethod construct (
+
+
+_fields_set : set [ str ] | None = None ** values : Any
+
+) → Self
+
+
+
+copy (
+
+
+* include : AbstractSetIntStr | MappingIntStrAny | None = None exclude : AbstractSetIntStr | MappingIntStrAny | None = None update : Dict [ str , Any ] | None = None deep : bool = False
+
+) → Self Returns a copy of the model.
+
+!!! warning “Deprecated” This method is now deprecated; use model_copy instead.
+
+If you need include or exclude , use:
+`python {test="skip" lint="skip"}
+data = self.model_dump(include=include, exclude=exclude, round_trip=True)
+data = {**data, **(update or {})}
+copied = self.model_validate(data)
+`
+
+Parameters:
+
+include – Optional set or mapping specifying which fields to include in the copied model.
exclude – Optional set or mapping specifying which fields to exclude in the copied model.
update – Optional dictionary of field-value pairs to override field values in the copied model.
deep – If True, the values of fields that are Pydantic models will be deep-copied.
+Returns:
+A copy of the model with included, excluded and updated fields as specified.
+
+
+
+
+
+dict (
+
+
+* include : set [ int ] | set [ str ] | Mapping [ int , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | Mapping [ str , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | None = None exclude : set [ int ] | set [ str ] | Mapping [ int , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | Mapping [ str , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | None = None by_alias : bool = False exclude_unset : bool = False exclude_defaults : bool = False exclude_none : bool = False
+
+) → Dict [ str , Any ]
+
+
+
+classmethod from_dict ( data : dict ) [source]
+
+
+
+
+classmethod from_orm (
+
+
+obj : Any
+
+) → Self
+
+
+
+json (
+
+
+* include : set [ int ] | set [ str ] | Mapping [ int , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | Mapping [ str , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | None = None exclude : set [ int ] | set [ str ] | Mapping [ int , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | Mapping [ str , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | None = None by_alias : bool = False exclude_unset : bool = False exclude_defaults : bool = False exclude_none : bool = False encoder : Callable [ [ Any ] , Any ] | None = PydanticUndefined models_as_dict : bool = PydanticUndefined ** dumps_kwargs : Any
+
+) → str
+
+
+
+classmethod model_construct (
+
+
+_fields_set : set [ str ] | None = None ** values : Any
+
+) → Self Creates a new instance of the Model class with validated data.
+Creates a new model setting __dict__ and __pydantic_fields_set__ from trusted or pre-validated data.
+Default values are respected, but no other validation is performed.
+
+!!! note model_construct() generally respects the model_config.extra setting on the provided model.
+That is, if model_config.extra == ‘allow’ , then all extra passed values are added to the model instance’s __dict__
+and __pydantic_extra__ fields. If model_config.extra == ‘ignore’ (the default), then all extra passed values are ignored.
+Because no validation is performed with a call to model_construct() , having model_config.extra == ‘forbid’ does not result in
+an error if extra values are passed, but they will be ignored.
+
+
+Parameters:
+
+_fields_set – A set of field names that were originally explicitly set during instantiation. If provided,
+this is directly used for the [model_fields_set ][pydantic.BaseModel.model_fields_set] attribute.
+Otherwise, the field names from the values argument will be used.
values – Trusted or pre-validated data dictionary.
+Returns:
+A new instance of the Model class with validated data.
+
+
+
+
+
+model_copy (
+
+
+* update : Mapping [ str , Any ] | None = None deep : bool = False
+
+) → Self
+!!! abstract “Usage Documentation” [model_copy ](../concepts/serialization.md#model_copy)
+
+Returns a copy of the model.
+
+!!! note The underlying instance’s [__dict__ ][object.__dict__] attribute is copied. This
+might have unexpected side effects if you store anything in it, on top of the model
+fields (e.g. the value of [cached properties][functools.cached_property]).
+
+
+Parameters:
+
+
+Returns:
+New model instance.
+
+
+
+
+
+model_dump (
+
+
+* mode : Literal [ 'json' , 'python' ] | str = 'python' include : set [ int ] | set [ str ] | Mapping [ int , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | Mapping [ str , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | None = None exclude : set [ int ] | set [ str ] | Mapping [ int , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | Mapping [ str , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | None = None context : Any | None = None by_alias : bool | None = None exclude_unset : bool = False exclude_defaults : bool = False exclude_none : bool = False round_trip : bool = False warnings : bool | Literal [ 'none' , 'warn' , 'error' ] = True fallback : Callable [ [ Any ] , Any ] | None = None serialize_as_any : bool = False
+
+) → dict [ str , Any ]
+!!! abstract “Usage Documentation” [model_dump ](../concepts/serialization.md#modelmodel_dump)
+
+Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.
+
+Parameters:
+
+mode – The mode in which to_python should run.
+If mode is ‘json’, the output will only contain JSON serializable types.
+If mode is ‘python’, the output may contain non-JSON-serializable Python objects.
include – A set of fields to include in the output.
exclude – A set of fields to exclude from the output.
context – Additional context to pass to the serializer.
by_alias – Whether to use the field’s alias in the dictionary key if defined.
exclude_unset – Whether to exclude fields that have not been explicitly set.
exclude_defaults – Whether to exclude fields that are set to their default value.
exclude_none – Whether to exclude fields that have a value of None .
round_trip – If True, dumped values should be valid as input for non-idempotent types such as Json[T].
warnings – How to handle serialization errors. False/”none” ignores them, True/”warn” logs errors,
+“error” raises a [PydanticSerializationError ][pydantic_core.PydanticSerializationError].
fallback – A function to call when an unknown value is encountered. If not provided,
+a [PydanticSerializationError ][pydantic_core.PydanticSerializationError] error is raised.
serialize_as_any – Whether to serialize fields with duck-typing serialization behavior.
+Returns:
+A dictionary representation of the model.
+
+
+
+
+
+model_dump_json (
+
+
+* indent : int | None = None include : set [ int ] | set [ str ] | Mapping [ int , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | Mapping [ str , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | None = None exclude : set [ int ] | set [ str ] | Mapping [ int , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | Mapping [ str , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | None = None context : Any | None = None by_alias : bool | None = None exclude_unset : bool = False exclude_defaults : bool = False exclude_none : bool = False round_trip : bool = False warnings : bool | Literal [ 'none' , 'warn' , 'error' ] = True fallback : Callable [ [ Any ] , Any ] | None = None serialize_as_any : bool = False
+
+) → str
+!!! abstract “Usage Documentation” [model_dump_json ](../concepts/serialization.md#modelmodel_dump_json)
+
+Generates a JSON representation of the model using Pydantic’s to_json method.
+
+Parameters:
+
+indent – Indentation to use in the JSON output. If None is passed, the output will be compact.
include – Field(s) to include in the JSON output.
exclude – Field(s) to exclude from the JSON output.
context – Additional context to pass to the serializer.
by_alias – Whether to serialize using field aliases.
exclude_unset – Whether to exclude fields that have not been explicitly set.
exclude_defaults – Whether to exclude fields that are set to their default value.
exclude_none – Whether to exclude fields that have a value of None .
round_trip – If True, dumped values should be valid as input for non-idempotent types such as Json[T].
warnings – How to handle serialization errors. False/”none” ignores them, True/”warn” logs errors,
+“error” raises a [PydanticSerializationError ][pydantic_core.PydanticSerializationError].
fallback – A function to call when an unknown value is encountered. If not provided,
+a [PydanticSerializationError ][pydantic_core.PydanticSerializationError] error is raised.
serialize_as_any – Whether to serialize fields with duck-typing serialization behavior.
+Returns:
+A JSON string representation of the model.
+
+
+
+
+
+classmethod model_json_schema (
+
+
+by_alias: bool = True ref_template: str = '#/$defs/{model}' schema_generator: type[~pydantic.json_schema.GenerateJsonSchema] = <class 'pydantic.json_schema.GenerateJsonSchema'> mode: ~typing.Literal['validation' 'serialization'] = 'validation'
+
+) → dict [ str , Any ] Generates a JSON schema for a model class.
+
+Parameters:
+
+by_alias – Whether to use attribute aliases or not.
ref_template – The reference template.
schema_generator – To override the logic used to generate the JSON schema, as a subclass of
+GenerateJsonSchema with your desired modifications
mode – The mode in which to generate the schema.
+Returns:
+The JSON schema for the given model class.
+
+
+
+
+
+classmethod model_parametrized_name (
+
+
+params : tuple [ type [ Any ] , ... ]
+
+) → str Compute the class name for parametrizations of generic classes.
+This method can be overridden to achieve a custom naming scheme for generic BaseModels.
+
+Parameters:
+params – Tuple of types of the class. Given a generic class
+Model with 2 type variables and a concrete model Model[str, int] ,
+the value (str, int) would be passed to params .
+Returns:
+String representing the new class where params are passed to cls as type variables.
+Raises:
+TypeError – Raised when trying to generate concrete names for non-generic models.
+
+
+
+
+
+model_post_init (
+
+
+_SaveHiddenStatesDecodingConfig__context
+
+) [source] This function is meant to behave like a BaseModel method to initialise private attributes.
+It takes context as an argument since that’s what pydantic-core passes when calling it.
+
+Parameters:
+
+
+
+
+
+
+
+classmethod model_rebuild (
+
+
+* force : bool = False raise_errors : bool = True _parent_namespace_depth : int = 2 _types_namespace : MappingNamespace | None = None
+
+) → bool | None Try to rebuild the pydantic-core schema for the model.
+This may be necessary when one of the annotations is a ForwardRef which could not be resolved during
+the initial attempt to build the schema, and automatic rebuilding fails.
+
+Parameters:
+
+force – Whether to force the rebuilding of the model schema, defaults to False .
raise_errors – Whether to raise errors, defaults to True .
_parent_namespace_depth – The depth level of the parent namespace, defaults to 2.
_types_namespace – The types namespace, defaults to None .
+Returns:
+Returns None if the schema is already “complete” and rebuilding was not required.
+If rebuilding _was_ required, returns True if rebuilding was successful, otherwise False .
+
+
+
+
+
+classmethod model_validate (
+
+
+obj : Any * strict : bool | None = None from_attributes : bool | None = None context : Any | None = None by_alias : bool | None = None by_name : bool | None = None
+
+) → Self Validate a pydantic model instance.
+
+Parameters:
+
+obj – The object to validate.
strict – Whether to enforce types strictly.
from_attributes – Whether to extract data from object attributes.
context – Additional context to pass to the validator.
by_alias – Whether to use the field’s alias when validating against the provided input data.
by_name – Whether to use the field’s name when validating against the provided input data.
+Raises:
+ValidationError – If the object could not be validated.
+Returns:
+The validated model instance.
+
+
+
+
+
+classmethod model_validate_json (
+
+
+json_data : str | bytes | bytearray * strict : bool | None = None context : Any | None = None by_alias : bool | None = None by_name : bool | None = None
+
+) → Self
+!!! abstract “Usage Documentation” [JSON Parsing](../concepts/json.md#json-parsing)
+
+Validate the given JSON data against the Pydantic model.
+
+Parameters:
+
+json_data – The JSON data to validate.
strict – Whether to enforce types strictly.
context – Extra variables to pass to the validator.
by_alias – Whether to use the field’s alias when validating against the provided input data.
by_name – Whether to use the field’s name when validating against the provided input data.
+Returns:
+The validated Pydantic model.
+Raises:
+ValidationError – If json_data is not a JSON string or the object could not be validated.
+
+
+
+
+
+classmethod model_validate_strings (
+
+
+obj : Any * strict : bool | None = None context : Any | None = None by_alias : bool | None = None by_name : bool | None = None
+
+) → Self Validate the given object with string data against the Pydantic model.
+
+Parameters:
+
+obj – The object containing string data to validate.
strict – Whether to enforce types strictly.
context – Extra variables to pass to the validator.
by_alias – Whether to use the field’s alias when validating against the provided input data.
by_name – Whether to use the field’s name when validating against the provided input data.
+Returns:
+The validated Pydantic model.
+
+
+
+
+
+classmethod parse_file (
+
+
+path : str | Path * content_type : str | None = None encoding : str = 'utf8' proto : DeprecatedParseProtocol | None = None allow_pickle : bool = False
+
+) → Self
+
+
+
+classmethod parse_obj (
+
+
+obj : Any
+
+) → Self
+
+
+
+classmethod parse_raw (
+
+
+b : str | bytes * content_type : str | None = None encoding : str = 'utf8' proto : DeprecatedParseProtocol | None = None allow_pickle : bool = False
+
+) → Self
+
+
+
+classmethod schema (
+
+
+by_alias : bool = True ref_template : str = '#/$defs/{model}'
+
+) → Dict [ str , Any ]
+
+
+
+classmethod schema_json (
+
+
+* by_alias : bool = True ref_template : str = '#/$defs/{model}' ** dumps_kwargs : Any
+
+) → str
+
+
+
+supports_backend ( backend : str ) → bool
+Override if the speculation algorithm does not support
+a subset of the possible backends.
+
+
+
+
+classmethod update_forward_refs (
+
+
+** localns : Any
+
+) → None
+
+
+
+validate ( ) → None [source]
+Do any additional error checking here.
+
+
+
+
+decoding_type : ClassVar [ str ] = 'SaveState'
+
+
+
+
+model_computed_fields = {}
+
+
+
+
+model_config : ClassVar [ ConfigDict ] = {'extra': 'forbid'}
+Configuration for the model, should be a dictionary conforming to [ConfigDict ][pydantic.config.ConfigDict].
+
+
+
+
+Get extra fields set during validation.
+
+Returns:
+A dictionary of extra fields, or None if config.extra is not set to “allow” .
+
+
+
+
+
+model_fields = {'acceptance_length_threshold': FieldInfo(annotation=Union[float, NoneType], required=False, default=None), 'acceptance_window': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'eagle3_layers_to_capture': FieldInfo(annotation=Union[Set[int], NoneType], required=False, default=None), 'eagle_choices': FieldInfo(annotation=Union[List[List[int]], NoneType], required=False, default=None, init=False), 'file_prefix': FieldInfo(annotation=str, required=False, default='data'), 'load_format': FieldInfo(annotation=Union[str, NoneType], required=False, default=None), 'max_concurrency': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_draft_len': FieldInfo(annotation=Union[int, NoneType], required=False, default=None), 'max_total_draft_tokens': FieldInfo(annotation=Union[int, NoneType], required=False, default=1, init=False), 'output_directory': FieldInfo(annotation=str, required=True), 'speculative_model_dir': FieldInfo(annotation=Union[str, Path, NoneType], required=False, default=None), 'write_interval': FieldInfo(annotation=int, required=False, default=20)}
+
+
+
+
+property model_fields_set : set [ str ]
+Returns the set of fields that have been explicitly set on this model instance.
+
+Returns:
+
+A set of strings representing the fields that have been set, i.e. that were not filled from defaults.
+
+
+
+
+
+
+
+property num_capture_layers
+Returns the number of layers to capture of the target model.
+If eagle3_layers_to_capture is not None, return the length of the set.
+Otherwise, assume Eagle3 base set and return 3 + 1 (for post norm last hidden state).
+
+
+
+
+property spec_dec_mode
+
+
+
+
+
+
+class tensorrt_llm.llmapi. RocketSparseAttentionConfig (
+
+
+* algorithm : Literal [ 'rocket' ] = 'rocket' window_size : int | None = None kernel_size : int | None = None topr : int | float | None = 76 topk : int | None = 128 prompt_budget : int | None = 1266 page_size : int | None = 3
+
+) [source] Bases: BaseSparseAttentionConfig
+Configuration for rocket sparse attention.
+
+
+field algorithm : Literal [ 'rocket' ] = 'rocket'
+The algorithm for sparse attention.
+
+
+
+
+field kernel_size : int | None = None
+The kernel size for snap KV.
+
+
+
+
+field page_size : int | None = 3
+Page size
+
+
+
+
+field prompt_budget : int | None = 1266
+Prompt budget
+
+
+
+
+field topk : int | None = 128
+Top-k
+
+
+
+
+field topr : int | float | None = 76
+Top-r
+
+
+
+
+field window_size : int | None = None
+The window size for snap KV.
+
+
+
+
+class Config
+Bases: object
+
+
+
+
+
+
+
+
+__init__ ( ** data : Any ) → None
+Create a new model by parsing and validating input data from keyword arguments.
+Raises [ValidationError ][pydantic_core.ValidationError] if the input data cannot be
+validated to form a valid model.
+self is explicitly positional-only to allow self as a field name.
+
+
+
+
+classmethod construct (
+
+
+_fields_set : set [ str ] | None = None ** values : Any
+
+) → Self
+
+
+
+copy (
+
+
+* include : AbstractSetIntStr | MappingIntStrAny | None = None exclude : AbstractSetIntStr | MappingIntStrAny | None = None update : Dict [ str , Any ] | None = None deep : bool = False
+
+) → Self Returns a copy of the model.
+
+!!! warning “Deprecated” This method is now deprecated; use model_copy instead.
+
+If you need include or exclude , use:
+`python {test="skip" lint="skip"}
+data = self.model_dump(include=include, exclude=exclude, round_trip=True)
+data = {**data, **(update or {})}
+copied = self.model_validate(data)
+`
+
+Parameters:
+
+include – Optional set or mapping specifying which fields to include in the copied model.
exclude – Optional set or mapping specifying which fields to exclude in the copied model.
update – Optional dictionary of field-value pairs to override field values in the copied model.
deep – If True, the values of fields that are Pydantic models will be deep-copied.
+Returns:
+A copy of the model with included, excluded and updated fields as specified.
+
+
+
+
+
+dict (
+
+
+* include : set [ int ] | set [ str ] | Mapping [ int , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | Mapping [ str , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | None = None exclude : set [ int ] | set [ str ] | Mapping [ int , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | Mapping [ str , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | None = None by_alias : bool = False exclude_unset : bool = False exclude_defaults : bool = False exclude_none : bool = False
+
+) → Dict [ str , Any ]
+
+
+
+classmethod from_dict ( data : dict ) [source]
+
+
+
+
+classmethod from_orm ( obj : Any ) → Self
+
+
+
+
+json (
+
+
+* include : set [ int ] | set [ str ] | Mapping [ int , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | Mapping [ str , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | None = None exclude : set [ int ] | set [ str ] | Mapping [ int , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | Mapping [ str , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | None = None by_alias : bool = False exclude_unset : bool = False exclude_defaults : bool = False exclude_none : bool = False encoder : Callable [ [ Any ] , Any ] | None = PydanticUndefined models_as_dict : bool = PydanticUndefined ** dumps_kwargs : Any
+
+) → str
+
+
+
+classmethod model_construct (
+
+
+_fields_set : set [ str ] | None = None ** values : Any
+
+) → Self Creates a new instance of the Model class with validated data.
+Creates a new model setting __dict__ and __pydantic_fields_set__ from trusted or pre-validated data.
+Default values are respected, but no other validation is performed.
+
+!!! note model_construct() generally respects the model_config.extra setting on the provided model.
+That is, if model_config.extra == ‘allow’ , then all extra passed values are added to the model instance’s __dict__
+and __pydantic_extra__ fields. If model_config.extra == ‘ignore’ (the default), then all extra passed values are ignored.
+Because no validation is performed with a call to model_construct() , having model_config.extra == ‘forbid’ does not result in
+an error if extra values are passed, but they will be ignored.
+
+
+Parameters:
+
+_fields_set – A set of field names that were originally explicitly set during instantiation. If provided,
+this is directly used for the [model_fields_set ][pydantic.BaseModel.model_fields_set] attribute.
+Otherwise, the field names from the values argument will be used.
values – Trusted or pre-validated data dictionary.
+Returns:
+A new instance of the Model class with validated data.
+
+
+
+
+
+model_copy (
+
+
+* update : Mapping [ str , Any ] | None = None deep : bool = False
+
+) → Self
+!!! abstract “Usage Documentation” [model_copy ](../concepts/serialization.md#model_copy)
+
+Returns a copy of the model.
+
+!!! note The underlying instance’s [__dict__ ][object.__dict__] attribute is copied. This
+might have unexpected side effects if you store anything in it, on top of the model
+fields (e.g. the value of [cached properties][functools.cached_property]).
+
+
+Parameters:
+
+
+Returns:
+New model instance.
+
+
+
+
+
+model_dump (
+
+
+* mode : Literal [ 'json' , 'python' ] | str = 'python' include : set [ int ] | set [ str ] | Mapping [ int , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | Mapping [ str , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | None = None exclude : set [ int ] | set [ str ] | Mapping [ int , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | Mapping [ str , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | None = None context : Any | None = None by_alias : bool | None = None exclude_unset : bool = False exclude_defaults : bool = False exclude_none : bool = False round_trip : bool = False warnings : bool | Literal [ 'none' , 'warn' , 'error' ] = True fallback : Callable [ [ Any ] , Any ] | None = None serialize_as_any : bool = False
+
+) → dict [ str , Any ]
+!!! abstract “Usage Documentation” [model_dump ](../concepts/serialization.md#modelmodel_dump)
+
+Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.
+
+Parameters:
+
+mode – The mode in which to_python should run.
+If mode is ‘json’, the output will only contain JSON serializable types.
+If mode is ‘python’, the output may contain non-JSON-serializable Python objects.
include – A set of fields to include in the output.
exclude – A set of fields to exclude from the output.
context – Additional context to pass to the serializer.
by_alias – Whether to use the field’s alias in the dictionary key if defined.
exclude_unset – Whether to exclude fields that have not been explicitly set.
exclude_defaults – Whether to exclude fields that are set to their default value.
exclude_none – Whether to exclude fields that have a value of None .
round_trip – If True, dumped values should be valid as input for non-idempotent types such as Json[T].
warnings – How to handle serialization errors. False/”none” ignores them, True/”warn” logs errors,
+“error” raises a [PydanticSerializationError ][pydantic_core.PydanticSerializationError].
fallback – A function to call when an unknown value is encountered. If not provided,
+a [PydanticSerializationError ][pydantic_core.PydanticSerializationError] error is raised.
serialize_as_any – Whether to serialize fields with duck-typing serialization behavior.
+Returns:
+A dictionary representation of the model.
+
+
+
+
+
+model_dump_json (
+
+
+* indent : int | None = None include : set [ int ] | set [ str ] | Mapping [ int , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | Mapping [ str , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | None = None exclude : set [ int ] | set [ str ] | Mapping [ int , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | Mapping [ str , set [ int ] | set [ str ] | Mapping [ int , IncEx | bool ] | Mapping [ str , IncEx | bool ] | bool ] | None = None context : Any | None = None by_alias : bool | None = None exclude_unset : bool = False exclude_defaults : bool = False exclude_none : bool = False round_trip : bool = False warnings : bool | Literal [ 'none' , 'warn' , 'error' ] = True fallback : Callable [ [ Any ] , Any ] | None = None serialize_as_any : bool = False
+
+) → str
+!!! abstract “Usage Documentation” [model_dump_json ](../concepts/serialization.md#modelmodel_dump_json)
+
+Generates a JSON representation of the model using Pydantic’s to_json method.
+
+Parameters:
+
+indent – Indentation to use in the JSON output. If None is passed, the output will be compact.
include – Field(s) to include in the JSON output.
exclude – Field(s) to exclude from the JSON output.
context – Additional context to pass to the serializer.
by_alias – Whether to serialize using field aliases.
exclude_unset – Whether to exclude fields that have not been explicitly set.
exclude_defaults – Whether to exclude fields that are set to their default value.
exclude_none – Whether to exclude fields that have a value of None .
round_trip – If True, dumped values should be valid as input for non-idempotent types such as Json[T].
warnings – How to handle serialization errors. False/”none” ignores them, True/”warn” logs errors,
+“error” raises a [PydanticSerializationError ][pydantic_core.PydanticSerializationError].
fallback – A function to call when an unknown value is encountered. If not provided,
+a [PydanticSerializationError ][pydantic_core.PydanticSerializationError] error is raised.
serialize_as_any – Whether to serialize fields with duck-typing serialization behavior.
+Returns:
+A JSON string representation of the model.
+
+
+
+
+
+classmethod model_json_schema (
+
+
+by_alias: bool = True ref_template: str = '#/$defs/{model}' schema_generator: type[~pydantic.json_schema.GenerateJsonSchema] = <class 'pydantic.json_schema.GenerateJsonSchema'> mode: ~typing.Literal['validation' 'serialization'] = 'validation'
+
+) → dict [ str , Any ] Generates a JSON schema for a model class.
+
+Parameters:
+
+by_alias – Whether to use attribute aliases or not.
ref_template – The reference template.
schema_generator – To override the logic used to generate the JSON schema, as a subclass of
+GenerateJsonSchema with your desired modifications
mode – The mode in which to generate the schema.
+Returns:
+The JSON schema for the given model class.
+
+
+
+
+
+classmethod model_parametrized_name (
+
+
+params : tuple [ type [ Any ] , ... ]
+
+) → str Compute the class name for parametrizations of generic classes.
+This method can be overridden to achieve a custom naming scheme for generic BaseModels.
+
+Parameters:
+params – Tuple of types of the class. Given a generic class
+Model with 2 type variables and a concrete model Model[str, int] ,
+the value (str, int) would be passed to params .
+Returns:
+String representing the new class where params are passed to cls as type variables.
+Raises:
+TypeError – Raised when trying to generate concrete names for non-generic models.
+
+
+
+
+
+model_post_init (
+
+
+context : Any /
+
+) → None Override this method to perform additional initialization after __init__ and model_construct .
+This is useful if you want to do some validation that requires the entire model to be initialized.
+
+
+
+
+classmethod model_rebuild (
+
+
+* force : bool = False raise_errors : bool = True _parent_namespace_depth : int = 2 _types_namespace : MappingNamespace | None = None
+
+) → bool | None Try to rebuild the pydantic-core schema for the model.
+This may be necessary when one of the annotations is a ForwardRef which could not be resolved during
+the initial attempt to build the schema, and automatic rebuilding fails.
+
+Parameters:
+
+force – Whether to force the rebuilding of the model schema, defaults to False .
raise_errors – Whether to raise errors, defaults to True .
_parent_namespace_depth – The depth level of the parent namespace, defaults to 2.
_types_namespace – The types namespace, defaults to None .
+Returns:
+Returns None if the schema is already “complete” and rebuilding was not required.
+If rebuilding _was_ required, returns True if rebuilding was successful, otherwise False .
+
+
+
+
+
+classmethod model_validate (
+
+
+obj : Any * strict : bool | None = None from_attributes : bool | None = None context : Any | None = None by_alias : bool | None = None by_name : bool | None = None
+
+) → Self Validate a pydantic model instance.
+
+Parameters:
+
+obj – The object to validate.
strict – Whether to enforce types strictly.
from_attributes – Whether to extract data from object attributes.
context – Additional context to pass to the validator.
by_alias – Whether to use the field’s alias when validating against the provided input data.
by_name – Whether to use the field’s name when validating against the provided input data.
+Raises:
+ValidationError – If the object could not be validated.
+Returns:
+The validated model instance.
+
+
+
+
+
+classmethod model_validate_json (
+
+
+json_data : str | bytes | bytearray * strict : bool | None = None context : Any | None = None by_alias : bool | None = None by_name : bool | None = None
+
+) → Self
+!!! abstract “Usage Documentation” [JSON Parsing](../concepts/json.md#json-parsing)
+
+Validate the given JSON data against the Pydantic model.
+
+Parameters:
+
+json_data – The JSON data to validate.
strict – Whether to enforce types strictly.
context – Extra variables to pass to the validator.
by_alias – Whether to use the field’s alias when validating against the provided input data.
by_name – Whether to use the field’s name when validating against the provided input data.
+Returns:
+The validated Pydantic model.
+Raises:
+ValidationError – If json_data is not a JSON string or the object could not be validated.
+
+
+
+
+
+classmethod model_validate_strings (
+
+
+obj : Any * strict : bool | None = None context : Any | None = None by_alias : bool | None = None by_name : bool | None = None
+
+) → Self Validate the given object with string data against the Pydantic model.
+
+Parameters:
+
+obj – The object containing string data to validate.
strict – Whether to enforce types strictly.
context – Extra variables to pass to the validator.
by_alias – Whether to use the field’s alias when validating against the provided input data.
by_name – Whether to use the field’s name when validating against the provided input data.
+Returns:
+The validated Pydantic model.
+
+
+
+
+
+classmethod parse_file (
+
+
+path : str | Path * content_type : str | None = None encoding : str = 'utf8' proto : DeprecatedParseProtocol | None = None allow_pickle : bool = False
+
+) → Self
+
+
+
+classmethod parse_obj (
+
+
+obj : Any
+
+) → Self
+
+
+
+classmethod parse_raw (
+
+
+b : str | bytes * content_type : str | None = None encoding : str = 'utf8' proto : DeprecatedParseProtocol | None = None allow_pickle : bool = False
+
+) → Self
+
+
+
+classmethod schema (
+
+
+by_alias : bool = True ref_template : str = '#/$defs/{model}'
+
+) → Dict [ str , Any ]
+
+
+
+classmethod schema_json (
+
+
+* by_alias : bool = True ref_template : str = '#/$defs/{model}' ** dumps_kwargs : Any
+
+) → str
+
+
+
+supports_backend ( backend : str ) → bool [source]
+Override if the speculation algorithm does not support
+a subset of the possible backends.
+
+
+
+
+classmethod update_forward_refs (
+
+
+** localns : Any
+
+) → None
+
+
+
+classmethod validate (
+
+
+value : Any
+
+) → Self
+
+
+
+model_computed_fields = {}
+
+
+
+
+model_config : ClassVar [ ConfigDict ] = {'extra': 'forbid'}
+Configuration for the model, should be a dictionary conforming to [ConfigDict ][pydantic.config.ConfigDict].
+
+
+
+
+Get extra fields set during validation.
+
+Returns:
+A dictionary of extra fields, or None if config.extra is not set to “allow” .
+
+
+
+
+
+model_fields = {'algorithm': FieldInfo(annotation=Literal['rocket'], required=False, default='rocket', description='The algorithm for sparse attention.'), 'kernel_size': FieldInfo(annotation=Union[int, NoneType], required=False, default=None, description='The kernel size for snap KV.'), 'page_size': FieldInfo(annotation=Union[int, NoneType], required=False, default=3, description='Page size'), 'prompt_budget': FieldInfo(annotation=Union[int, NoneType], required=False, default=1266, description='Prompt budget'), 'topk': FieldInfo(annotation=Union[int, NoneType], required=False, default=128, description='Top-k'), 'topr': FieldInfo(annotation=Union[int, float, NoneType], required=False, default=76, description='Top-r'), 'window_size': FieldInfo(annotation=Union[int, NoneType], required=False, default=None, description='The window size for snap KV.')}
+
+
+
+
+property model_fields_set : set [ str ]
+Returns the set of fields that have been explicitly set on this model instance.
+
+Returns:
+
+A set of strings representing the fields that have been set, i.e. that were not filled from defaults.
+
+
+
+
+
+
+
@@ -19368,6 +21016,8 @@ If rebuilding _was_ required, returns True if rebuilding was succes
logprobs_diff text_diff __init__() additional_context_outputs additional_generation_outputs cumulative_logprob disaggregated_params finish_reason True if rebuilding was succes
count() index() client_id disaggregated_params error is_final metrics request_perf_metrics res True if rebuilding was succes
SamplingParams LookaheadDecodingConfig MedusaDecodingConfig EagleDecodingConfig
+acceptance_length_threshold acceptance_window dynamic_tree_max_topK eagle3_layers_to_capture eagle3_one_model True if rebuilding was succes
MTPDecodingConfig NGramDecodingConfig UserProvidedDecodingConfig DraftTargetDecodingConfig
+acceptance_length_threshold acceptance_window load_format max_concurrency max_draft_len True if rebuilding was succes
moe_tensor_parallel_size mpi_session num_postprocess_workers orchestrator_type peft_cache_config perf_metrics_max_requests pipeline_parallel_size True if rebuilding was succes
sampler_type scheduler_config skip_tokenizer_init sparse_attention_config speculative_config stream_interval tensor_parallel_size True if rebuilding was succes
mpi_session normalize_log_probs num_postprocess_workers orchestrator_type peft_cache_config pipeline_parallel_size postprocess_tokenizer_dir True if rebuilding was succes
revision scheduler_config skip_tokenizer_init sparse_attention_config speculative_config tensor_parallel_size tokenizer True if rebuilding was succes
AutoDecodingConfig SaveHiddenStatesDecodingConfig RocketSparseAttentionConfig
@@ -21040,9 +22811,9 @@ If rebuilding _was_ required, returns True if rebuilding was succes
diff --git a/latest/models/adding-new-model.html b/latest/models/adding-new-model.html
index af19d8fdf8..96b5e27917 100644
--- a/latest/models/adding-new-model.html
+++ b/latest/models/adding-new-model.html
@@ -59,7 +59,7 @@
@@ -71,7 +71,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -425,6 +428,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -409,6 +411,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
+Qwen3NextForCausalLM
Qwen3Next
Qwen/Qwen3-Next-80B-A3B-Thinking
@@ -640,7 +648,23 @@
Yes
Yes
-Llama4ForConditionalGeneration
Qwen3NextForCausalLM
Yes
Yes
No
Untested
Yes
No
No
No
Yes
Yes
No
No
Untested
Untested
Llama4ForConditionalGeneration
Yes
Yes
Yes
Yes
Yes
GPT-OSS
GPT-OSS
Yes
Yes
Yes
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -405,6 +407,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -421,6 +424,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -409,6 +411,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -425,6 +428,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers This is the starting point to try out TensorRT LLM. Specifically, this Quick Start Guide enables you to quickly get set up and send HTTP requests using TensorRT LLM.
Launch Docker on a node with NVIDIA GPUs deployed
- run --rm -it --ipc host --gpus all --ulimit memlock = -1 --ulimit stack = 67108864 -p 8000 :8000 nvcr.io/nvidia/tensorrt-llm/release:1.2.0rc0
+ run --rm -it --ipc host --gpus all --ulimit memlock = -1 --ulimit stack = 67108864 -p 8000 :8000 nvcr.io/nvidia/tensorrt-llm/release:1.2.0rc1
diff --git a/latest/release-notes.html b/latest/release-notes.html
index 831fe331ba..8a29b45b39 100644
--- a/latest/release-notes.html
+++ b/latest/release-notes.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
-
+
@@ -335,6 +335,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters
+
Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading
@@ -365,6 +366,7 @@
Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware
+
Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -411,6 +413,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -427,6 +430,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers (c++ struct)": [[0, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsIbEE", false]], "tensorrt_llm::executor::typetraits::value (c++ member)": [[0, "_CPPv4N12tensorrt_llm8executor10TypeTraitsIbE5valueE", false]], "tensorrt_llm::executor::typetraits (c++ struct)": [[0, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsIfEE", false]], "tensorrt_llm::executor::typetraits::value (c++ member)": [[0, "_CPPv4N12tensorrt_llm8executor10TypeTraitsIfE5valueE", false]], "tensorrt_llm::executor::typetraits (c++ struct)": [[0, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsI4halfEE", false]], "tensorrt_llm::executor::typetraits::value (c++ member)": [[0, "_CPPv4N12tensorrt_llm8executor10TypeTraitsI4halfE5valueE", false]], "tensorrt_llm::executor::typetraits (c++ struct)": [[0, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsINSt7int32_tEEE", false]], "tensorrt_llm::executor::typetraits::value (c++ member)": [[0, "_CPPv4N12tensorrt_llm8executor10TypeTraitsINSt7int32_tEE5valueE", false]], "tensorrt_llm::executor::typetraits (c++ struct)": [[0, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsINSt7int64_tEEE", false]], "tensorrt_llm::executor::typetraits::value (c++ member)": [[0, "_CPPv4N12tensorrt_llm8executor10TypeTraitsINSt7int64_tEE5valueE", false]], "tensorrt_llm::executor::typetraits (c++ struct)": [[0, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsINSt6int8_tEEE", false]], "tensorrt_llm::executor::typetraits::value (c++ member)": [[0, "_CPPv4N12tensorrt_llm8executor10TypeTraitsINSt6int8_tEE5valueE", false]], "tensorrt_llm::executor::typetraits (c++ struct)": [[0, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsINSt7uint8_tEEE", false]], "tensorrt_llm::executor::typetraits::value (c++ member)": [[0, "_CPPv4N12tensorrt_llm8executor10TypeTraitsINSt7uint8_tEE5valueE", false]], "tensorrt_llm::executor::typetraits (c++ struct)": [[0, "_CPPv4I0EN12tensorrt_llm8executor10TypeTraitsIP1TEE", false]], "tensorrt_llm::executor::typetraits::value (c++ member)": [[0, "_CPPv4N12tensorrt_llm8executor10TypeTraitsIP1TE5valueE", false]], "tensorrt_llm::executor::veclogprobs (c++ type)": [[0, "_CPPv4N12tensorrt_llm8executor11VecLogProbsE", false]], "tensorrt_llm::executor::vectokenextraids (c++ type)": [[0, "_CPPv4N12tensorrt_llm8executor16VecTokenExtraIdsE", false]], "tensorrt_llm::executor::vectokens (c++ type)": [[0, "_CPPv4N12tensorrt_llm8executor9VecTokensE", false]], "tensorrt_llm::executor::version (c++ function)": [[0, "_CPPv4N12tensorrt_llm8executor7versionEv", false]], "tensorrt_llm::layers (c++ type)": [[1, "_CPPv4N12tensorrt_llm6layersE", false]], "tensorrt_llm::mpi (c++ type)": [[0, "_CPPv4N12tensorrt_llm3mpiE", false]], "tensorrt_llm::runtime (c++ type)": [[0, "_CPPv4N12tensorrt_llm7runtimeE", false], [1, "_CPPv4N12tensorrt_llm7runtimeE", false]], "tensorrt_llm::runtime::allreducebuffers (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffersE", false]], "tensorrt_llm::runtime::allreducebuffers::allreducebuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers16AllReduceBuffersE10SizeType3210SizeType3210SizeType3210SizeType32RK13BufferManagerRK11WorldConfigKb", false]], "tensorrt_llm::runtime::allreducebuffers::mallreducecommptrs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers18mAllReduceCommPtrsE", false]], "tensorrt_llm::runtime::allreducebuffers::mflagptrs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers9mFlagPtrsE", false]], "tensorrt_llm::runtime::allreducebuffers::mipcmemoryhandles (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers17mIpcMemoryHandlesE", false]], "tensorrt_llm::runtime::allreducebuffers::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers9TensorPtrE", false]], "tensorrt_llm::runtime::buffercast (c++ function)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime10bufferCastEP1TR7IBuffer", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime10bufferCastEPK1TRK7IBuffer", false]], "tensorrt_llm::runtime::buffercastornull (c++ function)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKN7IBuffer9SharedPtrE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKN7ITensor9SharedPtrE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKNSt8optionalIN7IBuffer9SharedPtrEEE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKNSt8optionalIN7ITensor9SharedPtrEEE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKN7IBuffer14SharedConstPtrE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKN7ITensor14SharedConstPtrE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKNSt8optionalIN7IBuffer14SharedConstPtrEEE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKNSt8optionalIN7ITensor14SharedConstPtrEEE", false]], "tensorrt_llm::runtime::bufferdatatype (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime14BufferDataTypeE", false]], "tensorrt_llm::runtime::bufferdatatype::bufferdatatype (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType14BufferDataTypeEN8nvinfer18DataTypeEbb", false]], "tensorrt_llm::runtime::bufferdatatype::getdatatype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType11getDataTypeEv", false]], "tensorrt_llm::runtime::bufferdatatype::getsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType7getSizeEv", false]], "tensorrt_llm::runtime::bufferdatatype::getsizeinbits (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType13getSizeInBitsEv", false]], "tensorrt_llm::runtime::bufferdatatype::ispointer (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType9isPointerEv", false]], "tensorrt_llm::runtime::bufferdatatype::isunsigned (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType10isUnsignedEv", false]], "tensorrt_llm::runtime::bufferdatatype::ktrtpointertype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType15kTrtPointerTypeE", false]], "tensorrt_llm::runtime::bufferdatatype::mdatatype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType9mDataTypeE", false]], "tensorrt_llm::runtime::bufferdatatype::mpointer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType8mPointerE", false]], "tensorrt_llm::runtime::bufferdatatype::munsigned (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType9mUnsignedE", false]], "tensorrt_llm::runtime::bufferdatatype::operator nvinfer1::datatype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataTypecvN8nvinfer18DataTypeEEv", false]], "tensorrt_llm::runtime::buffermanager (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManagerE", false]], "tensorrt_llm::runtime::buffermanager::allocate (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeN8nvinfer14DimsEN8nvinfer18DataTypeE", false], [1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeNSt6size_tEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::buffermanager (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager13BufferManagerE13CudaStreamPtrb", false]], "tensorrt_llm::runtime::buffermanager::copy (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyEPKvR7IBuffer", false], [1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyEPKvR7IBuffer10MemoryType", false], [1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferPv", false], [1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferPv10MemoryType", false], [1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferR7IBuffer", false]], "tensorrt_llm::runtime::buffermanager::copyfrom (c++ function)": [[1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10IBufferPtrRKNSt6vectorI1TEE10MemoryType", false], [1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrP1TN8nvinfer14DimsE10MemoryType", false], [1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrRKNSt6vectorI1TEEN8nvinfer14DimsE10MemoryType", false], [1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8copyFromERK7IBuffer10MemoryType", false], [1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8copyFromERK7ITensor10MemoryType", false]], "tensorrt_llm::runtime::buffermanager::cpu (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager3cpuEN8nvinfer14DimsEN8nvinfer18DataTypeE", false], [1, "_CPPv4N12tensorrt_llm7runtime13BufferManager3cpuENSt6size_tEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::cudamempoolptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager14CudaMemPoolPtrE", false]], "tensorrt_llm::runtime::buffermanager::cudastreamptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager13CudaStreamPtrE", false]], "tensorrt_llm::runtime::buffermanager::emptybuffer (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager11emptyBufferE10MemoryTypeN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::emptytensor (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager11emptyTensorE10MemoryTypeN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::getstream (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager9getStreamEv", false]], "tensorrt_llm::runtime::buffermanager::gpu (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager3gpuEN8nvinfer14DimsEN8nvinfer18DataTypeE", false], [1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager3gpuENSt6size_tEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::gpusync (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7gpuSyncEN8nvinfer14DimsEN8nvinfer18DataTypeE", false], [1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7gpuSyncENSt6size_tEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::ibufferptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10IBufferPtrE", false]], "tensorrt_llm::runtime::buffermanager::ipcnvls (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7ipcNvlsENSt3setIiEEN8nvinfer14DimsEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::itensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10ITensorPtrE", false]], "tensorrt_llm::runtime::buffermanager::kbyte_type (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10kBYTE_TYPEE", false]], "tensorrt_llm::runtime::buffermanager::managed (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7managedEN8nvinfer14DimsEN8nvinfer18DataTypeE", false], [1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7managedENSt6size_tEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::memorypoolfree (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager14memoryPoolFreeEv", false]], "tensorrt_llm::runtime::buffermanager::memorypoolreserved (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager18memoryPoolReservedEv", false]], "tensorrt_llm::runtime::buffermanager::memorypooltrimto (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager16memoryPoolTrimToENSt6size_tE", false]], "tensorrt_llm::runtime::buffermanager::memorypoolused (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager14memoryPoolUsedEv", false]], "tensorrt_llm::runtime::buffermanager::mpool (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager5mPoolE", false]], "tensorrt_llm::runtime::buffermanager::mstream (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7mStreamE", false]], "tensorrt_llm::runtime::buffermanager::mtrimpool (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager9mTrimPoolE", false]], "tensorrt_llm::runtime::buffermanager::pinned (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager6pinnedEN8nvinfer14DimsEN8nvinfer18DataTypeE", false], [1, "_CPPv4N12tensorrt_llm7runtime13BufferManager6pinnedENSt6size_tEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::pinnedpool (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10pinnedPoolEN8nvinfer14DimsEN8nvinfer18DataTypeE", false], [1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10pinnedPoolENSt6size_tEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::setmem (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager6setMemER7IBuffer7int32_t", false]], "tensorrt_llm::runtime::buffermanager::setzero (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager7setZeroER7IBuffer", false]], "tensorrt_llm::runtime::buffermanager::~buffermanager (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManagerD0Ev", false]], "tensorrt_llm::runtime::bufferrange (c++ class)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime11BufferRangeE", false]], "tensorrt_llm::runtime::bufferrange::base (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime11BufferRange4BaseE", false]], "tensorrt_llm::runtime::bufferrange::bufferrange (c++ function)": [[1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI1UEEbEEEN12tensorrt_llm7runtime11BufferRange11BufferRangeERK7IBuffer", false], [1, "_CPPv4I0_NSt11enable_if_tIXntNSt10is_const_vI1UEEEbEEEN12tensorrt_llm7runtime11BufferRange11BufferRangeER7IBuffer", false], [1, "_CPPv4N12tensorrt_llm7runtime11BufferRange11BufferRangeEP1T9size_type", false]], "tensorrt_llm::runtime::cachesaltidtype (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime15CacheSaltIDTypeE", false]], "tensorrt_llm::runtime::canaccesspeer (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13canAccessPeerERK11WorldConfig", false]], "tensorrt_llm::runtime::clearvirtualmemoryallocator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime27clearVirtualMemoryAllocatorEv", false]], "tensorrt_llm::runtime::constpointercast (c++ function)": [[1, "_CPPv4I00EN12tensorrt_llm7runtime16constPointerCastENSt10shared_ptrINSt14remove_const_tI1TEEEERRNSt10unique_ptrI1T1DEE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime16constPointerCastENSt10shared_ptrINSt14remove_const_tI1TEEEERKNSt10shared_ptrI1TEE", false]], "tensorrt_llm::runtime::cudaevent (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime9CudaEventE", false]], "tensorrt_llm::runtime::cudaevent::cudaevent (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent9CudaEventE7pointerb", false], [1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent9CudaEventEj", false]], "tensorrt_llm::runtime::cudaevent::deleter (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7DeleterE", false]], "tensorrt_llm::runtime::cudaevent::deleter::deleter (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7Deleter7DeleterEb", false], [1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7Deleter7DeleterEv", false]], "tensorrt_llm::runtime::cudaevent::deleter::mownsevent (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7Deleter10mOwnsEventE", false]], "tensorrt_llm::runtime::cudaevent::deleter::operator() (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9CudaEvent7DeleterclE7pointer", false]], "tensorrt_llm::runtime::cudaevent::element_type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent12element_typeE", false]], "tensorrt_llm::runtime::cudaevent::eventptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent8EventPtrE", false]], "tensorrt_llm::runtime::cudaevent::get (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9CudaEvent3getEv", false]], "tensorrt_llm::runtime::cudaevent::mevent (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent6mEventE", false]], "tensorrt_llm::runtime::cudaevent::pointer (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7pointerE", false]], "tensorrt_llm::runtime::cudaevent::synchronize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9CudaEvent11synchronizeEv", false]], "tensorrt_llm::runtime::cudastream (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime10CudaStreamE", false]], "tensorrt_llm::runtime::cudastream::cudastream (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamE12cudaStream_t", false], [1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamE12cudaStream_tib", false], [1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamEji", false]], "tensorrt_llm::runtime::cudastream::deleter (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7DeleterE", false]], "tensorrt_llm::runtime::cudastream::deleter::deleter (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7Deleter7DeleterEb", false], [1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7Deleter7DeleterEv", false]], "tensorrt_llm::runtime::cudastream::deleter::mownsstream (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7Deleter11mOwnsStreamE", false]], "tensorrt_llm::runtime::cudastream::deleter::operator() (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream7DeleterclE12cudaStream_t", false]], "tensorrt_llm::runtime::cudastream::get (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream3getEv", false]], "tensorrt_llm::runtime::cudastream::getdevice (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream9getDeviceEv", false]], "tensorrt_llm::runtime::cudastream::mdevice (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7mDeviceE", false]], "tensorrt_llm::runtime::cudastream::mstream (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7mStreamE", false]], "tensorrt_llm::runtime::cudastream::record (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream6recordEN9CudaEvent7pointerE", false], [1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream6recordERK9CudaEvent", false]], "tensorrt_llm::runtime::cudastream::streamptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime10CudaStream9StreamPtrE", false]], "tensorrt_llm::runtime::cudastream::synchronize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream11synchronizeEv", false]], "tensorrt_llm::runtime::cudastream::wait (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream4waitEN9CudaEvent7pointerE", false], [1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream4waitERK9CudaEvent", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocatorE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::allocate (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator8allocateEP7PointerNSt6size_tEi", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13ConfigurationE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::backgroundconfiguration (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration23backgroundConfigurationE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::configuration (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtr", false], [1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtrb", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::mbackground (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration11mBackgroundE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::mbackstream (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration11mBackStreamE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::mmanager (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration8mManagerE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::mmode (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration5mModeE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::mpagesize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration9mPageSizeE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::mtag (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration4mTagE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::pagealigned (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration11pageAlignedENSt6size_tE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::setvirtualmemoryallocator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration25setVirtualMemoryAllocatorERKNSt6stringE11RestoreModeNSt10shared_ptrI10CudaStreamEE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::cudastreamptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13CudaStreamPtrE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::cudavirtualmemoryallocator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator26CudaVirtualMemoryAllocatorENSt10shared_ptrI13ConfigurationEE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::deallocate (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator10deallocateE7PointerNSt6size_tE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::mconfig (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator7mConfigE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::operator bool (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocatorcvbEv", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::pointer (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator7PointerE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::restoremode (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreModeE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::restoremode::cpu (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode3CPUE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::restoremode::memset (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode6MEMSETE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::restoremode::none (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode4NONEE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::restoremode::pinned (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode6PINNEDE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunkE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::_release (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk8_releaseEb", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::configurator (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12ConfiguratorE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::configurator::configurator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator12ConfiguratorERK12Configurator", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator12ConfiguratorERR12Configurator", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator12ConfiguratorEv", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::configurator::operator= (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12ConfiguratoraSERK12Configurator", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12ConfiguratoraSERR12Configurator", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::configurator::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator5setupE28CUmemGenericAllocationHandle", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::configurator::teardown (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator8teardownE28CUmemGenericAllocationHandleb", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::configurator::~configurator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12ConfiguratorD0Ev", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::configuratorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk15ConfiguratorPtrE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::configurators (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk13ConfiguratorsE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::creator (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7CreatorE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::creator::create (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator6createEv", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::creator::creator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7CreatorERK7Creator", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7CreatorERR7Creator", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7CreatorEv", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::creator::operator= (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7CreatoraSERK7Creator", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7CreatoraSERR7Creator", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::creator::release (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7releaseE28CUmemGenericAllocationHandleb", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::creator::~creator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7CreatorD0Ev", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::creatorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk10CreatorPtrE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::cudavirtualmemorychunk (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkERK22CUDAVirtualMemoryChunk", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkERR10CreatorPtrRR13Configurators", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkERR22CUDAVirtualMemoryChunk", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkEv", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::invalid_state (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk13INVALID_STATEE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::materialize (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk11materializeEv", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::mconfigurators (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk14mConfiguratorsE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::mcreator (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk8mCreatorE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::mhandle (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7mHandleE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::mstate (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6mStateE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::operator bool (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime22CUDAVirtualMemoryChunkcvbEv", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::operator= (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunkaSERK22CUDAVirtualMemoryChunk", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunkaSERR22CUDAVirtualMemoryChunk", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::release (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7releaseEv", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::status (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6StatusE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::status (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6statusEv", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::status::errored (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status7ERROREDE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::status::invalid (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status7INVALIDE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::status::materialized (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status12MATERIALIZEDE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::status::released (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status8RELEASEDE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::~cudavirtualmemorychunk (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunkD0Ev", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManagerE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::add (c++ function)": [[1, "_CPPv4IDpEN12tensorrt_llm7runtime24CudaVirtualMemoryManager3addEv9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrEDpRR13Configurators", false], [1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERR22CUDAVirtualMemoryChunk", false], [1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrERRN22CUDAVirtualMemoryChunk13ConfiguratorsE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::addbadhandle (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager12addBadHandleE9uintptr_t", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::entry (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager5EntryE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::entry::mentryit (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager5Entry8mEntryItE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::entry::mmemory (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager5Entry7mMemoryE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::materializewithtag (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager18materializeWithTagERKNSt6stringE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::mbadhandles (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager11mBadHandlesE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::mentries (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager8mEntriesE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::mmemories (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager9mMemoriesE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::mmutex (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager6mMutexE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::pointermemorymap (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager16PointerMemoryMapE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::releasewithtag (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager14releaseWithTagERKNSt6stringE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::remove (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager6removeE9uintptr_t", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::retrievebadhandles (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager18retrieveBadHandlesEv", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::tagentrymap (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager11TagEntryMapE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::unsaferemove (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager12unsafeRemoveE9uintptr_t", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4I_N8nvinfer18DataTypeE_b_bEN12tensorrt_llm7runtime14DataTypeTraitsE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4I_N8nvinfer18DataTypeE_bEN12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEE4typeE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4I_bEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kBOOLE9kUnsignedEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kBOOLE9kUnsignedE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kBOOLE9kUnsignedE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kBOOLE9kUnsignedE4typeE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kFLOATEEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kFLOATEE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kFLOATEE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kFLOATEE4typeE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kHALFEEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kHALFEE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kHALFEE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kHALFEE4typeE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EXL1EEEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EXL1EEE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EXL1EEE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EXL1EEE4typeE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EE4typeE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EXL1EEEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EXL1EEE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EXL1EEE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EXL1EEE4typeE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EE4typeE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kINT8EEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kINT8EE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kINT8EE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kINT8EE4typeE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4I_bEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kUINT8E9kUnsignedEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kUINT8E9kUnsignedE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kUINT8E9kUnsignedE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kUINT8E9kUnsignedE4typeE", false]], "tensorrt_llm::runtime::decoder (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoderE", false]], "tensorrt_llm::runtime::decoder::beamsearchbuffers (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffersE", false]], "tensorrt_llm::runtime::decoder::beamsearchbuffers::beamsearchbuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers17BeamSearchBuffersERK13BufferManager", false]], "tensorrt_llm::runtime::decoder::beamsearchbuffers::mcumlogprobstmp (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers15mCumLogProbsTmpE", false]], "tensorrt_llm::runtime::decoder::beamsearchbuffers::mnumsms (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers7mNumSMsE", false]], "tensorrt_llm::runtime::decoder::beamsearchbuffers::moutputbeamhypotheses (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers21mOutputBeamHypothesesE", false]], "tensorrt_llm::runtime::decoder::beamsearchbuffers::reshape (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers7reshapeE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::decoder::decoderstate (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderStateE", false]], "tensorrt_llm::runtime::decoder::decoderstate::decoderstate (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12DecoderStateEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::decodinginputptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState16DecodingInputPtrE", false]], "tensorrt_llm::runtime::decoder::decoderstate::decodingoutputptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState17DecodingOutputPtrE", false]], "tensorrt_llm::runtime::decoder::decoderstate::disablelookahead (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState16disableLookaheadERK13RequestVector", false]], "tensorrt_llm::runtime::decoder::decoderstate::getacceptedlengthscumsum (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState24getAcceptedLengthsCumSumEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getacceptedpackedpaths (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState22getAcceptedPackedPathsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getallnewtokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState15getAllNewTokensEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getbeamsearchbuffers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState20getBeamSearchBuffersEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getcacheindirectioninput (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState24getCacheIndirectionInputEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getcacheindirectionoutput (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState25getCacheIndirectionOutputEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getcumlogprobs (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getCumLogProbsE10SizeType32", false], [1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getCumLogProbsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::geteaglebuffers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState15getEagleBuffersEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getexplicitdrafttokensbuffers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState29getExplicitDraftTokensBuffersEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getfinishedsum (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getFinishedSumEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getfinishreasons (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState16getFinishReasonsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getgatheredids (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getGatheredIdsE10SizeType32", false], [1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getGatheredIdsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getgenerationsteps (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getGenerationStepsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getids (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState6getIdsE10SizeType32", false], [1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState6getIdsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getjointdecodinginput (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState21getJointDecodingInputEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getjointdecodingoutput (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState22getJointDecodingOutputEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getlogprobs (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState11getLogProbsE10SizeType32", false], [1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState11getLogProbsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getlookaheadbuffers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState19getLookaheadBuffersEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getmaxbeamwidth (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState15getMaxBeamWidthEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getmaxdecodingdecodertokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState27getMaxDecodingDecoderTokensEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getmaxdecodingenginetokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState26getMaxDecodingEngineTokensEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getmaxnumsequences (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getMaxNumSequencesEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getmaxsequencelength (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState20getMaxSequenceLengthEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getnextdrafttokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getNextDraftTokensEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getnextdrafttokenslengths (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState25getNextDraftTokensLengthsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getnumdecodingenginetokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState26getNumDecodingEngineTokensE10SizeType32", false], [1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState26getNumDecodingEngineTokensEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getparentids (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState12getParentIdsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getprevdrafttokenslengths (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState25getPrevDraftTokensLengthsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getsequencelengths (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getSequenceLengthsE10SizeType32", false], [1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getSequenceLengthsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getspeculativedecodingmode (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState26getSpeculativeDecodingModeEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::llmrequestptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState13LlmRequestPtrE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mbeamsearchbuffers (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState18mBeamSearchBuffersE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mjointdecodinginput (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState19mJointDecodingInputE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mjointdecodingoutput (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState20mJointDecodingOutputE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mmaxbeamwidth (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState13mMaxBeamWidthE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mmaxdecodingdecodertokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState25mMaxDecodingDecoderTokensE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mmaxdecodingenginetokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24mMaxDecodingEngineTokensE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mmaxnumsequences (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState16mMaxNumSequencesE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mmaxsequencelength (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState18mMaxSequenceLengthE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mnumdecodingenginetokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24mNumDecodingEngineTokensE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mspeculativedecodingmode (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24mSpeculativeDecodingModeE", false]], "tensorrt_llm::runtime::decoder::decoderstate::requestvector (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState13RequestVectorE", false]], "tensorrt_llm::runtime::decoder::decoderstate::reshapebuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", false]], "tensorrt_llm::runtime::decoder::decoderstate::reshapecacheindirectionbuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState30reshapeCacheIndirectionBuffersE10SizeType3210SizeType3210SizeType32", false]], "tensorrt_llm::runtime::decoder::decoderstate::reshapespeculativedecodingbuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState33reshapeSpeculativeDecodingBuffersERK23SpeculativeDecodingMode10SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", false]], "tensorrt_llm::runtime::decoder::decoderstate::setbeamwidth (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12setBeamWidthE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::decoder::decoderstate::setgenerationsteps (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState18setGenerationStepsERKNSt6vectorI10SizeType32EE", false]], "tensorrt_llm::runtime::decoder::decoderstate::setnumdecodingenginetokens (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState26setNumDecodingEngineTokensE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::decoder::decoderstate::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", false]], "tensorrt_llm::runtime::decoder::decoderstate::setupbuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12setupBuffersEN8nvinfer18DataTypeERK13BufferManager", false]], "tensorrt_llm::runtime::decoder::decoderstate::setupcacheindirection (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState21setupCacheIndirectionE10SizeType3210SizeType3210SizeType32RK13BufferManager", false]], "tensorrt_llm::runtime::decoder::decoderstate::setupcacheindirectionbuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState28setupCacheIndirectionBuffersERK13BufferManager", false]], "tensorrt_llm::runtime::decoder::decoderstate::setupspeculativedecoding (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24setupSpeculativeDecodingERK23SpeculativeDecodingMode10SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", false]], "tensorrt_llm::runtime::decoder::decoderstate::setupspeculativedecodingbuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState31setupSpeculativeDecodingBuffersE23SpeculativeDecodingModeN8nvinfer18DataTypeERK13BufferManager", false]], "tensorrt_llm::runtime::decoder::decoderstate::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState9TensorPtrE", false]], "tensorrt_llm::runtime::decoder_batch (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime13decoder_batchE", false]], "tensorrt_llm::runtime::decoder_batch::input (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5InputE", false]], "tensorrt_llm::runtime::decoder_batch::input::batchslots (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input10batchSlotsE", false]], "tensorrt_llm::runtime::decoder_batch::input::input (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input5InputERKNSt6vectorI14TensorConstPtrEE", false], [1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input5InputERKNSt6vectorINSt6vectorI14TensorConstPtrEEEE10SizeType32", false]], "tensorrt_llm::runtime::decoder_batch::input::logits (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input6logitsE", false]], "tensorrt_llm::runtime::decoder_batch::input::maxdecodersteps (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input15maxDecoderStepsE", false]], "tensorrt_llm::runtime::decoder_batch::input::tensorconstptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input14TensorConstPtrE", false]], "tensorrt_llm::runtime::decoder_batch::input::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input9TensorPtrE", false]], "tensorrt_llm::runtime::decodinginput (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInputE", false]], "tensorrt_llm::runtime::decodinginput::badwordslens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12badWordsLensE", false]], "tensorrt_llm::runtime::decodinginput::badwordslists (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13badWordsListsE", false]], "tensorrt_llm::runtime::decodinginput::badwordsptrs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12badWordsPtrsE", false]], "tensorrt_llm::runtime::decodinginput::batchsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput9batchSizeE", false]], "tensorrt_llm::runtime::decodinginput::batchslots (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput10batchSlotsE", false]], "tensorrt_llm::runtime::decodinginput::beamwidths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput10beamWidthsE", false]], "tensorrt_llm::runtime::decodinginput::cacheindirection (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput16cacheIndirectionE", false]], "tensorrt_llm::runtime::decodinginput::decodinginput (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13DecodingInputEv", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11eagleInputsE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputsE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::acceptedlens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs12acceptedLensE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::acceptedpathids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs15acceptedPathIdsE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::acceptedtokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs14acceptedTokensE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::chunkedcontextnexttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs24chunkedContextNextTokensE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::lastdraftlens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs13lastDraftLensE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::lastdraftpaths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs14lastDraftPathsE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::lastdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs15lastDraftTokensE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::nextdraftlens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs13nextDraftLensE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::nextdraftpaths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs14nextDraftPathsE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::nextdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs15nextDraftTokensE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::seqslots (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs8seqSlotsE", false]], "tensorrt_llm::runtime::decodinginput::embeddingbias (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13embeddingBiasE", false]], "tensorrt_llm::runtime::decodinginput::endids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput6endIdsE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputsE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25explicitDraftTokensInputsE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::bestpathindices (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs15bestPathIndicesE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::bestpathlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs15bestPathLengthsE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::lastdraftindices (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs16lastDraftIndicesE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::lastdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs15lastDraftTokensE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::lastgenerationlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs21lastGenerationLengthsE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::lastpositionidsbase (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs19lastPositionIdsBaseE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::masks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs5masksE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::maxgenlengthdevice (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs18maxGenLengthDeviceE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::nextdraftindices (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs16nextDraftIndicesE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::nextdraftprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs14nextDraftProbsE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::nextdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs15nextDraftTokensE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::nextflattokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs14nextFlatTokensE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::nextgenerationlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs21nextGenerationLengthsE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::packedpositionids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs17packedPositionIdsE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::seqslots (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs8seqSlotsE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputsE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25externalDraftTokensInputsE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::constantthreshold (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs17constantThresholdE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::draftlogits (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs11draftLogitsE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::draftlogitshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs15draftLogitsHostE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::draftprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs10draftProbsE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::drafttokenids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs13draftTokenIdsE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::drafttokenidshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs17draftTokenIdsHostE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::numdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs14numDraftTokensE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::numdrafttokenshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs18numDraftTokensHostE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::step (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs4stepE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::targetprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs11targetProbsE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::usedraftlogits (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs14useDraftLogitsE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::usedraftlogitshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs18useDraftLogitsHostE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::userandomacceptancethreshold (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs28useRandomAcceptanceThresholdE", false]], "tensorrt_llm::runtime::decodinginput::finishreasons (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13finishReasonsE", false]], "tensorrt_llm::runtime::decodinginput::generationsteps (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15generationStepsE", false]], "tensorrt_llm::runtime::decodinginput::lengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput7lengthsE", false]], "tensorrt_llm::runtime::decodinginput::logitsvec (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput9logitsVecE", false]], "tensorrt_llm::runtime::decodinginput::lookaheadinputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15lookaheadInputsE", false]], "tensorrt_llm::runtime::decodinginput::lookaheadinputs (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15LookaheadInputsE", false]], "tensorrt_llm::runtime::decodinginput::lookaheadinputs::tokensperstep (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15LookaheadInputs13tokensPerStepE", false]], "tensorrt_llm::runtime::decodinginput::maxattentionwindow (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput18maxAttentionWindowE", false]], "tensorrt_llm::runtime::decodinginput::maxbadwordslen (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput14maxBadWordsLenE", false]], "tensorrt_llm::runtime::decodinginput::maxlength (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput9maxLengthE", false]], "tensorrt_llm::runtime::decodinginput::maxstopwordslen (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15maxStopWordsLenE", false]], "tensorrt_llm::runtime::decodinginput::medusainputs (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputsE", false]], "tensorrt_llm::runtime::decodinginput::medusainputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12medusaInputsE", false]], "tensorrt_llm::runtime::decodinginput::medusainputs::medusacurtokensperstep (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs22medusaCurTokensPerStepE", false]], "tensorrt_llm::runtime::decodinginput::medusainputs::medusalogits (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs12medusaLogitsE", false]], "tensorrt_llm::runtime::decodinginput::medusainputs::medusapaths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs11medusaPathsE", false]], "tensorrt_llm::runtime::decodinginput::medusainputs::medusatargettokensperstep (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs25medusaTargetTokensPerStepE", false]], "tensorrt_llm::runtime::decodinginput::medusainputs::medusatreeids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs13medusaTreeIdsE", false]], "tensorrt_llm::runtime::decodinginput::norepeatngramsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput17noRepeatNgramSizeE", false]], "tensorrt_llm::runtime::decodinginput::sequencelimitlength (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput19sequenceLimitLengthE", false]], "tensorrt_llm::runtime::decodinginput::sinktokenlength (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15sinkTokenLengthE", false]], "tensorrt_llm::runtime::decodinginput::step (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput4stepE", false]], "tensorrt_llm::runtime::decodinginput::stopwordslens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13stopWordsLensE", false]], "tensorrt_llm::runtime::decodinginput::stopwordslists (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput14stopWordsListsE", false]], "tensorrt_llm::runtime::decodinginput::stopwordsptrs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13stopWordsPtrsE", false]], "tensorrt_llm::runtime::decodinginput::tensorconstptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput14TensorConstPtrE", false]], "tensorrt_llm::runtime::decodinginput::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput9TensorPtrE", false]], "tensorrt_llm::runtime::decodingoutput (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutputE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypothesesE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14beamHypothesesE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::batchdones (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses10batchDonesE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::cumlogprobscba (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses14cumLogProbsCBAE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::empty (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses5emptyERK13BufferManager", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::init (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses4initERK13BufferManager11TokenIdType", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::logprobscba (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses11logProbsCBAE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::minnormedscorescba (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses18minNormedScoresCBAE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::normedscorescba (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses15normedScoresCBAE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::numbeamscba (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses11numBeamsCBAE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::outputidscba (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses12outputIdsCBAE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::release (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses7releaseEv", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::reshape (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses7reshapeE10SizeType3210SizeType3210SizeType32", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::sequencelengthscba (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses18sequenceLengthsCBAE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::slice (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses5sliceE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::decodingoutput::cacheindirection (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput16cacheIndirectionE", false]], "tensorrt_llm::runtime::decodingoutput::cumlogprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput11cumLogProbsE", false]], "tensorrt_llm::runtime::decodingoutput::decodingoutput (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14DecodingOutputEv", false]], "tensorrt_llm::runtime::decodingoutput::eaglebuffers (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput12eagleBuffersE", false]], "tensorrt_llm::runtime::decodingoutput::explicitdrafttokensbuffers (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26explicitDraftTokensBuffersE", false]], "tensorrt_llm::runtime::decodingoutput::finishedsum (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput11finishedSumE", false]], "tensorrt_llm::runtime::decodingoutput::finishreasons (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput13finishReasonsE", false]], "tensorrt_llm::runtime::decodingoutput::gatheredids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput11gatheredIdsE", false]], "tensorrt_llm::runtime::decodingoutput::ids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput3idsE", false]], "tensorrt_llm::runtime::decodingoutput::knegativeinfinity (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput17kNegativeInfinityE", false]], "tensorrt_llm::runtime::decodingoutput::lengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput7lengthsE", false]], "tensorrt_llm::runtime::decodingoutput::logprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput8logProbsE", false]], "tensorrt_llm::runtime::decodingoutput::logprobstiled (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput13logProbsTiledE", false]], "tensorrt_llm::runtime::decodingoutput::lookaheadoutputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput16lookaheadOutputsE", false]], "tensorrt_llm::runtime::decodingoutput::newtokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput9newTokensE", false]], "tensorrt_llm::runtime::decodingoutput::newtokenssteps (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14newTokensStepsE", false]], "tensorrt_llm::runtime::decodingoutput::newtokensvec (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput12newTokensVecE", false]], "tensorrt_llm::runtime::decodingoutput::parentids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput9parentIdsE", false]], "tensorrt_llm::runtime::decodingoutput::speculativedecodingoutputs (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputsE", false]], "tensorrt_llm::runtime::decodingoutput::speculativedecodingoutputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26speculativeDecodingOutputsE", false]], "tensorrt_llm::runtime::decodingoutput::speculativedecodingoutputs::acceptedlengthscumsum (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs21acceptedLengthsCumSumE", false]], "tensorrt_llm::runtime::decodingoutput::speculativedecodingoutputs::acceptedtokenslen (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs17acceptedTokensLenE", false]], "tensorrt_llm::runtime::decodingoutput::speculativedecodingoutputs::nextdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs15nextDraftTokensE", false]], "tensorrt_llm::runtime::decodingoutput::speculativedecodingoutputs::nextdrafttokenslen (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs18nextDraftTokensLenE", false]], "tensorrt_llm::runtime::decodingoutput::speculativedecodingoutputs::pathsoffsets (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs12pathsOffsetsE", false]], "tensorrt_llm::runtime::decodingoutput::speculativedecodingoutputs::prevdrafttokenslen (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs18prevDraftTokensLenE", false]], "tensorrt_llm::runtime::decodingoutput::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput9TensorPtrE", false]], "tensorrt_llm::runtime::deviceallocationnvls (c++ class)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime20DeviceAllocationNvlsE", false]], "tensorrt_llm::runtime::deviceallocationnvls::_capacity (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls9_capacityE", false]], "tensorrt_llm::runtime::deviceallocationnvls::_handle (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls7_handleE", false]], "tensorrt_llm::runtime::deviceallocationnvls::deviceallocationnvls (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls20DeviceAllocationNvlsEv", false]], "tensorrt_llm::runtime::deviceallocationnvls::free (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls4freeEv", false]], "tensorrt_llm::runtime::deviceallocationnvls::getcapacity (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime20DeviceAllocationNvls11getCapacityEv", false]], "tensorrt_llm::runtime::deviceallocationnvls::getipcunicastpointers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls21getIpcUnicastPointersEv", false]], "tensorrt_llm::runtime::deviceallocationnvls::getmulticastpointer (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime20DeviceAllocationNvls19getMulticastPointerEv", false]], "tensorrt_llm::runtime::deviceallocationnvls::getunicastpointer (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime20DeviceAllocationNvls17getUnicastPointerEv", false]], "tensorrt_llm::runtime::deviceallocationnvls::reset (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls5resetE6size_tNSt3setIiEE", false]], "tensorrt_llm::runtime::deviceallocationnvls::~deviceallocationnvls (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvlsD0Ev", false]], "tensorrt_llm::runtime::eaglebuffers (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffersE", false]], "tensorrt_llm::runtime::eaglebuffers::bufferptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers9BufferPtrE", false]], "tensorrt_llm::runtime::eaglebuffers::chunkedcontextnexttokenshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers28chunkedContextNextTokensHostE", false]], "tensorrt_llm::runtime::eaglebuffers::cumsumgenerationlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers23cumSumGenerationLengthsE", false]], "tensorrt_llm::runtime::eaglebuffers::eaglebuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12EagleBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN8executor14DecodingConfigE", false]], "tensorrt_llm::runtime::eaglebuffers::engineinputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12engineInputsE", false]], "tensorrt_llm::runtime::eaglebuffers::engineoutputs (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputsE", false]], "tensorrt_llm::runtime::eaglebuffers::engineoutputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13engineOutputsE", false]], "tensorrt_llm::runtime::eaglebuffers::engineoutputs::acceptedlens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs12acceptedLensE", false]], "tensorrt_llm::runtime::eaglebuffers::engineoutputs::acceptedpaths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs13acceptedPathsE", false]], "tensorrt_llm::runtime::eaglebuffers::engineoutputs::acceptedtokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs14acceptedTokensE", false]], "tensorrt_llm::runtime::eaglebuffers::engineoutputs::chunkedcontextnexttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs24chunkedContextNextTokensE", false]], "tensorrt_llm::runtime::eaglebuffers::engineoutputs::nextdraftlens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs13nextDraftLensE", false]], "tensorrt_llm::runtime::eaglebuffers::engineoutputs::nextdraftpaths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs14nextDraftPathsE", false]], "tensorrt_llm::runtime::eaglebuffers::engineoutputs::nextdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs15nextDraftTokensE", false]], "tensorrt_llm::runtime::eaglebuffers::greedysamplinghost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers18greedySamplingHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6InputsE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::alllayersdrafttokenids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs22allLayersDraftTokenIdsE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::alllayersdrafttokenidspredecessor (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs33allLayersDraftTokenIdsPredecessorE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::alllayersscores (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs15allLayersScoresE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::chunkedcontextnexttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs24chunkedContextNextTokensE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::create (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs6createE10SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfig", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::currentexpandindices (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs20currentExpandIndicesE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::draftlens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs9draftLensE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::draftpaths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs10draftPathsE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::draftpathshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs14draftPathsHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::drafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs11draftTokensE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::dynamictreemaxtopkhost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs22dynamicTreeMaxTopKHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::eaglenetctxcontextlengthshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs29eagleNetCtxContextLengthsHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::eaglenetctxpastkeyvaluelengthshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs34eagleNetCtxPastKeyValueLengthsHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::eaglenetctxrequesttypeshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs27eagleNetCtxRequestTypesHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::eaglenetgencontextlengthshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs29eagleNetGenContextLengthsHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::eaglenetgenpastkeyvaluelengthshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs34eagleNetGenPastKeyValueLengthsHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::eaglenetgenrequesttypeshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs27eagleNetGenRequestTypesHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::inputgentokenshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs18inputGenTokensHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::posterioralpha (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs14posteriorAlphaE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::posteriorthreshold (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs18posteriorThresholdE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::prevscores (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs10prevScoresE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::randomdatasample (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs16randomDataSampleE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::randomdatavalidation (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs20randomDataValidationE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::specdecodinggenerationlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs29specDecodingGenerationLengthsE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::specdecodinggenerationlengthshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs33specDecodingGenerationLengthsHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::specdecodingpackedmasks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs23specDecodingPackedMasksE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::specdecodingpositionoffsets (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs27specDecodingPositionOffsetsE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::temperatures (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs12temperaturesE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::usedynamictreehost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs18useDynamicTreeHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::usespecdecoding (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs15useSpecDecodingE", false]], "tensorrt_llm::runtime::eaglebuffers::insertinputtensors (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", false]], "tensorrt_llm::runtime::eaglebuffers::itensor (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers7ITensorE", false]], "tensorrt_llm::runtime::eaglebuffers::llmrequestptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13LlmRequestPtrE", false]], "tensorrt_llm::runtime::eaglebuffers::maxgenerationlength (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers19maxGenerationLengthE", false]], "tensorrt_llm::runtime::eaglebuffers::mdefaultposteriorthreshold (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers26mDefaultPosteriorThresholdE", false]], "tensorrt_llm::runtime::eaglebuffers::mdogreedysampling (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers17mDoGreedySamplingE", false]], "tensorrt_llm::runtime::eaglebuffers::posterioralphahost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers18posteriorAlphaHostE", false]], "tensorrt_llm::runtime::eaglebuffers::posteriorthresholdhost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers22posteriorThresholdHostE", false]], "tensorrt_llm::runtime::eaglebuffers::requestvector (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13RequestVectorE", false]], "tensorrt_llm::runtime::eaglebuffers::reshape (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", false]], "tensorrt_llm::runtime::eaglebuffers::scanreducetempstorage (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers21scanReduceTempStorageE", false]], "tensorrt_llm::runtime::eaglebuffers::scanreducetempstoragebytes (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers26scanReduceTempStorageBytesE", false]], "tensorrt_llm::runtime::eaglebuffers::setfrominputs (c++ function)": [[1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", false], [1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", false]], "tensorrt_llm::runtime::eaglebuffers::sizetype32 (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers10SizeType32E", false]], "tensorrt_llm::runtime::eaglebuffers::tensormap (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers9TensorMapE", false]], "tensorrt_llm::runtime::eaglebuffers::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers9TensorPtrE", false]], "tensorrt_llm::runtime::eaglemodule (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime11EagleModuleE", false]], "tensorrt_llm::runtime::eaglemodule::eaglemodule (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11EagleModule11EagleModuleE10SizeType3210SizeType3210SizeType3210SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime11EagleModule11EagleModuleEv", false]], "tensorrt_llm::runtime::eaglemodule::getdefaulteaglechoices (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11EagleModule22getDefaultEagleChoicesEv", false]], "tensorrt_llm::runtime::eaglemodule::getmaxnonleafnodesperlayer (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11EagleModule26getMaxNonLeafNodesPerLayerEv", false]], "tensorrt_llm::runtime::eaglemodule::getnumtransformerlayers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11EagleModule23getNumTransformerLayersEv", false]], "tensorrt_llm::runtime::eaglemodule::mdefaulteaglechoices (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11EagleModule20mDefaultEagleChoicesE", false]], "tensorrt_llm::runtime::eaglemodule::mmaxnonleafnodesperlayer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11EagleModule24mMaxNonLeafNodesPerLayerE", false]], "tensorrt_llm::runtime::eaglemodule::mnumtransformerslayer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11EagleModule21mNumTransformersLayerE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffersE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::bufferptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers9BufferPtrE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::cumsumgenerationlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers23cumSumGenerationLengthsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineinputs (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers12EngineInputsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineinputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers12engineInputsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineinputs::positionoffsets (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers12EngineInputs15positionOffsetsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineinputs::requesttypesdevice (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers12EngineInputs18requestTypesDeviceE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13engineOutputsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::bestpathindices (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs15bestPathIndicesE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::bestpathlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs15bestPathLengthsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::masks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs5masksE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::maxgentoken (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs11maxGenTokenE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::nextdraftindices (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs16nextDraftIndicesE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::nextdraftprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs14nextDraftProbsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::nextdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs15nextDraftTokensE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::nextflattokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs14nextFlatTokensE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::nextgenerationlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs21nextGenerationLengthsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::nextpositionoffsets (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs19nextPositionOffsetsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::packedpositionids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs17packedPositionIdsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::totalgentoken (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs13totalGenTokenE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::explicitdrafttokensbuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers26ExplicitDraftTokensBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6InputsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::create (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs6createE10SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::draftindices (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs12draftIndicesE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::draftprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs10draftProbsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::drafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs11draftTokensE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::generationlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs17generationLengthsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::generationlengthshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs21generationLengthsHostE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::maxgenlengthhost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs16maxGenLengthHostE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::packedmasks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs11packedMasksE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::positionids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs11positionIdsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::positionidsbase (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs15positionIdsBaseE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::randomdatasample (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs16randomDataSampleE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::randomdatavalidation (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs20randomDataValidationE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::temperatures (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs12temperaturesE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::usespecdecoding (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs15useSpecDecodingE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::insertinputtensors (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::itensor (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers7ITensorE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::reshape (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::scantempstorage (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers15scanTempStorageE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::scantempstoragebytes (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers20scanTempStorageBytesE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::setfrominputs (c++ function)": [[1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", false], [1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::sizetype32 (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers10SizeType32E", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::tensormap (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers9TensorMapE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers9TensorPtrE", false]], "tensorrt_llm::runtime::genericprompttuningparams (c++ class)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime25GenericPromptTuningParamsE", false]], "tensorrt_llm::runtime::genericprompttuningparams::embeddingtable (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams14embeddingTableE", false]], "tensorrt_llm::runtime::genericprompttuningparams::genericprompttuningparams (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams25GenericPromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", false]], "tensorrt_llm::runtime::genericprompttuningparams::prompttuningenabled (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams19promptTuningEnabledE", false]], "tensorrt_llm::runtime::genericprompttuningparams::sizetype32 (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams10SizeType32E", false]], "tensorrt_llm::runtime::genericprompttuningparams::tasks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams5tasksE", false]], "tensorrt_llm::runtime::genericprompttuningparams::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams9TensorPtrE", false]], "tensorrt_llm::runtime::genericprompttuningparams::vocabsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams9vocabSizeE", false]], "tensorrt_llm::runtime::getdefaultbatchslots (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20getDefaultBatchSlotsEN7runtime10SizeType32E", false]], "tensorrt_llm::runtime::getvirtualmemoryallocator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25getVirtualMemoryAllocatorEv", false]], "tensorrt_llm::runtime::getvirtualmemorymanager (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23getVirtualMemoryManagerEv", false]], "tensorrt_llm::runtime::gptdecoder (c++ class)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime10GptDecoderE", false]], "tensorrt_llm::runtime::gptdecoder::cudastreamptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder13CudaStreamPtrE", false]], "tensorrt_llm::runtime::gptdecoder::disablelookahead (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", false]], "tensorrt_llm::runtime::gptdecoder::forwardasync (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder12forwardAsyncER14DecodingOutputRK13DecodingInput", false]], "tensorrt_llm::runtime::gptdecoder::forwardsync (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder11forwardSyncER14DecodingOutputRK13DecodingInput", false]], "tensorrt_llm::runtime::gptdecoder::getsamplingconfig (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder17getSamplingConfigEv", false]], "tensorrt_llm::runtime::gptdecoder::gptdecoder (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10GptDecoderERKN8executor12DecodingModeE6size_t6size_t6size_t6size_tRK13CudaStreamPtrNSt10shared_ptrIK25SpeculativeDecodingModuleEE", false]], "tensorrt_llm::runtime::gptdecoder::mdecodinglayerworkspace (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder23mDecodingLayerWorkspaceE", false]], "tensorrt_llm::runtime::gptdecoder::mdecodingmode (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder13mDecodingModeE", false]], "tensorrt_llm::runtime::gptdecoder::mdynamicdecodelayer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder19mDynamicDecodeLayerE", false]], "tensorrt_llm::runtime::gptdecoder::mmanager (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder8mManagerE", false]], "tensorrt_llm::runtime::gptdecoder::mmaxnumsequences (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder16mMaxNumSequencesE", false]], "tensorrt_llm::runtime::gptdecoder::msamplingconfig (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder15mSamplingConfigE", false]], "tensorrt_llm::runtime::gptdecoder::mvocabsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10mVocabSizeE", false]], "tensorrt_llm::runtime::gptdecoder::mvocabsizepadded (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder16mVocabSizePaddedE", false]], "tensorrt_llm::runtime::gptdecoder::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", false]], "tensorrt_llm::runtime::gptdecoder::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder9TensorPtrE", false]], "tensorrt_llm::runtime::gptdecoderbatched (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatchedE", false]], "tensorrt_llm::runtime::gptdecoderbatched::cudastreamptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched13CudaStreamPtrE", false]], "tensorrt_llm::runtime::gptdecoderbatched::disablelookahead (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched16disableLookaheadERK13RequestVectorRK9TensorPtr", false]], "tensorrt_llm::runtime::gptdecoderbatched::finalize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", false]], "tensorrt_llm::runtime::gptdecoderbatched::forward (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched7forwardERKN7decoder12DecoderStateERKN13decoder_batch5InputE", false]], "tensorrt_llm::runtime::gptdecoderbatched::forwardasync (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched12forwardAsyncERKN7decoder12DecoderStateERKN13decoder_batch5InputE", false]], "tensorrt_llm::runtime::gptdecoderbatched::forwarddispatch (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched15forwardDispatchERKN7decoder12DecoderStateERKN13decoder_batch5InputE", false]], "tensorrt_llm::runtime::gptdecoderbatched::getbuffermanager (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched16getBufferManagerEv", false]], "tensorrt_llm::runtime::gptdecoderbatched::getdecoderstream (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched16getDecoderStreamEv", false]], "tensorrt_llm::runtime::gptdecoderbatched::getunderlyingdecoder (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched20getUnderlyingDecoderEv", false]], "tensorrt_llm::runtime::gptdecoderbatched::gptdecoderbatched (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched17GptDecoderBatchedE13CudaStreamPtr", false]], "tensorrt_llm::runtime::gptdecoderbatched::gptdecoderptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched13GptDecoderPtrE", false]], "tensorrt_llm::runtime::gptdecoderbatched::llmrequestptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched13LlmRequestPtrE", false]], "tensorrt_llm::runtime::gptdecoderbatched::mbuffermanager (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched14mBufferManagerE", false]], "tensorrt_llm::runtime::gptdecoderbatched::mdecoder (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched8mDecoderE", false]], "tensorrt_llm::runtime::gptdecoderbatched::mdecoderstream (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched14mDecoderStreamE", false]], "tensorrt_llm::runtime::gptdecoderbatched::mruntimestream (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched14mRuntimeStreamE", false]], "tensorrt_llm::runtime::gptdecoderbatched::requestvector (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched13RequestVectorE", false]], "tensorrt_llm::runtime::gptdecoderbatched::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", false]], "tensorrt_llm::runtime::gptdecoderbatched::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched9TensorPtrE", false]], "tensorrt_llm::runtime::gptjsonconfig (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfigE", false]], "tensorrt_llm::runtime::gptjsonconfig::enginefilename (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14engineFilenameERK11WorldConfig", false], [1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14engineFilenameERK11WorldConfigRKNSt6stringE", false]], "tensorrt_llm::runtime::gptjsonconfig::getcontextparallelism (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig21getContextParallelismEv", false]], "tensorrt_llm::runtime::gptjsonconfig::getgpuspernode (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14getGpusPerNodeEv", false]], "tensorrt_llm::runtime::gptjsonconfig::getmodelconfig (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14getModelConfigEv", false]], "tensorrt_llm::runtime::gptjsonconfig::getmodelconfigmutable (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig21getModelConfigMutableEv", false]], "tensorrt_llm::runtime::gptjsonconfig::getname (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig7getNameEv", false]], "tensorrt_llm::runtime::gptjsonconfig::getpipelineparallelism (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig22getPipelineParallelismEv", false]], "tensorrt_llm::runtime::gptjsonconfig::getprecision (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig12getPrecisionEv", false]], "tensorrt_llm::runtime::gptjsonconfig::getruntimedefaults (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig18getRuntimeDefaultsEv", false]], "tensorrt_llm::runtime::gptjsonconfig::gettensorparallelism (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig20getTensorParallelismEv", false]], "tensorrt_llm::runtime::gptjsonconfig::getversion (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig10getVersionEv", false]], "tensorrt_llm::runtime::gptjsonconfig::getworldsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig12getWorldSizeEv", false]], "tensorrt_llm::runtime::gptjsonconfig::gptjsonconfig (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", false]], "tensorrt_llm::runtime::gptjsonconfig::mcontextparallelism (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig19mContextParallelismE", false]], "tensorrt_llm::runtime::gptjsonconfig::mgpuspernode (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig12mGpusPerNodeE", false]], "tensorrt_llm::runtime::gptjsonconfig::mmodelconfig (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig12mModelConfigE", false]], "tensorrt_llm::runtime::gptjsonconfig::mname (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5mNameE", false]], "tensorrt_llm::runtime::gptjsonconfig::mpipelineparallelism (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig20mPipelineParallelismE", false]], "tensorrt_llm::runtime::gptjsonconfig::mprecision (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig10mPrecisionE", false]], "tensorrt_llm::runtime::gptjsonconfig::mruntimedefaults (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig16mRuntimeDefaultsE", false]], "tensorrt_llm::runtime::gptjsonconfig::mtensorparallelism (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig18mTensorParallelismE", false]], "tensorrt_llm::runtime::gptjsonconfig::mversion (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig8mVersionE", false]], "tensorrt_llm::runtime::gptjsonconfig::parse (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5parseERKNSt10filesystem4pathE", false], [1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5parseERKNSt6stringE", false], [1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5parseERNSt7istreamE", false]], "tensorrt_llm::runtime::ibuffer (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBufferE", false]], "tensorrt_llm::runtime::ibuffer::data (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4dataENSt6size_tE", false], [1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4dataEv", false], [1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer4dataENSt6size_tE", false], [1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer4dataEv", false]], "tensorrt_llm::runtime::ibuffer::datatype (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer8DataTypeE", false]], "tensorrt_llm::runtime::ibuffer::getcapacity (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer11getCapacityEv", false]], "tensorrt_llm::runtime::ibuffer::getdatatype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer11getDataTypeEv", false]], "tensorrt_llm::runtime::ibuffer::getdatatypename (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer15getDataTypeNameE8DataType", false], [1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer15getDataTypeNameEv", false]], "tensorrt_llm::runtime::ibuffer::getmemorytype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer13getMemoryTypeEv", false]], "tensorrt_llm::runtime::ibuffer::getmemorytypename (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer17getMemoryTypeNameEv", false]], "tensorrt_llm::runtime::ibuffer::getsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer7getSizeEv", false]], "tensorrt_llm::runtime::ibuffer::getsizeinbytes (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer14getSizeInBytesEv", false]], "tensorrt_llm::runtime::ibuffer::ibuffer (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer7IBufferERK7IBuffer", false], [1, "_CPPv4N12tensorrt_llm7runtime7IBuffer7IBufferEv", false]], "tensorrt_llm::runtime::ibuffer::memorytype (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer10memoryTypeEPKv", false]], "tensorrt_llm::runtime::ibuffer::operator= (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBufferaSERK7IBuffer", false]], "tensorrt_llm::runtime::ibuffer::release (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer7releaseEv", false]], "tensorrt_llm::runtime::ibuffer::resize (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer6resizeENSt6size_tE", false]], "tensorrt_llm::runtime::ibuffer::sharedconstptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer14SharedConstPtrE", false]], "tensorrt_llm::runtime::ibuffer::sharedptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer9SharedPtrE", false]], "tensorrt_llm::runtime::ibuffer::slice (c++ function)": [[1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", false], [1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", false], [1, "_CPPv4N12tensorrt_llm7runtime7IBuffer5sliceE9SharedPtrNSt6size_tE", false], [1, "_CPPv4N12tensorrt_llm7runtime7IBuffer5sliceE9SharedPtrNSt6size_tENSt6size_tE", false]], "tensorrt_llm::runtime::ibuffer::tobytes (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer7toBytesENSt6size_tE", false]], "tensorrt_llm::runtime::ibuffer::uniqueconstptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer14UniqueConstPtrE", false]], "tensorrt_llm::runtime::ibuffer::uniqueptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer9UniquePtrE", false]], "tensorrt_llm::runtime::ibuffer::view (c++ function)": [[1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer4viewE14UniqueConstPtrRR9TConstPtrNSt6size_tE", false], [1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4viewE9SharedPtr", false], [1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4viewE9SharedPtrNSt6size_tE", false]], "tensorrt_llm::runtime::ibuffer::wrap (c++ function)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tENSt6size_tE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrRNSt6vectorI1TEE", false], [1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tE", false], [1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tENSt6size_tE", false]], "tensorrt_llm::runtime::ibuffer::~ibuffer (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBufferD0Ev", false]], "tensorrt_llm::runtime::igptdecoder (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoderE", false]], "tensorrt_llm::runtime::igptdecoder::create (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", false]], "tensorrt_llm::runtime::igptdecoder::disablelookahead (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", false]], "tensorrt_llm::runtime::igptdecoder::forwardasync (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder12forwardAsyncER14DecodingOutputRK13DecodingInput", false]], "tensorrt_llm::runtime::igptdecoder::forwardsync (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder11forwardSyncER14DecodingOutputRK13DecodingInput", false]], "tensorrt_llm::runtime::igptdecoder::getsamplingconfig (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder17getSamplingConfigEv", false]], "tensorrt_llm::runtime::igptdecoder::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", false]], "tensorrt_llm::runtime::igptdecoder::tensorconstptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder14TensorConstPtrE", false]], "tensorrt_llm::runtime::igptdecoder::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder9TensorPtrE", false]], "tensorrt_llm::runtime::igptdecoder::~igptdecoder (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoderD0Ev", false]], "tensorrt_llm::runtime::igptdecoderbatched (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatchedE", false]], "tensorrt_llm::runtime::igptdecoderbatched::cudastreamptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched13CudaStreamPtrE", false]], "tensorrt_llm::runtime::igptdecoderbatched::disablelookahead (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched16disableLookaheadERK13RequestVectorRK9TensorPtr", false]], "tensorrt_llm::runtime::igptdecoderbatched::finalize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime18IGptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", false]], "tensorrt_llm::runtime::igptdecoderbatched::forward (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched7forwardERKN7decoder12DecoderStateERKN13decoder_batch5InputE", false]], "tensorrt_llm::runtime::igptdecoderbatched::forwardasync (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched12forwardAsyncERKN7decoder12DecoderStateERKN13decoder_batch5InputE", false]], "tensorrt_llm::runtime::igptdecoderbatched::igptdecoderbatched (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched18IGptDecoderBatchedEv", false]], "tensorrt_llm::runtime::igptdecoderbatched::llmrequestptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched13LlmRequestPtrE", false]], "tensorrt_llm::runtime::igptdecoderbatched::requestvector (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched13RequestVectorE", false]], "tensorrt_llm::runtime::igptdecoderbatched::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", false]], "tensorrt_llm::runtime::igptdecoderbatched::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched9TensorPtrE", false]], "tensorrt_llm::runtime::igptdecoderbatched::~igptdecoderbatched (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatchedD0Ev", false]], "tensorrt_llm::runtime::ipcmemory (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemoryE", false]], "tensorrt_llm::runtime::ipcmemory::allocateipcmemory (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory17allocateIpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfig", false]], "tensorrt_llm::runtime::ipcmemory::bufferptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9BufferPtrE", false]], "tensorrt_llm::runtime::ipcmemory::destroyipcmemory (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory16destroyIpcMemoryEv", false]], "tensorrt_llm::runtime::ipcmemory::flags_size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory10FLAGS_SIZEE", false]], "tensorrt_llm::runtime::ipcmemory::getcommptrs (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9IpcMemory11getCommPtrsEv", false]], "tensorrt_llm::runtime::ipcmemory::ipcmemory (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfigb", false], [1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryERK9IpcMemory", false], [1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryERR9IpcMemory", false]], "tensorrt_llm::runtime::ipcmemory::mbuffer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory7mBufferE", false]], "tensorrt_llm::runtime::ipcmemory::mcommptrs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9mCommPtrsE", false]], "tensorrt_llm::runtime::ipcmemory::mopenipc (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory8mOpenIpcE", false]], "tensorrt_llm::runtime::ipcmemory::mtprank (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory7mTpRankE", false]], "tensorrt_llm::runtime::ipcmemory::operator= (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemoryaSERK9IpcMemory", false], [1, "_CPPv4N12tensorrt_llm7runtime9IpcMemoryaSERR9IpcMemory", false]], "tensorrt_llm::runtime::ipcmemory::~ipcmemory (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemoryD0Ev", false]], "tensorrt_llm::runtime::ipcnvlsallocate (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime15ipcNvlsAllocateE6size_tNSt3setIiEE", false]], "tensorrt_llm::runtime::ipcnvlsfree (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ipcNvlsFreeEP13IpcNvlsHandle", false]], "tensorrt_llm::runtime::ipcnvlshandle (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandleE", false]], "tensorrt_llm::runtime::ipcnvlshandle::ipc_uc_handles (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle14ipc_uc_handlesE", false]], "tensorrt_llm::runtime::ipcnvlshandle::ipc_uc_ptrs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle11ipc_uc_ptrsE", false]], "tensorrt_llm::runtime::ipcnvlshandle::ipc_uc_vas (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle10ipc_uc_vasE", false]], "tensorrt_llm::runtime::ipcnvlshandle::mc_handle (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle9mc_handleE", false]], "tensorrt_llm::runtime::ipcnvlshandle::mc_ptr (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle6mc_ptrE", false]], "tensorrt_llm::runtime::ipcnvlshandle::mc_va (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle5mc_vaE", false]], "tensorrt_llm::runtime::ipcnvlshandle::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle4sizeE", false]], "tensorrt_llm::runtime::ipcnvlshandle::uc_handle (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle9uc_handleE", false]], "tensorrt_llm::runtime::ipcnvlshandle::uc_ptr (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle6uc_ptrE", false]], "tensorrt_llm::runtime::ipcnvlshandle::uc_va (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle5uc_vaE", false]], "tensorrt_llm::runtime::ipcnvlssupported (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime16ipcNvlsSupportedEv", false]], "tensorrt_llm::runtime::itensor (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensorE", false]], "tensorrt_llm::runtime::itensor::at (c++ function)": [[1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atE14UniqueConstPtrRR9TConstPtrRK5Shape", false], [1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atEN7ITensor14UniqueConstPtrERR9TConstPtrRKNSt16initializer_listI9DimType64EE", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor2atE9SharedPtrRK5Shape", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor2atE9SharedPtrRKNSt16initializer_listI9DimType64EE", false]], "tensorrt_llm::runtime::itensor::castsize (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor8castSizeE6size_t", false]], "tensorrt_llm::runtime::itensor::dimtype64 (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor9DimType64E", false]], "tensorrt_llm::runtime::itensor::flattenn (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor8flattenNE9SharedPtrNSt7int64_tE", false]], "tensorrt_llm::runtime::itensor::getdimension (c++ function)": [[1, "_CPPv4I_10SizeType32ENK12tensorrt_llm7runtime7ITensor12getDimensionE9DimType64v", false]], "tensorrt_llm::runtime::itensor::getshape (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7ITensor8getShapeEv", false]], "tensorrt_llm::runtime::itensor::itensor (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor7ITensorERK7ITensor", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor7ITensorEv", false]], "tensorrt_llm::runtime::itensor::makeshape (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor9makeShapeERKNSt16initializer_listI9DimType64EE", false]], "tensorrt_llm::runtime::itensor::operator= (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensoraSERK7ITensor", false]], "tensorrt_llm::runtime::itensor::reshape (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor7reshapeERK5Shape", false]], "tensorrt_llm::runtime::itensor::resize (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor6resizeENSt6size_tE", false]], "tensorrt_llm::runtime::itensor::shape (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor5ShapeE", false]], "tensorrt_llm::runtime::itensor::shapeequals (c++ function)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor11shapeEqualsEbRK5ShapePK1T10SizeType32", false], [1, "_CPPv4I0ENK12tensorrt_llm7runtime7ITensor11shapeEqualsEbPK1T10SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor11shapeEqualsERK5ShapeRK5Shape", false], [1, "_CPPv4NK12tensorrt_llm7runtime7ITensor11shapeEqualsERK5Shape", false], [1, "_CPPv4NK12tensorrt_llm7runtime7ITensor11shapeEqualsERKNSt16initializer_listI10SizeType32EE", false]], "tensorrt_llm::runtime::itensor::sharedconstptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor14SharedConstPtrE", false]], "tensorrt_llm::runtime::itensor::sharedptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor9SharedPtrE", false]], "tensorrt_llm::runtime::itensor::slice (c++ function)": [[1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", false], [1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", false], [1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5Shape", false], [1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5ShapeNSt6size_tE", false], [1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EE", false], [1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EENSt6size_tE", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrNSt6size_tE", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrNSt6size_tENSt6size_tE", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRK5Shape", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRK5Shape9DimType64", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRKNSt16initializer_listI9DimType64EE", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRKNSt16initializer_listI9DimType64EE9DimType64", false]], "tensorrt_llm::runtime::itensor::squeeze (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor7squeezeE10SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor7squeezeERK5Shape10SizeType32", false]], "tensorrt_llm::runtime::itensor::strides (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor7stridesERK5Shape", false]], "tensorrt_llm::runtime::itensor::tensormap (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor9TensorMapE", false]], "tensorrt_llm::runtime::itensor::tostring (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor8toStringERK5Shape", false]], "tensorrt_llm::runtime::itensor::uniqueconstptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor14UniqueConstPtrE", false]], "tensorrt_llm::runtime::itensor::uniqueptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor9UniquePtrE", false]], "tensorrt_llm::runtime::itensor::unsqueeze (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor9unsqueezeE10SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor9unsqueezeERK5Shape10SizeType32", false]], "tensorrt_llm::runtime::itensor::view (c++ function)": [[1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor4viewE14UniqueConstPtrRR9TConstPtrRK5Shape", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor4viewE9SharedPtr", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor4viewEN7IBuffer9SharedPtrERK5Shape", false]], "tensorrt_llm::runtime::itensor::volume (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor6volumeERK5Shape", false]], "tensorrt_llm::runtime::itensor::volumenonnegative (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor17volumeNonNegativeERK5Shape", false]], "tensorrt_llm::runtime::itensor::wrap (c++ function)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5Shape", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5ShapeNSt6size_tE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrRNSt6vectorI1TEERK5Shape", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5Shape", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5ShapeNSt6size_tE", false]], "tensorrt_llm::runtime::itensor::~itensor (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensorD0Ev", false]], "tensorrt_llm::runtime::lamportinitializeall (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20lamportInitializeAllEPvPvPv6size_t", false]], "tensorrt_llm::runtime::localcreator (c++ struct)": [[1, "_CPPv4I_bEN12tensorrt_llm7runtime12LocalCreatorE", false]], "tensorrt_llm::runtime::localcreator::create (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator6createEv", false]], "tensorrt_llm::runtime::localcreator::localcreator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator12LocalCreatorERK19CUmemAllocationProp6size_t", false]], "tensorrt_llm::runtime::localcreator::mprop (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator5mPropE", false]], "tensorrt_llm::runtime::localcreator::msize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator5mSizeE", false]], "tensorrt_llm::runtime::localcreator::release (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator7releaseE28CUmemGenericAllocationHandleb", false]], "tensorrt_llm::runtime::lookaheaddecodingbuffers (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffersE", false]], "tensorrt_llm::runtime::lookaheaddecodingbuffers::generationlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers17generationLengthsE", false]], "tensorrt_llm::runtime::lookaheaddecodingbuffers::lookaheaddecodingbuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers24LookaheadDecodingBuffersE10SizeType3210SizeType32RK13BufferManager", false]], "tensorrt_llm::runtime::lookaheaddecodingbuffers::packedmasks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers11packedMasksE", false]], "tensorrt_llm::runtime::lookaheaddecodingbuffers::positionids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers11positionIdsE", false]], "tensorrt_llm::runtime::lookaheaddecodingbuffers::positionoffsets (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers15positionOffsetsE", false]], "tensorrt_llm::runtime::lookaheaddecodingbuffers::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers9TensorPtrE", false]], "tensorrt_llm::runtime::lookaheadmodule (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModuleE", false]], "tensorrt_llm::runtime::lookaheadmodule::getexecutionconfig (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime15LookaheadModule18getExecutionConfigEv", false]], "tensorrt_llm::runtime::lookaheadmodule::lookaheadmodule (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule15LookaheadModuleE10SizeType3210SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule15LookaheadModuleEv", false]], "tensorrt_llm::runtime::lookaheadmodule::mexecutionconfig (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule16mExecutionConfigE", false]], "tensorrt_llm::runtime::lookaheadmodule::setexecutionconfig (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule18setExecutionConfigERKN8executor23LookaheadDecodingConfigE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffersE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::batchslotshostcopy (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers18batchSlotsHostCopyE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::cumsumlength (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers12cumSumLengthE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::disablelookaheaddecoding (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers24disableLookaheadDecodingEv", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::enablelookaheaddecoding (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23enableLookaheadDecodingE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::generationlengthsdevice (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23generationLengthsDeviceE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::generationlengthshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers21generationLengthsHostE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::generationlengthshostcopy (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers25generationLengthsHostCopyE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::insertinputtensors (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers18insertInputTensorsER9TensorMapR9TensorMapRK11WorldConfig", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::lookaheadruntimebuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23LookaheadRuntimeBuffersE10SizeType3210SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfigRKN8executor14DecodingConfigERK11TllmRuntime", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::packedmaskhost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers14packedMaskHostE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::packedmaskhostcopy (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers18packedMaskHostCopyE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::packedmasksdevice (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers17packedMasksDeviceE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::positionidsdevice (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers17positionIdsDeviceE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::positionidshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers15positionIdsHostE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::positionidshostcopy (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers19positionIdsHostCopyE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::positionoffsetsdevice (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers21positionOffsetsDeviceE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::positionoffsetshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers19positionOffsetsHostE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::positionoffsetshostcopy (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23positionOffsetsHostCopyE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::reshape (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers7reshapeE10SizeType3210SizeType3210SizeType32", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::setfrominputs (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::tensormap (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers9TensorMapE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers9TensorPtrE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::usespecdecoding (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers15useSpecDecodingE", false]], "tensorrt_llm::runtime::loracache (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCacheE", false]], "tensorrt_llm::runtime::loracache::bump (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache4bumpE10TaskIdType", false]], "tensorrt_llm::runtime::loracache::bumptaskinprogress (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache18bumpTaskInProgressE10TaskIdType", false]], "tensorrt_llm::runtime::loracache::claimpageswithevict (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache19claimPagesWithEvictE10SizeType32", false]], "tensorrt_llm::runtime::loracache::copytask (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache8copyTaskE10TaskIdTypeR9LoraCacheb", false]], "tensorrt_llm::runtime::loracache::copytaskmappages (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache16copyTaskMapPagesER9TaskValueRK9TaskValueRKNSt6vectorI6size_tEERK9LoraCache", false]], "tensorrt_llm::runtime::loracache::copytopages (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", false]], "tensorrt_llm::runtime::loracache::determinenumpages (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache17determineNumPagesE10TaskIdType", false], [1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache17determineNumPagesE9TensorPtr", false]], "tensorrt_llm::runtime::loracache::fits (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache4fitsE9TensorPtr", false]], "tensorrt_llm::runtime::loracache::get (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache3getE10TaskIdType", false]], "tensorrt_llm::runtime::loracache::getnumpages (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache11getNumPagesEv", false]], "tensorrt_llm::runtime::loracache::getpageptr (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache10getPagePtrE6size_t", false]], "tensorrt_llm::runtime::loracache::getstatus (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache9getStatusE10TaskIdType", false]], "tensorrt_llm::runtime::loracache::has (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache3hasE10TaskIdType", false]], "tensorrt_llm::runtime::loracache::isdone (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache6isDoneE10TaskIdType", false]], "tensorrt_llm::runtime::loracache::isloaded (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache8isLoadedE10TaskIdType", false]], "tensorrt_llm::runtime::loracache::loadweights (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsE10TaskIdType9TensorPtr9TensorPtr", false], [1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsER9TaskValue9TensorPtr9TensorPtr", false]], "tensorrt_llm::runtime::loracache::loracache (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9LoraCacheERK26LoraCachePageManagerConfigRK11ModelConfigRK11WorldConfigRK13BufferManager", false]], "tensorrt_llm::runtime::loracache::markalldone (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11markAllDoneEv", false]], "tensorrt_llm::runtime::loracache::marktaskdone (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache12markTaskDoneE10TaskIdType", false]], "tensorrt_llm::runtime::loracache::mbuffermanager (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache14mBufferManagerE", false]], "tensorrt_llm::runtime::loracache::mcachemap (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9mCacheMapE", false]], "tensorrt_llm::runtime::loracache::mcachemutex (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11mCacheMutexE", false]], "tensorrt_llm::runtime::loracache::mcachepagemanager (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17mCachePageManagerE", false]], "tensorrt_llm::runtime::loracache::mdevicebuffermanagers (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21mDeviceBufferManagersE", false]], "tensorrt_llm::runtime::loracache::mdonetasks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache10mDoneTasksE", false]], "tensorrt_llm::runtime::loracache::minprogresstasks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache16mInProgressTasksE", false]], "tensorrt_llm::runtime::loracache::mmodelconfig (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache12mModelConfigE", false]], "tensorrt_llm::runtime::loracache::mmoduleidtomodule (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17mModuleIdToModuleE", false]], "tensorrt_llm::runtime::loracache::mpagemanagerconfig (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache18mPageManagerConfigE", false]], "tensorrt_llm::runtime::loracache::mpagesmutex (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11mPagesMutexE", false]], "tensorrt_llm::runtime::loracache::mworldconfig (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache12mWorldConfigE", false]], "tensorrt_llm::runtime::loracache::put (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache3putE10TaskIdType9TensorPtr9TensorPtrb", false]], "tensorrt_llm::runtime::loracache::splittransposecpu (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17splitTransposeCpuER7ITensorRK7ITensor10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::loracache::splittransposecpuinner (c++ function)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime9LoraCache22splitTransposeCpuInnerEvR7ITensorRK7ITensor10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::loracache::taskidtype (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache10TaskIdTypeE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfigE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::adaptersize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig11adapterSizeE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::insize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig6inSizeE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::layerid (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig7layerIdE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::moduleid (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig8moduleIdE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::numslots (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig8numSlotsE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::operator== (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfigeqERKN9LoraCache21TaskLayerModuleConfigE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::outsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig7outSizeE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::pageid (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig6pageIdE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::scalingvecpointer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig17scalingVecPointerE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::slotidx (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig7slotIdxE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::tostring (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig8toStringEv", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::weightsinpointer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig16weightsInPointerE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::weightsoutpointer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig17weightsOutPointerE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfiglistptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache28TaskLayerModuleConfigListPtrE", false]], "tensorrt_llm::runtime::loracache::taskvalue (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValueE", false]], "tensorrt_llm::runtime::loracache::taskvalue::configs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue7configsE", false]], "tensorrt_llm::runtime::loracache::taskvalue::done (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue4doneE", false]], "tensorrt_llm::runtime::loracache::taskvalue::inprogress (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue10inProgressE", false]], "tensorrt_llm::runtime::loracache::taskvalue::it (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue2itE", false]], "tensorrt_llm::runtime::loracache::taskvalue::loaded (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue6loadedE", false]], "tensorrt_llm::runtime::loracache::taskvalue::loadinprogress (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue14loadInProgressE", false]], "tensorrt_llm::runtime::loracache::taskvalue::operator= (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValueaSERR9TaskValue", false]], "tensorrt_llm::runtime::loracache::taskvalue::pageids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue7pageIdsE", false]], "tensorrt_llm::runtime::loracache::taskvalue::taskvalue (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERKNSt6vectorINSt6size_tEEERK28TaskLayerModuleConfigListPtrNSt4listI10TaskIdTypeE8iteratorEbbbb", false], [1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERR9TaskValue", false], [1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueEv", false]], "tensorrt_llm::runtime::loracache::taskvalue::~taskvalue (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValueD0Ev", false]], "tensorrt_llm::runtime::loracache::taskvalueptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache12TaskValuePtrE", false]], "tensorrt_llm::runtime::loracache::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TensorPtrE", false]], "tensorrt_llm::runtime::loracache::valuestatus (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatusE", false]], "tensorrt_llm::runtime::loracache::valuestatus::kvalue_status_loaded (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatus20kVALUE_STATUS_LOADEDE", false]], "tensorrt_llm::runtime::loracache::valuestatus::kvalue_status_missing (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatus21kVALUE_STATUS_MISSINGE", false]], "tensorrt_llm::runtime::loracache::valuestatus::kvalue_status_processing (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatus24kVALUE_STATUS_PROCESSINGE", false]], "tensorrt_llm::runtime::loracachefullexception (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime22LoraCacheFullExceptionE", false]], "tensorrt_llm::runtime::loracachefullexception::loracachefullexception (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22LoraCacheFullException22LoraCacheFullExceptionERKNSt6stringE", false]], "tensorrt_llm::runtime::loracachefullexception::~loracachefullexception (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22LoraCacheFullExceptionD0Ev", false]], "tensorrt_llm::runtime::loracachepagemanager (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManagerE", false]], "tensorrt_llm::runtime::loracachepagemanager::blockptr (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime20LoraCachePageManager8blockPtrE10SizeType32", false]], "tensorrt_llm::runtime::loracachepagemanager::claimpages (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager10claimPagesE10SizeType32", false]], "tensorrt_llm::runtime::loracachepagemanager::initialize (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager10initializeERK13BufferManager", false]], "tensorrt_llm::runtime::loracachepagemanager::loracachepagemanager (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager20LoraCachePageManagerERK26LoraCachePageManagerConfigRK13BufferManager", false]], "tensorrt_llm::runtime::loracachepagemanager::mconfig (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager7mConfigE", false]], "tensorrt_llm::runtime::loracachepagemanager::mfreepageids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager12mFreePageIdsE", false]], "tensorrt_llm::runtime::loracachepagemanager::mispagefree (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager11mIsPageFreeE", false]], "tensorrt_llm::runtime::loracachepagemanager::mpageblocks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager11mPageBlocksE", false]], "tensorrt_llm::runtime::loracachepagemanager::mutablepageptr (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager14mutablePagePtrENSt6size_tE", false]], "tensorrt_llm::runtime::loracachepagemanager::numavailablepages (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime20LoraCachePageManager17numAvailablePagesEv", false]], "tensorrt_llm::runtime::loracachepagemanager::pageptr (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime20LoraCachePageManager7pagePtrENSt6size_tE", false]], "tensorrt_llm::runtime::loracachepagemanager::releasepages (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager12releasePagesERKNSt6vectorINSt6size_tEEE", false]], "tensorrt_llm::runtime::loracachepagemanager::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager9TensorPtrE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfigE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::getdatatype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig11getDataTypeEv", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::getinittozero (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig13getInitToZeroEv", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::getmaxpagesperblock (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig19getMaxPagesPerBlockEv", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::getmemorytype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig13getMemoryTypeEv", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::getnumcopystreams (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig17getNumCopyStreamsEv", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::getpagewidth (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig12getPageWidthEv", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::getslotsperpage (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig15getSlotsPerPageEv", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::gettotalnumpages (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig16getTotalNumPagesEv", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::loracachepagemanagerconfig (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig26LoraCachePageManagerConfigEN7runtime10MemoryTypeEN8nvinfer18DataTypeE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::mdatatype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig9mDataTypeE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::minittozero (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig11mInitToZeroE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::mmaxpagesperblock (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig17mMaxPagesPerBlockE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::mmemorytype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig11mMemoryTypeE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::mnumcopystreams (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig15mNumCopyStreamsE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::mpagewidth (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig10mPageWidthE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::mslotsperpage (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig13mSlotsPerPageE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::mtotalnumpages (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig14mTotalNumPagesE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::setdatatype (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig11setDataTypeERKN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::setinittozero (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig13setInitToZeroEb", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::setmaxpagesperblock (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig19setMaxPagesPerBlockERK10SizeType32", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::setmemorytype (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig13setMemoryTypeERKN7runtime10MemoryTypeE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::setnumcopystreams (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig17setNumCopyStreamsE10SizeType32", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::setpagewidth (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig12setPageWidthERK10SizeType32", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::setslotsperpage (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig15setSlotsPerPageERK10SizeType32", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::settotalnumpage (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig15setTotalNumPageERK10SizeType32", false]], "tensorrt_llm::runtime::loraexpectedexception (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime21LoraExpectedExceptionE", false]], "tensorrt_llm::runtime::loraexpectedexception::loraexpectedexception (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime21LoraExpectedException21LoraExpectedExceptionERKNSt6stringE", false]], "tensorrt_llm::runtime::loraexpectedexception::~loraexpectedexception (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime21LoraExpectedExceptionD0Ev", false]], "tensorrt_llm::runtime::loramodule (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModuleE", false]], "tensorrt_llm::runtime::loramodule::createloramodules (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", false]], "tensorrt_llm::runtime::loramodule::flattenedinoutsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule18flattenedInOutSizeE10SizeType32b", false]], "tensorrt_llm::runtime::loramodule::indim (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule5inDimEv", false]], "tensorrt_llm::runtime::loramodule::indimfirst (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule10inDimFirstEv", false]], "tensorrt_llm::runtime::loramodule::insize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule6inSizeE10SizeType32", false]], "tensorrt_llm::runtime::loramodule::intpsplitdim (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule12inTpSplitDimEv", false]], "tensorrt_llm::runtime::loramodule::localinadaptersize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule18localInAdapterSizeE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::loramodule::localindim (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule10localInDimE10SizeType32", false]], "tensorrt_llm::runtime::loramodule::localinoutsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule14localInOutSizeE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::loramodule::localinsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule11localInSizeE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::loramodule::localoutadaptersize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule19localOutAdapterSizeE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::loramodule::localoutdim (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule11localOutDimE10SizeType32", false]], "tensorrt_llm::runtime::loramodule::localoutsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule12localOutSizeE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::loramodule::localscalessize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule15localScalesSizeE10SizeType32b", false]], "tensorrt_llm::runtime::loramodule::localtotalsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule14localTotalSizeE10SizeType3210SizeType32b", false]], "tensorrt_llm::runtime::loramodule::loramodule (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10LoraModule", false], [1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10ModuleType10SizeType3210SizeType32bb10SizeType3210SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleEv", false]], "tensorrt_llm::runtime::loramodule::mindim (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule6mInDimE", false]], "tensorrt_llm::runtime::loramodule::mindimfirst (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule11mInDimFirstE", false]], "tensorrt_llm::runtime::loramodule::mintpsplitdim (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule13mInTpSplitDimE", false]], "tensorrt_llm::runtime::loramodule::moduletype (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleTypeE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kattn_dense (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType11kATTN_DENSEE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kattn_k (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType7kATTN_KE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kattn_q (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType7kATTN_QE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kattn_qkv (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType9kATTN_QKVE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kattn_v (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType7kATTN_VE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kcross_attn_dense (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType17kCROSS_ATTN_DENSEE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kcross_attn_k (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType13kCROSS_ATTN_KE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kcross_attn_q (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType13kCROSS_ATTN_QE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kcross_attn_qkv (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType15kCROSS_ATTN_QKVE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kcross_attn_v (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType13kCROSS_ATTN_VE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kinvalid (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType8kINVALIDE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kmlp_4h_to_h (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMLP_4H_TO_HE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kmlp_gate (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType9kMLP_GATEE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kmlp_gate_up (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMLP_GATE_UPE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kmlp_h_to_4h (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMLP_H_TO_4HE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kmlp_router (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType11kMLP_ROUTERE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kmoe_4h_to_h (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMOE_4H_TO_HE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kmoe_gate (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType9kMOE_GATEE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kmoe_h_to_4h (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMOE_H_TO_4HE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kmoe_router (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType11kMOE_ROUTERE", false]], "tensorrt_llm::runtime::loramodule::moutdim (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule7mOutDimE", false]], "tensorrt_llm::runtime::loramodule::moutdimfirst (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12mOutDimFirstE", false]], "tensorrt_llm::runtime::loramodule::mouttpsplitdim (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule14mOutTpSplitDimE", false]], "tensorrt_llm::runtime::loramodule::mtype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule5mTypeE", false]], "tensorrt_llm::runtime::loramodule::name (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule4nameEv", false]], "tensorrt_llm::runtime::loramodule::operator= (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModuleaSERK10LoraModule", false]], "tensorrt_llm::runtime::loramodule::outdim (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule6outDimEv", false]], "tensorrt_llm::runtime::loramodule::outdimfirst (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule11outDimFirstEv", false]], "tensorrt_llm::runtime::loramodule::outsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule7outSizeE10SizeType32", false]], "tensorrt_llm::runtime::loramodule::outtpsplitdim (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule13outTpSplitDimEv", false]], "tensorrt_llm::runtime::loramodule::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule9TensorPtrE", false]], "tensorrt_llm::runtime::loramodule::tomodulename (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12toModuleNameE10ModuleType", false], [1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12toModuleNameE10SizeType32", false]], "tensorrt_llm::runtime::loramodule::tomoduletype (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12toModuleTypeERKNSt11string_viewE", false]], "tensorrt_llm::runtime::loramodule::value (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule5valueEv", false]], "tensorrt_llm::runtime::lorataskidtype (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14LoraTaskIdTypeE", false]], "tensorrt_llm::runtime::medusamodule (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime12MedusaModuleE", false]], "tensorrt_llm::runtime::medusamodule::getmedusachoices (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime12MedusaModule16getMedusaChoicesEv", false]], "tensorrt_llm::runtime::medusamodule::mdefaultmedusachoices (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule21mDefaultMedusaChoicesE", false]], "tensorrt_llm::runtime::medusamodule::medusachoices (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule13MedusaChoicesE", false]], "tensorrt_llm::runtime::medusamodule::medusamodule (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule12MedusaModuleE10SizeType3210SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule12MedusaModuleEv", false]], "tensorrt_llm::runtime::medusamodule::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule9TensorPtrE", false]], "tensorrt_llm::runtime::memorycounters (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCountersE", false]], "tensorrt_llm::runtime::memorycounters::allocate (c++ function)": [[1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime14MemoryCounters8allocateEv10SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8allocateE10MemoryType10SizeType32", false]], "tensorrt_llm::runtime::memorycounters::bytestostring (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters13bytesToStringE10SizeType32i", false], [1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters13bytesToStringE8DiffTypei", false]], "tensorrt_llm::runtime::memorycounters::deallocate (c++ function)": [[1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime14MemoryCounters10deallocateEv10SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters10deallocateE10MemoryType10SizeType32", false]], "tensorrt_llm::runtime::memorycounters::difftype (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8DiffTypeE", false]], "tensorrt_llm::runtime::memorycounters::getcpu (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters6getCpuEv", false]], "tensorrt_llm::runtime::memorycounters::getcpudiff (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters10getCpuDiffEv", false]], "tensorrt_llm::runtime::memorycounters::getgpu (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters6getGpuEv", false]], "tensorrt_llm::runtime::memorycounters::getgpudiff (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters10getGpuDiffEv", false]], "tensorrt_llm::runtime::memorycounters::getinstance (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters11getInstanceEv", false]], "tensorrt_llm::runtime::memorycounters::getpinned (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters9getPinnedEv", false]], "tensorrt_llm::runtime::memorycounters::getpinneddiff (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters13getPinnedDiffEv", false]], "tensorrt_llm::runtime::memorycounters::getpinnedpool (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters13getPinnedPoolEv", false]], "tensorrt_llm::runtime::memorycounters::getpinnedpooldiff (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters17getPinnedPoolDiffEv", false]], "tensorrt_llm::runtime::memorycounters::getuvm (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters6getUVMEv", false]], "tensorrt_llm::runtime::memorycounters::getuvmdiff (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters10getUVMDiffEv", false]], "tensorrt_llm::runtime::memorycounters::mcpu (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters4mCpuE", false]], "tensorrt_llm::runtime::memorycounters::mcpudiff (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8mCpuDiffE", false]], "tensorrt_llm::runtime::memorycounters::memorycounters (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters14MemoryCountersEv", false]], "tensorrt_llm::runtime::memorycounters::mgpu (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters4mGpuE", false]], "tensorrt_llm::runtime::memorycounters::mgpudiff (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8mGpuDiffE", false]], "tensorrt_llm::runtime::memorycounters::mpinned (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters7mPinnedE", false]], "tensorrt_llm::runtime::memorycounters::mpinneddiff (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters11mPinnedDiffE", false]], "tensorrt_llm::runtime::memorycounters::mpinnedpool (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters11mPinnedPoolE", false]], "tensorrt_llm::runtime::memorycounters::mpinnedpooldiff (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters15mPinnedPoolDiffE", false]], "tensorrt_llm::runtime::memorycounters::muvm (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters4mUVME", false]], "tensorrt_llm::runtime::memorycounters::muvmdiff (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8mUVMDiffE", false]], "tensorrt_llm::runtime::memorycounters::sizetype32 (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters10SizeType32E", false]], "tensorrt_llm::runtime::memorycounters::tostring (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters8toStringEv", false]], "tensorrt_llm::runtime::memorytype (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime10MemoryTypeE", false]], "tensorrt_llm::runtime::memorytype::kcpu (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10MemoryType4kCPUE", false]], "tensorrt_llm::runtime::memorytype::kgpu (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10MemoryType4kGPUE", false]], "tensorrt_llm::runtime::memorytype::kpinned (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10MemoryType7kPINNEDE", false]], "tensorrt_llm::runtime::memorytype::kpinnedpool (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10MemoryType11kPINNEDPOOLE", false]], "tensorrt_llm::runtime::memorytype::kuvm (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10MemoryType4kUVME", false]], "tensorrt_llm::runtime::memorytypestring (c++ struct)": [[1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime16MemoryTypeStringE", false]], "tensorrt_llm::runtime::memorytypestring (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kCPUEEE", false]], "tensorrt_llm::runtime::memorytypestring::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kCPUEE5valueE", false]], "tensorrt_llm::runtime::memorytypestring (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kGPUEEE", false]], "tensorrt_llm::runtime::memorytypestring::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kGPUEE5valueE", false]], "tensorrt_llm::runtime::memorytypestring (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType7kPINNEDEEE", false]], "tensorrt_llm::runtime::memorytypestring::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType7kPINNEDEE5valueE", false]], "tensorrt_llm::runtime::memorytypestring (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType11kPINNEDPOOLEEE", false]], "tensorrt_llm::runtime::memorytypestring::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType11kPINNEDPOOLEE5valueE", false]], "tensorrt_llm::runtime::memorytypestring (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kUVMEEE", false]], "tensorrt_llm::runtime::memorytypestring::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kUVMEE5valueE", false]], "tensorrt_llm::runtime::memsetconfigurator (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfiguratorE", false]], "tensorrt_llm::runtime::memsetconfigurator::maddress (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator8mAddressE", false]], "tensorrt_llm::runtime::memsetconfigurator::memsetconfigurator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator18MemsetConfiguratorE11CUdeviceptr6size_t7uint8_t8CUstream", false]], "tensorrt_llm::runtime::memsetconfigurator::mfirsttime (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator10mFirstTimeE", false]], "tensorrt_llm::runtime::memsetconfigurator::msize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator5mSizeE", false]], "tensorrt_llm::runtime::memsetconfigurator::mstream (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator7mStreamE", false]], "tensorrt_llm::runtime::memsetconfigurator::mvalue (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator6mValueE", false]], "tensorrt_llm::runtime::memsetconfigurator::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator5setupE28CUmemGenericAllocationHandle", false]], "tensorrt_llm::runtime::memsetconfigurator::teardown (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator8teardownE28CUmemGenericAllocationHandleb", false]], "tensorrt_llm::runtime::modelconfig (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfigE", false]], "tensorrt_llm::runtime::modelconfig::computecontextlogits (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20computeContextLogitsEb", false], [1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20computeContextLogitsEv", false]], "tensorrt_llm::runtime::modelconfig::computegenerationlogits (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig23computeGenerationLogitsEb", false], [1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig23computeGenerationLogitsEv", false]], "tensorrt_llm::runtime::modelconfig::countlocallayers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16countLocalLayersE9LayerType10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::modelconfig::countlowerranklayers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20countLowerRankLayersE9LayerType10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::modelconfig::disableseamlesslookaheaddecoding (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig32disableSeamlessLookaheadDecodingEv", false]], "tensorrt_llm::runtime::modelconfig::enableseamlesslookaheaddecoding (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig31enableSeamlessLookaheadDecodingE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::getcontextfmha (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getContextFMHAEv", false]], "tensorrt_llm::runtime::modelconfig::getdatatype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig11getDataTypeEv", false]], "tensorrt_llm::runtime::modelconfig::getencoderhiddensize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getEncoderHiddenSizeEv", false]], "tensorrt_llm::runtime::modelconfig::getfirstlocallayer (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18getFirstLocalLayerE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::modelconfig::getgemmallreducedtype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21getGemmAllReduceDtypeEv", false]], "tensorrt_llm::runtime::modelconfig::gethiddensize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13getHiddenSizeEv", false]], "tensorrt_llm::runtime::modelconfig::getkvcachetype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getKVCacheTypeEv", false]], "tensorrt_llm::runtime::modelconfig::getkvdatatype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13getKvDataTypeEv", false]], "tensorrt_llm::runtime::modelconfig::getlayertypes (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13getLayerTypesEv", false]], "tensorrt_llm::runtime::modelconfig::getlogitsdtype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getLogitsDtypeEv", false]], "tensorrt_llm::runtime::modelconfig::getloramodules (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getLoraModulesEv", false]], "tensorrt_llm::runtime::modelconfig::getmanageweightstype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getManageWeightsTypeEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxbatchsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getMaxBatchSizeEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxbeamwidth (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getMaxBeamWidthEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxdecodingdrafttokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig25getMaxDecodingDraftTokensEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxdecodingtokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getMaxDecodingTokensEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxencoderlen (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16getMaxEncoderLenEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxinputlen (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getMaxInputLenEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxlorarank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getMaxLoraRankEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxnumtokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getMaxNumTokensEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxpositionembeddings (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig24getMaxPositionEmbeddingsEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxpromptembeddingtablesize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig30getMaxPromptEmbeddingTableSizeEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxsequencelen (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig17getMaxSequenceLenEv", false]], "tensorrt_llm::runtime::modelconfig::getmlphiddensize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16getMlpHiddenSizeEv", false]], "tensorrt_llm::runtime::modelconfig::getmodelname (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getModelNameEv", false]], "tensorrt_llm::runtime::modelconfig::getmodelvariant (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getModelVariantEv", false]], "tensorrt_llm::runtime::modelconfig::getnbattentionlayers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getNbAttentionLayersE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::modelconfig::getnbheads (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig10getNbHeadsEv", false]], "tensorrt_llm::runtime::modelconfig::getnbkvheads (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getNbKvHeadsE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::getnblayers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig11getNbLayersE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::modelconfig::getnbrnnlayers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getNbRnnLayersE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::modelconfig::getnumkvheadsforgivenlayers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig27getNumKvHeadsForGivenLayersERKNSt6vectorI10SizeType32EEb", false]], "tensorrt_llm::runtime::modelconfig::getnumkvheadsperlayer (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21getNumKvHeadsPerLayerEv", false]], "tensorrt_llm::runtime::modelconfig::getnumkvheadsperlayerlocalrange (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig31getNumKvHeadsPerLayerLocalRangeE10SizeType3210SizeType32b", false]], "tensorrt_llm::runtime::modelconfig::getnumlanguages (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getNumLanguagesEv", false]], "tensorrt_llm::runtime::modelconfig::getoptprofilessplitpoints (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig25getOptProfilesSplitPointsEv", false]], "tensorrt_llm::runtime::modelconfig::getpagedcontextfmha (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig19getPagedContextFMHAEv", false]], "tensorrt_llm::runtime::modelconfig::getppreducescatter (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18getPpReduceScatterEv", false]], "tensorrt_llm::runtime::modelconfig::getquantmode (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getQuantModeEv", false]], "tensorrt_llm::runtime::modelconfig::getrnnconfig (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getRnnConfigEv", false]], "tensorrt_llm::runtime::modelconfig::getrotaryembeddingdim (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21getRotaryEmbeddingDimEv", false]], "tensorrt_llm::runtime::modelconfig::getsizeperhead (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getSizePerHeadEv", false]], "tensorrt_llm::runtime::modelconfig::getspeculativedecodingmode (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig26getSpeculativeDecodingModeEv", false]], "tensorrt_llm::runtime::modelconfig::getspeculativedecodingmodule (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig28getSpeculativeDecodingModuleEv", false]], "tensorrt_llm::runtime::modelconfig::getspeculativedecodingmoduleptr (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig31getSpeculativeDecodingModulePtrEv", false], [1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig31getSpeculativeDecodingModulePtrEv", false]], "tensorrt_llm::runtime::modelconfig::gettokensperblock (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig17getTokensPerBlockEv", false]], "tensorrt_llm::runtime::modelconfig::getvocabsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getVocabSizeEv", false]], "tensorrt_llm::runtime::modelconfig::getvocabsizepadded (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18getVocabSizePaddedE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::hasrnnconfig (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12hasRnnConfigEv", false]], "tensorrt_llm::runtime::modelconfig::hasspeculativedecodingmodule (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig28hasSpeculativeDecodingModuleEv", false]], "tensorrt_llm::runtime::modelconfig::iscontinuouskvcache (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig19isContinuousKVCacheEv", false]], "tensorrt_llm::runtime::modelconfig::iskvcacheenabled (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16isKVCacheEnabledEv", false]], "tensorrt_llm::runtime::modelconfig::ismultimodal (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12isMultiModalEv", false]], "tensorrt_llm::runtime::modelconfig::ispagedkvcache (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14isPagedKVCacheEv", false]], "tensorrt_llm::runtime::modelconfig::isrnnbased (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig10isRnnBasedEv", false]], "tensorrt_llm::runtime::modelconfig::istransformerbased (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18isTransformerBasedEv", false]], "tensorrt_llm::runtime::modelconfig::iswhisper (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig9isWhisperEv", false]], "tensorrt_llm::runtime::modelconfig::kdefault_num_tokens_per_block (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig29kDEFAULT_NUM_TOKENS_PER_BLOCKE", false]], "tensorrt_llm::runtime::modelconfig::kopt_profiles_split_points (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26kOPT_PROFILES_SPLIT_POINTSE", false]], "tensorrt_llm::runtime::modelconfig::kvcachetype (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11KVCacheTypeE", false]], "tensorrt_llm::runtime::modelconfig::kvcachetype::kcontinuous (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11KVCacheType11kCONTINUOUSE", false]], "tensorrt_llm::runtime::modelconfig::kvcachetype::kdisabled (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11KVCacheType9kDISABLEDE", false]], "tensorrt_llm::runtime::modelconfig::kvcachetype::kpaged (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11KVCacheType6kPAGEDE", false]], "tensorrt_llm::runtime::modelconfig::kvcachetypefromstring (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21KVCacheTypeFromStringENSt6stringE", false]], "tensorrt_llm::runtime::modelconfig::layertype (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerTypeE", false]], "tensorrt_llm::runtime::modelconfig::layertype::kattention (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerType10kATTENTIONE", false]], "tensorrt_llm::runtime::modelconfig::layertype::klinear (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerType7kLINEARE", false]], "tensorrt_llm::runtime::modelconfig::layertype::knoop (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerType5kNOOPE", false]], "tensorrt_llm::runtime::modelconfig::layertype::krecurrent (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerType10kRECURRENTE", false]], "tensorrt_llm::runtime::modelconfig::manageweightstype (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17ManageWeightsTypeE", false]], "tensorrt_llm::runtime::modelconfig::manageweightstype::kdisabled (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17ManageWeightsType9kDisabledE", false]], "tensorrt_llm::runtime::modelconfig::manageweightstype::kenabled (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17ManageWeightsType8kEnabledE", false]], "tensorrt_llm::runtime::modelconfig::mcomputecontextlogits (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21mComputeContextLogitsE", false]], "tensorrt_llm::runtime::modelconfig::mcomputegenerationlogits (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24mComputeGenerationLogitsE", false]], "tensorrt_llm::runtime::modelconfig::mcontextfmha (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mContextFMHAE", false]], "tensorrt_llm::runtime::modelconfig::mdatatype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9mDataTypeE", false]], "tensorrt_llm::runtime::modelconfig::mencoderhiddensize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mEncoderHiddenSizeE", false]], "tensorrt_llm::runtime::modelconfig::mgemmallreducedtype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig19mGemmAllReduceDtypeE", false]], "tensorrt_llm::runtime::modelconfig::mhiddensize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11mHiddenSizeE", false]], "tensorrt_llm::runtime::modelconfig::minputpacked (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mInputPackedE", false]], "tensorrt_llm::runtime::modelconfig::mkvcachetype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mKVCacheTypeE", false]], "tensorrt_llm::runtime::modelconfig::mlayertypes (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11mLayerTypesE", false]], "tensorrt_llm::runtime::modelconfig::mlogitsdtype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mLogitsDtypeE", false]], "tensorrt_llm::runtime::modelconfig::mloramodules (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mLoraModulesE", false]], "tensorrt_llm::runtime::modelconfig::mmanageweightstype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mManageWeightsTypeE", false]], "tensorrt_llm::runtime::modelconfig::mmaxbatchsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mMaxBatchSizeE", false]], "tensorrt_llm::runtime::modelconfig::mmaxbeamwidth (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mMaxBeamWidthE", false]], "tensorrt_llm::runtime::modelconfig::mmaxencoderlen (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14mMaxEncoderLenE", false]], "tensorrt_llm::runtime::modelconfig::mmaxinputlen (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mMaxInputLenE", false]], "tensorrt_llm::runtime::modelconfig::mmaxlorarank (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mMaxLoraRankE", false]], "tensorrt_llm::runtime::modelconfig::mmaxnumtokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mMaxNumTokensE", false]], "tensorrt_llm::runtime::modelconfig::mmaxpositionembeddings (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22mMaxPositionEmbeddingsE", false]], "tensorrt_llm::runtime::modelconfig::mmaxpromptembeddingtablesize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig28mMaxPromptEmbeddingTableSizeE", false]], "tensorrt_llm::runtime::modelconfig::mmaxsequencelen (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15mMaxSequenceLenE", false]], "tensorrt_llm::runtime::modelconfig::mmlphiddensize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14mMlpHiddenSizeE", false]], "tensorrt_llm::runtime::modelconfig::mmodelname (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig10mModelNameE", false]], "tensorrt_llm::runtime::modelconfig::mmodelvariant (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mModelVariantE", false]], "tensorrt_llm::runtime::modelconfig::mnbattentionlayers (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mNbAttentionLayersE", false]], "tensorrt_llm::runtime::modelconfig::mnbheads (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig8mNbHeadsE", false]], "tensorrt_llm::runtime::modelconfig::mnblayers (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9mNbLayersE", false]], "tensorrt_llm::runtime::modelconfig::mnbrnnlayers (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mNbRnnLayersE", false]], "tensorrt_llm::runtime::modelconfig::mnumkvheadsperattentionlayer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig28mNumKvHeadsPerAttentionLayerE", false]], "tensorrt_llm::runtime::modelconfig::mnumkvheadspercrossattentionlayer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig33mNumKvHeadsPerCrossAttentionLayerE", false]], "tensorrt_llm::runtime::modelconfig::mnumlanguages (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mNumLanguagesE", false]], "tensorrt_llm::runtime::modelconfig::modelconfig (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11ModelConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::modelconfig::modelvariant (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariantE", false]], "tensorrt_llm::runtime::modelconfig::modelvariant::kchatglm (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant8kChatGlmE", false]], "tensorrt_llm::runtime::modelconfig::modelvariant::kencdec (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant7kEncDecE", false]], "tensorrt_llm::runtime::modelconfig::modelvariant::kglm (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant4kGlmE", false]], "tensorrt_llm::runtime::modelconfig::modelvariant::kgpt (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant4kGptE", false]], "tensorrt_llm::runtime::modelconfig::modelvariant::kmamba (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant6kMambaE", false]], "tensorrt_llm::runtime::modelconfig::modelvariant::krecurrentgemma (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant15kRecurrentGemmaE", false]], "tensorrt_llm::runtime::modelconfig::mpagedcontextfmha (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17mPagedContextFMHAE", false]], "tensorrt_llm::runtime::modelconfig::mpagedstate (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11mPagedStateE", false]], "tensorrt_llm::runtime::modelconfig::mppreducescatter (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig16mPpReduceScatterE", false]], "tensorrt_llm::runtime::modelconfig::mquantmode (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig10mQuantModeE", false]], "tensorrt_llm::runtime::modelconfig::mrnnconfig (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig10mRnnConfigE", false]], "tensorrt_llm::runtime::modelconfig::mrotaryembeddingdim (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig19mRotaryEmbeddingDimE", false]], "tensorrt_llm::runtime::modelconfig::msizeperhead (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mSizePerHeadE", false]], "tensorrt_llm::runtime::modelconfig::mskipcrossattnblocks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20mSkipCrossAttnBlocksE", false]], "tensorrt_llm::runtime::modelconfig::mspeculativedecodingmode (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24mSpeculativeDecodingModeE", false]], "tensorrt_llm::runtime::modelconfig::mspeculativedecodingmodule (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26mSpeculativeDecodingModuleE", false]], "tensorrt_llm::runtime::modelconfig::mtokensperblock (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15mTokensPerBlockE", false]], "tensorrt_llm::runtime::modelconfig::musecrossattention (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mUseCrossAttentionE", false]], "tensorrt_llm::runtime::modelconfig::musegemmallreduceplugin (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig23mUseGemmAllReducePluginE", false]], "tensorrt_llm::runtime::modelconfig::musegptattentionplugin (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22mUseGptAttentionPluginE", false]], "tensorrt_llm::runtime::modelconfig::museloraplugin (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14mUseLoraPluginE", false]], "tensorrt_llm::runtime::modelconfig::musemambaconv1dplugin (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21mUseMambaConv1dPluginE", false]], "tensorrt_llm::runtime::modelconfig::musemrope (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9mUseMropeE", false]], "tensorrt_llm::runtime::modelconfig::musepositionembedding (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21mUsePositionEmbeddingE", false]], "tensorrt_llm::runtime::modelconfig::museshapeinference (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mUseShapeInferenceE", false]], "tensorrt_llm::runtime::modelconfig::musetokentypeembedding (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22mUseTokenTypeEmbeddingE", false]], "tensorrt_llm::runtime::modelconfig::mvocabsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig10mVocabSizeE", false]], "tensorrt_llm::runtime::modelconfig::resetspeculativedecodingmodule (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig30resetSpeculativeDecodingModuleEv", false]], "tensorrt_llm::runtime::modelconfig::rnnconfig (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfigE", false]], "tensorrt_llm::runtime::modelconfig::rnnconfig::convkernel (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig10convKernelE", false]], "tensorrt_llm::runtime::modelconfig::rnnconfig::rnnconvdimsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig14rnnConvDimSizeE", false]], "tensorrt_llm::runtime::modelconfig::rnnconfig::rnnheadsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig11rnnHeadSizeE", false]], "tensorrt_llm::runtime::modelconfig::rnnconfig::rnnhiddensize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig13rnnHiddenSizeE", false]], "tensorrt_llm::runtime::modelconfig::rnnconfig::statesize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig9stateSizeE", false]], "tensorrt_llm::runtime::modelconfig::setcontextfmha (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setContextFMHAEb", false]], "tensorrt_llm::runtime::modelconfig::setencoderhiddensize (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setEncoderHiddenSizeE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setgemmallreducedtype (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21setGemmAllReduceDtypeEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::modelconfig::setkvcachetype (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setKVCacheTypeE11KVCacheType", false]], "tensorrt_llm::runtime::modelconfig::setlayertypes (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13setLayerTypesERKNSt6vectorI9LayerTypeEE", false]], "tensorrt_llm::runtime::modelconfig::setlogitsdtype (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setLogitsDtypeEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::modelconfig::setloramodules (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setLoraModulesERKNSt6vectorI10LoraModuleEE", false]], "tensorrt_llm::runtime::modelconfig::setmanageweightstype (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setManageWeightsTypeEK17ManageWeightsType", false]], "tensorrt_llm::runtime::modelconfig::setmaxbatchsize (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setMaxBatchSizeE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setmaxbeamwidth (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setMaxBeamWidthE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setmaxencoderlen (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig16setMaxEncoderLenE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setmaxinputlen (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setMaxInputLenE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setmaxlorarank (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setMaxLoraRankE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setmaxnumtokens (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setMaxNumTokensENSt8optionalI10SizeType32EE", false]], "tensorrt_llm::runtime::modelconfig::setmaxpositionembeddings (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24setMaxPositionEmbeddingsE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setmaxpromptembeddingtablesize (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig30setMaxPromptEmbeddingTableSizeE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setmaxsequencelen (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17setMaxSequenceLenE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setmlphiddensize (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig16setMlpHiddenSizeE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setmodelname (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setModelNameERKNSt6stringE", false]], "tensorrt_llm::runtime::modelconfig::setmodelvariant (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setModelVariantE12ModelVariant", false]], "tensorrt_llm::runtime::modelconfig::setnbcrosskvheads (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17setNbCrossKvHeadsE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setnbkvheads (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setNbKvHeadsE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setnumkvheadspercrosslayer (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26setNumKvHeadsPerCrossLayerERKNSt6vectorI10SizeType32EE", false]], "tensorrt_llm::runtime::modelconfig::setnumkvheadsperlayer (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21setNumKvHeadsPerLayerERKNSt6vectorI10SizeType32EE", false]], "tensorrt_llm::runtime::modelconfig::setnumlanguages (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setNumLanguagesENSt8optionalI10SizeType32EE", false]], "tensorrt_llm::runtime::modelconfig::setpagedcontextfmha (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig19setPagedContextFMHAEb", false]], "tensorrt_llm::runtime::modelconfig::setppreducescatter (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18setPpReduceScatterEb", false]], "tensorrt_llm::runtime::modelconfig::setquantmode (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setQuantModeEN6common9QuantModeE", false]], "tensorrt_llm::runtime::modelconfig::setrnnconfig (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setRnnConfigERK9RnnConfig", false]], "tensorrt_llm::runtime::modelconfig::setrotaryembeddingdim (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21setRotaryEmbeddingDimE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setsizeperhead (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setSizePerHeadE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setskipcrossattnblocks (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22setSkipCrossAttnBlocksEb", false]], "tensorrt_llm::runtime::modelconfig::setspeculativedecodingmode (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26setSpeculativeDecodingModeE23SpeculativeDecodingMode", false]], "tensorrt_llm::runtime::modelconfig::setspeculativedecodingmodule (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig28setSpeculativeDecodingModuleERKNSt10shared_ptrI25SpeculativeDecodingModuleEE", false]], "tensorrt_llm::runtime::modelconfig::settokensperblock (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17setTokensPerBlockE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setusecrossattention (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setUseCrossAttentionEb", false]], "tensorrt_llm::runtime::modelconfig::setusemrope (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11setUseMropeEb", false]], "tensorrt_llm::runtime::modelconfig::setusepositionembedding (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig23setUsePositionEmbeddingEb", false]], "tensorrt_llm::runtime::modelconfig::setuseshapeinference (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setUseShapeInferenceEb", false]], "tensorrt_llm::runtime::modelconfig::setusetokentypeembedding (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24setUseTokenTypeEmbeddingEb", false]], "tensorrt_llm::runtime::modelconfig::skipcrossattnblocks (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig19skipCrossAttnBlocksEv", false]], "tensorrt_llm::runtime::modelconfig::supportsinflightbatching (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig24supportsInflightBatchingEv", false]], "tensorrt_llm::runtime::modelconfig::usecrossattention (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig17useCrossAttentionEv", false]], "tensorrt_llm::runtime::modelconfig::usegemmallreduceplugin (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22useGemmAllReducePluginEb", false], [1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig22useGemmAllReducePluginEv", false]], "tensorrt_llm::runtime::modelconfig::usegptattentionplugin (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21useGptAttentionPluginEb", false], [1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21useGptAttentionPluginEv", false]], "tensorrt_llm::runtime::modelconfig::uselanguageadapter (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18useLanguageAdapterEv", false]], "tensorrt_llm::runtime::modelconfig::useloraplugin (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13useLoraPluginEb", false], [1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13useLoraPluginEv", false]], "tensorrt_llm::runtime::modelconfig::usemambaconv1dplugin (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20useMambaConv1dPluginEb", false], [1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20useMambaConv1dPluginEv", false]], "tensorrt_llm::runtime::modelconfig::usemrope (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig8useMropeEv", false]], "tensorrt_llm::runtime::modelconfig::usepackedinput (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14usePackedInputEb", false], [1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14usePackedInputEv", false]], "tensorrt_llm::runtime::modelconfig::usepagedstate (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13usePagedStateEb", false], [1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13usePagedStateEv", false]], "tensorrt_llm::runtime::modelconfig::usepositionembedding (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20usePositionEmbeddingEv", false]], "tensorrt_llm::runtime::modelconfig::useprompttuning (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15usePromptTuningEv", false]], "tensorrt_llm::runtime::modelconfig::useshapeinference (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig17useShapeInferenceEv", false]], "tensorrt_llm::runtime::modelconfig::usetokentypeembedding (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21useTokenTypeEmbeddingEv", false]], "tensorrt_llm::runtime::mpi_group_barrier (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime17MPI_group_barrierENSt3setIiEE", false]], "tensorrt_llm::runtime::multicastconfigurator (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfiguratorE", false]], "tensorrt_llm::runtime::multicastconfigurator::mbindoffset (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator11mBindOffsetE", false]], "tensorrt_llm::runtime::multicastconfigurator::mdevice (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator7mDeviceE", false]], "tensorrt_llm::runtime::multicastconfigurator::mmulticast (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator10mMulticastE", false]], "tensorrt_llm::runtime::multicastconfigurator::msize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator5mSizeE", false]], "tensorrt_llm::runtime::multicastconfigurator::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator5setupE28CUmemGenericAllocationHandle", false]], "tensorrt_llm::runtime::multicastconfigurator::teardown (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator8teardownE28CUmemGenericAllocationHandleb", false]], "tensorrt_llm::runtime::offloadconfigurator (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfiguratorE", false]], "tensorrt_llm::runtime::offloadconfigurator::maddress (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator8mAddressE", false]], "tensorrt_llm::runtime::offloadconfigurator::mbackedstorage (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator14mBackedStorageE", false]], "tensorrt_llm::runtime::offloadconfigurator::mbacktype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator9mBackTypeE", false]], "tensorrt_llm::runtime::offloadconfigurator::mondemand (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator9mOndemandE", false]], "tensorrt_llm::runtime::offloadconfigurator::msize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator5mSizeE", false]], "tensorrt_llm::runtime::offloadconfigurator::mstream (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator7mStreamE", false]], "tensorrt_llm::runtime::offloadconfigurator::offloadconfigurator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator19OffloadConfiguratorE11CUdeviceptr6size_t10MemoryType8CUstreamb", false]], "tensorrt_llm::runtime::offloadconfigurator::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator5setupE28CUmemGenericAllocationHandle", false]], "tensorrt_llm::runtime::offloadconfigurator::teardown (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator8teardownE28CUmemGenericAllocationHandleb", false]], "tensorrt_llm::runtime::operator<< (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK10LoraModule", false], [1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK26LoraCachePageManagerConfig", false], [1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK7IBuffer", false], [1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK7ITensor", false], [1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERKN7ITensor5ShapeE", false], [1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERKN9LoraCache21TaskLayerModuleConfigE", false]], "tensorrt_llm::runtime::pointerelementtype (c++ type)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime18PointerElementTypeE", false]], "tensorrt_llm::runtime::prompttuningparams (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParamsE", false]], "tensorrt_llm::runtime::prompttuningparams::filltaskstensor (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams15fillTasksTensorE9TensorPtr10SizeType3210SizeType32RKNSt6vectorI10SizeType32EERKNSt6vectorI10SizeType32EERK13BufferManagerb", false]], "tensorrt_llm::runtime::prompttuningparams::prompttuningparams (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams18PromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", false]], "tensorrt_llm::runtime::prompttuningparams::sizetype32 (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams10SizeType32E", false]], "tensorrt_llm::runtime::prompttuningparams::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams9TensorPtrE", false]], "tensorrt_llm::runtime::rawengine (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngineE", false]], "tensorrt_llm::runtime::rawengine::getaddress (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine10getAddressEv", false]], "tensorrt_llm::runtime::rawengine::gethostmemory (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine13getHostMemoryEv", false]], "tensorrt_llm::runtime::rawengine::getmanagedweightsmapopt (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine23getManagedWeightsMapOptEv", false]], "tensorrt_llm::runtime::rawengine::getpath (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine7getPathEv", false]], "tensorrt_llm::runtime::rawengine::getpathopt (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine10getPathOptEv", false]], "tensorrt_llm::runtime::rawengine::getsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine7getSizeEv", false]], "tensorrt_llm::runtime::rawengine::gettype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine7getTypeEv", false]], "tensorrt_llm::runtime::rawengine::mengineaddr (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine11mEngineAddrE", false]], "tensorrt_llm::runtime::rawengine::menginebuffer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine13mEngineBufferE", false]], "tensorrt_llm::runtime::rawengine::menginepath (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine11mEnginePathE", false]], "tensorrt_llm::runtime::rawengine::menginesize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine11mEngineSizeE", false]], "tensorrt_llm::runtime::rawengine::mmanagedweightsmap (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine18mManagedWeightsMapE", false]], "tensorrt_llm::runtime::rawengine::mtype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine5mTypeE", false]], "tensorrt_llm::runtime::rawengine::rawengine (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineENSt10filesystem4pathE", false], [1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineEPKN8nvinfer111IHostMemoryE", false], [1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineEPKvNSt6size_tE", false]], "tensorrt_llm::runtime::rawengine::setmanagedweightsmap (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine20setManagedWeightsMapENSt3mapINSt6stringEN12tensorrt_llm8executor6TensorEEE", false]], "tensorrt_llm::runtime::rawengine::setpath (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine7setPathENSt10filesystem4pathE", false]], "tensorrt_llm::runtime::rawengine::type (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4TypeE", false]], "tensorrt_llm::runtime::rawengine::type::addresswithsize (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4Type15AddressWithSizeE", false]], "tensorrt_llm::runtime::rawengine::type::filepath (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4Type8FilePathE", false]], "tensorrt_llm::runtime::rawengine::type::hostmemory (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4Type10HostMemoryE", false]], "tensorrt_llm::runtime::requesttype (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime11RequestTypeE", false]], "tensorrt_llm::runtime::requesttype::kcontext (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11RequestType8kCONTEXTE", false]], "tensorrt_llm::runtime::requesttype::kgeneration (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11RequestType11kGENERATIONE", false]], "tensorrt_llm::runtime::runtimedefaults (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaultsE", false]], "tensorrt_llm::runtime::runtimedefaults::maxattentionwindowvec (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults21maxAttentionWindowVecE", false]], "tensorrt_llm::runtime::runtimedefaults::runtimedefaults (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults15RuntimeDefaultsENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalI10SizeType32EE", false], [1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults15RuntimeDefaultsEv", false]], "tensorrt_llm::runtime::runtimedefaults::sinktokenlength (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults15sinkTokenLengthE", false]], "tensorrt_llm::runtime::samplingconfig (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfigE", false]], "tensorrt_llm::runtime::samplingconfig::beamsearchdiversityrate (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig23beamSearchDiversityRateE", false]], "tensorrt_llm::runtime::samplingconfig::beamwidth (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig9beamWidthE", false]], "tensorrt_llm::runtime::samplingconfig::beamwidtharray (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14beamWidthArrayE", false]], "tensorrt_llm::runtime::samplingconfig::cumlogprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig11cumLogProbsE", false]], "tensorrt_llm::runtime::samplingconfig::draftacceptancethreshold (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig24draftAcceptanceThresholdE", false]], "tensorrt_llm::runtime::samplingconfig::earlystopping (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig13earlyStoppingE", false]], "tensorrt_llm::runtime::samplingconfig::floattype (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig9FloatTypeE", false]], "tensorrt_llm::runtime::samplingconfig::frequencypenalty (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig16frequencyPenaltyE", false]], "tensorrt_llm::runtime::samplingconfig::fusevalues (c++ function)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig10fuseValuesE6OptVecI1TERKNSt6vectorI14SamplingConfigEENSt8functionIF6OptVecI1TE6size_tEEE1T", false]], "tensorrt_llm::runtime::samplingconfig::getmaxbeamwidth (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14SamplingConfig15getMaxBeamWidthEv", false]], "tensorrt_llm::runtime::samplingconfig::getnumreturnbeams (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14SamplingConfig17getNumReturnBeamsEv", false]], "tensorrt_llm::runtime::samplingconfig::lengthpenalty (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig13lengthPenaltyE", false]], "tensorrt_llm::runtime::samplingconfig::minlength (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig9minLengthE", false]], "tensorrt_llm::runtime::samplingconfig::minp (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig4minPE", false]], "tensorrt_llm::runtime::samplingconfig::norepeatngramsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig17noRepeatNgramSizeE", false]], "tensorrt_llm::runtime::samplingconfig::normalizelogprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig17normalizeLogProbsE", false]], "tensorrt_llm::runtime::samplingconfig::numreturnsequences (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig18numReturnSequencesE", false]], "tensorrt_llm::runtime::samplingconfig::operator== (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14SamplingConfigeqERK14SamplingConfig", false]], "tensorrt_llm::runtime::samplingconfig::optvec (c++ type)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig6OptVecE", false]], "tensorrt_llm::runtime::samplingconfig::originaltemperature (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig19originalTemperatureE", false]], "tensorrt_llm::runtime::samplingconfig::outputlogprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14outputLogProbsE", false]], "tensorrt_llm::runtime::samplingconfig::presencepenalty (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig15presencePenaltyE", false]], "tensorrt_llm::runtime::samplingconfig::randomseed (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig10randomSeedE", false]], "tensorrt_llm::runtime::samplingconfig::repetitionpenalty (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig17repetitionPenaltyE", false]], "tensorrt_llm::runtime::samplingconfig::samplingconfig (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigE10SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigERKN8executor14SamplingConfigERKNSt8optionalIN8executor25ExternalDraftTokensConfigEEE", false], [1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigERKNSt6vectorI14SamplingConfigEE", false]], "tensorrt_llm::runtime::samplingconfig::temperature (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig11temperatureE", false]], "tensorrt_llm::runtime::samplingconfig::topk (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig4topKE", false]], "tensorrt_llm::runtime::samplingconfig::topkmedusaheads (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig15topKMedusaHeadsE", false]], "tensorrt_llm::runtime::samplingconfig::topp (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig4topPE", false]], "tensorrt_llm::runtime::samplingconfig::toppdecay (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig9topPDecayE", false]], "tensorrt_llm::runtime::samplingconfig::toppmin (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig7topPMinE", false]], "tensorrt_llm::runtime::samplingconfig::toppresetids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig12topPResetIdsE", false]], "tensorrt_llm::runtime::samplingconfig::usedefaultvalues (c++ function)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig16useDefaultValuesEbRK6OptVecI1TE1T", false]], "tensorrt_llm::runtime::samplingconfig::validate (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig8validateEv", false]], "tensorrt_llm::runtime::samplingconfig::validatevec (c++ function)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig11validateVecEbNSt6stringERK6OptVecI1TE1TNSt8optionalI1TEE", false]], "tensorrt_llm::runtime::setvirtualmemoryallocator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25setVirtualMemoryAllocatorERKNSt6stringEN26CudaVirtualMemoryAllocator11RestoreModeENSt10shared_ptrI10CudaStreamEE", false]], "tensorrt_llm::runtime::sizetype32 (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime10SizeType32E", false]], "tensorrt_llm::runtime::sizetype64 (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime10SizeType64E", false]], "tensorrt_llm::runtime::speculativedecodingmode (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingModeE", false]], "tensorrt_llm::runtime::speculativedecodingmode::allbitset (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode9allBitSetE14UnderlyingType", false]], "tensorrt_llm::runtime::speculativedecodingmode::anybitset (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode9anyBitSetE14UnderlyingType", false]], "tensorrt_llm::runtime::speculativedecodingmode::drafttokensexternal (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode19DraftTokensExternalEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::eagle (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode5EagleEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::explicitdrafttokens (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode19ExplicitDraftTokensEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::hasdraftlogits (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode14hasDraftLogitsEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::isdrafttokensexternal (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode21isDraftTokensExternalEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::iseagle (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode7isEagleEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::isexplicitdrafttokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode21isExplicitDraftTokensEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::islookaheaddecoding (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode19isLookaheadDecodingEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::ismedusa (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode8isMedusaEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::isnone (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode6isNoneEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::kdrafttokensexternal (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode20kDraftTokensExternalE", false]], "tensorrt_llm::runtime::speculativedecodingmode::keagle (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode6kEagleE", false]], "tensorrt_llm::runtime::speculativedecodingmode::kexplicitdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode20kExplicitDraftTokensE", false]], "tensorrt_llm::runtime::speculativedecodingmode::klookaheaddecoding (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode18kLookaheadDecodingE", false]], "tensorrt_llm::runtime::speculativedecodingmode::kmedusa (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode7kMedusaE", false]], "tensorrt_llm::runtime::speculativedecodingmode::knone (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode5kNoneE", false]], "tensorrt_llm::runtime::speculativedecodingmode::lookaheaddecoding (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode17LookaheadDecodingEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::medusa (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode6MedusaEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::mstate (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode6mStateE", false]], "tensorrt_llm::runtime::speculativedecodingmode::needsdecoderprologue (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode20needsDecoderPrologueEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::needskvcacherewind (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode18needsKVCacheRewindEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::none (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode4NoneEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::operator== (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingModeeqERK23SpeculativeDecodingMode", false]], "tensorrt_llm::runtime::speculativedecodingmode::predictsdrafttokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode19predictsDraftTokensEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::requiresattentionmask (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode21requiresAttentionMaskEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::speculativedecodingmode (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode23SpeculativeDecodingModeE14UnderlyingType", false]], "tensorrt_llm::runtime::speculativedecodingmode::underlyingtype (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode14UnderlyingTypeE", false]], "tensorrt_llm::runtime::speculativedecodingmode::updatespositionids (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode18updatesPositionIdsEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::variabledraftlength (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode19variableDraftLengthEv", false]], "tensorrt_llm::runtime::speculativedecodingmodule (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModuleE", false]], "tensorrt_llm::runtime::speculativedecodingmodule::computenumpackedmasks (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule21computeNumPackedMasksEv", false]], "tensorrt_llm::runtime::speculativedecodingmodule::getmaxdecodingdrafttokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule25getMaxDecodingDraftTokensEv", false]], "tensorrt_llm::runtime::speculativedecodingmodule::getmaxdecodingtokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule20getMaxDecodingTokensEv", false]], "tensorrt_llm::runtime::speculativedecodingmodule::getmaxdraftpathlen (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule18getMaxDraftPathLenEv", false]], "tensorrt_llm::runtime::speculativedecodingmodule::getmaxnumpaths (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule14getMaxNumPathsEv", false]], "tensorrt_llm::runtime::speculativedecodingmodule::getmaxpathlen (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule13getMaxPathLenEv", false]], "tensorrt_llm::runtime::speculativedecodingmodule::getnumpackedmasks (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule17getNumPackedMasksEv", false]], "tensorrt_llm::runtime::speculativedecodingmodule::mmaxdecodingdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule23mMaxDecodingDraftTokensE", false]], "tensorrt_llm::runtime::speculativedecodingmodule::mmaxdraftpathlen (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule16mMaxDraftPathLenE", false]], "tensorrt_llm::runtime::speculativedecodingmodule::mmaxnumpackedmasks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule18mMaxNumPackedMasksE", false]], "tensorrt_llm::runtime::speculativedecodingmodule::mmaxnumpaths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule12mMaxNumPathsE", false]], "tensorrt_llm::runtime::speculativedecodingmodule::operator= (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModuleaSERK25SpeculativeDecodingModule", false]], "tensorrt_llm::runtime::speculativedecodingmodule::setmaxdraftpathlen (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule18setMaxDraftPathLenE10SizeType32", false]], "tensorrt_llm::runtime::speculativedecodingmodule::setmaxdrafttokens (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule17setMaxDraftTokensE10SizeType32", false]], "tensorrt_llm::runtime::speculativedecodingmodule::setmaxnumpaths (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule14setMaxNumPathsE10SizeType32", false]], "tensorrt_llm::runtime::speculativedecodingmodule::speculativedecodingmodule (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleE10SizeType3210SizeType3210SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleERK25SpeculativeDecodingModule", false], [1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleEv", false]], "tensorrt_llm::runtime::speculativedecodingmodule::~speculativedecodingmodule (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModuleD0Ev", false]], "tensorrt_llm::runtime::stringptrmap (c++ type)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime12StringPtrMapE", false]], "tensorrt_llm::runtime::tllmlogger (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime10TllmLoggerE", false]], "tensorrt_llm::runtime::tllmlogger::getlevel (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10TllmLogger8getLevelEv", false]], "tensorrt_llm::runtime::tllmlogger::log (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10TllmLogger3logE8SeverityPKN8nvinfer19AsciiCharE", false]], "tensorrt_llm::runtime::tllmlogger::setlevel (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10TllmLogger8setLevelE8Severity", false]], "tensorrt_llm::runtime::to_string (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9to_stringERK26LoraCachePageManagerConfig", false], [1, "_CPPv4N12tensorrt_llm7runtime9to_stringERKN9LoraCache21TaskLayerModuleConfigE", false]], "tensorrt_llm::runtime::tokenextraidtype (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime16TokenExtraIdTypeE", false]], "tensorrt_llm::runtime::tokenidtype (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime11TokenIdTypeE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4I0_bEN12tensorrt_llm7runtime11TRTDataTypeE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIbEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIbE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIfEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIfE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeI4halfEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeI4halfE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIN7kernels13FinishedStateEEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIN7kernels13FinishedStateEE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIN7kernels12KVCacheIndexEEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIN7kernels12KVCacheIndexEE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIN7runtime11RequestTypeEEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIN7runtime11RequestTypeEE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt7int32_tEEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt7int32_tEE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt7int64_tEEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt7int64_tEE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt6int8_tEEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt6int8_tEE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt8uint32_tEEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt8uint32_tEE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt8uint64_tEEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt8uint64_tEE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt7uint8_tEEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt7uint8_tEE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime11TRTDataTypeIP1TEE", false]], "tensorrt_llm::runtime::trtdatatype::kunderlyingtype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIP1TE15kUnderlyingTypeE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIP1TE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIPvEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIPvE5valueE", false]], "tensorrt_llm::runtime::unicastconfigurator (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfiguratorE", false]], "tensorrt_llm::runtime::unicastconfigurator::maddress (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator8mAddressE", false]], "tensorrt_llm::runtime::unicastconfigurator::mdesc (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator5mDescE", false]], "tensorrt_llm::runtime::unicastconfigurator::msize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator5mSizeE", false]], "tensorrt_llm::runtime::unicastconfigurator::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator5setupE28CUmemGenericAllocationHandle", false]], "tensorrt_llm::runtime::unicastconfigurator::teardown (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator8teardownE28CUmemGenericAllocationHandleb", false]], "tensorrt_llm::runtime::unicastconfigurator::unicastconfigurator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator19UnicastConfiguratorE11CUdeviceptr6size_tRK15CUmemAccessDesc", false]], "tensorrt_llm::runtime::uniquetoken (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime11UniqueTokenE", false]], "tensorrt_llm::runtime::uniquetoken::operator== (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11UniqueTokeneqERK11UniqueToken", false]], "tensorrt_llm::runtime::uniquetoken::tokenextraid (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11UniqueToken12tokenExtraIdE", false]], "tensorrt_llm::runtime::uniquetoken::tokenid (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11UniqueToken7tokenIdE", false]], "tensorrt_llm::runtime::vectokenextraids (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime16VecTokenExtraIdsE", false]], "tensorrt_llm::runtime::vecuniquetokens (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime15VecUniqueTokensE", false]], "tensorrt_llm::runtime::worldconfig (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfigE", false]], "tensorrt_llm::runtime::worldconfig::enableattentiondp (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig17enableAttentionDPEv", false]], "tensorrt_llm::runtime::worldconfig::getcontextparallelgroup (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig23getContextParallelGroupEv", false]], "tensorrt_llm::runtime::worldconfig::getcontextparallelism (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig21getContextParallelismEv", false]], "tensorrt_llm::runtime::worldconfig::getcontextparallelrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig22getContextParallelRankEv", false]], "tensorrt_llm::runtime::worldconfig::getdevice (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig9getDeviceEv", false]], "tensorrt_llm::runtime::worldconfig::getdeviceof (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig11getDeviceOfE10SizeType32", false]], "tensorrt_llm::runtime::worldconfig::getgpuspergroup (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig15getGpusPerGroupEv", false]], "tensorrt_llm::runtime::worldconfig::getgpuspernode (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig14getGpusPerNodeEv", false]], "tensorrt_llm::runtime::worldconfig::getlastrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig11getLastRankEv", false]], "tensorrt_llm::runtime::worldconfig::getlocalrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig12getLocalRankEv", false]], "tensorrt_llm::runtime::worldconfig::getnoderank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig11getNodeRankEv", false]], "tensorrt_llm::runtime::worldconfig::getnoderankof (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig13getNodeRankOfE10SizeType32", false]], "tensorrt_llm::runtime::worldconfig::getpipelineparallelgroup (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig24getPipelineParallelGroupEv", false]], "tensorrt_llm::runtime::worldconfig::getpipelineparallelism (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig22getPipelineParallelismEv", false]], "tensorrt_llm::runtime::worldconfig::getpipelineparallelrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig23getPipelineParallelRankEv", false]], "tensorrt_llm::runtime::worldconfig::getrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig7getRankEv", false]], "tensorrt_llm::runtime::worldconfig::getsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig7getSizeEv", false]], "tensorrt_llm::runtime::worldconfig::gettensorparallelgroup (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig22getTensorParallelGroupEv", false]], "tensorrt_llm::runtime::worldconfig::gettensorparallelism (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig20getTensorParallelismEv", false]], "tensorrt_llm::runtime::worldconfig::gettensorparallelrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig21getTensorParallelRankEv", false]], "tensorrt_llm::runtime::worldconfig::iscontextparallel (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig17isContextParallelEv", false]], "tensorrt_llm::runtime::worldconfig::isfirstcontextparallelrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig26isFirstContextParallelRankEv", false]], "tensorrt_llm::runtime::worldconfig::isfirstpipelineparallelrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig27isFirstPipelineParallelRankEv", false]], "tensorrt_llm::runtime::worldconfig::isfirsttensorparallelrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig25isFirstTensorParallelRankEv", false]], "tensorrt_llm::runtime::worldconfig::islastpipelineparallelrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig26isLastPipelineParallelRankEv", false]], "tensorrt_llm::runtime::worldconfig::ispipelineparallel (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig18isPipelineParallelEv", false]], "tensorrt_llm::runtime::worldconfig::istensorparallel (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig16isTensorParallelEv", false]], "tensorrt_llm::runtime::worldconfig::kdefaultgpuspernode (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig19kDefaultGpusPerNodeE", false]], "tensorrt_llm::runtime::worldconfig::mcontextparallelism (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig19mContextParallelismE", false]], "tensorrt_llm::runtime::worldconfig::mdeviceids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig10mDeviceIdsE", false]], "tensorrt_llm::runtime::worldconfig::menableattentiondp (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig18mEnableAttentionDPE", false]], "tensorrt_llm::runtime::worldconfig::mgpuspernode (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig12mGpusPerNodeE", false]], "tensorrt_llm::runtime::worldconfig::mpi (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig3mpiE10SizeType32NSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEEb", false]], "tensorrt_llm::runtime::worldconfig::mpipelineparallelism (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig20mPipelineParallelismE", false]], "tensorrt_llm::runtime::worldconfig::mrank (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig5mRankE", false]], "tensorrt_llm::runtime::worldconfig::mtensorparallelism (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig18mTensorParallelismE", false]], "tensorrt_llm::runtime::worldconfig::validmpiconfig (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig14validMpiConfigEv", false]], "tensorrt_llm::runtime::worldconfig::worldconfig (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig11WorldConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalINSt6vectorI10SizeType32EEEEb", false]], "text (tensorrt_llm.llmapi.completionoutput attribute)": [[143, "tensorrt_llm.llmapi.CompletionOutput.text", false]], "text_diff (tensorrt_llm.llmapi.completionoutput attribute)": [[143, "tensorrt_llm.llmapi.CompletionOutput.text_diff", false]], "text_diff (tensorrt_llm.llmapi.completionoutput property)": [[143, "id4", false]], "timeout_iters (tensorrt_llm.llmapi.attentiondpconfig attribute)": [[143, "tensorrt_llm.llmapi.AttentionDpConfig.timeout_iters", false]], "timestepembedding (class in tensorrt_llm.layers.embedding)": [[130, "tensorrt_llm.layers.embedding.TimestepEmbedding", false]], "timesteps (class in tensorrt_llm.layers.embedding)": [[130, "tensorrt_llm.layers.embedding.Timesteps", false]], "title() (tensorrt_llm.llmapi.batchingtype method)": [[143, "tensorrt_llm.llmapi.BatchingType.title", false]], "title() (tensorrt_llm.llmapi.capacityschedulerpolicy method)": [[143, "tensorrt_llm.llmapi.CapacitySchedulerPolicy.title", false]], "title() (tensorrt_llm.llmapi.contextchunkingpolicy method)": [[143, "tensorrt_llm.llmapi.ContextChunkingPolicy.title", false]], "title() (tensorrt_llm.llmapi.quantalgo method)": [[143, "tensorrt_llm.llmapi.QuantAlgo.title", false]], "to_dict() (tensorrt_llm.llmapi.buildconfig method)": [[143, "tensorrt_llm.llmapi.BuildConfig.to_dict", false]], "to_dict() (tensorrt_llm.llmapi.calibconfig method)": [[143, "tensorrt_llm.llmapi.CalibConfig.to_dict", false]], "to_dict() (tensorrt_llm.llmapi.quantconfig method)": [[143, "tensorrt_llm.llmapi.QuantConfig.to_dict", false]], "to_dict() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.to_dict", false]], "to_dict() (tensorrt_llm.llmapi.trtllmargs method)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.to_dict", false]], "to_dict() (tensorrt_llm.models.chatglmconfig method)": [[131, "tensorrt_llm.models.ChatGLMConfig.to_dict", false]], "to_dict() (tensorrt_llm.models.cogvlmconfig method)": [[131, "tensorrt_llm.models.CogVLMConfig.to_dict", false]], "to_dict() (tensorrt_llm.models.dbrxconfig method)": [[131, "tensorrt_llm.models.DbrxConfig.to_dict", false]], "to_dict() (tensorrt_llm.models.falconconfig method)": [[131, "tensorrt_llm.models.FalconConfig.to_dict", false]], "to_dict() (tensorrt_llm.models.gemmaconfig method)": [[131, "tensorrt_llm.models.GemmaConfig.to_dict", false]], "to_dict() (tensorrt_llm.models.gptconfig method)": [[131, "tensorrt_llm.models.GPTConfig.to_dict", false]], "to_dict() (tensorrt_llm.models.gptjconfig method)": [[131, "tensorrt_llm.models.GPTJConfig.to_dict", false]], "to_dict() (tensorrt_llm.models.llamaconfig method)": [[131, "tensorrt_llm.models.LLaMAConfig.to_dict", false]], "to_dict() (tensorrt_llm.models.medusaconfig method)": [[131, "tensorrt_llm.models.MedusaConfig.to_dict", false]], "to_dict() (tensorrt_llm.models.pretrainedconfig method)": [[131, "tensorrt_llm.models.PretrainedConfig.to_dict", false]], "to_json_file() (tensorrt_llm.models.pretrainedconfig method)": [[131, "tensorrt_llm.models.PretrainedConfig.to_json_file", false]], "to_layer_quant_config() (tensorrt_llm.models.pretrainedconfig method)": [[131, "tensorrt_llm.models.PretrainedConfig.to_layer_quant_config", false]], "to_legacy_setting() (tensorrt_llm.plugin.pluginconfig method)": [[132, "tensorrt_llm.plugin.PluginConfig.to_legacy_setting", false]], "token_drop() (tensorrt_llm.layers.embedding.labelembedding method)": [[130, "tensorrt_llm.layers.embedding.LabelEmbedding.token_drop", false]], "token_end (tensorrt_llm.llmapi.kvcacheretentionconfig.tokenrangeretentionconfig property)": [[143, "tensorrt_llm.llmapi.KvCacheRetentionConfig.TokenRangeRetentionConfig.token_end", false]], "token_ids (tensorrt_llm.llmapi.completionoutput attribute)": [[143, "tensorrt_llm.llmapi.CompletionOutput.token_ids", false]], "token_ids_diff (tensorrt_llm.llmapi.completionoutput attribute)": [[143, "tensorrt_llm.llmapi.CompletionOutput.token_ids_diff", false]], "token_ids_diff (tensorrt_llm.llmapi.completionoutput property)": [[143, "id5", false]], "token_range_retention_configs (tensorrt_llm.llmapi.kvcacheretentionconfig property)": [[143, "tensorrt_llm.llmapi.KvCacheRetentionConfig.token_range_retention_configs", false]], "token_start (tensorrt_llm.llmapi.kvcacheretentionconfig.tokenrangeretentionconfig property)": [[143, "tensorrt_llm.llmapi.KvCacheRetentionConfig.TokenRangeRetentionConfig.token_start", false]], "tokenizer (tensorrt_llm.llmapi.llm attribute)": [[143, "tensorrt_llm.llmapi.LLM.tokenizer", false]], "tokenizer (tensorrt_llm.llmapi.llm property)": [[143, "id1", false]], "tokenizer (tensorrt_llm.llmapi.multimodalencoder property)": [[143, "tensorrt_llm.llmapi.MultimodalEncoder.tokenizer", false]], "tokenizer (tensorrt_llm.llmapi.torchllmargs attribute)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.tokenizer", false]], "tokenizer (tensorrt_llm.llmapi.trtllmargs attribute)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.tokenizer", false]], "tokenizer_image_token() (tensorrt_llm.runtime.multimodalmodelrunner static method)": [[134, "tensorrt_llm.runtime.MultimodalModelRunner.tokenizer_image_token", false]], "tokenizer_max_seq_length (tensorrt_llm.llmapi.calibconfig attribute)": [[143, "tensorrt_llm.llmapi.CalibConfig.tokenizer_max_seq_length", false]], "tokenizer_mode (tensorrt_llm.llmapi.torchllmargs attribute)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.tokenizer_mode", false]], "tokenizer_mode (tensorrt_llm.llmapi.trtllmargs attribute)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.tokenizer_mode", false]], "tokenizer_revision (tensorrt_llm.llmapi.torchllmargs attribute)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.tokenizer_revision", false]], "tokenizer_revision (tensorrt_llm.llmapi.trtllmargs attribute)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.tokenizer_revision", false]], "tokens_per_block (tensorrt_llm.llmapi.kvcacheconfig attribute)": [[143, "tensorrt_llm.llmapi.KvCacheConfig.tokens_per_block", false]], "tokens_per_block (tensorrt_llm.runtime.generationsession property)": [[134, "tensorrt_llm.runtime.GenerationSession.tokens_per_block", false]], "tokens_per_block (tensorrt_llm.runtime.modelconfig attribute)": [[134, "tensorrt_llm.runtime.ModelConfig.tokens_per_block", false]], "top_k (tensorrt_llm.llmapi.samplingparams attribute)": [[143, "tensorrt_llm.llmapi.SamplingParams.top_k", false]], "top_k (tensorrt_llm.runtime.samplingconfig attribute)": [[134, "tensorrt_llm.runtime.SamplingConfig.top_k", false]], "top_p (tensorrt_llm.llmapi.samplingparams attribute)": [[143, "tensorrt_llm.llmapi.SamplingParams.top_p", false]], "top_p (tensorrt_llm.runtime.samplingconfig attribute)": [[134, "tensorrt_llm.runtime.SamplingConfig.top_p", false]], "top_p_decay (tensorrt_llm.llmapi.samplingparams attribute)": [[143, "tensorrt_llm.llmapi.SamplingParams.top_p_decay", false]], "top_p_decay (tensorrt_llm.runtime.samplingconfig attribute)": [[134, "tensorrt_llm.runtime.SamplingConfig.top_p_decay", false]], "top_p_min (tensorrt_llm.llmapi.samplingparams attribute)": [[143, "tensorrt_llm.llmapi.SamplingParams.top_p_min", false]], "top_p_min (tensorrt_llm.runtime.samplingconfig attribute)": [[134, "tensorrt_llm.runtime.SamplingConfig.top_p_min", false]], "top_p_reset_ids (tensorrt_llm.llmapi.samplingparams attribute)": [[143, "tensorrt_llm.llmapi.SamplingParams.top_p_reset_ids", false]], "top_p_reset_ids (tensorrt_llm.runtime.samplingconfig attribute)": [[134, "tensorrt_llm.runtime.SamplingConfig.top_p_reset_ids", false]], "topk() (in module tensorrt_llm.functional)": [[129, "tensorrt_llm.functional.topk", false]], "torch_compile_config (tensorrt_llm.llmapi.torchllmargs attribute)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.torch_compile_config", false]], "torchcompileconfig (class in tensorrt_llm.llmapi)": [[143, "tensorrt_llm.llmapi.TorchCompileConfig", false]], "torchcompileconfig.config (class in tensorrt_llm.llmapi)": [[143, "tensorrt_llm.llmapi.TorchCompileConfig.Config", false]], "torchllmargs (class in tensorrt_llm.llmapi)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs", false]], "torchllmargs.config (class in tensorrt_llm.llmapi)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.Config", false]], "tp_split_dim() (tensorrt_llm.layers.linear.linear class method)": [[130, "tensorrt_llm.layers.linear.Linear.tp_split_dim", false]], "tp_split_dim() (tensorrt_llm.layers.linear.linearbase class method)": [[130, "tensorrt_llm.layers.linear.LinearBase.tp_split_dim", false]], "tp_split_dim() (tensorrt_llm.layers.linear.rowlinear class method)": [[130, "tensorrt_llm.layers.linear.RowLinear.tp_split_dim", false]], "transfer_mode (tensorrt_llm.llmapi.kvcacheretentionconfig property)": [[143, "tensorrt_llm.llmapi.KvCacheRetentionConfig.transfer_mode", false]], "translate() (tensorrt_llm.llmapi.batchingtype method)": [[143, "tensorrt_llm.llmapi.BatchingType.translate", false]], "translate() (tensorrt_llm.llmapi.capacityschedulerpolicy method)": [[143, "tensorrt_llm.llmapi.CapacitySchedulerPolicy.translate", false]], "translate() (tensorrt_llm.llmapi.contextchunkingpolicy method)": [[143, "tensorrt_llm.llmapi.ContextChunkingPolicy.translate", false]], "translate() (tensorrt_llm.llmapi.quantalgo method)": [[143, "tensorrt_llm.llmapi.QuantAlgo.translate", false]], "transpose() (in module tensorrt_llm.functional)": [[129, "tensorrt_llm.functional.transpose", false]], "transpose() (tensorrt_llm.functional.tensor method)": [[129, "tensorrt_llm.functional.Tensor.transpose", false]], "trtllm-bench command line option": [[21, "cmdoption-trtllm-bench-log_level", false], [21, "cmdoption-trtllm-bench-m", false], [21, "cmdoption-trtllm-bench-model_path", false], [21, "cmdoption-trtllm-bench-w", false]], "trtllm-bench-build command line option": [[21, "cmdoption-trtllm-bench-build-dataset", false], [21, "cmdoption-trtllm-bench-build-max_batch_size", false], [21, "cmdoption-trtllm-bench-build-max_num_tokens", false], [21, "cmdoption-trtllm-bench-build-max_seq_len", false], [21, "cmdoption-trtllm-bench-build-no_weights_loading", false], [21, "cmdoption-trtllm-bench-build-pp", false], [21, "cmdoption-trtllm-bench-build-q", false], [21, "cmdoption-trtllm-bench-build-target_input_len", false], [21, "cmdoption-trtllm-bench-build-target_output_len", false], [21, "cmdoption-trtllm-bench-build-tp", false], [21, "cmdoption-trtllm-bench-build-trust_remote_code", false]], "trtllm-bench-latency command line option": [[21, "cmdoption-trtllm-bench-latency-backend", false], [21, "cmdoption-trtllm-bench-latency-beam_width", false], [21, "cmdoption-trtllm-bench-latency-concurrency", false], [21, "cmdoption-trtllm-bench-latency-dataset", false], [21, "cmdoption-trtllm-bench-latency-engine_dir", false], [21, "cmdoption-trtllm-bench-latency-ep", false], [21, "cmdoption-trtllm-bench-latency-extra_llm_api_options", false], [21, "cmdoption-trtllm-bench-latency-iteration_log", false], [21, "cmdoption-trtllm-bench-latency-kv_cache_free_gpu_mem_fraction", false], [21, "cmdoption-trtllm-bench-latency-max_input_len", false], [21, "cmdoption-trtllm-bench-latency-max_seq_len", false], [21, "cmdoption-trtllm-bench-latency-medusa_choices", false], [21, "cmdoption-trtllm-bench-latency-modality", false], [21, "cmdoption-trtllm-bench-latency-num_requests", false], [21, "cmdoption-trtllm-bench-latency-pp", false], [21, "cmdoption-trtllm-bench-latency-report_json", false], [21, "cmdoption-trtllm-bench-latency-sampler_options", false], [21, "cmdoption-trtllm-bench-latency-tp", false], [21, "cmdoption-trtllm-bench-latency-warmup", false]], "trtllm-bench-throughput command line option": [[21, "cmdoption-trtllm-bench-throughput-backend", false], [21, "cmdoption-trtllm-bench-throughput-beam_width", false], [21, "cmdoption-trtllm-bench-throughput-cluster_size", false], [21, "cmdoption-trtllm-bench-throughput-concurrency", false], [21, "cmdoption-trtllm-bench-throughput-custom_module_dirs", false], [21, "cmdoption-trtllm-bench-throughput-data_device", false], [21, "cmdoption-trtllm-bench-throughput-dataset", false], [21, "cmdoption-trtllm-bench-throughput-enable_chunked_context", false], [21, "cmdoption-trtllm-bench-throughput-engine_dir", false], [21, "cmdoption-trtllm-bench-throughput-eos_id", false], [21, "cmdoption-trtllm-bench-throughput-ep", false], [21, "cmdoption-trtllm-bench-throughput-extra_llm_api_options", false], [21, "cmdoption-trtllm-bench-throughput-image_data_format", false], [21, "cmdoption-trtllm-bench-throughput-iteration_log", false], [21, "cmdoption-trtllm-bench-throughput-kv_cache_free_gpu_mem_fraction", false], [21, "cmdoption-trtllm-bench-throughput-max_batch_size", false], [21, "cmdoption-trtllm-bench-throughput-max_input_len", false], [21, "cmdoption-trtllm-bench-throughput-max_num_tokens", false], [21, "cmdoption-trtllm-bench-throughput-max_seq_len", false], [21, "cmdoption-trtllm-bench-throughput-modality", false], [21, "cmdoption-trtllm-bench-throughput-no_skip_tokenizer_init", false], [21, "cmdoption-trtllm-bench-throughput-num_requests", false], [21, "cmdoption-trtllm-bench-throughput-output_json", false], [21, "cmdoption-trtllm-bench-throughput-pp", false], [21, "cmdoption-trtllm-bench-throughput-report_json", false], [21, "cmdoption-trtllm-bench-throughput-request_json", false], [21, "cmdoption-trtllm-bench-throughput-sampler_options", false], [21, "cmdoption-trtllm-bench-throughput-scheduler_policy", false], [21, "cmdoption-trtllm-bench-throughput-streaming", false], [21, "cmdoption-trtllm-bench-throughput-target_input_len", false], [21, "cmdoption-trtllm-bench-throughput-target_output_len", false], [21, "cmdoption-trtllm-bench-throughput-tp", false], [21, "cmdoption-trtllm-bench-throughput-warmup", false]], "trtllm-eval command line option": [[23, "cmdoption-trtllm-eval-backend", false], [23, "cmdoption-trtllm-eval-disable_kv_cache_reuse", false], [23, "cmdoption-trtllm-eval-ep_size", false], [23, "cmdoption-trtllm-eval-extra_llm_api_options", false], [23, "cmdoption-trtllm-eval-gpus_per_node", false], [23, "cmdoption-trtllm-eval-kv_cache_free_gpu_memory_fraction", false], [23, "cmdoption-trtllm-eval-log_level", false], [23, "cmdoption-trtllm-eval-max_batch_size", false], [23, "cmdoption-trtllm-eval-max_beam_width", false], [23, "cmdoption-trtllm-eval-max_num_tokens", false], [23, "cmdoption-trtllm-eval-max_seq_len", false], [23, "cmdoption-trtllm-eval-model", false], [23, "cmdoption-trtllm-eval-pp_size", false], [23, "cmdoption-trtllm-eval-tokenizer", false], [23, "cmdoption-trtllm-eval-tp_size", false], [23, "cmdoption-trtllm-eval-trust_remote_code", false]], "trtllm-eval-cnn_dailymail command line option": [[23, "cmdoption-trtllm-eval-cnn_dailymail-apply_chat_template", false], [23, "cmdoption-trtllm-eval-cnn_dailymail-dataset_path", false], [23, "cmdoption-trtllm-eval-cnn_dailymail-max_input_length", false], [23, "cmdoption-trtllm-eval-cnn_dailymail-max_output_length", false], [23, "cmdoption-trtllm-eval-cnn_dailymail-num_samples", false], [23, "cmdoption-trtllm-eval-cnn_dailymail-random_seed", false], [23, "cmdoption-trtllm-eval-cnn_dailymail-rouge_path", false], [23, "cmdoption-trtllm-eval-cnn_dailymail-system_prompt", false]], "trtllm-eval-gpqa_diamond command line option": [[23, "cmdoption-trtllm-eval-gpqa_diamond-apply_chat_template", false], [23, "cmdoption-trtllm-eval-gpqa_diamond-dataset_path", false], [23, "cmdoption-trtllm-eval-gpqa_diamond-max_input_length", false], [23, "cmdoption-trtllm-eval-gpqa_diamond-max_output_length", false], [23, "cmdoption-trtllm-eval-gpqa_diamond-num_samples", false], [23, "cmdoption-trtllm-eval-gpqa_diamond-random_seed", false], [23, "cmdoption-trtllm-eval-gpqa_diamond-system_prompt", false]], "trtllm-eval-gpqa_extended command line option": [[23, "cmdoption-trtllm-eval-gpqa_extended-apply_chat_template", false], [23, "cmdoption-trtllm-eval-gpqa_extended-dataset_path", false], [23, "cmdoption-trtllm-eval-gpqa_extended-max_input_length", false], [23, "cmdoption-trtllm-eval-gpqa_extended-max_output_length", false], [23, "cmdoption-trtllm-eval-gpqa_extended-num_samples", false], [23, "cmdoption-trtllm-eval-gpqa_extended-random_seed", false], [23, "cmdoption-trtllm-eval-gpqa_extended-system_prompt", false]], "trtllm-eval-gpqa_main command line option": [[23, "cmdoption-trtllm-eval-gpqa_main-apply_chat_template", false], [23, "cmdoption-trtllm-eval-gpqa_main-dataset_path", false], [23, "cmdoption-trtllm-eval-gpqa_main-max_input_length", false], [23, "cmdoption-trtllm-eval-gpqa_main-max_output_length", false], [23, "cmdoption-trtllm-eval-gpqa_main-num_samples", false], [23, "cmdoption-trtllm-eval-gpqa_main-random_seed", false], [23, "cmdoption-trtllm-eval-gpqa_main-system_prompt", false]], "trtllm-eval-gsm8k command line option": [[23, "cmdoption-trtllm-eval-gsm8k-apply_chat_template", false], [23, "cmdoption-trtllm-eval-gsm8k-dataset_path", false], [23, "cmdoption-trtllm-eval-gsm8k-fewshot_as_multiturn", false], [23, "cmdoption-trtllm-eval-gsm8k-max_input_length", false], [23, "cmdoption-trtllm-eval-gsm8k-max_output_length", false], [23, "cmdoption-trtllm-eval-gsm8k-num_samples", false], [23, "cmdoption-trtllm-eval-gsm8k-random_seed", false], [23, "cmdoption-trtllm-eval-gsm8k-system_prompt", false]], "trtllm-eval-json_mode_eval command line option": [[23, "cmdoption-trtllm-eval-json_mode_eval-dataset_path", false], [23, "cmdoption-trtllm-eval-json_mode_eval-max_input_length", false], [23, "cmdoption-trtllm-eval-json_mode_eval-max_output_length", false], [23, "cmdoption-trtllm-eval-json_mode_eval-num_samples", false], [23, "cmdoption-trtllm-eval-json_mode_eval-random_seed", false], [23, "cmdoption-trtllm-eval-json_mode_eval-system_prompt", false]], "trtllm-eval-mmlu command line option": [[23, "cmdoption-trtllm-eval-mmlu-accuracy_threshold", false], [23, "cmdoption-trtllm-eval-mmlu-apply_chat_template", false], [23, "cmdoption-trtllm-eval-mmlu-check_accuracy", false], [23, "cmdoption-trtllm-eval-mmlu-dataset_path", false], [23, "cmdoption-trtllm-eval-mmlu-max_input_length", false], [23, "cmdoption-trtllm-eval-mmlu-max_output_length", false], [23, "cmdoption-trtllm-eval-mmlu-num_fewshot", false], [23, "cmdoption-trtllm-eval-mmlu-num_samples", false], [23, "cmdoption-trtllm-eval-mmlu-random_seed", false], [23, "cmdoption-trtllm-eval-mmlu-system_prompt", false]], "trtllm-eval-mmmu command line option": [[23, "cmdoption-trtllm-eval-mmmu-dataset_path", false], [23, "cmdoption-trtllm-eval-mmmu-max_input_length", false], [23, "cmdoption-trtllm-eval-mmmu-max_output_length", false], [23, "cmdoption-trtllm-eval-mmmu-num_samples", false], [23, "cmdoption-trtllm-eval-mmmu-random_seed", false], [23, "cmdoption-trtllm-eval-mmmu-system_prompt", false]], "trtllm-serve-disaggregated command line option": [[26, "cmdoption-trtllm-serve-disaggregated-c", false], [26, "cmdoption-trtllm-serve-disaggregated-l", false], [26, "cmdoption-trtllm-serve-disaggregated-m", false], [26, "cmdoption-trtllm-serve-disaggregated-metrics-log-interval", false], [26, "cmdoption-trtllm-serve-disaggregated-r", false], [26, "cmdoption-trtllm-serve-disaggregated-t", false]], "trtllm-serve-disaggregated_mpi_worker command line option": [[26, "cmdoption-trtllm-serve-disaggregated_mpi_worker-c", false], [26, "cmdoption-trtllm-serve-disaggregated_mpi_worker-log_level", false]], "trtllm-serve-mm_embedding_serve command line option": [[26, "cmdoption-trtllm-serve-mm_embedding_serve-arg-MODEL", false], [26, "cmdoption-trtllm-serve-mm_embedding_serve-extra_encoder_options", false], [26, "cmdoption-trtllm-serve-mm_embedding_serve-gpus_per_node", false], [26, "cmdoption-trtllm-serve-mm_embedding_serve-host", false], [26, "cmdoption-trtllm-serve-mm_embedding_serve-log_level", false], [26, "cmdoption-trtllm-serve-mm_embedding_serve-max_batch_size", false], [26, "cmdoption-trtllm-serve-mm_embedding_serve-max_num_tokens", false], [26, "cmdoption-trtllm-serve-mm_embedding_serve-metadata_server_config_file", false], [26, "cmdoption-trtllm-serve-mm_embedding_serve-port", false], [26, "cmdoption-trtllm-serve-mm_embedding_serve-trust_remote_code", false]], "trtllm-serve-serve command line option": [[26, "cmdoption-trtllm-serve-serve-arg-MODEL", false], [26, "cmdoption-trtllm-serve-serve-backend", false], [26, "cmdoption-trtllm-serve-serve-cluster_size", false], [26, "cmdoption-trtllm-serve-serve-ep_size", false], [26, "cmdoption-trtllm-serve-serve-extra_llm_api_options", false], [26, "cmdoption-trtllm-serve-serve-fail_fast_on_attention_window_too_large", false], [26, "cmdoption-trtllm-serve-serve-gpus_per_node", false], [26, "cmdoption-trtllm-serve-serve-host", false], [26, "cmdoption-trtllm-serve-serve-kv_cache_free_gpu_memory_fraction", false], [26, "cmdoption-trtllm-serve-serve-log_level", false], [26, "cmdoption-trtllm-serve-serve-max_batch_size", false], [26, "cmdoption-trtllm-serve-serve-max_beam_width", false], [26, "cmdoption-trtllm-serve-serve-max_num_tokens", false], [26, "cmdoption-trtllm-serve-serve-max_seq_len", false], [26, "cmdoption-trtllm-serve-serve-metadata_server_config_file", false], [26, "cmdoption-trtllm-serve-serve-num_postprocess_workers", false], [26, "cmdoption-trtllm-serve-serve-port", false], [26, "cmdoption-trtllm-serve-serve-pp_size", false], [26, "cmdoption-trtllm-serve-serve-reasoning_parser", false], [26, "cmdoption-trtllm-serve-serve-server_role", false], [26, "cmdoption-trtllm-serve-serve-tokenizer", false], [26, "cmdoption-trtllm-serve-serve-tp_size", false], [26, "cmdoption-trtllm-serve-serve-trust_remote_code", false]], "trtllm_modules_to_hf_modules (tensorrt_llm.runtime.modelconfig attribute)": [[134, "tensorrt_llm.runtime.ModelConfig.trtllm_modules_to_hf_modules", false]], "trtllmargs (class in tensorrt_llm.llmapi)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs", false]], "trtllmargs.config (class in tensorrt_llm.llmapi)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.Config", false]], "truncate_prompt_tokens (tensorrt_llm.llmapi.samplingparams attribute)": [[143, "tensorrt_llm.llmapi.SamplingParams.truncate_prompt_tokens", false]], "trust_remote_code (tensorrt_llm.llmapi.torchllmargs attribute)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.trust_remote_code", false]], "trust_remote_code (tensorrt_llm.llmapi.trtllmargs attribute)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.trust_remote_code", false]], "twoshot (tensorrt_llm.functional.allreducestrategy attribute)": [[129, "tensorrt_llm.functional.AllReduceStrategy.TWOSHOT", false]], "ub (tensorrt_llm.functional.allreducestrategy attribute)": [[129, "tensorrt_llm.functional.AllReduceStrategy.UB", false]], "unary() (in module tensorrt_llm.functional)": [[129, "tensorrt_llm.functional.unary", false]], "unbind() (in module tensorrt_llm.functional)": [[129, "tensorrt_llm.functional.unbind", false]], "unbind() (tensorrt_llm.functional.tensor method)": [[129, "tensorrt_llm.functional.Tensor.unbind", false]], "unfuse_qkv_projections() (tensorrt_llm.models.sd3transformer2dmodel method)": [[131, "tensorrt_llm.models.SD3Transformer2DModel.unfuse_qkv_projections", false]], "unpatchify() (tensorrt_llm.models.dit method)": [[131, "tensorrt_llm.models.DiT.unpatchify", false]], "unsqueeze() (in module tensorrt_llm.functional)": [[129, "tensorrt_llm.functional.unsqueeze", false]], "unsqueeze() (tensorrt_llm.functional.tensor method)": [[129, "tensorrt_llm.functional.Tensor.unsqueeze", false]], "update() (tensorrt_llm.llmapi.buildconfig method)": [[143, "tensorrt_llm.llmapi.BuildConfig.update", false]], "update() (tensorrt_llm.runtime.samplingconfig method)": [[134, "tensorrt_llm.runtime.SamplingConfig.update", false]], "update_forward_refs() (tensorrt_llm.llmapi.attentiondpconfig class method)": [[143, "tensorrt_llm.llmapi.AttentionDpConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.autodecodingconfig class method)": [[143, "tensorrt_llm.llmapi.AutoDecodingConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.cachetransceiverconfig class method)": [[143, "tensorrt_llm.llmapi.CacheTransceiverConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.calibconfig class method)": [[143, "tensorrt_llm.llmapi.CalibConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.cudagraphconfig class method)": [[143, "tensorrt_llm.llmapi.CudaGraphConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.drafttargetdecodingconfig class method)": [[143, "tensorrt_llm.llmapi.DraftTargetDecodingConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.dynamicbatchconfig class method)": [[143, "tensorrt_llm.llmapi.DynamicBatchConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.eagledecodingconfig class method)": [[143, "tensorrt_llm.llmapi.EagleDecodingConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.extendedruntimeperfknobconfig class method)": [[143, "tensorrt_llm.llmapi.ExtendedRuntimePerfKnobConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.kvcacheconfig class method)": [[143, "tensorrt_llm.llmapi.KvCacheConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.lookaheaddecodingconfig class method)": [[143, "tensorrt_llm.llmapi.LookaheadDecodingConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.medusadecodingconfig class method)": [[143, "tensorrt_llm.llmapi.MedusaDecodingConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.moeconfig class method)": [[143, "tensorrt_llm.llmapi.MoeConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.mtpdecodingconfig class method)": [[143, "tensorrt_llm.llmapi.MTPDecodingConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.ngramdecodingconfig class method)": [[143, "tensorrt_llm.llmapi.NGramDecodingConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.schedulerconfig class method)": [[143, "tensorrt_llm.llmapi.SchedulerConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.torchcompileconfig class method)": [[143, "tensorrt_llm.llmapi.TorchCompileConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.userprovideddecodingconfig class method)": [[143, "tensorrt_llm.llmapi.UserProvidedDecodingConfig.update_forward_refs", false]], "update_from_dict() (tensorrt_llm.llmapi.buildconfig method)": [[143, "tensorrt_llm.llmapi.BuildConfig.update_from_dict", false]], "update_kv_cache_type() (tensorrt_llm.llmapi.buildconfig method)": [[143, "tensorrt_llm.llmapi.BuildConfig.update_kv_cache_type", false]], "update_output_ids_by_offset() (tensorrt_llm.runtime.generationsession method)": [[134, "tensorrt_llm.runtime.GenerationSession.update_output_ids_by_offset", false]], "update_strategy() (tensorrt_llm.functional.allreduceparams method)": [[129, "tensorrt_llm.functional.AllReduceParams.update_strategy", false]], "upper() (tensorrt_llm.llmapi.batchingtype method)": [[143, "tensorrt_llm.llmapi.BatchingType.upper", false]], "upper() (tensorrt_llm.llmapi.capacityschedulerpolicy method)": [[143, "tensorrt_llm.llmapi.CapacitySchedulerPolicy.upper", false]], "upper() (tensorrt_llm.llmapi.contextchunkingpolicy method)": [[143, "tensorrt_llm.llmapi.ContextChunkingPolicy.upper", false]], "upper() (tensorrt_llm.llmapi.quantalgo method)": [[143, "tensorrt_llm.llmapi.QuantAlgo.upper", false]], "use_beam_hyps (tensorrt_llm.runtime.samplingconfig attribute)": [[134, "tensorrt_llm.runtime.SamplingConfig.use_beam_hyps", false]], "use_beam_search (tensorrt_llm.llmapi.samplingparams attribute)": [[143, "tensorrt_llm.llmapi.SamplingParams.use_beam_search", false]], "use_dynamic_tree (tensorrt_llm.llmapi.eagledecodingconfig attribute)": [[143, "tensorrt_llm.llmapi.EagleDecodingConfig.use_dynamic_tree", false]], "use_gemm_allreduce_plugin (tensorrt_llm.runtime.generationsession property)": [[134, "tensorrt_llm.runtime.GenerationSession.use_gemm_allreduce_plugin", false]], "use_gpt_attention_plugin (tensorrt_llm.runtime.generationsession property)": [[134, "tensorrt_llm.runtime.GenerationSession.use_gpt_attention_plugin", false]], "use_kv_cache (tensorrt_llm.runtime.generationsession property)": [[134, "tensorrt_llm.runtime.GenerationSession.use_kv_cache", false]], "use_lora() (tensorrt_llm.models.decodermodel method)": [[131, "tensorrt_llm.models.DecoderModel.use_lora", false]], "use_lora() (tensorrt_llm.models.encodermodel method)": [[131, "tensorrt_llm.models.EncoderModel.use_lora", false]], "use_lora() (tensorrt_llm.models.gemmaforcausallm method)": [[131, "tensorrt_llm.models.GemmaForCausalLM.use_lora", false]], "use_lora() (tensorrt_llm.models.gptforcausallm method)": [[131, "tensorrt_llm.models.GPTForCausalLM.use_lora", false]], "use_lora() (tensorrt_llm.models.llamaforcausallm method)": [[131, "tensorrt_llm.models.LLaMAForCausalLM.use_lora", false]], "use_lora() (tensorrt_llm.models.mllamaforcausallm method)": [[131, "tensorrt_llm.models.MLLaMAForCausalLM.use_lora", false]], "use_lora() (tensorrt_llm.models.phi3forcausallm method)": [[131, "tensorrt_llm.models.Phi3ForCausalLM.use_lora", false]], "use_lora() (tensorrt_llm.models.phiforcausallm method)": [[131, "tensorrt_llm.models.PhiForCausalLM.use_lora", false]], "use_lora_plugin (tensorrt_llm.runtime.generationsession property)": [[134, "tensorrt_llm.runtime.GenerationSession.use_lora_plugin", false]], "use_lora_plugin (tensorrt_llm.runtime.modelrunner property)": [[134, "tensorrt_llm.runtime.ModelRunner.use_lora_plugin", false]], "use_low_precision_moe_combine (tensorrt_llm.llmapi.moeconfig attribute)": [[143, "tensorrt_llm.llmapi.MoeConfig.use_low_precision_moe_combine", false]], "use_mamba_conv1d_plugin (tensorrt_llm.runtime.generationsession property)": [[134, "tensorrt_llm.runtime.GenerationSession.use_mamba_conv1d_plugin", false]], "use_meta_recipe (tensorrt_llm.llmapi.quantconfig attribute)": [[143, "tensorrt_llm.llmapi.QuantConfig.use_meta_recipe", false]], "use_mrope (tensorrt_llm.llmapi.buildconfig attribute)": [[143, "tensorrt_llm.llmapi.BuildConfig.use_mrope", false]], "use_mtp_vanilla (tensorrt_llm.llmapi.mtpdecodingconfig attribute)": [[143, "tensorrt_llm.llmapi.MTPDecodingConfig.use_mtp_vanilla", false]], "use_prompt_tuning() (tensorrt_llm.models.encodermodel method)": [[131, "tensorrt_llm.models.EncoderModel.use_prompt_tuning", false]], "use_refit (tensorrt_llm.llmapi.buildconfig attribute)": [[143, "tensorrt_llm.llmapi.BuildConfig.use_refit", false]], "use_relaxed_acceptance_for_thinking (tensorrt_llm.llmapi.mtpdecodingconfig attribute)": [[143, "tensorrt_llm.llmapi.MTPDecodingConfig.use_relaxed_acceptance_for_thinking", false]], "use_strip_plan (tensorrt_llm.llmapi.buildconfig attribute)": [[143, "tensorrt_llm.llmapi.BuildConfig.use_strip_plan", false]], "use_uvm (tensorrt_llm.llmapi.kvcacheconfig attribute)": [[143, "tensorrt_llm.llmapi.KvCacheConfig.use_uvm", false]], "user_provided (tensorrt_llm.models.speculativedecodingmode attribute)": [[131, "tensorrt_llm.models.SpeculativeDecodingMode.USER_PROVIDED", false]], "userprovideddecodingconfig (class in tensorrt_llm.llmapi)": [[143, "tensorrt_llm.llmapi.UserProvidedDecodingConfig", false]], "userprovideddecodingconfig.config (class in tensorrt_llm.llmapi)": [[143, "tensorrt_llm.llmapi.UserProvidedDecodingConfig.Config", false]], "validate() (tensorrt_llm.llmapi.attentiondpconfig class method)": [[143, "tensorrt_llm.llmapi.AttentionDpConfig.validate", false]], "validate() (tensorrt_llm.llmapi.autodecodingconfig method)": [[143, "tensorrt_llm.llmapi.AutoDecodingConfig.validate", false]], "validate() (tensorrt_llm.llmapi.cachetransceiverconfig class method)": [[143, "tensorrt_llm.llmapi.CacheTransceiverConfig.validate", false]], "validate() (tensorrt_llm.llmapi.calibconfig class method)": [[143, "tensorrt_llm.llmapi.CalibConfig.validate", false]], "validate() (tensorrt_llm.llmapi.cudagraphconfig class method)": [[143, "tensorrt_llm.llmapi.CudaGraphConfig.validate", false]], "validate() (tensorrt_llm.llmapi.drafttargetdecodingconfig method)": [[143, "tensorrt_llm.llmapi.DraftTargetDecodingConfig.validate", false]], "validate() (tensorrt_llm.llmapi.dynamicbatchconfig class method)": [[143, "tensorrt_llm.llmapi.DynamicBatchConfig.validate", false]], "validate() (tensorrt_llm.llmapi.eagledecodingconfig method)": [[143, "tensorrt_llm.llmapi.EagleDecodingConfig.validate", false]], "validate() (tensorrt_llm.llmapi.extendedruntimeperfknobconfig class method)": [[143, "tensorrt_llm.llmapi.ExtendedRuntimePerfKnobConfig.validate", false]], "validate() (tensorrt_llm.llmapi.kvcacheconfig class method)": [[143, "tensorrt_llm.llmapi.KvCacheConfig.validate", false]], "validate() (tensorrt_llm.llmapi.lookaheaddecodingconfig method)": [[143, "tensorrt_llm.llmapi.LookaheadDecodingConfig.validate", false]], "validate() (tensorrt_llm.llmapi.medusadecodingconfig method)": [[143, "tensorrt_llm.llmapi.MedusaDecodingConfig.validate", false]], "validate() (tensorrt_llm.llmapi.moeconfig class method)": [[143, "tensorrt_llm.llmapi.MoeConfig.validate", false]], "validate() (tensorrt_llm.llmapi.mtpdecodingconfig method)": [[143, "tensorrt_llm.llmapi.MTPDecodingConfig.validate", false]], "validate() (tensorrt_llm.llmapi.ngramdecodingconfig method)": [[143, "tensorrt_llm.llmapi.NGramDecodingConfig.validate", false]], "validate() (tensorrt_llm.llmapi.schedulerconfig class method)": [[143, "tensorrt_llm.llmapi.SchedulerConfig.validate", false]], "validate() (tensorrt_llm.llmapi.torchcompileconfig class method)": [[143, "tensorrt_llm.llmapi.TorchCompileConfig.validate", false]], "validate() (tensorrt_llm.llmapi.userprovideddecodingconfig method)": [[143, "tensorrt_llm.llmapi.UserProvidedDecodingConfig.validate", false]], "validate_and_init_tokenizer() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_and_init_tokenizer", false]], "validate_and_init_tokenizer() (tensorrt_llm.llmapi.trtllmargs method)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.validate_and_init_tokenizer", false]], "validate_attention_dp_config() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_attention_dp_config", false]], "validate_auto_parallel() (tensorrt_llm.llmapi.trtllmargs method)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.validate_auto_parallel", false]], "validate_batch_wait_max_tokens_ratio() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_batch_wait_max_tokens_ratio", false]], "validate_batch_wait_timeout_iters() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_batch_wait_timeout_iters", false]], "validate_batch_wait_timeout_ms() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_batch_wait_timeout_ms", false]], "validate_build_config_remaining() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_build_config_remaining", false]], "validate_build_config_remaining() (tensorrt_llm.llmapi.trtllmargs method)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.validate_build_config_remaining", false]], "validate_build_config_with_runtime_params() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_build_config_with_runtime_params", false]], "validate_build_config_with_runtime_params() (tensorrt_llm.llmapi.trtllmargs method)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.validate_build_config_with_runtime_params", false]], "validate_capture_num_tokens() (tensorrt_llm.llmapi.torchcompileconfig class method)": [[143, "tensorrt_llm.llmapi.TorchCompileConfig.validate_capture_num_tokens", false]], "validate_checkpoint_format() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_checkpoint_format", false]], "validate_cuda_graph_config() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_cuda_graph_config", false]], "validate_cuda_graph_max_batch_size() (tensorrt_llm.llmapi.cudagraphconfig class method)": [[143, "tensorrt_llm.llmapi.CudaGraphConfig.validate_cuda_graph_max_batch_size", false]], "validate_dtype() (tensorrt_llm.llmapi.torchllmargs class method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_dtype", false]], "validate_dtype() (tensorrt_llm.llmapi.trtllmargs class method)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.validate_dtype", false]], "validate_enable_build_cache() (tensorrt_llm.llmapi.trtllmargs method)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.validate_enable_build_cache", false]], "validate_gpus_per_node() (tensorrt_llm.llmapi.torchllmargs class method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_gpus_per_node", false]], "validate_gpus_per_node() (tensorrt_llm.llmapi.trtllmargs class method)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.validate_gpus_per_node", false]], "validate_kv_cache_dtype() (tensorrt_llm.llmapi.trtllmargs method)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.validate_kv_cache_dtype", false]], "validate_load_balancer() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_load_balancer", false]], "validate_lora_config_consistency() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_lora_config_consistency", false]], "validate_lora_config_consistency() (tensorrt_llm.llmapi.trtllmargs method)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.validate_lora_config_consistency", false]], "validate_max_attention_window() (tensorrt_llm.llmapi.kvcacheconfig class method)": [[143, "tensorrt_llm.llmapi.KvCacheConfig.validate_max_attention_window", false]], "validate_max_gpu_total_bytes() (tensorrt_llm.llmapi.kvcacheconfig class method)": [[143, "tensorrt_llm.llmapi.KvCacheConfig.validate_max_gpu_total_bytes", false]], "validate_model() (tensorrt_llm.llmapi.torchllmargs class method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_model", false]], "validate_model() (tensorrt_llm.llmapi.trtllmargs class method)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.validate_model", false]], "validate_model_format_misc() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_model_format_misc", false]], "validate_model_format_misc() (tensorrt_llm.llmapi.trtllmargs method)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.validate_model_format_misc", false]], "validate_parallel_config() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_parallel_config", false]], "validate_parallel_config() (tensorrt_llm.llmapi.trtllmargs method)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.validate_parallel_config", false]], "validate_peft_cache_config() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_peft_cache_config", false]], "validate_peft_cache_config() (tensorrt_llm.llmapi.trtllmargs method)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.validate_peft_cache_config", false]], "validate_positive_values() (tensorrt_llm.llmapi.lookaheaddecodingconfig class method)": [[143, "tensorrt_llm.llmapi.LookaheadDecodingConfig.validate_positive_values", false]], "validate_quant_config() (tensorrt_llm.llmapi.trtllmargs class method)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.validate_quant_config", false]], "validate_runtime_args() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_runtime_args", false]], "validate_runtime_args() (tensorrt_llm.llmapi.trtllmargs method)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.validate_runtime_args", false]], "validate_speculative_config() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_speculative_config", false]], "validate_speculative_config() (tensorrt_llm.llmapi.trtllmargs method)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.validate_speculative_config", false]], "validate_stream_interval() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.validate_stream_interval", false]], "validate_torch_compile_max_num_streams() (tensorrt_llm.llmapi.torchcompileconfig class method)": [[143, "tensorrt_llm.llmapi.TorchCompileConfig.validate_torch_compile_max_num_streams", false]], "verbatim (tensorrt_llm.models.gemmaconfig attribute)": [[131, "tensorrt_llm.models.GemmaConfig.VERBATIM", false]], "video_preprocess() (tensorrt_llm.runtime.multimodalmodelrunner method)": [[134, "tensorrt_llm.runtime.MultimodalModelRunner.video_preprocess", false]], "view() (in module tensorrt_llm.functional)": [[129, "tensorrt_llm.functional.view", false]], "view() (tensorrt_llm.functional.tensor method)": [[129, "tensorrt_llm.functional.Tensor.view", false]], "view() (tensorrt_llm.runtime.tensorinfo method)": [[134, "tensorrt_llm.runtime.TensorInfo.view", false]], "visual_engine_dir (tensorrt_llm.runtime.multimodalmodelrunner property)": [[134, "tensorrt_llm.runtime.MultimodalModelRunner.visual_engine_dir", false]], "visualize_network (tensorrt_llm.llmapi.buildconfig attribute)": [[143, "tensorrt_llm.llmapi.BuildConfig.visualize_network", false]], "vocab_size (tensorrt_llm.runtime.generationsession property)": [[134, "tensorrt_llm.runtime.GenerationSession.vocab_size", false]], "vocab_size (tensorrt_llm.runtime.modelconfig attribute)": [[134, "tensorrt_llm.runtime.ModelConfig.vocab_size", false]], "vocab_size (tensorrt_llm.runtime.modelrunner property)": [[134, "tensorrt_llm.runtime.ModelRunner.vocab_size", false]], "vocab_size (tensorrt_llm.runtime.modelrunnercpp property)": [[134, "tensorrt_llm.runtime.ModelRunnerCpp.vocab_size", false]], "vocab_size_padded (tensorrt_llm.runtime.modelrunner property)": [[134, "tensorrt_llm.runtime.ModelRunner.vocab_size_padded", false]], "vocab_size_padded (tensorrt_llm.runtime.modelrunnercpp property)": [[134, "tensorrt_llm.runtime.ModelRunnerCpp.vocab_size_padded", false]], "w4a16 (tensorrt_llm.llmapi.quantalgo attribute)": [[143, "tensorrt_llm.llmapi.QuantAlgo.W4A16", false]], "w4a16_awq (tensorrt_llm.llmapi.quantalgo attribute)": [[143, "tensorrt_llm.llmapi.QuantAlgo.W4A16_AWQ", false]], "w4a16_gptq (tensorrt_llm.llmapi.quantalgo attribute)": [[143, "tensorrt_llm.llmapi.QuantAlgo.W4A16_GPTQ", false]], "w4a16_mxfp4 (tensorrt_llm.llmapi.quantalgo attribute)": [[143, "tensorrt_llm.llmapi.QuantAlgo.W4A16_MXFP4", false]], "w4a8_awq (tensorrt_llm.llmapi.quantalgo attribute)": [[143, "tensorrt_llm.llmapi.QuantAlgo.W4A8_AWQ", false]], "w4a8_mxfp4_fp8 (tensorrt_llm.llmapi.quantalgo attribute)": [[143, "tensorrt_llm.llmapi.QuantAlgo.W4A8_MXFP4_FP8", false]], "w4a8_mxfp4_mxfp8 (tensorrt_llm.llmapi.quantalgo attribute)": [[143, "tensorrt_llm.llmapi.QuantAlgo.W4A8_MXFP4_MXFP8", false]], "w4a8_nvfp4_fp8 (tensorrt_llm.llmapi.quantalgo attribute)": [[143, "tensorrt_llm.llmapi.QuantAlgo.W4A8_NVFP4_FP8", false]], "w4a8_qserve_per_channel (tensorrt_llm.llmapi.quantalgo attribute)": [[143, "tensorrt_llm.llmapi.QuantAlgo.W4A8_QSERVE_PER_CHANNEL", false]], "w4a8_qserve_per_group (tensorrt_llm.llmapi.quantalgo attribute)": [[143, "tensorrt_llm.llmapi.QuantAlgo.W4A8_QSERVE_PER_GROUP", false]], "w8a16 (tensorrt_llm.llmapi.quantalgo attribute)": [[143, "tensorrt_llm.llmapi.QuantAlgo.W8A16", false]], "w8a16_gptq (tensorrt_llm.llmapi.quantalgo attribute)": [[143, "tensorrt_llm.llmapi.QuantAlgo.W8A16_GPTQ", false]], "w8a8_sq_per_channel (tensorrt_llm.llmapi.quantalgo attribute)": [[143, "tensorrt_llm.llmapi.QuantAlgo.W8A8_SQ_PER_CHANNEL", false]], "w8a8_sq_per_channel_per_tensor_plugin (tensorrt_llm.llmapi.quantalgo attribute)": [[143, "tensorrt_llm.llmapi.QuantAlgo.W8A8_SQ_PER_CHANNEL_PER_TENSOR_PLUGIN", false]], "w8a8_sq_per_channel_per_token_plugin (tensorrt_llm.llmapi.quantalgo attribute)": [[143, "tensorrt_llm.llmapi.QuantAlgo.W8A8_SQ_PER_CHANNEL_PER_TOKEN_PLUGIN", false]], "w8a8_sq_per_tensor_per_token_plugin (tensorrt_llm.llmapi.quantalgo attribute)": [[143, "tensorrt_llm.llmapi.QuantAlgo.W8A8_SQ_PER_TENSOR_PER_TOKEN_PLUGIN", false]], "w8a8_sq_per_tensor_plugin (tensorrt_llm.llmapi.quantalgo attribute)": [[143, "tensorrt_llm.llmapi.QuantAlgo.W8A8_SQ_PER_TENSOR_PLUGIN", false]], "warn_on_unstable_feature_usage() (tensorrt_llm.llmapi.torchllmargs method)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.warn_on_unstable_feature_usage", false]], "weight_loader() (tensorrt_llm.layers.attention.deepseekv2attention method)": [[130, "tensorrt_llm.layers.attention.DeepseekV2Attention.weight_loader", false]], "weight_loader() (tensorrt_llm.layers.embedding.embedding method)": [[130, "tensorrt_llm.layers.embedding.Embedding.weight_loader", false]], "weight_loader() (tensorrt_llm.layers.linear.linearbase method)": [[130, "tensorrt_llm.layers.linear.LinearBase.weight_loader", false]], "weight_sparsity (tensorrt_llm.llmapi.buildconfig attribute)": [[143, "tensorrt_llm.llmapi.BuildConfig.weight_sparsity", false]], "weight_streaming (tensorrt_llm.llmapi.buildconfig attribute)": [[143, "tensorrt_llm.llmapi.BuildConfig.weight_streaming", false]], "where() (in module tensorrt_llm.functional)": [[129, "tensorrt_llm.functional.where", false]], "whisperencoder (class in tensorrt_llm.models)": [[131, "tensorrt_llm.models.WhisperEncoder", false]], "with_traceback() (tensorrt_llm.llmapi.requesterror method)": [[143, "tensorrt_llm.llmapi.RequestError.with_traceback", false]], "workspace (tensorrt_llm.llmapi.trtllmargs attribute)": [[143, "tensorrt_llm.llmapi.TrtLlmArgs.workspace", false]], "wrapped_property (tensorrt_llm.llmapi.torchllmargs attribute)": [[143, "tensorrt_llm.llmapi.TorchLlmArgs.wrapped_property", false]], "wrapped_property (tensorrt_llm.llmapi.trtllmargs attribute)": [[143, "id17", false], [143, "id20", false], [143, "tensorrt_llm.llmapi.TrtLlmArgs.wrapped_property", false]], "yarn (tensorrt_llm.functional.positionembeddingtype attribute)": [[129, "tensorrt_llm.functional.PositionEmbeddingType.yarn", false]], "yarn (tensorrt_llm.functional.rotaryscalingtype attribute)": [[129, "tensorrt_llm.functional.RotaryScalingType.yarn", false]], "zfill() (tensorrt_llm.llmapi.batchingtype method)": [[143, "tensorrt_llm.llmapi.BatchingType.zfill", false]], "zfill() (tensorrt_llm.llmapi.capacityschedulerpolicy method)": [[143, "tensorrt_llm.llmapi.CapacitySchedulerPolicy.zfill", false]], "zfill() (tensorrt_llm.llmapi.contextchunkingpolicy method)": [[143, "tensorrt_llm.llmapi.ContextChunkingPolicy.zfill", false]], "zfill() (tensorrt_llm.llmapi.quantalgo method)": [[143, "tensorrt_llm.llmapi.QuantAlgo.zfill", false]]}, "objects": {"": [[1, 0, 1, "c.FMT_DIM", "FMT_DIM"], [1, 0, 1, "c.SET_FROM_OPTIONAL", "SET_FROM_OPTIONAL"], [1, 1, 1, "_CPPv48nvinfer1", "nvinfer1"], [0, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [0, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [0, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [0, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [0, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [0, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [0, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [0, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [0, 1, 1, "_CPPv4N12tensorrt_llm13batch_managerE", "tensorrt_llm::batch_manager"], [1, 1, 1, "_CPPv4N12tensorrt_llm13batch_managerE", "tensorrt_llm::batch_manager"], [1, 1, 1, "_CPPv4N12tensorrt_llm13batch_managerE", "tensorrt_llm::batch_manager"], [1, 1, 1, "_CPPv4N12tensorrt_llm13batch_managerE", "tensorrt_llm::batch_manager"], [1, 1, 1, "_CPPv4N12tensorrt_llm13batch_managerE", "tensorrt_llm::batch_manager"], [0, 1, 1, "_CPPv4N12tensorrt_llm13batch_manager16kv_cache_managerE", "tensorrt_llm::batch_manager::kv_cache_manager"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executorE", "tensorrt_llm::executor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executorE", "tensorrt_llm::executor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executorE", "tensorrt_llm::executor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executorE", "tensorrt_llm::executor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executorE", "tensorrt_llm::executor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executorE", "tensorrt_llm::executor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executorE", "tensorrt_llm::executor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executorE", "tensorrt_llm::executor"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor21AdditionalModelOutputE", "tensorrt_llm::executor::AdditionalModelOutput"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor21AdditionalModelOutput21AdditionalModelOutputENSt6stringEb", "tensorrt_llm::executor::AdditionalModelOutput::AdditionalModelOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor21AdditionalModelOutput21AdditionalModelOutputENSt6stringEb", "tensorrt_llm::executor::AdditionalModelOutput::AdditionalModelOutput::gatherContext"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor21AdditionalModelOutput21AdditionalModelOutputENSt6stringEb", "tensorrt_llm::executor::AdditionalModelOutput::AdditionalModelOutput::name"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor21AdditionalModelOutput13gatherContextE", "tensorrt_llm::executor::AdditionalModelOutput::gatherContext"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor21AdditionalModelOutput4nameE", "tensorrt_llm::executor::AdditionalModelOutput::name"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor21AdditionalModelOutputeqERK21AdditionalModelOutput", "tensorrt_llm::executor::AdditionalModelOutput::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor21AdditionalModelOutputeqERK21AdditionalModelOutput", "tensorrt_llm::executor::AdditionalModelOutput::operator==::other"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutputE", "tensorrt_llm::executor::AdditionalOutput"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutput16AdditionalOutputENSt6stringE6Tensor", "tensorrt_llm::executor::AdditionalOutput::AdditionalOutput"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutput16AdditionalOutputERK16AdditionalOutput", "tensorrt_llm::executor::AdditionalOutput::AdditionalOutput"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutput16AdditionalOutputERR16AdditionalOutput", "tensorrt_llm::executor::AdditionalOutput::AdditionalOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutput16AdditionalOutputENSt6stringE6Tensor", "tensorrt_llm::executor::AdditionalOutput::AdditionalOutput::name"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutput16AdditionalOutputERK16AdditionalOutput", "tensorrt_llm::executor::AdditionalOutput::AdditionalOutput::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutput16AdditionalOutputERR16AdditionalOutput", "tensorrt_llm::executor::AdditionalOutput::AdditionalOutput::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutput16AdditionalOutputENSt6stringE6Tensor", "tensorrt_llm::executor::AdditionalOutput::AdditionalOutput::output"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutput4nameE", "tensorrt_llm::executor::AdditionalOutput::name"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutputaSERK16AdditionalOutput", "tensorrt_llm::executor::AdditionalOutput::operator="], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutputaSERR16AdditionalOutput", "tensorrt_llm::executor::AdditionalOutput::operator="], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutputaSERK16AdditionalOutput", "tensorrt_llm::executor::AdditionalOutput::operator=::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutputaSERR16AdditionalOutput", "tensorrt_llm::executor::AdditionalOutput::operator=::other"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutput6outputE", "tensorrt_llm::executor::AdditionalOutput::output"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutputD0Ev", "tensorrt_llm::executor::AdditionalOutput::~AdditionalOutput"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor12BatchingTypeE", "tensorrt_llm::executor::BatchingType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12BatchingType9kINFLIGHTE", "tensorrt_llm::executor::BatchingType::kINFLIGHT"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12BatchingType7kSTATICE", "tensorrt_llm::executor::BatchingType::kSTATIC"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor10BeamTokensE", "tensorrt_llm::executor::BeamTokens"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor10BufferViewE", "tensorrt_llm::executor::BufferView"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor15CacheSaltIDTypeE", "tensorrt_llm::executor::CacheSaltIDType"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfigE", "tensorrt_llm::executor::CacheTransceiverConfig"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig11BackendTypeE", "tensorrt_llm::executor::CacheTransceiverConfig::BackendType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig11BackendType7DEFAULTE", "tensorrt_llm::executor::CacheTransceiverConfig::BackendType::DEFAULT"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig11BackendType3MPIE", "tensorrt_llm::executor::CacheTransceiverConfig::BackendType::MPI"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig11BackendType4NIXLE", "tensorrt_llm::executor::CacheTransceiverConfig::BackendType::NIXL"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig11BackendType3UCXE", "tensorrt_llm::executor::CacheTransceiverConfig::BackendType::UCX"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig22CacheTransceiverConfigENSt8optionalI11BackendTypeEENSt8optionalI6size_tEE", "tensorrt_llm::executor::CacheTransceiverConfig::CacheTransceiverConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig22CacheTransceiverConfigENSt8optionalI11BackendTypeEENSt8optionalI6size_tEE", "tensorrt_llm::executor::CacheTransceiverConfig::CacheTransceiverConfig::backendType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig22CacheTransceiverConfigENSt8optionalI11BackendTypeEENSt8optionalI6size_tEE", "tensorrt_llm::executor::CacheTransceiverConfig::CacheTransceiverConfig::maxNumTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22CacheTransceiverConfig14getBackendTypeEv", "tensorrt_llm::executor::CacheTransceiverConfig::getBackendType"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22CacheTransceiverConfig20getMaxTokensInBufferEv", "tensorrt_llm::executor::CacheTransceiverConfig::getMaxTokensInBuffer"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig12mBackendTypeE", "tensorrt_llm::executor::CacheTransceiverConfig::mBackendType"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig18mMaxTokensInBufferE", "tensorrt_llm::executor::CacheTransceiverConfig::mMaxTokensInBuffer"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22CacheTransceiverConfigeqERK22CacheTransceiverConfig", "tensorrt_llm::executor::CacheTransceiverConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor22CacheTransceiverConfigeqERK22CacheTransceiverConfig", "tensorrt_llm::executor::CacheTransceiverConfig::operator==::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig14setBackendTypeENSt8optionalI11BackendTypeEE", "tensorrt_llm::executor::CacheTransceiverConfig::setBackendType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig14setBackendTypeENSt8optionalI11BackendTypeEE", "tensorrt_llm::executor::CacheTransceiverConfig::setBackendType::backendType"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig20setMaxTokensInBufferENSt8optionalI6size_tEE", "tensorrt_llm::executor::CacheTransceiverConfig::setMaxTokensInBuffer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig20setMaxTokensInBufferENSt8optionalI6size_tEE", "tensorrt_llm::executor::CacheTransceiverConfig::setMaxTokensInBuffer::maxTokensInBuffer"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor23CapacitySchedulerPolicyE", "tensorrt_llm::executor::CapacitySchedulerPolicy"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor23CapacitySchedulerPolicy20kGUARANTEED_NO_EVICTE", "tensorrt_llm::executor::CapacitySchedulerPolicy::kGUARANTEED_NO_EVICT"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor23CapacitySchedulerPolicy16kMAX_UTILIZATIONE", "tensorrt_llm::executor::CapacitySchedulerPolicy::kMAX_UTILIZATION"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor23CapacitySchedulerPolicy13kSTATIC_BATCHE", "tensorrt_llm::executor::CapacitySchedulerPolicy::kSTATIC_BATCH"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor17CommunicationModeE", "tensorrt_llm::executor::CommunicationMode"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor17CommunicationMode7kLEADERE", "tensorrt_llm::executor::CommunicationMode::kLEADER"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor17CommunicationMode13kORCHESTRATORE", "tensorrt_llm::executor::CommunicationMode::kORCHESTRATOR"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor17CommunicationTypeE", "tensorrt_llm::executor::CommunicationType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor17CommunicationType4kMPIE", "tensorrt_llm::executor::CommunicationType::kMPI"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor21ContextChunkingPolicyE", "tensorrt_llm::executor::ContextChunkingPolicy"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor21ContextChunkingPolicy15kEQUAL_PROGRESSE", "tensorrt_llm::executor::ContextChunkingPolicy::kEQUAL_PROGRESS"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor21ContextChunkingPolicy24kFIRST_COME_FIRST_SERVEDE", "tensorrt_llm::executor::ContextChunkingPolicy::kFIRST_COME_FIRST_SERVED"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParamsE", "tensorrt_llm::executor::ContextPhaseParams"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypeNSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypePvNSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypeRKNSt6vectorIcEENSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsERK18ContextPhaseParams", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsERR18ContextPhaseParams", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypeNSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::draftTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypePvNSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::draftTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypeRKNSt6vectorIcEENSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::draftTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypeNSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::firstGenTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypePvNSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::firstGenTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypeRKNSt6vectorIcEENSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::firstGenTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypeNSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::reqId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypePvNSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::reqId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypeRKNSt6vectorIcEENSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::reqId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypeRKNSt6vectorIcEENSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::serializedState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypePvNSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::state"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams13RequestIdTypeE", "tensorrt_llm::executor::ContextPhaseParams::RequestIdType"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams8StatePtrE", "tensorrt_llm::executor::ContextPhaseParams::StatePtr"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams7deleterEPKv", "tensorrt_llm::executor::ContextPhaseParams::deleter"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams7deleterEPKv", "tensorrt_llm::executor::ContextPhaseParams::deleter::data"], [0, 3, 1, "_CPPv4NKR12tensorrt_llm8executor18ContextPhaseParams14getDraftTokensEv", "tensorrt_llm::executor::ContextPhaseParams::getDraftTokens"], [0, 3, 1, "_CPPv4NKR12tensorrt_llm8executor18ContextPhaseParams17getFirstGenTokensEv", "tensorrt_llm::executor::ContextPhaseParams::getFirstGenTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18ContextPhaseParams8getReqIdEv", "tensorrt_llm::executor::ContextPhaseParams::getReqId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18ContextPhaseParams18getSerializedStateEv", "tensorrt_llm::executor::ContextPhaseParams::getSerializedState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams8getStateEv", "tensorrt_llm::executor::ContextPhaseParams::getState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18ContextPhaseParams8getStateEv", "tensorrt_llm::executor::ContextPhaseParams::getState"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams12mDraftTokensE", "tensorrt_llm::executor::ContextPhaseParams::mDraftTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams15mFirstGenTokensE", "tensorrt_llm::executor::ContextPhaseParams::mFirstGenTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams6mReqIdE", "tensorrt_llm::executor::ContextPhaseParams::mReqId"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams6mStateE", "tensorrt_llm::executor::ContextPhaseParams::mState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParamsaSERK18ContextPhaseParams", "tensorrt_llm::executor::ContextPhaseParams::operator="], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParamsaSERR18ContextPhaseParams", "tensorrt_llm::executor::ContextPhaseParams::operator="], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18ContextPhaseParamseqERK18ContextPhaseParams", "tensorrt_llm::executor::ContextPhaseParams::operator=="], [0, 3, 1, "_CPPv4NO12tensorrt_llm8executor18ContextPhaseParams17popFirstGenTokensEv", "tensorrt_llm::executor::ContextPhaseParams::popFirstGenTokens"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams12releaseStateEv", "tensorrt_llm::executor::ContextPhaseParams::releaseState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParamsD0Ev", "tensorrt_llm::executor::ContextPhaseParams::~ContextPhaseParams"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverStateE", "tensorrt_llm::executor::DataTransceiverState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState20DataTransceiverStateEN8kv_cache10CacheStateEN8kv_cache9CommStateE", "tensorrt_llm::executor::DataTransceiverState::DataTransceiverState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState20DataTransceiverStateEv", "tensorrt_llm::executor::DataTransceiverState::DataTransceiverState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState20DataTransceiverStateEN8kv_cache10CacheStateEN8kv_cache9CommStateE", "tensorrt_llm::executor::DataTransceiverState::DataTransceiverState::cacheState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState20DataTransceiverStateEN8kv_cache10CacheStateEN8kv_cache9CommStateE", "tensorrt_llm::executor::DataTransceiverState::DataTransceiverState::commState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20DataTransceiverState13getCacheStateEv", "tensorrt_llm::executor::DataTransceiverState::getCacheState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20DataTransceiverState12getCommStateEv", "tensorrt_llm::executor::DataTransceiverState::getCommState"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState11mCacheStateE", "tensorrt_llm::executor::DataTransceiverState::mCacheState"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState10mCommStateE", "tensorrt_llm::executor::DataTransceiverState::mCommState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20DataTransceiverStateeqERK20DataTransceiverState", "tensorrt_llm::executor::DataTransceiverState::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor20DataTransceiverStateeqERK20DataTransceiverState", "tensorrt_llm::executor::DataTransceiverState::operator==::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState13setCacheStateEN8kv_cache10CacheStateE", "tensorrt_llm::executor::DataTransceiverState::setCacheState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState13setCacheStateEN8kv_cache10CacheStateE", "tensorrt_llm::executor::DataTransceiverState::setCacheState::state"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState12setCommStateEN8kv_cache9CommStateE", "tensorrt_llm::executor::DataTransceiverState::setCommState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState12setCommStateEN8kv_cache9CommStateE", "tensorrt_llm::executor::DataTransceiverState::setCommState::state"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20DataTransceiverState8toStringEv", "tensorrt_llm::executor::DataTransceiverState::toString"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor8DataTypeE", "tensorrt_llm::executor::DataType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType5kBF16E", "tensorrt_llm::executor::DataType::kBF16"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType5kBOOLE", "tensorrt_llm::executor::DataType::kBOOL"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType5kFP16E", "tensorrt_llm::executor::DataType::kFP16"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType5kFP32E", "tensorrt_llm::executor::DataType::kFP32"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType4kFP8E", "tensorrt_llm::executor::DataType::kFP8"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType6kINT32E", "tensorrt_llm::executor::DataType::kINT32"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType6kINT64E", "tensorrt_llm::executor::DataType::kINT64"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType5kINT8E", "tensorrt_llm::executor::DataType::kINT8"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType6kUINT8E", "tensorrt_llm::executor::DataType::kUINT8"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType8kUNKNOWNE", "tensorrt_llm::executor::DataType::kUNKNOWN"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfigE", "tensorrt_llm::executor::DebugConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig11DebugConfigEbb9StringVec10SizeType32", "tensorrt_llm::executor::DebugConfig::DebugConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig11DebugConfigEbb9StringVec10SizeType32", "tensorrt_llm::executor::DebugConfig::DebugConfig::debugInputTensors"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig11DebugConfigEbb9StringVec10SizeType32", "tensorrt_llm::executor::DebugConfig::DebugConfig::debugOutputTensors"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig11DebugConfigEbb9StringVec10SizeType32", "tensorrt_llm::executor::DebugConfig::DebugConfig::debugTensorNames"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig11DebugConfigEbb9StringVec10SizeType32", "tensorrt_llm::executor::DebugConfig::DebugConfig::debugTensorsMaxIterations"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig9StringVecE", "tensorrt_llm::executor::DebugConfig::StringVec"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11DebugConfig20getDebugInputTensorsEv", "tensorrt_llm::executor::DebugConfig::getDebugInputTensors"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11DebugConfig21getDebugOutputTensorsEv", "tensorrt_llm::executor::DebugConfig::getDebugOutputTensors"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11DebugConfig19getDebugTensorNamesEv", "tensorrt_llm::executor::DebugConfig::getDebugTensorNames"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11DebugConfig28getDebugTensorsMaxIterationsEv", "tensorrt_llm::executor::DebugConfig::getDebugTensorsMaxIterations"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig18mDebugInputTensorsE", "tensorrt_llm::executor::DebugConfig::mDebugInputTensors"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig19mDebugOutputTensorsE", "tensorrt_llm::executor::DebugConfig::mDebugOutputTensors"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig17mDebugTensorNamesE", "tensorrt_llm::executor::DebugConfig::mDebugTensorNames"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig26mDebugTensorsMaxIterationsE", "tensorrt_llm::executor::DebugConfig::mDebugTensorsMaxIterations"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11DebugConfigeqERK11DebugConfig", "tensorrt_llm::executor::DebugConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor11DebugConfigeqERK11DebugConfig", "tensorrt_llm::executor::DebugConfig::operator==::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig20setDebugInputTensorsEb", "tensorrt_llm::executor::DebugConfig::setDebugInputTensors"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig20setDebugInputTensorsEb", "tensorrt_llm::executor::DebugConfig::setDebugInputTensors::debugInputTensors"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig21setDebugOutputTensorsEb", "tensorrt_llm::executor::DebugConfig::setDebugOutputTensors"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig21setDebugOutputTensorsEb", "tensorrt_llm::executor::DebugConfig::setDebugOutputTensors::debugOutputTensors"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig19setDebugTensorNamesERK9StringVec", "tensorrt_llm::executor::DebugConfig::setDebugTensorNames"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig19setDebugTensorNamesERK9StringVec", "tensorrt_llm::executor::DebugConfig::setDebugTensorNames::debugTensorNames"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig28setDebugTensorsMaxIterationsE10SizeType32", "tensorrt_llm::executor::DebugConfig::setDebugTensorsMaxIterations"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig28setDebugTensorsMaxIterationsE10SizeType32", "tensorrt_llm::executor::DebugConfig::setDebugTensorsMaxIterations::debugTensorsMaxIterations"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor24DebugTensorsPerIterationE", "tensorrt_llm::executor::DebugTensorsPerIteration"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor24DebugTensorsPerIteration12debugTensorsE", "tensorrt_llm::executor::DebugTensorsPerIteration::debugTensors"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor24DebugTensorsPerIteration4iterE", "tensorrt_llm::executor::DebugTensorsPerIteration::iter"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfigE", "tensorrt_llm::executor::DecodingConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig14DecodingConfigENSt8optionalI12DecodingModeEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI13MedusaChoicesEENSt8optionalI11EagleConfigEE", "tensorrt_llm::executor::DecodingConfig::DecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig14DecodingConfigENSt8optionalI12DecodingModeEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI13MedusaChoicesEENSt8optionalI11EagleConfigEE", "tensorrt_llm::executor::DecodingConfig::DecodingConfig::decodingMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig14DecodingConfigENSt8optionalI12DecodingModeEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI13MedusaChoicesEENSt8optionalI11EagleConfigEE", "tensorrt_llm::executor::DecodingConfig::DecodingConfig::eagleConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig14DecodingConfigENSt8optionalI12DecodingModeEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI13MedusaChoicesEENSt8optionalI11EagleConfigEE", "tensorrt_llm::executor::DecodingConfig::DecodingConfig::lookaheadDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig14DecodingConfigENSt8optionalI12DecodingModeEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI13MedusaChoicesEENSt8optionalI11EagleConfigEE", "tensorrt_llm::executor::DecodingConfig::DecodingConfig::medusaChoices"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig31enableSeamlessLookaheadDecodingEv", "tensorrt_llm::executor::DecodingConfig::enableSeamlessLookaheadDecoding"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14DecodingConfig15getDecodingModeEv", "tensorrt_llm::executor::DecodingConfig::getDecodingMode"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14DecodingConfig14getEagleConfigEv", "tensorrt_llm::executor::DecodingConfig::getEagleConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14DecodingConfig26getLookaheadDecodingConfigEv", "tensorrt_llm::executor::DecodingConfig::getLookaheadDecodingConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14DecodingConfig33getLookaheadDecodingMaxNumRequestEv", "tensorrt_llm::executor::DecodingConfig::getLookaheadDecodingMaxNumRequest"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14DecodingConfig16getMedusaChoicesEv", "tensorrt_llm::executor::DecodingConfig::getMedusaChoices"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig13mDecodingModeE", "tensorrt_llm::executor::DecodingConfig::mDecodingMode"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig12mEagleConfigE", "tensorrt_llm::executor::DecodingConfig::mEagleConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig24mLookaheadDecodingConfigE", "tensorrt_llm::executor::DecodingConfig::mLookaheadDecodingConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig31mLookaheadDecodingMaxNumRequestE", "tensorrt_llm::executor::DecodingConfig::mLookaheadDecodingMaxNumRequest"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig14mMedusaChoicesE", "tensorrt_llm::executor::DecodingConfig::mMedusaChoices"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14DecodingConfigeqERK14DecodingConfig", "tensorrt_llm::executor::DecodingConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor14DecodingConfigeqERK14DecodingConfig", "tensorrt_llm::executor::DecodingConfig::operator==::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig15setDecodingModeERK12DecodingMode", "tensorrt_llm::executor::DecodingConfig::setDecodingMode"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig14setEagleConfigERK11EagleConfig", "tensorrt_llm::executor::DecodingConfig::setEagleConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig26setLookaheadDecodingConfigERK23LookaheadDecodingConfig", "tensorrt_llm::executor::DecodingConfig::setLookaheadDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig26setLookaheadDecodingConfigERK23LookaheadDecodingConfig", "tensorrt_llm::executor::DecodingConfig::setLookaheadDecodingConfig::lookaheadDecodingConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig16setMedusaChoicesERK13MedusaChoices", "tensorrt_llm::executor::DecodingConfig::setMedusaChoices"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor12DecodingModeE", "tensorrt_llm::executor::DecodingMode"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode4AutoEv", "tensorrt_llm::executor::DecodingMode::Auto"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode10BeamSearchEv", "tensorrt_llm::executor::DecodingMode::BeamSearch"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode12DecodingModeE14UnderlyingType", "tensorrt_llm::executor::DecodingMode::DecodingMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode12DecodingModeE14UnderlyingType", "tensorrt_llm::executor::DecodingMode::DecodingMode::state"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode5EagleEv", "tensorrt_llm::executor::DecodingMode::Eagle"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode19ExplicitDraftTokensEv", "tensorrt_llm::executor::DecodingMode::ExplicitDraftTokens"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode19ExternalDraftTokensEv", "tensorrt_llm::executor::DecodingMode::ExternalDraftTokens"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode9LookaheadEv", "tensorrt_llm::executor::DecodingMode::Lookahead"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode6MedusaEv", "tensorrt_llm::executor::DecodingMode::Medusa"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode4TopKEv", "tensorrt_llm::executor::DecodingMode::TopK"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode8TopKTopPEv", "tensorrt_llm::executor::DecodingMode::TopKTopP"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode4TopPEv", "tensorrt_llm::executor::DecodingMode::TopP"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode14UnderlyingTypeE", "tensorrt_llm::executor::DecodingMode::UnderlyingType"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode9allBitSetE14UnderlyingType", "tensorrt_llm::executor::DecodingMode::allBitSet"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode9allBitSetE14UnderlyingType", "tensorrt_llm::executor::DecodingMode::allBitSet::bits"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode9anyBitSetE14UnderlyingType", "tensorrt_llm::executor::DecodingMode::anyBitSet"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode9anyBitSetE14UnderlyingType", "tensorrt_llm::executor::DecodingMode::anyBitSet::bits"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode7getNameEv", "tensorrt_llm::executor::DecodingMode::getName"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode8getStateEv", "tensorrt_llm::executor::DecodingMode::getState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode6isAutoEv", "tensorrt_llm::executor::DecodingMode::isAuto"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode12isBeamSearchEv", "tensorrt_llm::executor::DecodingMode::isBeamSearch"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode7isEagleEv", "tensorrt_llm::executor::DecodingMode::isEagle"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode21isExplicitDraftTokensEv", "tensorrt_llm::executor::DecodingMode::isExplicitDraftTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode21isExternalDraftTokensEv", "tensorrt_llm::executor::DecodingMode::isExternalDraftTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode11isLookaheadEv", "tensorrt_llm::executor::DecodingMode::isLookahead"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode8isMedusaEv", "tensorrt_llm::executor::DecodingMode::isMedusa"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode6isTopKEv", "tensorrt_llm::executor::DecodingMode::isTopK"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode13isTopKandTopPEv", "tensorrt_llm::executor::DecodingMode::isTopKandTopP"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode12isTopKorTopPEv", "tensorrt_llm::executor::DecodingMode::isTopKorTopP"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode6isTopPEv", "tensorrt_llm::executor::DecodingMode::isTopP"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode14isUseBanTokensEv", "tensorrt_llm::executor::DecodingMode::isUseBanTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode13isUseBanWordsEv", "tensorrt_llm::executor::DecodingMode::isUseBanWords"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode20isUseExplicitEosStopEv", "tensorrt_llm::executor::DecodingMode::isUseExplicitEosStop"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode21isUseFrequencyPenaltyEv", "tensorrt_llm::executor::DecodingMode::isUseFrequencyPenalty"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode18isUseMaxLengthStopEv", "tensorrt_llm::executor::DecodingMode::isUseMaxLengthStop"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode14isUseMinLengthEv", "tensorrt_llm::executor::DecodingMode::isUseMinLength"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode9isUseMinPEv", "tensorrt_llm::executor::DecodingMode::isUseMinP"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode22isUseNoRepeatNgramSizeEv", "tensorrt_llm::executor::DecodingMode::isUseNoRepeatNgramSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode22isUseOccurrencePenaltyEv", "tensorrt_llm::executor::DecodingMode::isUseOccurrencePenalty"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode12isUsePenaltyEv", "tensorrt_llm::executor::DecodingMode::isUsePenalty"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode20isUsePresencePenaltyEv", "tensorrt_llm::executor::DecodingMode::isUsePresencePenalty"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode22isUseRepetitionPenaltyEv", "tensorrt_llm::executor::DecodingMode::isUseRepetitionPenalty"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode17isUseStopCriteriaEv", "tensorrt_llm::executor::DecodingMode::isUseStopCriteria"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode14isUseStopWordsEv", "tensorrt_llm::executor::DecodingMode::isUseStopWords"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode16isUseTemperatureEv", "tensorrt_llm::executor::DecodingMode::isUseTemperature"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode28isUseVariableBeamWidthSearchEv", "tensorrt_llm::executor::DecodingMode::isUseVariableBeamWidthSearch"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode5kAutoE", "tensorrt_llm::executor::DecodingMode::kAuto"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode11kBeamSearchE", "tensorrt_llm::executor::DecodingMode::kBeamSearch"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode6kEagleE", "tensorrt_llm::executor::DecodingMode::kEagle"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode20kExplicitDraftTokensE", "tensorrt_llm::executor::DecodingMode::kExplicitDraftTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode20kExternalDraftTokensE", "tensorrt_llm::executor::DecodingMode::kExternalDraftTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode10kLookaheadE", "tensorrt_llm::executor::DecodingMode::kLookahead"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode7kMedusaE", "tensorrt_llm::executor::DecodingMode::kMedusa"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode9kNumFlagsE", "tensorrt_llm::executor::DecodingMode::kNumFlags"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode5kTopKE", "tensorrt_llm::executor::DecodingMode::kTopK"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode9kTopKTopPE", "tensorrt_llm::executor::DecodingMode::kTopKTopP"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode5kTopPE", "tensorrt_llm::executor::DecodingMode::kTopP"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode13kUseBanTokensE", "tensorrt_llm::executor::DecodingMode::kUseBanTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode12kUseBanWordsE", "tensorrt_llm::executor::DecodingMode::kUseBanWords"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode19kUseExplicitEosStopE", "tensorrt_llm::executor::DecodingMode::kUseExplicitEosStop"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode22kUseFrequencyPenaltiesE", "tensorrt_llm::executor::DecodingMode::kUseFrequencyPenalties"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode17kUseMaxLengthStopE", "tensorrt_llm::executor::DecodingMode::kUseMaxLengthStop"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode13kUseMinLengthE", "tensorrt_llm::executor::DecodingMode::kUseMinLength"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode8kUseMinPE", "tensorrt_llm::executor::DecodingMode::kUseMinP"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode21kUseNoRepeatNgramSizeE", "tensorrt_llm::executor::DecodingMode::kUseNoRepeatNgramSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode23kUseOccurrencePenaltiesE", "tensorrt_llm::executor::DecodingMode::kUseOccurrencePenalties"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode13kUsePenaltiesE", "tensorrt_llm::executor::DecodingMode::kUsePenalties"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode21kUsePresencePenaltiesE", "tensorrt_llm::executor::DecodingMode::kUsePresencePenalties"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode23kUseRepetitionPenaltiesE", "tensorrt_llm::executor::DecodingMode::kUseRepetitionPenalties"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode24kUseStandardStopCriteriaE", "tensorrt_llm::executor::DecodingMode::kUseStandardStopCriteria"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode13kUseStopWordsE", "tensorrt_llm::executor::DecodingMode::kUseStopWords"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode15kUseTemperatureE", "tensorrt_llm::executor::DecodingMode::kUseTemperature"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode27kUseVariableBeamWidthSearchE", "tensorrt_llm::executor::DecodingMode::kUseVariableBeamWidthSearch"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode6mStateE", "tensorrt_llm::executor::DecodingMode::mState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingModeeqERK12DecodingMode", "tensorrt_llm::executor::DecodingMode::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingModeeqERK12DecodingMode", "tensorrt_llm::executor::DecodingMode::operator==::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode8setBitToE14UnderlyingTypeb", "tensorrt_llm::executor::DecodingMode::setBitTo"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode8setBitToE14UnderlyingTypeb", "tensorrt_llm::executor::DecodingMode::setBitTo::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode8setBitToE14UnderlyingTypeb", "tensorrt_llm::executor::DecodingMode::setBitTo::x"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode12useBanTokensEb", "tensorrt_llm::executor::DecodingMode::useBanTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode12useBanTokensEb", "tensorrt_llm::executor::DecodingMode::useBanTokens::banTokens"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode11useBanWordsEb", "tensorrt_llm::executor::DecodingMode::useBanWords"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode11useBanWordsEb", "tensorrt_llm::executor::DecodingMode::useBanWords::banWords"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode18useExplicitEosStopEb", "tensorrt_llm::executor::DecodingMode::useExplicitEosStop"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode18useExplicitEosStopEb", "tensorrt_llm::executor::DecodingMode::useExplicitEosStop::explicitEosStop"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode19useFrequencyPenaltyEb", "tensorrt_llm::executor::DecodingMode::useFrequencyPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode19useFrequencyPenaltyEb", "tensorrt_llm::executor::DecodingMode::useFrequencyPenalty::usePenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode16useMaxLengthStopEb", "tensorrt_llm::executor::DecodingMode::useMaxLengthStop"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode16useMaxLengthStopEb", "tensorrt_llm::executor::DecodingMode::useMaxLengthStop::maxLengthStop"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode12useMinLengthEb", "tensorrt_llm::executor::DecodingMode::useMinLength"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode12useMinLengthEb", "tensorrt_llm::executor::DecodingMode::useMinLength::useMinLen"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode7useMinPEb", "tensorrt_llm::executor::DecodingMode::useMinP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode7useMinPEb", "tensorrt_llm::executor::DecodingMode::useMinP::useMinP"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode20useNoRepeatNgramSizeEb", "tensorrt_llm::executor::DecodingMode::useNoRepeatNgramSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode20useNoRepeatNgramSizeEb", "tensorrt_llm::executor::DecodingMode::useNoRepeatNgramSize::noRepeatNgramSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode22useOccurrencePenaltiesEb", "tensorrt_llm::executor::DecodingMode::useOccurrencePenalties"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode22useOccurrencePenaltiesEb", "tensorrt_llm::executor::DecodingMode::useOccurrencePenalties::usePenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode18usePresencePenaltyEb", "tensorrt_llm::executor::DecodingMode::usePresencePenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode18usePresencePenaltyEb", "tensorrt_llm::executor::DecodingMode::usePresencePenalty::usePenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode20useRepetitionPenaltyEb", "tensorrt_llm::executor::DecodingMode::useRepetitionPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode20useRepetitionPenaltyEb", "tensorrt_llm::executor::DecodingMode::useRepetitionPenalty::usePenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode12useStopWordsEb", "tensorrt_llm::executor::DecodingMode::useStopWords"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode12useStopWordsEb", "tensorrt_llm::executor::DecodingMode::useStopWords::stopWords"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode14useTemperatureEb", "tensorrt_llm::executor::DecodingMode::useTemperature"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode14useTemperatureEb", "tensorrt_llm::executor::DecodingMode::useTemperature::useTemp"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode26useVariableBeamWidthSearchEb", "tensorrt_llm::executor::DecodingMode::useVariableBeamWidthSearch"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode26useVariableBeamWidthSearchEb", "tensorrt_llm::executor::DecodingMode::useVariableBeamWidthSearch::useVariableBeamWidthSearch"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor22DisServingRequestStatsE", "tensorrt_llm::executor::DisServingRequestStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22DisServingRequestStats11kvCacheSizeE", "tensorrt_llm::executor::DisServingRequestStats::kvCacheSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22DisServingRequestStats17kvCacheTransferMSE", "tensorrt_llm::executor::DisServingRequestStats::kvCacheTransferMS"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfigE", "tensorrt_llm::executor::DynamicBatchConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig18DynamicBatchConfigEbb10SizeType32NSt6vectorINSt4pairI10SizeType3210SizeType32EEEE", "tensorrt_llm::executor::DynamicBatchConfig::DynamicBatchConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig18DynamicBatchConfigEbb10SizeType32NSt6vectorINSt4pairI10SizeType3210SizeType32EEEE", "tensorrt_llm::executor::DynamicBatchConfig::DynamicBatchConfig::batchSizeTable"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig18DynamicBatchConfigEbb10SizeType32NSt6vectorINSt4pairI10SizeType3210SizeType32EEEE", "tensorrt_llm::executor::DynamicBatchConfig::DynamicBatchConfig::dynamicBatchMovingAverageWindow"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig18DynamicBatchConfigEbb10SizeType32NSt6vectorINSt4pairI10SizeType3210SizeType32EEEE", "tensorrt_llm::executor::DynamicBatchConfig::DynamicBatchConfig::enableBatchSizeTuning"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig18DynamicBatchConfigEbb10SizeType32NSt6vectorINSt4pairI10SizeType3210SizeType32EEEE", "tensorrt_llm::executor::DynamicBatchConfig::DynamicBatchConfig::enableMaxNumTokensTuning"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18DynamicBatchConfig17getBatchSizeTableEv", "tensorrt_llm::executor::DynamicBatchConfig::getBatchSizeTable"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18DynamicBatchConfig34getDynamicBatchMovingAverageWindowEv", "tensorrt_llm::executor::DynamicBatchConfig::getDynamicBatchMovingAverageWindow"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18DynamicBatchConfig24getEnableBatchSizeTuningEv", "tensorrt_llm::executor::DynamicBatchConfig::getEnableBatchSizeTuning"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18DynamicBatchConfig27getEnableMaxNumTokensTuningEv", "tensorrt_llm::executor::DynamicBatchConfig::getEnableMaxNumTokensTuning"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig22kDefaultBatchSizeTableE", "tensorrt_llm::executor::DynamicBatchConfig::kDefaultBatchSizeTable"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig39kDefaultDynamicBatchMovingAverageWindowE", "tensorrt_llm::executor::DynamicBatchConfig::kDefaultDynamicBatchMovingAverageWindow"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig15mBatchSizeTableE", "tensorrt_llm::executor::DynamicBatchConfig::mBatchSizeTable"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig32mDynamicBatchMovingAverageWindowE", "tensorrt_llm::executor::DynamicBatchConfig::mDynamicBatchMovingAverageWindow"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig22mEnableBatchSizeTuningE", "tensorrt_llm::executor::DynamicBatchConfig::mEnableBatchSizeTuning"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig25mEnableMaxNumTokensTuningE", "tensorrt_llm::executor::DynamicBatchConfig::mEnableMaxNumTokensTuning"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor12EagleChoicesE", "tensorrt_llm::executor::EagleChoices"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfigE", "tensorrt_llm::executor::EagleConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig11EagleConfigENSt8optionalI12EagleChoicesEEbNSt8optionalIfEEbNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::EagleConfig::EagleConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig11EagleConfigENSt8optionalI12EagleChoicesEEbNSt8optionalIfEEbNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::EagleConfig::EagleConfig::dynamicTreeMaxTopK"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig11EagleConfigENSt8optionalI12EagleChoicesEEbNSt8optionalIfEEbNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::EagleConfig::EagleConfig::eagleChoices"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig11EagleConfigENSt8optionalI12EagleChoicesEEbNSt8optionalIfEEbNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::EagleConfig::EagleConfig::greedySampling"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig11EagleConfigENSt8optionalI12EagleChoicesEEbNSt8optionalIfEEbNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::EagleConfig::EagleConfig::posteriorThreshold"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig11EagleConfigENSt8optionalI12EagleChoicesEEbNSt8optionalIfEEbNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::EagleConfig::EagleConfig::useDynamicTree"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig19checkPosteriorValueERKNSt8optionalIfEE", "tensorrt_llm::executor::EagleConfig::checkPosteriorValue"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig19checkPosteriorValueERKNSt8optionalIfEE", "tensorrt_llm::executor::EagleConfig::checkPosteriorValue::value"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11EagleConfig21getDynamicTreeMaxTopKEv", "tensorrt_llm::executor::EagleConfig::getDynamicTreeMaxTopK"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11EagleConfig15getEagleChoicesEv", "tensorrt_llm::executor::EagleConfig::getEagleChoices"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11EagleConfig21getPosteriorThresholdEv", "tensorrt_llm::executor::EagleConfig::getPosteriorThreshold"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11EagleConfig16isGreedySamplingEv", "tensorrt_llm::executor::EagleConfig::isGreedySampling"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig19mDynamicTreeMaxTopKE", "tensorrt_llm::executor::EagleConfig::mDynamicTreeMaxTopK"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig13mEagleChoicesE", "tensorrt_llm::executor::EagleConfig::mEagleChoices"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig15mGreedySamplingE", "tensorrt_llm::executor::EagleConfig::mGreedySampling"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig19mPosteriorThresholdE", "tensorrt_llm::executor::EagleConfig::mPosteriorThreshold"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig15mUseDynamicTreeE", "tensorrt_llm::executor::EagleConfig::mUseDynamicTree"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11EagleConfigeqERK11EagleConfig", "tensorrt_llm::executor::EagleConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor11EagleConfigeqERK11EagleConfig", "tensorrt_llm::executor::EagleConfig::operator==::other"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11EagleConfig14useDynamicTreeEv", "tensorrt_llm::executor::EagleConfig::useDynamicTree"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8ExecutorE", "tensorrt_llm::executor::Executor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorENSt10shared_ptrI5ModelEENSt10shared_ptrI5ModelEERK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorENSt10shared_ptrI5ModelEERK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfigRKNSt8optionalINSt3mapINSt6stringE6TensorEEEE", "tensorrt_llm::executor::Executor::Executor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK8Executor", "tensorrt_llm::executor::Executor::Executor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERKNSt10filesystem4pathE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERKNSt10filesystem4pathERKNSt10filesystem4pathE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERR8Executor", "tensorrt_llm::executor::Executor::Executor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::decoderEngineBuffer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::decoderJsonConfigStr"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorENSt10shared_ptrI5ModelEENSt10shared_ptrI5ModelEERK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::decoderModel"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERKNSt10filesystem4pathERKNSt10filesystem4pathE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::decoderModelPath"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::encoderEngineBuffer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::encoderJsonConfigStr"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorENSt10shared_ptrI5ModelEENSt10shared_ptrI5ModelEERK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::encoderModel"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERKNSt10filesystem4pathERKNSt10filesystem4pathE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::encoderModelPath"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfigRKNSt8optionalINSt3mapINSt6stringE6TensorEEEE", "tensorrt_llm::executor::Executor::Executor::engineBuffer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK8Executor", "tensorrt_llm::executor::Executor::Executor::executor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorENSt10shared_ptrI5ModelEENSt10shared_ptrI5ModelEERK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::executorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorENSt10shared_ptrI5ModelEERK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::executorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfigRKNSt8optionalINSt3mapINSt6stringE6TensorEEEE", "tensorrt_llm::executor::Executor::Executor::executorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::executorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERKNSt10filesystem4pathE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::executorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERKNSt10filesystem4pathERKNSt10filesystem4pathE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::executorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfigRKNSt8optionalINSt3mapINSt6stringE6TensorEEEE", "tensorrt_llm::executor::Executor::Executor::jsonConfigStr"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfigRKNSt8optionalINSt3mapINSt6stringE6TensorEEEE", "tensorrt_llm::executor::Executor::Executor::managedWeights"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorENSt10shared_ptrI5ModelEERK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::model"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERKNSt10filesystem4pathE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::modelPath"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfigRKNSt8optionalINSt3mapINSt6stringE6TensorEEEE", "tensorrt_llm::executor::Executor::Executor::modelType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::modelType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERKNSt10filesystem4pathE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::modelType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERKNSt10filesystem4pathERKNSt10filesystem4pathE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::modelType"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor14awaitResponsesERK6IdTypeRKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::Executor::awaitResponses"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor14awaitResponsesERKNSt6vectorI6IdTypeEERKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::Executor::awaitResponses"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor14awaitResponsesERKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::Executor::awaitResponses"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor14awaitResponsesERK6IdTypeRKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::Executor::awaitResponses::requestId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor14awaitResponsesERKNSt6vectorI6IdTypeEERKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::Executor::awaitResponses::requestIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor14awaitResponsesERK6IdTypeRKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::Executor::awaitResponses::timeout"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor14awaitResponsesERKNSt6vectorI6IdTypeEERKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::Executor::awaitResponses::timeout"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor14awaitResponsesERKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::Executor::awaitResponses::timeout"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8Executor18canEnqueueRequestsEv", "tensorrt_llm::executor::Executor::canEnqueueRequests"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor13cancelRequestE6IdType", "tensorrt_llm::executor::Executor::cancelRequest"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor13cancelRequestE6IdType", "tensorrt_llm::executor::Executor::cancelRequest::requestId"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor14enqueueRequestERK7Request", "tensorrt_llm::executor::Executor::enqueueRequest"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor14enqueueRequestERK7Request", "tensorrt_llm::executor::Executor::enqueueRequest::request"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor15enqueueRequestsERKNSt6vectorI7RequestEE", "tensorrt_llm::executor::Executor::enqueueRequests"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor15enqueueRequestsERKNSt6vectorI7RequestEE", "tensorrt_llm::executor::Executor::enqueueRequests::requests"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8Executor22getKVCacheEventManagerEv", "tensorrt_llm::executor::Executor::getKVCacheEventManager"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor21getLatestDebugTensorsEv", "tensorrt_llm::executor::Executor::getLatestDebugTensors"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor23getLatestIterationStatsEv", "tensorrt_llm::executor::Executor::getLatestIterationStats"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor21getLatestRequestStatsEv", "tensorrt_llm::executor::Executor::getLatestRequestStats"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8Executor20getNumResponsesReadyERKNSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Executor::getNumResponsesReady"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8Executor20getNumResponsesReadyERKNSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Executor::getNumResponsesReady::requestId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8Executor13isParticipantEv", "tensorrt_llm::executor::Executor::isParticipant"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8Executor5mImplE", "tensorrt_llm::executor::Executor::mImpl"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8ExecutoraSERK8Executor", "tensorrt_llm::executor::Executor::operator="], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8ExecutoraSERR8Executor", "tensorrt_llm::executor::Executor::operator="], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8ExecutoraSERK8Executor", "tensorrt_llm::executor::Executor::operator=::executor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor8shutdownEv", "tensorrt_llm::executor::Executor::shutdown"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8ExecutorD0Ev", "tensorrt_llm::executor::Executor::~Executor"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfigE", "tensorrt_llm::executor::ExecutorConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::additionalModelOutputs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::batchingType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::cacheTransceiverConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::debugConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::decodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::enableChunkedContext"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::enableTrtOverlap"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::extendedRuntimePerfKnobConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::failFastOnAttentionWindowTooLarge"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::gatherGenerationLogits"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::gpuWeightsPercent"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::guidedDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::iterStatsMaxIterations"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::kvCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::logitsPostProcessorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::maxBatchSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::maxBeamWidth"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::maxNumTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::maxQueueSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::maxSeqIdleMicroseconds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::normalizeLogProbs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::parallelConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::peftCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::promptTableOffloading"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::recvPollPeriodMs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::requestStatsMaxIterations"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::schedulerConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::specDecConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::useGpuDirectStorage"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig25getAdditionalModelOutputsEv", "tensorrt_llm::executor::ExecutorConfig::getAdditionalModelOutputs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig15getBatchingTypeEv", "tensorrt_llm::executor::ExecutorConfig::getBatchingType"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig25getCacheTransceiverConfigEv", "tensorrt_llm::executor::ExecutorConfig::getCacheTransceiverConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig14getDebugConfigEv", "tensorrt_llm::executor::ExecutorConfig::getDebugConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig17getDecodingConfigEv", "tensorrt_llm::executor::ExecutorConfig::getDecodingConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig23getEnableChunkedContextEv", "tensorrt_llm::executor::ExecutorConfig::getEnableChunkedContext"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig19getEnableTrtOverlapEv", "tensorrt_llm::executor::ExecutorConfig::getEnableTrtOverlap"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig32getExtendedRuntimePerfKnobConfigEv", "tensorrt_llm::executor::ExecutorConfig::getExtendedRuntimePerfKnobConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig36getFailFastOnAttentionWindowTooLargeEv", "tensorrt_llm::executor::ExecutorConfig::getFailFastOnAttentionWindowTooLarge"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig25getGatherGenerationLogitsEv", "tensorrt_llm::executor::ExecutorConfig::getGatherGenerationLogits"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig20getGpuWeightsPercentEv", "tensorrt_llm::executor::ExecutorConfig::getGpuWeightsPercent"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig23getGuidedDecodingConfigEv", "tensorrt_llm::executor::ExecutorConfig::getGuidedDecodingConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig25getIterStatsMaxIterationsEv", "tensorrt_llm::executor::ExecutorConfig::getIterStatsMaxIterations"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig16getKvCacheConfigEv", "tensorrt_llm::executor::ExecutorConfig::getKvCacheConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig19getKvCacheConfigRefEv", "tensorrt_llm::executor::ExecutorConfig::getKvCacheConfigRef"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig28getLogitsPostProcessorConfigEv", "tensorrt_llm::executor::ExecutorConfig::getLogitsPostProcessorConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig15getMaxBatchSizeEv", "tensorrt_llm::executor::ExecutorConfig::getMaxBatchSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig15getMaxBeamWidthEv", "tensorrt_llm::executor::ExecutorConfig::getMaxBeamWidth"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig15getMaxNumTokensEv", "tensorrt_llm::executor::ExecutorConfig::getMaxNumTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig15getMaxQueueSizeEv", "tensorrt_llm::executor::ExecutorConfig::getMaxQueueSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig25getMaxSeqIdleMicrosecondsEv", "tensorrt_llm::executor::ExecutorConfig::getMaxSeqIdleMicroseconds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig20getNormalizeLogProbsEv", "tensorrt_llm::executor::ExecutorConfig::getNormalizeLogProbs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig17getParallelConfigEv", "tensorrt_llm::executor::ExecutorConfig::getParallelConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig18getPeftCacheConfigEv", "tensorrt_llm::executor::ExecutorConfig::getPeftCacheConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig24getPromptTableOffloadingEv", "tensorrt_llm::executor::ExecutorConfig::getPromptTableOffloading"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig19getRecvPollPeriodMsEv", "tensorrt_llm::executor::ExecutorConfig::getRecvPollPeriodMs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig28getRequestStatsMaxIterationsEv", "tensorrt_llm::executor::ExecutorConfig::getRequestStatsMaxIterations"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig18getSchedulerConfigEv", "tensorrt_llm::executor::ExecutorConfig::getSchedulerConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig21getSchedulerConfigRefEv", "tensorrt_llm::executor::ExecutorConfig::getSchedulerConfigRef"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig16getSpecDecConfigEv", "tensorrt_llm::executor::ExecutorConfig::getSpecDecConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig22getUseGpuDirectStorageEv", "tensorrt_llm::executor::ExecutorConfig::getUseGpuDirectStorage"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig30kDefaultIterStatsMaxIterationsE", "tensorrt_llm::executor::ExecutorConfig::kDefaultIterStatsMaxIterations"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig30kDefaultMaxSeqIdleMicrosecondsE", "tensorrt_llm::executor::ExecutorConfig::kDefaultMaxSeqIdleMicroseconds"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig33kDefaultRequestStatsMaxIterationsE", "tensorrt_llm::executor::ExecutorConfig::kDefaultRequestStatsMaxIterations"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig23mAdditionalModelOutputsE", "tensorrt_llm::executor::ExecutorConfig::mAdditionalModelOutputs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig13mBatchingTypeE", "tensorrt_llm::executor::ExecutorConfig::mBatchingType"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig23mCacheTransceiverConfigE", "tensorrt_llm::executor::ExecutorConfig::mCacheTransceiverConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig12mDebugConfigE", "tensorrt_llm::executor::ExecutorConfig::mDebugConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15mDecodingConfigE", "tensorrt_llm::executor::ExecutorConfig::mDecodingConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig21mEnableChunkedContextE", "tensorrt_llm::executor::ExecutorConfig::mEnableChunkedContext"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig17mEnableTrtOverlapE", "tensorrt_llm::executor::ExecutorConfig::mEnableTrtOverlap"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig30mExtendedRuntimePerfKnobConfigE", "tensorrt_llm::executor::ExecutorConfig::mExtendedRuntimePerfKnobConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig34mFailFastOnAttentionWindowTooLargeE", "tensorrt_llm::executor::ExecutorConfig::mFailFastOnAttentionWindowTooLarge"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig23mGatherGenerationLogitsE", "tensorrt_llm::executor::ExecutorConfig::mGatherGenerationLogits"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig18mGpuWeightsPercentE", "tensorrt_llm::executor::ExecutorConfig::mGpuWeightsPercent"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig21mGuidedDecodingConfigE", "tensorrt_llm::executor::ExecutorConfig::mGuidedDecodingConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig23mIterStatsMaxIterationsE", "tensorrt_llm::executor::ExecutorConfig::mIterStatsMaxIterations"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14mKvCacheConfigE", "tensorrt_llm::executor::ExecutorConfig::mKvCacheConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig26mLogitsPostProcessorConfigE", "tensorrt_llm::executor::ExecutorConfig::mLogitsPostProcessorConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig13mMaxBatchSizeE", "tensorrt_llm::executor::ExecutorConfig::mMaxBatchSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig13mMaxBeamWidthE", "tensorrt_llm::executor::ExecutorConfig::mMaxBeamWidth"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig13mMaxNumTokensE", "tensorrt_llm::executor::ExecutorConfig::mMaxNumTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig13mMaxQueueSizeE", "tensorrt_llm::executor::ExecutorConfig::mMaxQueueSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig23mMaxSeqIdleMicrosecondsE", "tensorrt_llm::executor::ExecutorConfig::mMaxSeqIdleMicroseconds"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig18mNormalizeLogProbsE", "tensorrt_llm::executor::ExecutorConfig::mNormalizeLogProbs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15mParallelConfigE", "tensorrt_llm::executor::ExecutorConfig::mParallelConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig16mPeftCacheConfigE", "tensorrt_llm::executor::ExecutorConfig::mPeftCacheConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig22mPromptTableOffloadingE", "tensorrt_llm::executor::ExecutorConfig::mPromptTableOffloading"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig17mRecvPollPeriodMsE", "tensorrt_llm::executor::ExecutorConfig::mRecvPollPeriodMs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig26mRequestStatsMaxIterationsE", "tensorrt_llm::executor::ExecutorConfig::mRequestStatsMaxIterations"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig16mSchedulerConfigE", "tensorrt_llm::executor::ExecutorConfig::mSchedulerConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig26mSpeculativeDecodingConfigE", "tensorrt_llm::executor::ExecutorConfig::mSpeculativeDecodingConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig20mUseGpuDirectStorageE", "tensorrt_llm::executor::ExecutorConfig::mUseGpuDirectStorage"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setAdditionalModelOutputsERKNSt6vectorI21AdditionalModelOutputEE", "tensorrt_llm::executor::ExecutorConfig::setAdditionalModelOutputs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setAdditionalModelOutputsERKNSt6vectorI21AdditionalModelOutputEE", "tensorrt_llm::executor::ExecutorConfig::setAdditionalModelOutputs::additionalModelOutputs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setBatchingTypeE12BatchingType", "tensorrt_llm::executor::ExecutorConfig::setBatchingType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setBatchingTypeE12BatchingType", "tensorrt_llm::executor::ExecutorConfig::setBatchingType::batchingType"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setCacheTransceiverConfigERK22CacheTransceiverConfig", "tensorrt_llm::executor::ExecutorConfig::setCacheTransceiverConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setCacheTransceiverConfigERK22CacheTransceiverConfig", "tensorrt_llm::executor::ExecutorConfig::setCacheTransceiverConfig::cacheTransceiverConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14setDebugConfigERK11DebugConfig", "tensorrt_llm::executor::ExecutorConfig::setDebugConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14setDebugConfigERK11DebugConfig", "tensorrt_llm::executor::ExecutorConfig::setDebugConfig::debugConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig17setDecodingConfigERK14DecodingConfig", "tensorrt_llm::executor::ExecutorConfig::setDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig17setDecodingConfigERK14DecodingConfig", "tensorrt_llm::executor::ExecutorConfig::setDecodingConfig::decodingConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig23setEnableChunkedContextEb", "tensorrt_llm::executor::ExecutorConfig::setEnableChunkedContext"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig23setEnableChunkedContextEb", "tensorrt_llm::executor::ExecutorConfig::setEnableChunkedContext::enableChunkedContext"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig19setEnableTrtOverlapEb", "tensorrt_llm::executor::ExecutorConfig::setEnableTrtOverlap"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig19setEnableTrtOverlapEb", "tensorrt_llm::executor::ExecutorConfig::setEnableTrtOverlap::enableTrtOverlap"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig32setExtendedRuntimePerfKnobConfigERK29ExtendedRuntimePerfKnobConfig", "tensorrt_llm::executor::ExecutorConfig::setExtendedRuntimePerfKnobConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig32setExtendedRuntimePerfKnobConfigERK29ExtendedRuntimePerfKnobConfig", "tensorrt_llm::executor::ExecutorConfig::setExtendedRuntimePerfKnobConfig::extendedRuntimePerfKnobConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig36setFailFastOnAttentionWindowTooLargeEb", "tensorrt_llm::executor::ExecutorConfig::setFailFastOnAttentionWindowTooLarge"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig36setFailFastOnAttentionWindowTooLargeEb", "tensorrt_llm::executor::ExecutorConfig::setFailFastOnAttentionWindowTooLarge::failFastOnAttentionWindowTooLarge"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setGatherGenerationLogitsEb", "tensorrt_llm::executor::ExecutorConfig::setGatherGenerationLogits"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setGatherGenerationLogitsEb", "tensorrt_llm::executor::ExecutorConfig::setGatherGenerationLogits::gatherGenerationLogits"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig20setGpuWeightsPercentERKf", "tensorrt_llm::executor::ExecutorConfig::setGpuWeightsPercent"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig20setGpuWeightsPercentERKf", "tensorrt_llm::executor::ExecutorConfig::setGpuWeightsPercent::gpuWeightsPercent"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig23setGuidedDecodingConfigERK20GuidedDecodingConfig", "tensorrt_llm::executor::ExecutorConfig::setGuidedDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig23setGuidedDecodingConfigERK20GuidedDecodingConfig", "tensorrt_llm::executor::ExecutorConfig::setGuidedDecodingConfig::guidedDecodingConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setIterStatsMaxIterationsE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setIterStatsMaxIterations"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setIterStatsMaxIterationsE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setIterStatsMaxIterations::iterStatsMaxIterations"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig16setKvCacheConfigERK13KvCacheConfig", "tensorrt_llm::executor::ExecutorConfig::setKvCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig16setKvCacheConfigERK13KvCacheConfig", "tensorrt_llm::executor::ExecutorConfig::setKvCacheConfig::kvCacheConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig28setLogitsPostProcessorConfigERK25LogitsPostProcessorConfig", "tensorrt_llm::executor::ExecutorConfig::setLogitsPostProcessorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig28setLogitsPostProcessorConfigERK25LogitsPostProcessorConfig", "tensorrt_llm::executor::ExecutorConfig::setLogitsPostProcessorConfig::logitsPostProcessorConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setMaxBatchSizeE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setMaxBatchSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setMaxBatchSizeE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setMaxBatchSize::maxBatchSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setMaxBeamWidthE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setMaxBeamWidth"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setMaxBeamWidthE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setMaxBeamWidth::maxBeamWidth"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setMaxNumTokensE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setMaxNumTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setMaxNumTokensE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setMaxNumTokens::maxNumTokens"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setMaxQueueSizeERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::ExecutorConfig::setMaxQueueSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setMaxQueueSizeERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::ExecutorConfig::setMaxQueueSize::maxQueueSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setMaxSeqIdleMicrosecondsE8uint64_t", "tensorrt_llm::executor::ExecutorConfig::setMaxSeqIdleMicroseconds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setMaxSeqIdleMicrosecondsE8uint64_t", "tensorrt_llm::executor::ExecutorConfig::setMaxSeqIdleMicroseconds::maxSeqIdleMicroseconds"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig20setNormalizeLogProbsEb", "tensorrt_llm::executor::ExecutorConfig::setNormalizeLogProbs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig20setNormalizeLogProbsEb", "tensorrt_llm::executor::ExecutorConfig::setNormalizeLogProbs::normalizeLogProbs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig17setParallelConfigERK14ParallelConfig", "tensorrt_llm::executor::ExecutorConfig::setParallelConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig17setParallelConfigERK14ParallelConfig", "tensorrt_llm::executor::ExecutorConfig::setParallelConfig::parallelConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig18setPeftCacheConfigERK15PeftCacheConfig", "tensorrt_llm::executor::ExecutorConfig::setPeftCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig18setPeftCacheConfigERK15PeftCacheConfig", "tensorrt_llm::executor::ExecutorConfig::setPeftCacheConfig::peftCacheConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig24setPromptTableOffloadingEb", "tensorrt_llm::executor::ExecutorConfig::setPromptTableOffloading"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig24setPromptTableOffloadingEb", "tensorrt_llm::executor::ExecutorConfig::setPromptTableOffloading::promptTableOffloading"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig19setRecvPollPeriodMsERK10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setRecvPollPeriodMs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig19setRecvPollPeriodMsERK10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setRecvPollPeriodMs::recvPollPeriodMs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig28setRequestStatsMaxIterationsE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setRequestStatsMaxIterations"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig28setRequestStatsMaxIterationsE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setRequestStatsMaxIterations::requestStatsMaxIterations"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig18setSchedulerConfigERK15SchedulerConfig", "tensorrt_llm::executor::ExecutorConfig::setSchedulerConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig18setSchedulerConfigERK15SchedulerConfig", "tensorrt_llm::executor::ExecutorConfig::setSchedulerConfig::schedulerConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig16setSpecDecConfigERK25SpeculativeDecodingConfig", "tensorrt_llm::executor::ExecutorConfig::setSpecDecConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig16setSpecDecConfigERK25SpeculativeDecodingConfig", "tensorrt_llm::executor::ExecutorConfig::setSpecDecConfig::specDecConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig22setUseGpuDirectStorageERKb", "tensorrt_llm::executor::ExecutorConfig::setUseGpuDirectStorage"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig22setUseGpuDirectStorageERKb", "tensorrt_llm::executor::ExecutorConfig::setUseGpuDirectStorage::useGpuDirectStorage"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfigE", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig29ExtendedRuntimePerfKnobConfigEbbb10SizeType32", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::ExtendedRuntimePerfKnobConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig29ExtendedRuntimePerfKnobConfigEbbb10SizeType32", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::ExtendedRuntimePerfKnobConfig::cudaGraphCacheSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig29ExtendedRuntimePerfKnobConfigEbbb10SizeType32", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::ExtendedRuntimePerfKnobConfig::cudaGraphMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig29ExtendedRuntimePerfKnobConfigEbbb10SizeType32", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::ExtendedRuntimePerfKnobConfig::enableContextFMHAFP32Acc"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig29ExtendedRuntimePerfKnobConfigEbbb10SizeType32", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::ExtendedRuntimePerfKnobConfig::multiBlockMode"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig21getCudaGraphCacheSizeEv", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::getCudaGraphCacheSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig16getCudaGraphModeEv", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::getCudaGraphMode"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig27getEnableContextFMHAFP32AccEv", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::getEnableContextFMHAFP32Acc"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig17getMultiBlockModeEv", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::getMultiBlockMode"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig19mCudaGraphCacheSizeE", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::mCudaGraphCacheSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig14mCudaGraphModeE", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::mCudaGraphMode"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig25mEnableContextFMHAFP32AccE", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::mEnableContextFMHAFP32Acc"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig15mMultiBlockModeE", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::mMultiBlockMode"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfigeqERK29ExtendedRuntimePerfKnobConfig", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfigeqERK29ExtendedRuntimePerfKnobConfig", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::operator==::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig21setCudaGraphCacheSizeE10SizeType32", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::setCudaGraphCacheSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig21setCudaGraphCacheSizeE10SizeType32", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::setCudaGraphCacheSize::cacheSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig16setCudaGraphModeEb", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::setCudaGraphMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig16setCudaGraphModeEb", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::setCudaGraphMode::cudaGraphMode"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig27setEnableContextFMHAFP32AccEb", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::setEnableContextFMHAFP32Acc"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig27setEnableContextFMHAFP32AccEb", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::setEnableContextFMHAFP32Acc::enableContextFMHAFP32Acc"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig17setMultiBlockModeEb", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::setMultiBlockMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig17setMultiBlockModeEb", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::setMultiBlockMode::multiBlockMode"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfigE", "tensorrt_llm::executor::ExternalDraftTokensConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfig25ExternalDraftTokensConfigE9VecTokensNSt8optionalI6TensorEERKNSt8optionalI9FloatTypeEERKNSt8optionalIbEE", "tensorrt_llm::executor::ExternalDraftTokensConfig::ExternalDraftTokensConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfig25ExternalDraftTokensConfigE9VecTokensNSt8optionalI6TensorEERKNSt8optionalI9FloatTypeEERKNSt8optionalIbEE", "tensorrt_llm::executor::ExternalDraftTokensConfig::ExternalDraftTokensConfig::acceptanceThreshold"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfig25ExternalDraftTokensConfigE9VecTokensNSt8optionalI6TensorEERKNSt8optionalI9FloatTypeEERKNSt8optionalIbEE", "tensorrt_llm::executor::ExternalDraftTokensConfig::ExternalDraftTokensConfig::fastLogits"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfig25ExternalDraftTokensConfigE9VecTokensNSt8optionalI6TensorEERKNSt8optionalI9FloatTypeEERKNSt8optionalIbEE", "tensorrt_llm::executor::ExternalDraftTokensConfig::ExternalDraftTokensConfig::logits"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfig25ExternalDraftTokensConfigE9VecTokensNSt8optionalI6TensorEERKNSt8optionalI9FloatTypeEERKNSt8optionalIbEE", "tensorrt_llm::executor::ExternalDraftTokensConfig::ExternalDraftTokensConfig::tokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor25ExternalDraftTokensConfig22getAcceptanceThresholdEv", "tensorrt_llm::executor::ExternalDraftTokensConfig::getAcceptanceThreshold"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor25ExternalDraftTokensConfig13getFastLogitsEv", "tensorrt_llm::executor::ExternalDraftTokensConfig::getFastLogits"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor25ExternalDraftTokensConfig9getLogitsEv", "tensorrt_llm::executor::ExternalDraftTokensConfig::getLogits"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor25ExternalDraftTokensConfig9getTokensEv", "tensorrt_llm::executor::ExternalDraftTokensConfig::getTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfig20mAcceptanceThresholdE", "tensorrt_llm::executor::ExternalDraftTokensConfig::mAcceptanceThreshold"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfig11mFastLogitsE", "tensorrt_llm::executor::ExternalDraftTokensConfig::mFastLogits"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfig7mLogitsE", "tensorrt_llm::executor::ExternalDraftTokensConfig::mLogits"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfig7mTokensE", "tensorrt_llm::executor::ExternalDraftTokensConfig::mTokens"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor12FinishReasonE", "tensorrt_llm::executor::FinishReason"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12FinishReason10kCANCELLEDE", "tensorrt_llm::executor::FinishReason::kCANCELLED"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12FinishReason7kEND_IDE", "tensorrt_llm::executor::FinishReason::kEND_ID"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12FinishReason7kLENGTHE", "tensorrt_llm::executor::FinishReason::kLENGTH"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12FinishReason13kNOT_FINISHEDE", "tensorrt_llm::executor::FinishReason::kNOT_FINISHED"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12FinishReason11kSTOP_WORDSE", "tensorrt_llm::executor::FinishReason::kSTOP_WORDS"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12FinishReason10kTIMED_OUTE", "tensorrt_llm::executor::FinishReason::kTIMED_OUT"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor9FloatTypeE", "tensorrt_llm::executor::FloatType"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfigE", "tensorrt_llm::executor::GuidedDecodingConfig"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig21GuidedDecodingBackendE", "tensorrt_llm::executor::GuidedDecodingConfig::GuidedDecodingBackend"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig21GuidedDecodingBackend11kLLGUIDANCEE", "tensorrt_llm::executor::GuidedDecodingConfig::GuidedDecodingBackend::kLLGUIDANCE"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig21GuidedDecodingBackend9kXGRAMMARE", "tensorrt_llm::executor::GuidedDecodingConfig::GuidedDecodingBackend::kXGRAMMAR"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig20GuidedDecodingConfigE21GuidedDecodingBackendNSt8optionalINSt6vectorINSt6stringEEEEENSt8optionalINSt6stringEEENSt8optionalINSt6vectorI11TokenIdTypeEEEE", "tensorrt_llm::executor::GuidedDecodingConfig::GuidedDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig20GuidedDecodingConfigE21GuidedDecodingBackendNSt8optionalINSt6vectorINSt6stringEEEEENSt8optionalINSt6stringEEENSt8optionalINSt6vectorI11TokenIdTypeEEEE", "tensorrt_llm::executor::GuidedDecodingConfig::GuidedDecodingConfig::backend"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig20GuidedDecodingConfigE21GuidedDecodingBackendNSt8optionalINSt6vectorINSt6stringEEEEENSt8optionalINSt6stringEEENSt8optionalINSt6vectorI11TokenIdTypeEEEE", "tensorrt_llm::executor::GuidedDecodingConfig::GuidedDecodingConfig::encodedVocab"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig20GuidedDecodingConfigE21GuidedDecodingBackendNSt8optionalINSt6vectorINSt6stringEEEEENSt8optionalINSt6stringEEENSt8optionalINSt6vectorI11TokenIdTypeEEEE", "tensorrt_llm::executor::GuidedDecodingConfig::GuidedDecodingConfig::stopTokenIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig20GuidedDecodingConfigE21GuidedDecodingBackendNSt8optionalINSt6vectorINSt6stringEEEEENSt8optionalINSt6stringEEENSt8optionalINSt6vectorI11TokenIdTypeEEEE", "tensorrt_llm::executor::GuidedDecodingConfig::GuidedDecodingConfig::tokenizerStr"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingConfig10getBackendEv", "tensorrt_llm::executor::GuidedDecodingConfig::getBackend"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingConfig15getEncodedVocabEv", "tensorrt_llm::executor::GuidedDecodingConfig::getEncodedVocab"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingConfig15getStopTokenIdsEv", "tensorrt_llm::executor::GuidedDecodingConfig::getStopTokenIds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingConfig15getTokenizerStrEv", "tensorrt_llm::executor::GuidedDecodingConfig::getTokenizerStr"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig8mBackendE", "tensorrt_llm::executor::GuidedDecodingConfig::mBackend"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig13mEncodedVocabE", "tensorrt_llm::executor::GuidedDecodingConfig::mEncodedVocab"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig13mStopTokenIdsE", "tensorrt_llm::executor::GuidedDecodingConfig::mStopTokenIds"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig13mTokenizerStrE", "tensorrt_llm::executor::GuidedDecodingConfig::mTokenizerStr"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingConfigeqERK20GuidedDecodingConfig", "tensorrt_llm::executor::GuidedDecodingConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingConfigeqERK20GuidedDecodingConfig", "tensorrt_llm::executor::GuidedDecodingConfig::operator==::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig10setBackendERK21GuidedDecodingBackend", "tensorrt_llm::executor::GuidedDecodingConfig::setBackend"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig10setBackendERK21GuidedDecodingBackend", "tensorrt_llm::executor::GuidedDecodingConfig::setBackend::backend"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig15setEncodedVocabERKNSt6vectorINSt6stringEEE", "tensorrt_llm::executor::GuidedDecodingConfig::setEncodedVocab"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig15setEncodedVocabERKNSt6vectorINSt6stringEEE", "tensorrt_llm::executor::GuidedDecodingConfig::setEncodedVocab::encodedVocab"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig15setStopTokenIdsERKNSt6vectorI11TokenIdTypeEE", "tensorrt_llm::executor::GuidedDecodingConfig::setStopTokenIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig15setStopTokenIdsERKNSt6vectorI11TokenIdTypeEE", "tensorrt_llm::executor::GuidedDecodingConfig::setStopTokenIds::stopTokenIds"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig15setTokenizerStrERKNSt6stringE", "tensorrt_llm::executor::GuidedDecodingConfig::setTokenizerStr"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig15setTokenizerStrERKNSt6stringE", "tensorrt_llm::executor::GuidedDecodingConfig::setTokenizerStr::tokenizerStr"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingConfig8validateEv", "tensorrt_llm::executor::GuidedDecodingConfig::validate"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParamsE", "tensorrt_llm::executor::GuidedDecodingParams"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams9GuideTypeE", "tensorrt_llm::executor::GuidedDecodingParams::GuideType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams9GuideType13kEBNF_GRAMMARE", "tensorrt_llm::executor::GuidedDecodingParams::GuideType::kEBNF_GRAMMAR"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams9GuideType5kJSONE", "tensorrt_llm::executor::GuidedDecodingParams::GuideType::kJSON"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams9GuideType12kJSON_SCHEMAE", "tensorrt_llm::executor::GuidedDecodingParams::GuideType::kJSON_SCHEMA"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams9GuideType6kREGEXE", "tensorrt_llm::executor::GuidedDecodingParams::GuideType::kREGEX"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams9GuideType15kSTRUCTURAL_TAGE", "tensorrt_llm::executor::GuidedDecodingParams::GuideType::kSTRUCTURAL_TAG"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams20GuidedDecodingParamsE9GuideTypeNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::GuidedDecodingParams::GuidedDecodingParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams20GuidedDecodingParamsE9GuideTypeNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::GuidedDecodingParams::GuidedDecodingParams::guide"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams20GuidedDecodingParamsE9GuideTypeNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::GuidedDecodingParams::GuidedDecodingParams::guideType"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingParams8getGuideEv", "tensorrt_llm::executor::GuidedDecodingParams::getGuide"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingParams12getGuideTypeEv", "tensorrt_llm::executor::GuidedDecodingParams::getGuideType"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams6mGuideE", "tensorrt_llm::executor::GuidedDecodingParams::mGuide"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams10mGuideTypeE", "tensorrt_llm::executor::GuidedDecodingParams::mGuideType"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingParamseqERK20GuidedDecodingParams", "tensorrt_llm::executor::GuidedDecodingParams::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingParamseqERK20GuidedDecodingParams", "tensorrt_llm::executor::GuidedDecodingParams::operator==::other"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor6IdTypeE", "tensorrt_llm::executor::IdType"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor21InflightBatchingStatsE", "tensorrt_llm::executor::InflightBatchingStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor21InflightBatchingStats26avgNumDecodedTokensPerIterE", "tensorrt_llm::executor::InflightBatchingStats::avgNumDecodedTokensPerIter"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor21InflightBatchingStats12microBatchIdE", "tensorrt_llm::executor::InflightBatchingStats::microBatchId"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor21InflightBatchingStats18numContextRequestsE", "tensorrt_llm::executor::InflightBatchingStats::numContextRequests"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor21InflightBatchingStats12numCtxTokensE", "tensorrt_llm::executor::InflightBatchingStats::numCtxTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor21InflightBatchingStats14numGenRequestsE", "tensorrt_llm::executor::InflightBatchingStats::numGenRequests"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor21InflightBatchingStats17numPausedRequestsE", "tensorrt_llm::executor::InflightBatchingStats::numPausedRequests"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor21InflightBatchingStats20numScheduledRequestsE", "tensorrt_llm::executor::InflightBatchingStats::numScheduledRequests"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor14IterationStatsE", "tensorrt_llm::executor::IterationStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats11cpuMemUsageE", "tensorrt_llm::executor::IterationStats::cpuMemUsage"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats17crossKvCacheStatsE", "tensorrt_llm::executor::IterationStats::crossKvCacheStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats11gpuMemUsageE", "tensorrt_llm::executor::IterationStats::gpuMemUsage"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats21inflightBatchingStatsE", "tensorrt_llm::executor::IterationStats::inflightBatchingStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats4iterE", "tensorrt_llm::executor::IterationStats::iter"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats13iterLatencyMSE", "tensorrt_llm::executor::IterationStats::iterLatencyMS"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats12kvCacheStatsE", "tensorrt_llm::executor::IterationStats::kvCacheStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats19maxBatchSizeRuntimeE", "tensorrt_llm::executor::IterationStats::maxBatchSizeRuntime"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats18maxBatchSizeStaticE", "tensorrt_llm::executor::IterationStats::maxBatchSizeStatic"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats28maxBatchSizeTunerRecommendedE", "tensorrt_llm::executor::IterationStats::maxBatchSizeTunerRecommended"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats20maxNumActiveRequestsE", "tensorrt_llm::executor::IterationStats::maxNumActiveRequests"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats19maxNumTokensRuntimeE", "tensorrt_llm::executor::IterationStats::maxNumTokensRuntime"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats18maxNumTokensStaticE", "tensorrt_llm::executor::IterationStats::maxNumTokensStatic"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats28maxNumTokensTunerRecommendedE", "tensorrt_llm::executor::IterationStats::maxNumTokensTunerRecommended"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats31newActiveRequestsQueueLatencyMSE", "tensorrt_llm::executor::IterationStats::newActiveRequestsQueueLatencyMS"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats17numActiveRequestsE", "tensorrt_llm::executor::IterationStats::numActiveRequests"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats20numCompletedRequestsE", "tensorrt_llm::executor::IterationStats::numCompletedRequests"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats20numNewActiveRequestsE", "tensorrt_llm::executor::IterationStats::numNewActiveRequests"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats17numQueuedRequestsE", "tensorrt_llm::executor::IterationStats::numQueuedRequests"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats14pinnedMemUsageE", "tensorrt_llm::executor::IterationStats::pinnedMemUsage"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats17specDecodingStatsE", "tensorrt_llm::executor::IterationStats::specDecodingStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats19staticBatchingStatsE", "tensorrt_llm::executor::IterationStats::staticBatchingStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats9timestampE", "tensorrt_llm::executor::IterationStats::timestamp"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor13IterationTypeE", "tensorrt_llm::executor::IterationType"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor17JsonSerializationE", "tensorrt_llm::executor::JsonSerialization"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor17JsonSerialization9toJsonStrERK12RequestStats", "tensorrt_llm::executor::JsonSerialization::toJsonStr"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor17JsonSerialization9toJsonStrERK14IterationStats", "tensorrt_llm::executor::JsonSerialization::toJsonStr"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor17JsonSerialization9toJsonStrERK24RequestStatsPerIteration", "tensorrt_llm::executor::JsonSerialization::toJsonStr"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor17JsonSerialization9toJsonStrERK14IterationStats", "tensorrt_llm::executor::JsonSerialization::toJsonStr::iterationStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor17JsonSerialization9toJsonStrERK12RequestStats", "tensorrt_llm::executor::JsonSerialization::toJsonStr::requestStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor17JsonSerialization9toJsonStrERK24RequestStatsPerIteration", "tensorrt_llm::executor::JsonSerialization::toJsonStr::requestStatsPerIter"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheCreatedDataE", "tensorrt_llm::executor::KVCacheCreatedData"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheCreatedData22numBlocksPerCacheLevelE", "tensorrt_llm::executor::KVCacheCreatedData::numBlocksPerCacheLevel"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEventE", "tensorrt_llm::executor::KVCacheEvent"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEvent12KVCacheEventE6IdType16KVCacheEventData10SizeType32NSt8optionalI10SizeType32EE", "tensorrt_llm::executor::KVCacheEvent::KVCacheEvent"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEvent12KVCacheEventE6IdType16KVCacheEventData10SizeType32NSt8optionalI10SizeType32EE", "tensorrt_llm::executor::KVCacheEvent::KVCacheEvent::attentionDpRank"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEvent12KVCacheEventE6IdType16KVCacheEventData10SizeType32NSt8optionalI10SizeType32EE", "tensorrt_llm::executor::KVCacheEvent::KVCacheEvent::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEvent12KVCacheEventE6IdType16KVCacheEventData10SizeType32NSt8optionalI10SizeType32EE", "tensorrt_llm::executor::KVCacheEvent::KVCacheEvent::eventId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEvent12KVCacheEventE6IdType16KVCacheEventData10SizeType32NSt8optionalI10SizeType32EE", "tensorrt_llm::executor::KVCacheEvent::KVCacheEvent::windowSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEvent15attentionDpRankE", "tensorrt_llm::executor::KVCacheEvent::attentionDpRank"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEvent4dataE", "tensorrt_llm::executor::KVCacheEvent::data"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEvent7eventIdE", "tensorrt_llm::executor::KVCacheEvent::eventId"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEvent10windowSizeE", "tensorrt_llm::executor::KVCacheEvent::windowSize"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor16KVCacheEventDataE", "tensorrt_llm::executor::KVCacheEventData"], [0, 2, 1, "_CPPv4I0EN12tensorrt_llm8executor16KVCacheEventDiffE", "tensorrt_llm::executor::KVCacheEventDiff"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor16KVCacheEventDiffE", "tensorrt_llm::executor::KVCacheEventDiff::T"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor16KVCacheEventDiff8newValueE", "tensorrt_llm::executor::KVCacheEventDiff::newValue"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor16KVCacheEventDiff8oldValueE", "tensorrt_llm::executor::KVCacheEventDiff::oldValue"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor19KVCacheEventManagerE", "tensorrt_llm::executor::KVCacheEventManager"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor19KVCacheEventManager19KVCacheEventManagerENSt10shared_ptrIN12tensorrt_llm13batch_manager16kv_cache_manager18BaseKVCacheManagerEEE", "tensorrt_llm::executor::KVCacheEventManager::KVCacheEventManager"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor19KVCacheEventManager19KVCacheEventManagerENSt10shared_ptrIN12tensorrt_llm13batch_manager16kv_cache_manager18BaseKVCacheManagerEEE", "tensorrt_llm::executor::KVCacheEventManager::KVCacheEventManager::kvCacheManager"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor19KVCacheEventManager15getLatestEventsENSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::KVCacheEventManager::getLatestEvents"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor19KVCacheEventManager15getLatestEventsENSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::KVCacheEventManager::getLatestEvents::timeout"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor19KVCacheEventManager14kvCacheManagerE", "tensorrt_llm::executor::KVCacheEventManager::kvCacheManager"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheRemovedDataE", "tensorrt_llm::executor::KVCacheRemovedData"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheRemovedData11blockHashesE", "tensorrt_llm::executor::KVCacheRemovedData::blockHashes"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockDataE", "tensorrt_llm::executor::KVCacheStoredBlockData"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData22KVCacheStoredBlockDataE6IdTypeN12tensorrt_llm7runtime15VecUniqueTokensENSt8optionalIN12tensorrt_llm7runtime14LoraTaskIdTypeEEE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheStoredBlockData::KVCacheStoredBlockData"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData22KVCacheStoredBlockDataE6IdTypeN12tensorrt_llm7runtime15VecUniqueTokensENSt8optionalIN12tensorrt_llm7runtime14LoraTaskIdTypeEEE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheStoredBlockData::KVCacheStoredBlockData::blockHash"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData22KVCacheStoredBlockDataE6IdTypeN12tensorrt_llm7runtime15VecUniqueTokensENSt8optionalIN12tensorrt_llm7runtime14LoraTaskIdTypeEEE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheStoredBlockData::KVCacheStoredBlockData::cacheLevel"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData22KVCacheStoredBlockDataE6IdTypeN12tensorrt_llm7runtime15VecUniqueTokensENSt8optionalIN12tensorrt_llm7runtime14LoraTaskIdTypeEEE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheStoredBlockData::KVCacheStoredBlockData::loraId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData22KVCacheStoredBlockDataE6IdTypeN12tensorrt_llm7runtime15VecUniqueTokensENSt8optionalIN12tensorrt_llm7runtime14LoraTaskIdTypeEEE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheStoredBlockData::KVCacheStoredBlockData::priority"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData22KVCacheStoredBlockDataE6IdTypeN12tensorrt_llm7runtime15VecUniqueTokensENSt8optionalIN12tensorrt_llm7runtime14LoraTaskIdTypeEEE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheStoredBlockData::KVCacheStoredBlockData::tokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData9blockHashE", "tensorrt_llm::executor::KVCacheStoredBlockData::blockHash"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData10cacheLevelE", "tensorrt_llm::executor::KVCacheStoredBlockData::cacheLevel"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData6loraIdE", "tensorrt_llm::executor::KVCacheStoredBlockData::loraId"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData8priorityE", "tensorrt_llm::executor::KVCacheStoredBlockData::priority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData6tokensE", "tensorrt_llm::executor::KVCacheStoredBlockData::tokens"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor17KVCacheStoredDataE", "tensorrt_llm::executor::KVCacheStoredData"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor17KVCacheStoredData6blocksE", "tensorrt_llm::executor::KVCacheStoredData::blocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor17KVCacheStoredData10parentHashE", "tensorrt_llm::executor::KVCacheStoredData::parentHash"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedDataE", "tensorrt_llm::executor::KVCacheUpdatedData"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData18KVCacheUpdatedDataE6IdType", "tensorrt_llm::executor::KVCacheUpdatedData::KVCacheUpdatedData"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData18KVCacheUpdatedDataE6IdTypeNSt8optionalI16KVCacheEventDiffI10SizeType32EEENSt8optionalI16KVCacheEventDiffI10SizeType32EEE", "tensorrt_llm::executor::KVCacheUpdatedData::KVCacheUpdatedData"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData18KVCacheUpdatedDataE6IdType", "tensorrt_llm::executor::KVCacheUpdatedData::KVCacheUpdatedData::blockHash"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData18KVCacheUpdatedDataE6IdTypeNSt8optionalI16KVCacheEventDiffI10SizeType32EEENSt8optionalI16KVCacheEventDiffI10SizeType32EEE", "tensorrt_llm::executor::KVCacheUpdatedData::KVCacheUpdatedData::blockHash"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData18KVCacheUpdatedDataE6IdTypeNSt8optionalI16KVCacheEventDiffI10SizeType32EEENSt8optionalI16KVCacheEventDiffI10SizeType32EEE", "tensorrt_llm::executor::KVCacheUpdatedData::KVCacheUpdatedData::cacheLevel"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData18KVCacheUpdatedDataE6IdTypeNSt8optionalI16KVCacheEventDiffI10SizeType32EEENSt8optionalI16KVCacheEventDiffI10SizeType32EEE", "tensorrt_llm::executor::KVCacheUpdatedData::KVCacheUpdatedData::priority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData9blockHashE", "tensorrt_llm::executor::KVCacheUpdatedData::blockHash"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData10cacheLevelE", "tensorrt_llm::executor::KVCacheUpdatedData::cacheLevel"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData17cacheLevelUpdatedE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheUpdatedData::cacheLevelUpdated"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData17cacheLevelUpdatedE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheUpdatedData::cacheLevelUpdated::newValue"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData17cacheLevelUpdatedE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheUpdatedData::cacheLevelUpdated::oldValue"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData8priorityE", "tensorrt_llm::executor::KVCacheUpdatedData::priority"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData15priorityUpdatedE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheUpdatedData::priorityUpdated"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData15priorityUpdatedE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheUpdatedData::priorityUpdated::newValue"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData15priorityUpdatedE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheUpdatedData::priorityUpdated::oldValue"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfigE", "tensorrt_llm::executor::KvCacheConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::attentionDpEventsGatherPeriodMs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::copyOnPartialReuse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::crossKvCacheFraction"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::enableBlockReuse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::enablePartialReuse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::eventBufferMaxSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::freeGpuMemoryFraction"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::hostCacheSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::maxAttentionWindowVec"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::maxGpuTotalBytes"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::maxTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::onboardBlocks"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::runtimeDefaults"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::secondaryOffloadMinPriority"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::sinkTokenLength"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::useUvm"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig34fillEmptyFieldsFromRuntimeDefaultsERKN12tensorrt_llm7runtime15RuntimeDefaultsE", "tensorrt_llm::executor::KvCacheConfig::fillEmptyFieldsFromRuntimeDefaults"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig34fillEmptyFieldsFromRuntimeDefaultsERKN12tensorrt_llm7runtime15RuntimeDefaultsE", "tensorrt_llm::executor::KvCacheConfig::fillEmptyFieldsFromRuntimeDefaults::runtimeDefaults"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig34getAttentionDpEventsGatherPeriodMsEv", "tensorrt_llm::executor::KvCacheConfig::getAttentionDpEventsGatherPeriodMs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig21getCopyOnPartialReuseEv", "tensorrt_llm::executor::KvCacheConfig::getCopyOnPartialReuse"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig23getCrossKvCacheFractionEv", "tensorrt_llm::executor::KvCacheConfig::getCrossKvCacheFraction"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig19getEnableBlockReuseEv", "tensorrt_llm::executor::KvCacheConfig::getEnableBlockReuse"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig21getEnablePartialReuseEv", "tensorrt_llm::executor::KvCacheConfig::getEnablePartialReuse"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig21getEventBufferMaxSizeEv", "tensorrt_llm::executor::KvCacheConfig::getEventBufferMaxSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig24getFreeGpuMemoryFractionEv", "tensorrt_llm::executor::KvCacheConfig::getFreeGpuMemoryFraction"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig16getHostCacheSizeEv", "tensorrt_llm::executor::KvCacheConfig::getHostCacheSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig24getMaxAttentionWindowVecEv", "tensorrt_llm::executor::KvCacheConfig::getMaxAttentionWindowVec"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig19getMaxGpuTotalBytesEv", "tensorrt_llm::executor::KvCacheConfig::getMaxGpuTotalBytes"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig12getMaxTokensEv", "tensorrt_llm::executor::KvCacheConfig::getMaxTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig16getOnboardBlocksEv", "tensorrt_llm::executor::KvCacheConfig::getOnboardBlocks"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig30getSecondaryOffloadMinPriorityEv", "tensorrt_llm::executor::KvCacheConfig::getSecondaryOffloadMinPriority"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig18getSinkTokenLengthEv", "tensorrt_llm::executor::KvCacheConfig::getSinkTokenLength"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig9getUseUvmEv", "tensorrt_llm::executor::KvCacheConfig::getUseUvm"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig22kDefaultGpuMemFractionE", "tensorrt_llm::executor::KvCacheConfig::kDefaultGpuMemFraction"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig32mAttentionDpEventsGatherPeriodMsE", "tensorrt_llm::executor::KvCacheConfig::mAttentionDpEventsGatherPeriodMs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig19mCopyOnPartialReuseE", "tensorrt_llm::executor::KvCacheConfig::mCopyOnPartialReuse"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig21mCrossKvCacheFractionE", "tensorrt_llm::executor::KvCacheConfig::mCrossKvCacheFraction"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig17mEnableBlockReuseE", "tensorrt_llm::executor::KvCacheConfig::mEnableBlockReuse"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig19mEnablePartialReuseE", "tensorrt_llm::executor::KvCacheConfig::mEnablePartialReuse"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig19mEventBufferMaxSizeE", "tensorrt_llm::executor::KvCacheConfig::mEventBufferMaxSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig22mFreeGpuMemoryFractionE", "tensorrt_llm::executor::KvCacheConfig::mFreeGpuMemoryFraction"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig14mHostCacheSizeE", "tensorrt_llm::executor::KvCacheConfig::mHostCacheSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig22mMaxAttentionWindowVecE", "tensorrt_llm::executor::KvCacheConfig::mMaxAttentionWindowVec"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig17mMaxGpuTotalBytesE", "tensorrt_llm::executor::KvCacheConfig::mMaxGpuTotalBytes"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig10mMaxTokensE", "tensorrt_llm::executor::KvCacheConfig::mMaxTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig14mOnboardBlocksE", "tensorrt_llm::executor::KvCacheConfig::mOnboardBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig28mSecondaryOffloadMinPriorityE", "tensorrt_llm::executor::KvCacheConfig::mSecondaryOffloadMinPriority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig16mSinkTokenLengthE", "tensorrt_llm::executor::KvCacheConfig::mSinkTokenLength"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig7mUseUvmE", "tensorrt_llm::executor::KvCacheConfig::mUseUvm"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig34setAttentionDpEventsGatherPeriodMsE10SizeType32", "tensorrt_llm::executor::KvCacheConfig::setAttentionDpEventsGatherPeriodMs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig34setAttentionDpEventsGatherPeriodMsE10SizeType32", "tensorrt_llm::executor::KvCacheConfig::setAttentionDpEventsGatherPeriodMs::attentionDpEventsGatherPeriodMs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig21setCopyOnPartialReuseEb", "tensorrt_llm::executor::KvCacheConfig::setCopyOnPartialReuse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig21setCopyOnPartialReuseEb", "tensorrt_llm::executor::KvCacheConfig::setCopyOnPartialReuse::copyOnPartialReuse"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig23setCrossKvCacheFractionE9FloatType", "tensorrt_llm::executor::KvCacheConfig::setCrossKvCacheFraction"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig23setCrossKvCacheFractionE9FloatType", "tensorrt_llm::executor::KvCacheConfig::setCrossKvCacheFraction::crossKvCacheFraction"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig19setEnableBlockReuseEb", "tensorrt_llm::executor::KvCacheConfig::setEnableBlockReuse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig19setEnableBlockReuseEb", "tensorrt_llm::executor::KvCacheConfig::setEnableBlockReuse::enableBlockReuse"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig21setEnablePartialReuseEb", "tensorrt_llm::executor::KvCacheConfig::setEnablePartialReuse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig21setEnablePartialReuseEb", "tensorrt_llm::executor::KvCacheConfig::setEnablePartialReuse::enablePartialReuse"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig21setEventBufferMaxSizeE6size_t", "tensorrt_llm::executor::KvCacheConfig::setEventBufferMaxSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig21setEventBufferMaxSizeE6size_t", "tensorrt_llm::executor::KvCacheConfig::setEventBufferMaxSize::eventBufferMaxSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig24setFreeGpuMemoryFractionE9FloatType", "tensorrt_llm::executor::KvCacheConfig::setFreeGpuMemoryFraction"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig24setFreeGpuMemoryFractionE9FloatType", "tensorrt_llm::executor::KvCacheConfig::setFreeGpuMemoryFraction::freeGpuMemoryFraction"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig16setHostCacheSizeE6size_t", "tensorrt_llm::executor::KvCacheConfig::setHostCacheSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig16setHostCacheSizeE6size_t", "tensorrt_llm::executor::KvCacheConfig::setHostCacheSize::hostCacheSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig24setMaxAttentionWindowVecENSt6vectorI10SizeType32EE", "tensorrt_llm::executor::KvCacheConfig::setMaxAttentionWindowVec"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig24setMaxAttentionWindowVecENSt6vectorI10SizeType32EE", "tensorrt_llm::executor::KvCacheConfig::setMaxAttentionWindowVec::maxAttentionWindowVec"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig19setMaxGpuTotalBytesE8uint64_t", "tensorrt_llm::executor::KvCacheConfig::setMaxGpuTotalBytes"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig19setMaxGpuTotalBytesE8uint64_t", "tensorrt_llm::executor::KvCacheConfig::setMaxGpuTotalBytes::maxGpuTotalBytes"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig12setMaxTokensENSt8optionalI10SizeType32EE", "tensorrt_llm::executor::KvCacheConfig::setMaxTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig12setMaxTokensENSt8optionalI10SizeType32EE", "tensorrt_llm::executor::KvCacheConfig::setMaxTokens::maxTokens"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig16setOnboardBlocksEb", "tensorrt_llm::executor::KvCacheConfig::setOnboardBlocks"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig16setOnboardBlocksEb", "tensorrt_llm::executor::KvCacheConfig::setOnboardBlocks::onboardBlocks"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig30setSecondaryOffloadMinPriorityENSt8optionalI17RetentionPriorityEE", "tensorrt_llm::executor::KvCacheConfig::setSecondaryOffloadMinPriority"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig30setSecondaryOffloadMinPriorityENSt8optionalI17RetentionPriorityEE", "tensorrt_llm::executor::KvCacheConfig::setSecondaryOffloadMinPriority::secondaryOffloadMinPriority"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig18setSinkTokenLengthE10SizeType32", "tensorrt_llm::executor::KvCacheConfig::setSinkTokenLength"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig18setSinkTokenLengthE10SizeType32", "tensorrt_llm::executor::KvCacheConfig::setSinkTokenLength::sinkTokenLength"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig9setUseUvmEb", "tensorrt_llm::executor::KvCacheConfig::setUseUvm"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig9setUseUvmEb", "tensorrt_llm::executor::KvCacheConfig::setUseUvm::useUvm"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfigE", "tensorrt_llm::executor::KvCacheRetentionConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig22KvCacheRetentionConfigERKNSt6vectorI25TokenRangeRetentionConfigEE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE19KvCacheTransferModeRKNSt6stringE", "tensorrt_llm::executor::KvCacheRetentionConfig::KvCacheRetentionConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig22KvCacheRetentionConfigEv", "tensorrt_llm::executor::KvCacheRetentionConfig::KvCacheRetentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig22KvCacheRetentionConfigERKNSt6vectorI25TokenRangeRetentionConfigEE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE19KvCacheTransferModeRKNSt6stringE", "tensorrt_llm::executor::KvCacheRetentionConfig::KvCacheRetentionConfig::decodeDurationMs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig22KvCacheRetentionConfigERKNSt6vectorI25TokenRangeRetentionConfigEE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE19KvCacheTransferModeRKNSt6stringE", "tensorrt_llm::executor::KvCacheRetentionConfig::KvCacheRetentionConfig::decodeRetentionPriority"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig22KvCacheRetentionConfigERKNSt6vectorI25TokenRangeRetentionConfigEE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE19KvCacheTransferModeRKNSt6stringE", "tensorrt_llm::executor::KvCacheRetentionConfig::KvCacheRetentionConfig::directory"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig22KvCacheRetentionConfigERKNSt6vectorI25TokenRangeRetentionConfigEE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE19KvCacheTransferModeRKNSt6stringE", "tensorrt_llm::executor::KvCacheRetentionConfig::KvCacheRetentionConfig::tokenRangeRetentionPriorities"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig22KvCacheRetentionConfigERKNSt6vectorI25TokenRangeRetentionConfigEE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE19KvCacheTransferModeRKNSt6stringE", "tensorrt_llm::executor::KvCacheRetentionConfig::KvCacheRetentionConfig::transferMode"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfigE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfig25TokenRangeRetentionConfigE10SizeType32NSt8optionalI10SizeType32EE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::TokenRangeRetentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfig25TokenRangeRetentionConfigE10SizeType32NSt8optionalI10SizeType32EE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::TokenRangeRetentionConfig::durationMs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfig25TokenRangeRetentionConfigE10SizeType32NSt8optionalI10SizeType32EE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::TokenRangeRetentionConfig::priority"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfig25TokenRangeRetentionConfigE10SizeType32NSt8optionalI10SizeType32EE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::TokenRangeRetentionConfig::tokenEnd"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfig25TokenRangeRetentionConfigE10SizeType32NSt8optionalI10SizeType32EE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::TokenRangeRetentionConfig::tokenStart"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfig10durationMsE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::durationMs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfigeqERK25TokenRangeRetentionConfig", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfigeqERK25TokenRangeRetentionConfig", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::operator==::other"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfig8priorityE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::priority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfig8tokenEndE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::tokenEnd"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfig10tokenStartE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::tokenStart"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig19getDecodeDurationMsEv", "tensorrt_llm::executor::KvCacheRetentionConfig::getDecodeDurationMs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig26getDecodeRetentionPriorityEv", "tensorrt_llm::executor::KvCacheRetentionConfig::getDecodeRetentionPriority"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig12getDirectoryEv", "tensorrt_llm::executor::KvCacheRetentionConfig::getDirectory"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig36getPerBlockRetentionPriorityDurationE10SizeType3210SizeType32", "tensorrt_llm::executor::KvCacheRetentionConfig::getPerBlockRetentionPriorityDuration"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig36getPerBlockRetentionPriorityDurationE10SizeType3210SizeType32", "tensorrt_llm::executor::KvCacheRetentionConfig::getPerBlockRetentionPriorityDuration::blockSize"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig36getPerBlockRetentionPriorityDurationE10SizeType3210SizeType32", "tensorrt_llm::executor::KvCacheRetentionConfig::getPerBlockRetentionPriorityDuration::seqLen"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig29getTokenRangeRetentionConfigsEv", "tensorrt_llm::executor::KvCacheRetentionConfig::getTokenRangeRetentionConfigs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig15getTransferModeEv", "tensorrt_llm::executor::KvCacheRetentionConfig::getTransferMode"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25kDefaultRetentionPriorityE", "tensorrt_llm::executor::KvCacheRetentionConfig::kDefaultRetentionPriority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig21kMaxRetentionPriorityE", "tensorrt_llm::executor::KvCacheRetentionConfig::kMaxRetentionPriority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig21kMinRetentionPriorityE", "tensorrt_llm::executor::KvCacheRetentionConfig::kMinRetentionPriority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig17mDecodeDurationMsE", "tensorrt_llm::executor::KvCacheRetentionConfig::mDecodeDurationMs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig24mDecodeRetentionPriorityE", "tensorrt_llm::executor::KvCacheRetentionConfig::mDecodeRetentionPriority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig10mDirectoryE", "tensorrt_llm::executor::KvCacheRetentionConfig::mDirectory"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig27mTokenRangeRetentionConfigsE", "tensorrt_llm::executor::KvCacheRetentionConfig::mTokenRangeRetentionConfigs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig13mTransferModeE", "tensorrt_llm::executor::KvCacheRetentionConfig::mTransferMode"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfigeqERK22KvCacheRetentionConfig", "tensorrt_llm::executor::KvCacheRetentionConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfigeqERK22KvCacheRetentionConfig", "tensorrt_llm::executor::KvCacheRetentionConfig::operator==::other"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStatsE", "tensorrt_llm::executor::KvCacheStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStats14allocNewBlocksE", "tensorrt_llm::executor::KvCacheStats::allocNewBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStats16allocTotalBlocksE", "tensorrt_llm::executor::KvCacheStats::allocTotalBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStats12cacheHitRateE", "tensorrt_llm::executor::KvCacheStats::cacheHitRate"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStats13freeNumBlocksE", "tensorrt_llm::executor::KvCacheStats::freeNumBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStats12maxNumBlocksE", "tensorrt_llm::executor::KvCacheStats::maxNumBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStats12missedBlocksE", "tensorrt_llm::executor::KvCacheStats::missedBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStats12reusedBlocksE", "tensorrt_llm::executor::KvCacheStats::reusedBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStats14tokensPerBlockE", "tensorrt_llm::executor::KvCacheStats::tokensPerBlock"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStats13usedNumBlocksE", "tensorrt_llm::executor::KvCacheStats::usedNumBlocks"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor19KvCacheTransferModeE", "tensorrt_llm::executor::KvCacheTransferMode"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor19KvCacheTransferMode4DRAME", "tensorrt_llm::executor::KvCacheTransferMode::DRAM"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor19KvCacheTransferMode3GDSE", "tensorrt_llm::executor::KvCacheTransferMode::GDS"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor19KvCacheTransferMode20POSIX_DEBUG_FALLBACKE", "tensorrt_llm::executor::KvCacheTransferMode::POSIX_DEBUG_FALLBACK"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor19LogitsPostProcessorE", "tensorrt_llm::executor::LogitsPostProcessor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor26LogitsPostProcessorBatchedE", "tensorrt_llm::executor::LogitsPostProcessorBatched"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfigE", "tensorrt_llm::executor::LogitsPostProcessorConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig25LogitsPostProcessorConfigENSt8optionalI22LogitsPostProcessorMapEENSt8optionalI26LogitsPostProcessorBatchedEEb", "tensorrt_llm::executor::LogitsPostProcessorConfig::LogitsPostProcessorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig25LogitsPostProcessorConfigENSt8optionalI22LogitsPostProcessorMapEENSt8optionalI26LogitsPostProcessorBatchedEEb", "tensorrt_llm::executor::LogitsPostProcessorConfig::LogitsPostProcessorConfig::processorBatched"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig25LogitsPostProcessorConfigENSt8optionalI22LogitsPostProcessorMapEENSt8optionalI26LogitsPostProcessorBatchedEEb", "tensorrt_llm::executor::LogitsPostProcessorConfig::LogitsPostProcessorConfig::processorMap"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig25LogitsPostProcessorConfigENSt8optionalI22LogitsPostProcessorMapEENSt8optionalI26LogitsPostProcessorBatchedEEb", "tensorrt_llm::executor::LogitsPostProcessorConfig::LogitsPostProcessorConfig::replicate"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor25LogitsPostProcessorConfig19getProcessorBatchedEv", "tensorrt_llm::executor::LogitsPostProcessorConfig::getProcessorBatched"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor25LogitsPostProcessorConfig15getProcessorMapEv", "tensorrt_llm::executor::LogitsPostProcessorConfig::getProcessorMap"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor25LogitsPostProcessorConfig12getReplicateEv", "tensorrt_llm::executor::LogitsPostProcessorConfig::getReplicate"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig17mProcessorBatchedE", "tensorrt_llm::executor::LogitsPostProcessorConfig::mProcessorBatched"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig13mProcessorMapE", "tensorrt_llm::executor::LogitsPostProcessorConfig::mProcessorMap"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig10mReplicateE", "tensorrt_llm::executor::LogitsPostProcessorConfig::mReplicate"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig19setProcessorBatchedERK26LogitsPostProcessorBatched", "tensorrt_llm::executor::LogitsPostProcessorConfig::setProcessorBatched"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig19setProcessorBatchedERK26LogitsPostProcessorBatched", "tensorrt_llm::executor::LogitsPostProcessorConfig::setProcessorBatched::processorBatched"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig15setProcessorMapERK22LogitsPostProcessorMap", "tensorrt_llm::executor::LogitsPostProcessorConfig::setProcessorMap"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig15setProcessorMapERK22LogitsPostProcessorMap", "tensorrt_llm::executor::LogitsPostProcessorConfig::setProcessorMap::processorMap"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig12setReplicateEb", "tensorrt_llm::executor::LogitsPostProcessorConfig::setReplicate"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig12setReplicateEb", "tensorrt_llm::executor::LogitsPostProcessorConfig::setReplicate::replicate"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor22LogitsPostProcessorMapE", "tensorrt_llm::executor::LogitsPostProcessorMap"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfigE", "tensorrt_llm::executor::LookaheadDecodingConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig23LookaheadDecodingConfigE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::LookaheadDecodingConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig23LookaheadDecodingConfigEv", "tensorrt_llm::executor::LookaheadDecodingConfig::LookaheadDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig23LookaheadDecodingConfigE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::LookaheadDecodingConfig::ngramSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig23LookaheadDecodingConfigE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::LookaheadDecodingConfig::verificationSetSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig23LookaheadDecodingConfigE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::LookaheadDecodingConfig::windowSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor23LookaheadDecodingConfig28calculateSpeculativeResourceEv", "tensorrt_llm::executor::LookaheadDecodingConfig::calculateSpeculativeResource"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig33calculateSpeculativeResourceTupleE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::calculateSpeculativeResourceTuple"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig33calculateSpeculativeResourceTupleE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::calculateSpeculativeResourceTuple::ngramSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig33calculateSpeculativeResourceTupleE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::calculateSpeculativeResourceTuple::verificationSetSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig33calculateSpeculativeResourceTupleE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::calculateSpeculativeResourceTuple::windowSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor23LookaheadDecodingConfig3getEv", "tensorrt_llm::executor::LookaheadDecodingConfig::get"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor23LookaheadDecodingConfig12getNgramSizeEv", "tensorrt_llm::executor::LookaheadDecodingConfig::getNgramSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor23LookaheadDecodingConfig22getVerificationSetSizeEv", "tensorrt_llm::executor::LookaheadDecodingConfig::getVerificationSetSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor23LookaheadDecodingConfig13getWindowSizeEv", "tensorrt_llm::executor::LookaheadDecodingConfig::getWindowSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor23LookaheadDecodingConfig4isLEERK23LookaheadDecodingConfig", "tensorrt_llm::executor::LookaheadDecodingConfig::isLE"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor23LookaheadDecodingConfig4isLEERK23LookaheadDecodingConfig", "tensorrt_llm::executor::LookaheadDecodingConfig::isLE::that"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig7isLegalE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::isLegal"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig7isLegalE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::isLegal::ngramSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig7isLegalE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::isLegal::verificationSetSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig7isLegalE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::isLegal::windowSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig30kDefaultLookaheadDecodingNgramE", "tensorrt_llm::executor::LookaheadDecodingConfig::kDefaultLookaheadDecodingNgram"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig40kDefaultLookaheadDecodingVerificationSetE", "tensorrt_llm::executor::LookaheadDecodingConfig::kDefaultLookaheadDecodingVerificationSet"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig31kDefaultLookaheadDecodingWindowE", "tensorrt_llm::executor::LookaheadDecodingConfig::kDefaultLookaheadDecodingWindow"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig10mNgramSizeE", "tensorrt_llm::executor::LookaheadDecodingConfig::mNgramSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig20mVerificationSetSizeE", "tensorrt_llm::executor::LookaheadDecodingConfig::mVerificationSetSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig11mWindowSizeE", "tensorrt_llm::executor::LookaheadDecodingConfig::mWindowSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor23LookaheadDecodingConfigeqERK23LookaheadDecodingConfig", "tensorrt_llm::executor::LookaheadDecodingConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor23LookaheadDecodingConfigeqERK23LookaheadDecodingConfig", "tensorrt_llm::executor::LookaheadDecodingConfig::operator==::other"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor10LoraConfigE", "tensorrt_llm::executor::LoraConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor10LoraConfig10LoraConfigE6IdTypeNSt8optionalI6TensorEENSt8optionalI6TensorEE", "tensorrt_llm::executor::LoraConfig::LoraConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor10LoraConfig10LoraConfigE6IdTypeNSt8optionalI6TensorEENSt8optionalI6TensorEE", "tensorrt_llm::executor::LoraConfig::LoraConfig::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor10LoraConfig10LoraConfigE6IdTypeNSt8optionalI6TensorEENSt8optionalI6TensorEE", "tensorrt_llm::executor::LoraConfig::LoraConfig::taskId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor10LoraConfig10LoraConfigE6IdTypeNSt8optionalI6TensorEENSt8optionalI6TensorEE", "tensorrt_llm::executor::LoraConfig::LoraConfig::weights"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor10LoraConfig9getConfigEv", "tensorrt_llm::executor::LoraConfig::getConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor10LoraConfig9getTaskIdEv", "tensorrt_llm::executor::LoraConfig::getTaskId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor10LoraConfig10getWeightsEv", "tensorrt_llm::executor::LoraConfig::getWeights"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10LoraConfig7mConfigE", "tensorrt_llm::executor::LoraConfig::mConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10LoraConfig7mTaskIdE", "tensorrt_llm::executor::LoraConfig::mTaskId"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10LoraConfig8mWeightsE", "tensorrt_llm::executor::LoraConfig::mWeights"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor13MedusaChoicesE", "tensorrt_llm::executor::MedusaChoices"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor10MemoryTypeE", "tensorrt_llm::executor::MemoryType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor10MemoryType4kCPUE", "tensorrt_llm::executor::MemoryType::kCPU"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor10MemoryType11kCPU_PINNEDE", "tensorrt_llm::executor::MemoryType::kCPU_PINNED"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor10MemoryType15kCPU_PINNEDPOOLE", "tensorrt_llm::executor::MemoryType::kCPU_PINNEDPOOL"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor10MemoryType4kGPUE", "tensorrt_llm::executor::MemoryType::kGPU"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor10MemoryType8kUNKNOWNE", "tensorrt_llm::executor::MemoryType::kUNKNOWN"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor10MemoryType4kUVME", "tensorrt_llm::executor::MemoryType::kUVM"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor16MillisecondsTypeE", "tensorrt_llm::executor::MillisecondsType"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor9ModelTypeE", "tensorrt_llm::executor::ModelType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor9ModelType13kDECODER_ONLYE", "tensorrt_llm::executor::ModelType::kDECODER_ONLY"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor9ModelType16kENCODER_DECODERE", "tensorrt_llm::executor::ModelType::kENCODER_DECODER"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor9ModelType13kENCODER_ONLYE", "tensorrt_llm::executor::ModelType::kENCODER_ONLY"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor11MropeConfigE", "tensorrt_llm::executor::MropeConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor11MropeConfig11MropeConfigE6Tensor10SizeType32", "tensorrt_llm::executor::MropeConfig::MropeConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11MropeConfig11MropeConfigE6Tensor10SizeType32", "tensorrt_llm::executor::MropeConfig::MropeConfig::mropePositionDeltas"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11MropeConfig11MropeConfigE6Tensor10SizeType32", "tensorrt_llm::executor::MropeConfig::MropeConfig::mropeRoratySinCos"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11MropeConfig22getMRopePositionDeltasEv", "tensorrt_llm::executor::MropeConfig::getMRopePositionDeltas"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11MropeConfig20getMRopeRotaryCosSinEv", "tensorrt_llm::executor::MropeConfig::getMRopeRotaryCosSin"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11MropeConfig20mMRopePositionDeltasE", "tensorrt_llm::executor::MropeConfig::mMRopePositionDeltas"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11MropeConfig18mMRopeRotaryCosSinE", "tensorrt_llm::executor::MropeConfig::mMRopeRotaryCosSin"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor15MultimodalInputE", "tensorrt_llm::executor::MultimodalInput"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15MultimodalInput15MultimodalInputENSt6vectorINSt6vectorI10SizeType32EEEENSt6vectorI10SizeType32EENSt6vectorI10SizeType32EE", "tensorrt_llm::executor::MultimodalInput::MultimodalInput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15MultimodalInput15MultimodalInputENSt6vectorINSt6vectorI10SizeType32EEEENSt6vectorI10SizeType32EENSt6vectorI10SizeType32EE", "tensorrt_llm::executor::MultimodalInput::MultimodalInput::multimodalHashes"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15MultimodalInput15MultimodalInputENSt6vectorINSt6vectorI10SizeType32EEEENSt6vectorI10SizeType32EENSt6vectorI10SizeType32EE", "tensorrt_llm::executor::MultimodalInput::MultimodalInput::multimodalLengths"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15MultimodalInput15MultimodalInputENSt6vectorINSt6vectorI10SizeType32EEEENSt6vectorI10SizeType32EENSt6vectorI10SizeType32EE", "tensorrt_llm::executor::MultimodalInput::MultimodalInput::multimodalPositions"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15MultimodalInput19getMultimodalHashesEv", "tensorrt_llm::executor::MultimodalInput::getMultimodalHashes"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15MultimodalInput20getMultimodalLengthsEv", "tensorrt_llm::executor::MultimodalInput::getMultimodalLengths"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15MultimodalInput22getMultimodalPositionsEv", "tensorrt_llm::executor::MultimodalInput::getMultimodalPositions"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15MultimodalInput17mMultimodalHashesE", "tensorrt_llm::executor::MultimodalInput::mMultimodalHashes"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15MultimodalInput18mMultimodalLengthsE", "tensorrt_llm::executor::MultimodalInput::mMultimodalLengths"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15MultimodalInput20mMultimodalPositionsE", "tensorrt_llm::executor::MultimodalInput::mMultimodalPositions"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfigE", "tensorrt_llm::executor::OrchestratorConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig18OrchestratorConfigEbNSt6stringENSt10shared_ptrIN3mpi7MpiCommEEEb", "tensorrt_llm::executor::OrchestratorConfig::OrchestratorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig18OrchestratorConfigEbNSt6stringENSt10shared_ptrIN3mpi7MpiCommEEEb", "tensorrt_llm::executor::OrchestratorConfig::OrchestratorConfig::isOrchestrator"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig18OrchestratorConfigEbNSt6stringENSt10shared_ptrIN3mpi7MpiCommEEEb", "tensorrt_llm::executor::OrchestratorConfig::OrchestratorConfig::orchLeaderComm"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig18OrchestratorConfigEbNSt6stringENSt10shared_ptrIN3mpi7MpiCommEEEb", "tensorrt_llm::executor::OrchestratorConfig::OrchestratorConfig::spawnProcesses"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig18OrchestratorConfigEbNSt6stringENSt10shared_ptrIN3mpi7MpiCommEEEb", "tensorrt_llm::executor::OrchestratorConfig::OrchestratorConfig::workerExecutablePath"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18OrchestratorConfig17getIsOrchestratorEv", "tensorrt_llm::executor::OrchestratorConfig::getIsOrchestrator"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18OrchestratorConfig17getOrchLeaderCommEv", "tensorrt_llm::executor::OrchestratorConfig::getOrchLeaderComm"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18OrchestratorConfig17getSpawnProcessesEv", "tensorrt_llm::executor::OrchestratorConfig::getSpawnProcesses"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18OrchestratorConfig23getWorkerExecutablePathEv", "tensorrt_llm::executor::OrchestratorConfig::getWorkerExecutablePath"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig15mIsOrchestratorE", "tensorrt_llm::executor::OrchestratorConfig::mIsOrchestrator"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig15mOrchLeaderCommE", "tensorrt_llm::executor::OrchestratorConfig::mOrchLeaderComm"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig15mSpawnProcessesE", "tensorrt_llm::executor::OrchestratorConfig::mSpawnProcesses"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig21mWorkerExecutablePathE", "tensorrt_llm::executor::OrchestratorConfig::mWorkerExecutablePath"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig17setIsOrchestratorEb", "tensorrt_llm::executor::OrchestratorConfig::setIsOrchestrator"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig17setIsOrchestratorEb", "tensorrt_llm::executor::OrchestratorConfig::setIsOrchestrator::isOrchestrator"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig17setOrchLeaderCommERKNSt10shared_ptrIN3mpi7MpiCommEEE", "tensorrt_llm::executor::OrchestratorConfig::setOrchLeaderComm"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig17setOrchLeaderCommERKNSt10shared_ptrIN3mpi7MpiCommEEE", "tensorrt_llm::executor::OrchestratorConfig::setOrchLeaderComm::orchLeaderComm"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig17setSpawnProcessesEb", "tensorrt_llm::executor::OrchestratorConfig::setSpawnProcesses"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig17setSpawnProcessesEb", "tensorrt_llm::executor::OrchestratorConfig::setSpawnProcesses::spawnProcesses"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig23setWorkerExecutablePathERKNSt6stringE", "tensorrt_llm::executor::OrchestratorConfig::setWorkerExecutablePath"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig23setWorkerExecutablePathERKNSt6stringE", "tensorrt_llm::executor::OrchestratorConfig::setWorkerExecutablePath::workerExecutablePath"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfigE", "tensorrt_llm::executor::OutputConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig12OutputConfigEbbbbbbNSt8optionalINSt6vectorI21AdditionalModelOutputEEEE", "tensorrt_llm::executor::OutputConfig::OutputConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig12OutputConfigEbbbbbbNSt8optionalINSt6vectorI21AdditionalModelOutputEEEE", "tensorrt_llm::executor::OutputConfig::OutputConfig::additionalModelOutputs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig12OutputConfigEbbbbbbNSt8optionalINSt6vectorI21AdditionalModelOutputEEEE", "tensorrt_llm::executor::OutputConfig::OutputConfig::excludeInputFromOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig12OutputConfigEbbbbbbNSt8optionalINSt6vectorI21AdditionalModelOutputEEEE", "tensorrt_llm::executor::OutputConfig::OutputConfig::returnContextLogits"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig12OutputConfigEbbbbbbNSt8optionalINSt6vectorI21AdditionalModelOutputEEEE", "tensorrt_llm::executor::OutputConfig::OutputConfig::returnEncoderOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig12OutputConfigEbbbbbbNSt8optionalINSt6vectorI21AdditionalModelOutputEEEE", "tensorrt_llm::executor::OutputConfig::OutputConfig::returnGenerationLogits"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig12OutputConfigEbbbbbbNSt8optionalINSt6vectorI21AdditionalModelOutputEEEE", "tensorrt_llm::executor::OutputConfig::OutputConfig::returnLogProbs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig12OutputConfigEbbbbbbNSt8optionalINSt6vectorI21AdditionalModelOutputEEEE", "tensorrt_llm::executor::OutputConfig::OutputConfig::returnPerfMetrics"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig22additionalModelOutputsE", "tensorrt_llm::executor::OutputConfig::additionalModelOutputs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig22excludeInputFromOutputE", "tensorrt_llm::executor::OutputConfig::excludeInputFromOutput"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig19returnContextLogitsE", "tensorrt_llm::executor::OutputConfig::returnContextLogits"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig19returnEncoderOutputE", "tensorrt_llm::executor::OutputConfig::returnEncoderOutput"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig22returnGenerationLogitsE", "tensorrt_llm::executor::OutputConfig::returnGenerationLogits"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig14returnLogProbsE", "tensorrt_llm::executor::OutputConfig::returnLogProbs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig17returnPerfMetricsE", "tensorrt_llm::executor::OutputConfig::returnPerfMetrics"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfigE", "tensorrt_llm::executor::ParallelConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig14ParallelConfigE17CommunicationType17CommunicationModeNSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI18OrchestratorConfigEENSt8optionalI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::ParallelConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig14ParallelConfigE17CommunicationType17CommunicationModeNSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI18OrchestratorConfigEENSt8optionalI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::ParallelConfig::commMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig14ParallelConfigE17CommunicationType17CommunicationModeNSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI18OrchestratorConfigEENSt8optionalI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::ParallelConfig::commType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig14ParallelConfigE17CommunicationType17CommunicationModeNSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI18OrchestratorConfigEENSt8optionalI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::ParallelConfig::deviceIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig14ParallelConfigE17CommunicationType17CommunicationModeNSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI18OrchestratorConfigEENSt8optionalI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::ParallelConfig::numNodes"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig14ParallelConfigE17CommunicationType17CommunicationModeNSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI18OrchestratorConfigEENSt8optionalI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::ParallelConfig::orchestratorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig14ParallelConfigE17CommunicationType17CommunicationModeNSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI18OrchestratorConfigEENSt8optionalI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::ParallelConfig::participantIds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ParallelConfig20getCommunicationModeEv", "tensorrt_llm::executor::ParallelConfig::getCommunicationMode"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ParallelConfig20getCommunicationTypeEv", "tensorrt_llm::executor::ParallelConfig::getCommunicationType"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ParallelConfig12getDeviceIdsEv", "tensorrt_llm::executor::ParallelConfig::getDeviceIds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ParallelConfig11getNumNodesEv", "tensorrt_llm::executor::ParallelConfig::getNumNodes"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ParallelConfig21getOrchestratorConfigEv", "tensorrt_llm::executor::ParallelConfig::getOrchestratorConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ParallelConfig17getParticipantIdsEv", "tensorrt_llm::executor::ParallelConfig::getParticipantIds"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig9mCommModeE", "tensorrt_llm::executor::ParallelConfig::mCommMode"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig9mCommTypeE", "tensorrt_llm::executor::ParallelConfig::mCommType"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig10mDeviceIdsE", "tensorrt_llm::executor::ParallelConfig::mDeviceIds"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig9mNumNodesE", "tensorrt_llm::executor::ParallelConfig::mNumNodes"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig19mOrchestratorConfigE", "tensorrt_llm::executor::ParallelConfig::mOrchestratorConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig15mParticipantIdsE", "tensorrt_llm::executor::ParallelConfig::mParticipantIds"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig20setCommunicationModeE17CommunicationMode", "tensorrt_llm::executor::ParallelConfig::setCommunicationMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig20setCommunicationModeE17CommunicationMode", "tensorrt_llm::executor::ParallelConfig::setCommunicationMode::mode"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig20setCommunicationTypeE17CommunicationType", "tensorrt_llm::executor::ParallelConfig::setCommunicationType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig20setCommunicationTypeE17CommunicationType", "tensorrt_llm::executor::ParallelConfig::setCommunicationType::type"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig12setDeviceIdsERKNSt6vectorI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::setDeviceIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig12setDeviceIdsERKNSt6vectorI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::setDeviceIds::deviceIds"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig11setNumNodesE10SizeType32", "tensorrt_llm::executor::ParallelConfig::setNumNodes"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig11setNumNodesE10SizeType32", "tensorrt_llm::executor::ParallelConfig::setNumNodes::numNodes"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig21setOrchestratorConfigERK18OrchestratorConfig", "tensorrt_llm::executor::ParallelConfig::setOrchestratorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig21setOrchestratorConfigERK18OrchestratorConfig", "tensorrt_llm::executor::ParallelConfig::setOrchestratorConfig::orchestratorConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig17setParticipantIdsERKNSt6vectorI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::setParticipantIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig17setParticipantIdsERKNSt6vectorI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::setParticipantIds::participantIds"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfigE", "tensorrt_llm::executor::PeftCacheConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::deviceCachePercent"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::hostCacheSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::loraPrefetchDir"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::maxAdapterSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::maxPagesPerBlockDevice"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::maxPagesPerBlockHost"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::numCopyStreams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::numDeviceModuleLayer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::numEnsureWorkers"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::numHostModuleLayer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::numPutWorkers"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::optimalAdapterSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig21getDeviceCachePercentEv", "tensorrt_llm::executor::PeftCacheConfig::getDeviceCachePercent"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig16getHostCacheSizeEv", "tensorrt_llm::executor::PeftCacheConfig::getHostCacheSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig18getLoraPrefetchDirEv", "tensorrt_llm::executor::PeftCacheConfig::getLoraPrefetchDir"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig17getMaxAdapterSizeEv", "tensorrt_llm::executor::PeftCacheConfig::getMaxAdapterSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig25getMaxPagesPerBlockDeviceEv", "tensorrt_llm::executor::PeftCacheConfig::getMaxPagesPerBlockDevice"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig23getMaxPagesPerBlockHostEv", "tensorrt_llm::executor::PeftCacheConfig::getMaxPagesPerBlockHost"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig17getNumCopyStreamsEv", "tensorrt_llm::executor::PeftCacheConfig::getNumCopyStreams"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig23getNumDeviceModuleLayerEv", "tensorrt_llm::executor::PeftCacheConfig::getNumDeviceModuleLayer"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig19getNumEnsureWorkersEv", "tensorrt_llm::executor::PeftCacheConfig::getNumEnsureWorkers"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig21getNumHostModuleLayerEv", "tensorrt_llm::executor::PeftCacheConfig::getNumHostModuleLayer"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig16getNumPutWorkersEv", "tensorrt_llm::executor::PeftCacheConfig::getNumPutWorkers"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig21getOptimalAdapterSizeEv", "tensorrt_llm::executor::PeftCacheConfig::getOptimalAdapterSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig22kDefaultMaxAdapterSizeE", "tensorrt_llm::executor::PeftCacheConfig::kDefaultMaxAdapterSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig30kDefaultMaxPagesPerBlockDeviceE", "tensorrt_llm::executor::PeftCacheConfig::kDefaultMaxPagesPerBlockDevice"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig28kDefaultMaxPagesPerBlockHostE", "tensorrt_llm::executor::PeftCacheConfig::kDefaultMaxPagesPerBlockHost"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig26kDefaultOptimalAdapterSizeE", "tensorrt_llm::executor::PeftCacheConfig::kDefaultOptimalAdapterSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig19mDeviceCachePercentE", "tensorrt_llm::executor::PeftCacheConfig::mDeviceCachePercent"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig14mHostCacheSizeE", "tensorrt_llm::executor::PeftCacheConfig::mHostCacheSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig16mLoraPrefetchDirE", "tensorrt_llm::executor::PeftCacheConfig::mLoraPrefetchDir"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15mMaxAdapterSizeE", "tensorrt_llm::executor::PeftCacheConfig::mMaxAdapterSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig23mMaxPagesPerBlockDeviceE", "tensorrt_llm::executor::PeftCacheConfig::mMaxPagesPerBlockDevice"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig21mMaxPagesPerBlockHostE", "tensorrt_llm::executor::PeftCacheConfig::mMaxPagesPerBlockHost"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15mNumCopyStreamsE", "tensorrt_llm::executor::PeftCacheConfig::mNumCopyStreams"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig21mNumDeviceModuleLayerE", "tensorrt_llm::executor::PeftCacheConfig::mNumDeviceModuleLayer"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig17mNumEnsureWorkersE", "tensorrt_llm::executor::PeftCacheConfig::mNumEnsureWorkers"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig19mNumHostModuleLayerE", "tensorrt_llm::executor::PeftCacheConfig::mNumHostModuleLayer"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig14mNumPutWorkersE", "tensorrt_llm::executor::PeftCacheConfig::mNumPutWorkers"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig19mOptimalAdapterSizeE", "tensorrt_llm::executor::PeftCacheConfig::mOptimalAdapterSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfigeqERK15PeftCacheConfig", "tensorrt_llm::executor::PeftCacheConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfigeqERK15PeftCacheConfig", "tensorrt_llm::executor::PeftCacheConfig::operator==::other"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor12PriorityTypeE", "tensorrt_llm::executor::PriorityType"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18PromptTuningConfigE", "tensorrt_llm::executor::PromptTuningConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18PromptTuningConfig18PromptTuningConfigE6TensorNSt8optionalI16VecTokenExtraIdsEE", "tensorrt_llm::executor::PromptTuningConfig::PromptTuningConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18PromptTuningConfig18PromptTuningConfigE6TensorNSt8optionalI16VecTokenExtraIdsEE", "tensorrt_llm::executor::PromptTuningConfig::PromptTuningConfig::embeddingTable"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18PromptTuningConfig18PromptTuningConfigE6TensorNSt8optionalI16VecTokenExtraIdsEE", "tensorrt_llm::executor::PromptTuningConfig::PromptTuningConfig::inputTokenExtraIds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18PromptTuningConfig17getEmbeddingTableEv", "tensorrt_llm::executor::PromptTuningConfig::getEmbeddingTable"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18PromptTuningConfig21getInputTokenExtraIdsEv", "tensorrt_llm::executor::PromptTuningConfig::getInputTokenExtraIds"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18PromptTuningConfig15mEmbeddingTableE", "tensorrt_llm::executor::PromptTuningConfig::mEmbeddingTable"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18PromptTuningConfig19mInputTokenExtraIdsE", "tensorrt_llm::executor::PromptTuningConfig::mInputTokenExtraIds"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor14RandomSeedTypeE", "tensorrt_llm::executor::RandomSeedType"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor7RequestE", "tensorrt_llm::executor::Request"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestERK7Request", "tensorrt_llm::executor::Request::Request"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestERR7Request", "tensorrt_llm::executor::Request::Request"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::allottedTimeMs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::badWords"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::cacheSaltID"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::clientId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::contextPhaseParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::crossAttentionMask"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::eagleConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::embeddingBias"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::encoderInputFeatures"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::encoderInputTokenIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::encoderOutputLength"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::endId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::externalDraftTokensConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::guidedDecodingParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::inputTokenIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::kvCacheRetentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::languageAdapterUid"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::logitsPostProcessor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::logitsPostProcessorName"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::lookaheadConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::loraConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::mRopeConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::maxTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::multimodalEmbedding"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::multimodalInput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::numReturnSequences"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestERK7Request", "tensorrt_llm::executor::Request::Request::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestERR7Request", "tensorrt_llm::executor::Request::Request::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::outputConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::pTuningConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::padId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::positionIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::priority"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::returnAllGeneratedTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::samplingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::skipCrossAttnBlocks"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::stopWords"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::streaming"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::type"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request24getAdditionalOutputNamesEv", "tensorrt_llm::executor::Request::getAdditionalOutputNames"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request17getAllottedTimeMsEv", "tensorrt_llm::executor::Request::getAllottedTimeMs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request11getBadWordsEv", "tensorrt_llm::executor::Request::getBadWords"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request14getCacheSaltIDEv", "tensorrt_llm::executor::Request::getCacheSaltID"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request11getClientIdEv", "tensorrt_llm::executor::Request::getClientId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request21getContextPhaseParamsEv", "tensorrt_llm::executor::Request::getContextPhaseParams"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request21getCrossAttentionMaskEv", "tensorrt_llm::executor::Request::getCrossAttentionMask"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request14getEagleConfigEv", "tensorrt_llm::executor::Request::getEagleConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request16getEmbeddingBiasEv", "tensorrt_llm::executor::Request::getEmbeddingBias"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request23getEncoderInputFeaturesEv", "tensorrt_llm::executor::Request::getEncoderInputFeatures"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request23getEncoderInputTokenIdsEv", "tensorrt_llm::executor::Request::getEncoderInputTokenIds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request22getEncoderOutputLengthEv", "tensorrt_llm::executor::Request::getEncoderOutputLength"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request8getEndIdEv", "tensorrt_llm::executor::Request::getEndId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request28getExternalDraftTokensConfigEv", "tensorrt_llm::executor::Request::getExternalDraftTokensConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request23getGuidedDecodingParamsEv", "tensorrt_llm::executor::Request::getGuidedDecodingParams"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request16getInputTokenIdsEv", "tensorrt_llm::executor::Request::getInputTokenIds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request25getKvCacheRetentionConfigEv", "tensorrt_llm::executor::Request::getKvCacheRetentionConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request21getLanguageAdapterUidEv", "tensorrt_llm::executor::Request::getLanguageAdapterUid"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request22getLogitsPostProcessorEv", "tensorrt_llm::executor::Request::getLogitsPostProcessor"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request26getLogitsPostProcessorNameEv", "tensorrt_llm::executor::Request::getLogitsPostProcessorName"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request18getLookaheadConfigEv", "tensorrt_llm::executor::Request::getLookaheadConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request13getLoraConfigEv", "tensorrt_llm::executor::Request::getLoraConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request12getMaxTokensEv", "tensorrt_llm::executor::Request::getMaxTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request14getMropeConfigEv", "tensorrt_llm::executor::Request::getMropeConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request22getMultimodalEmbeddingEv", "tensorrt_llm::executor::Request::getMultimodalEmbedding"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request18getMultimodalInputEv", "tensorrt_llm::executor::Request::getMultimodalInput"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request15getOutputConfigEv", "tensorrt_llm::executor::Request::getOutputConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request8getPadIdEv", "tensorrt_llm::executor::Request::getPadId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request14getPositionIdsEv", "tensorrt_llm::executor::Request::getPositionIds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request11getPriorityEv", "tensorrt_llm::executor::Request::getPriority"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request21getPromptTuningConfigEv", "tensorrt_llm::executor::Request::getPromptTuningConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request14getRequestTypeEv", "tensorrt_llm::executor::Request::getRequestType"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request27getReturnAllGeneratedTokensEv", "tensorrt_llm::executor::Request::getReturnAllGeneratedTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request17getSamplingConfigEv", "tensorrt_llm::executor::Request::getSamplingConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request22getSkipCrossAttnBlocksEv", "tensorrt_llm::executor::Request::getSkipCrossAttnBlocks"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request12getStopWordsEv", "tensorrt_llm::executor::Request::getStopWords"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request12getStreamingEv", "tensorrt_llm::executor::Request::getStreaming"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor7Request25kBatchedPostProcessorNameE", "tensorrt_llm::executor::Request::kBatchedPostProcessorName"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor7Request16kDefaultPriorityE", "tensorrt_llm::executor::Request::kDefaultPriority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor7Request31kDynamicPostProcessorNamePrefixE", "tensorrt_llm::executor::Request::kDynamicPostProcessorNamePrefix"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor7Request5mImplE", "tensorrt_llm::executor::Request::mImpl"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7RequestaSERK7Request", "tensorrt_llm::executor::Request::operator="], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7RequestaSERR7Request", "tensorrt_llm::executor::Request::operator="], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7RequestaSERK7Request", "tensorrt_llm::executor::Request::operator=::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7RequestaSERR7Request", "tensorrt_llm::executor::Request::operator=::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request17setAllottedTimeMsE16MillisecondsType", "tensorrt_llm::executor::Request::setAllottedTimeMs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request17setAllottedTimeMsE16MillisecondsType", "tensorrt_llm::executor::Request::setAllottedTimeMs::allottedTimeMs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request11setBadWordsERKNSt4listI9VecTokensEE", "tensorrt_llm::executor::Request::setBadWords"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request11setBadWordsERKNSt4listI9VecTokensEE", "tensorrt_llm::executor::Request::setBadWords::badWords"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request14setCacheSaltIDE15CacheSaltIDType", "tensorrt_llm::executor::Request::setCacheSaltID"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request14setCacheSaltIDE15CacheSaltIDType", "tensorrt_llm::executor::Request::setCacheSaltID::cacheSaltID"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request11setClientIdE6IdType", "tensorrt_llm::executor::Request::setClientId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request11setClientIdE6IdType", "tensorrt_llm::executor::Request::setClientId::clientId"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request21setContextPhaseParamsE18ContextPhaseParams", "tensorrt_llm::executor::Request::setContextPhaseParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request21setContextPhaseParamsE18ContextPhaseParams", "tensorrt_llm::executor::Request::setContextPhaseParams::contextPhaseParams"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request21setCrossAttentionMaskE6Tensor", "tensorrt_llm::executor::Request::setCrossAttentionMask"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request21setCrossAttentionMaskE6Tensor", "tensorrt_llm::executor::Request::setCrossAttentionMask::crossAttentionMask"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request14setEagleConfigERKNSt8optionalI11EagleConfigEE", "tensorrt_llm::executor::Request::setEagleConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request14setEagleConfigERKNSt8optionalI11EagleConfigEE", "tensorrt_llm::executor::Request::setEagleConfig::eagleConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request16setEmbeddingBiasERK6Tensor", "tensorrt_llm::executor::Request::setEmbeddingBias"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request16setEmbeddingBiasERK6Tensor", "tensorrt_llm::executor::Request::setEmbeddingBias::embeddingBias"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request23setEncoderInputFeaturesE6Tensor", "tensorrt_llm::executor::Request::setEncoderInputFeatures"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request23setEncoderInputFeaturesE6Tensor", "tensorrt_llm::executor::Request::setEncoderInputFeatures::encoderInputFeatures"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request23setEncoderInputTokenIdsERK9VecTokens", "tensorrt_llm::executor::Request::setEncoderInputTokenIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request23setEncoderInputTokenIdsERK9VecTokens", "tensorrt_llm::executor::Request::setEncoderInputTokenIds::encoderInputTokenIds"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request22setEncoderOutputLengthE10SizeType32", "tensorrt_llm::executor::Request::setEncoderOutputLength"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request22setEncoderOutputLengthE10SizeType32", "tensorrt_llm::executor::Request::setEncoderOutputLength::encoderOutputLength"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request8setEndIdE10SizeType32", "tensorrt_llm::executor::Request::setEndId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request8setEndIdE10SizeType32", "tensorrt_llm::executor::Request::setEndId::endId"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request28setExternalDraftTokensConfigERK25ExternalDraftTokensConfig", "tensorrt_llm::executor::Request::setExternalDraftTokensConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request28setExternalDraftTokensConfigERK25ExternalDraftTokensConfig", "tensorrt_llm::executor::Request::setExternalDraftTokensConfig::externalDraftTokensConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request23setGuidedDecodingParamsERK20GuidedDecodingParams", "tensorrt_llm::executor::Request::setGuidedDecodingParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request23setGuidedDecodingParamsERK20GuidedDecodingParams", "tensorrt_llm::executor::Request::setGuidedDecodingParams::guidedDecodingParams"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request25setKvCacheRetentionConfigERK22KvCacheRetentionConfig", "tensorrt_llm::executor::Request::setKvCacheRetentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request25setKvCacheRetentionConfigERK22KvCacheRetentionConfig", "tensorrt_llm::executor::Request::setKvCacheRetentionConfig::kvCacheRetentionConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request21setLanguageAdapterUidE10SizeType32", "tensorrt_llm::executor::Request::setLanguageAdapterUid"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request21setLanguageAdapterUidE10SizeType32", "tensorrt_llm::executor::Request::setLanguageAdapterUid::languageAdapterUid"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request22setLogitsPostProcessorERKNSt8optionalI19LogitsPostProcessorEE", "tensorrt_llm::executor::Request::setLogitsPostProcessor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request22setLogitsPostProcessorERKNSt8optionalI19LogitsPostProcessorEE", "tensorrt_llm::executor::Request::setLogitsPostProcessor::logitsPostProcessor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request26setLogitsPostProcessorNameERKNSt6stringE", "tensorrt_llm::executor::Request::setLogitsPostProcessorName"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request26setLogitsPostProcessorNameERKNSt6stringE", "tensorrt_llm::executor::Request::setLogitsPostProcessorName::logitsPostProcessorName"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request18setLookaheadConfigERK23LookaheadDecodingConfig", "tensorrt_llm::executor::Request::setLookaheadConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request18setLookaheadConfigERK23LookaheadDecodingConfig", "tensorrt_llm::executor::Request::setLookaheadConfig::lookaheadConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request13setLoraConfigERK10LoraConfig", "tensorrt_llm::executor::Request::setLoraConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request13setLoraConfigERK10LoraConfig", "tensorrt_llm::executor::Request::setLoraConfig::loraConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request14setMropeConfigERK11MropeConfig", "tensorrt_llm::executor::Request::setMropeConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request14setMropeConfigERK11MropeConfig", "tensorrt_llm::executor::Request::setMropeConfig::mRopeConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request22setMultimodalEmbeddingERK6Tensor", "tensorrt_llm::executor::Request::setMultimodalEmbedding"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request22setMultimodalEmbeddingERK6Tensor", "tensorrt_llm::executor::Request::setMultimodalEmbedding::multimodalEmbedding"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request18setMultimodalInputERK15MultimodalInput", "tensorrt_llm::executor::Request::setMultimodalInput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request18setMultimodalInputERK15MultimodalInput", "tensorrt_llm::executor::Request::setMultimodalInput::multimodalInput"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request15setOutputConfigERK12OutputConfig", "tensorrt_llm::executor::Request::setOutputConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request15setOutputConfigERK12OutputConfig", "tensorrt_llm::executor::Request::setOutputConfig::outputConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request8setPadIdE10SizeType32", "tensorrt_llm::executor::Request::setPadId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request8setPadIdE10SizeType32", "tensorrt_llm::executor::Request::setPadId::padId"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request14setPositionIdsERKNSt6vectorI10SizeType32EE", "tensorrt_llm::executor::Request::setPositionIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request14setPositionIdsERKNSt6vectorI10SizeType32EE", "tensorrt_llm::executor::Request::setPositionIds::positionIds"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request11setPriorityE12PriorityType", "tensorrt_llm::executor::Request::setPriority"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request11setPriorityE12PriorityType", "tensorrt_llm::executor::Request::setPriority::priority"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request21setPromptTuningConfigERK18PromptTuningConfig", "tensorrt_llm::executor::Request::setPromptTuningConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request21setPromptTuningConfigERK18PromptTuningConfig", "tensorrt_llm::executor::Request::setPromptTuningConfig::pTuningConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request14setRequestTypeERK11RequestType", "tensorrt_llm::executor::Request::setRequestType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request14setRequestTypeERK11RequestType", "tensorrt_llm::executor::Request::setRequestType::requestType"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request27setReturnAllGeneratedTokensEb", "tensorrt_llm::executor::Request::setReturnAllGeneratedTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request27setReturnAllGeneratedTokensEb", "tensorrt_llm::executor::Request::setReturnAllGeneratedTokens::returnAllGeneratedTokens"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request17setSamplingConfigERK14SamplingConfig", "tensorrt_llm::executor::Request::setSamplingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request17setSamplingConfigERK14SamplingConfig", "tensorrt_llm::executor::Request::setSamplingConfig::config"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request22setSkipCrossAttnBlocksE6Tensor", "tensorrt_llm::executor::Request::setSkipCrossAttnBlocks"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request22setSkipCrossAttnBlocksE6Tensor", "tensorrt_llm::executor::Request::setSkipCrossAttnBlocks::skipCrossAttnBlocks"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request12setStopWordsERKNSt4listI9VecTokensEE", "tensorrt_llm::executor::Request::setStopWords"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request12setStopWordsERKNSt4listI9VecTokensEE", "tensorrt_llm::executor::Request::setStopWords::stopWords"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request12setStreamingEb", "tensorrt_llm::executor::Request::setStreaming"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request12setStreamingEb", "tensorrt_llm::executor::Request::setStreaming::streaming"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7RequestD0Ev", "tensorrt_llm::executor::Request::~Request"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetricsE", "tensorrt_llm::executor::RequestPerfMetrics"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics14KvCacheMetricsE", "tensorrt_llm::executor::RequestPerfMetrics::KvCacheMetrics"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics14KvCacheMetrics14kvCacheHitRateE", "tensorrt_llm::executor::RequestPerfMetrics::KvCacheMetrics::kvCacheHitRate"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics14KvCacheMetrics15numMissedBlocksE", "tensorrt_llm::executor::RequestPerfMetrics::KvCacheMetrics::numMissedBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics14KvCacheMetrics21numNewAllocatedBlocksE", "tensorrt_llm::executor::RequestPerfMetrics::KvCacheMetrics::numNewAllocatedBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics14KvCacheMetrics15numReusedBlocksE", "tensorrt_llm::executor::RequestPerfMetrics::KvCacheMetrics::numReusedBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics14KvCacheMetrics23numTotalAllocatedBlocksE", "tensorrt_llm::executor::RequestPerfMetrics::KvCacheMetrics::numTotalAllocatedBlocks"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics26SpeculativeDecodingMetricsE", "tensorrt_llm::executor::RequestPerfMetrics::SpeculativeDecodingMetrics"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics26SpeculativeDecodingMetrics14acceptanceRateE", "tensorrt_llm::executor::RequestPerfMetrics::SpeculativeDecodingMetrics::acceptanceRate"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics26SpeculativeDecodingMetrics24totalAcceptedDraftTokensE", "tensorrt_llm::executor::RequestPerfMetrics::SpeculativeDecodingMetrics::totalAcceptedDraftTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics26SpeculativeDecodingMetrics16totalDraftTokensE", "tensorrt_llm::executor::RequestPerfMetrics::SpeculativeDecodingMetrics::totalDraftTokens"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics9TimePointE", "tensorrt_llm::executor::RequestPerfMetrics::TimePoint"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics13TimingMetricsE", "tensorrt_llm::executor::RequestPerfMetrics::TimingMetrics"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics13TimingMetrics11arrivalTimeE", "tensorrt_llm::executor::RequestPerfMetrics::TimingMetrics::arrivalTime"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics13TimingMetrics18firstScheduledTimeE", "tensorrt_llm::executor::RequestPerfMetrics::TimingMetrics::firstScheduledTime"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics13TimingMetrics14firstTokenTimeE", "tensorrt_llm::executor::RequestPerfMetrics::TimingMetrics::firstTokenTime"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics13TimingMetrics11kvCacheSizeE", "tensorrt_llm::executor::RequestPerfMetrics::TimingMetrics::kvCacheSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics13TimingMetrics18kvCacheTransferEndE", "tensorrt_llm::executor::RequestPerfMetrics::TimingMetrics::kvCacheTransferEnd"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics13TimingMetrics20kvCacheTransferStartE", "tensorrt_llm::executor::RequestPerfMetrics::TimingMetrics::kvCacheTransferStart"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics13TimingMetrics13lastTokenTimeE", "tensorrt_llm::executor::RequestPerfMetrics::TimingMetrics::lastTokenTime"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics9firstIterE", "tensorrt_llm::executor::RequestPerfMetrics::firstIter"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics4iterE", "tensorrt_llm::executor::RequestPerfMetrics::iter"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics14kvCacheMetricsE", "tensorrt_llm::executor::RequestPerfMetrics::kvCacheMetrics"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics8lastIterE", "tensorrt_llm::executor::RequestPerfMetrics::lastIter"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics19speculativeDecodingE", "tensorrt_llm::executor::RequestPerfMetrics::speculativeDecoding"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics13timingMetricsE", "tensorrt_llm::executor::RequestPerfMetrics::timingMetrics"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor12RequestStageE", "tensorrt_llm::executor::RequestStage"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12RequestStage20kCONTEXT_IN_PROGRESSE", "tensorrt_llm::executor::RequestStage::kCONTEXT_IN_PROGRESS"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12RequestStage20kENCODER_IN_PROGRESSE", "tensorrt_llm::executor::RequestStage::kENCODER_IN_PROGRESS"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12RequestStage20kGENERATION_COMPLETEE", "tensorrt_llm::executor::RequestStage::kGENERATION_COMPLETE"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12RequestStage23kGENERATION_IN_PROGRESSE", "tensorrt_llm::executor::RequestStage::kGENERATION_IN_PROGRESS"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12RequestStage7kQUEUEDE", "tensorrt_llm::executor::RequestStage::kQUEUED"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor12RequestStatsE", "tensorrt_llm::executor::RequestStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats24allocNewBlocksPerRequestE", "tensorrt_llm::executor::RequestStats::allocNewBlocksPerRequest"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats26allocTotalBlocksPerRequestE", "tensorrt_llm::executor::RequestStats::allocTotalBlocksPerRequest"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats26avgNumDecodedTokensPerIterE", "tensorrt_llm::executor::RequestStats::avgNumDecodedTokensPerIter"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats22contextPrefillPositionE", "tensorrt_llm::executor::RequestStats::contextPrefillPosition"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats15disServingStatsE", "tensorrt_llm::executor::RequestStats::disServingStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats2idE", "tensorrt_llm::executor::RequestStats::id"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats24kvCacheHitRatePerRequestE", "tensorrt_llm::executor::RequestStats::kvCacheHitRatePerRequest"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats22missedBlocksPerRequestE", "tensorrt_llm::executor::RequestStats::missedBlocksPerRequest"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats18numGeneratedTokensE", "tensorrt_llm::executor::RequestStats::numGeneratedTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats6pausedE", "tensorrt_llm::executor::RequestStats::paused"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats22reusedBlocksPerRequestE", "tensorrt_llm::executor::RequestStats::reusedBlocksPerRequest"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats9scheduledE", "tensorrt_llm::executor::RequestStats::scheduled"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats5stageE", "tensorrt_llm::executor::RequestStats::stage"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor24RequestStatsPerIterationE", "tensorrt_llm::executor::RequestStatsPerIteration"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor24RequestStatsPerIteration4iterE", "tensorrt_llm::executor::RequestStatsPerIteration::iter"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor24RequestStatsPerIteration12requestStatsE", "tensorrt_llm::executor::RequestStatsPerIteration::requestStats"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor11RequestTypeE", "tensorrt_llm::executor::RequestType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor11RequestType35REQUEST_TYPE_CONTEXT_AND_GENERATIONE", "tensorrt_llm::executor::RequestType::REQUEST_TYPE_CONTEXT_AND_GENERATION"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor11RequestType25REQUEST_TYPE_CONTEXT_ONLYE", "tensorrt_llm::executor::RequestType::REQUEST_TYPE_CONTEXT_ONLY"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor11RequestType28REQUEST_TYPE_GENERATION_ONLYE", "tensorrt_llm::executor::RequestType::REQUEST_TYPE_GENERATION_ONLY"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8ResponseE", "tensorrt_llm::executor::Response"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseE6IdType6ResultNSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Response::Response"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseE6IdTypeNSt6stringENSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Response::Response"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseERK8Response", "tensorrt_llm::executor::Response::Response"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseERR8Response", "tensorrt_llm::executor::Response::Response"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseE6IdType6ResultNSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Response::Response::Result"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseE6IdType6ResultNSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Response::Response::clientId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseE6IdTypeNSt6stringENSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Response::Response::clientId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseE6IdTypeNSt6stringENSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Response::Response::errorMsg"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseERK8Response", "tensorrt_llm::executor::Response::Response::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseERR8Response", "tensorrt_llm::executor::Response::Response::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseE6IdType6ResultNSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Response::Response::requestId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseE6IdTypeNSt6stringENSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Response::Response::requestId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8Response11getClientIdEv", "tensorrt_llm::executor::Response::getClientId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8Response11getErrorMsgEv", "tensorrt_llm::executor::Response::getErrorMsg"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8Response12getRequestIdEv", "tensorrt_llm::executor::Response::getRequestId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8Response9getResultEv", "tensorrt_llm::executor::Response::getResult"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8Response8hasErrorEv", "tensorrt_llm::executor::Response::hasError"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8Response5mImplE", "tensorrt_llm::executor::Response::mImpl"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8ResponseaSERK8Response", "tensorrt_llm::executor::Response::operator="], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8ResponseaSERR8Response", "tensorrt_llm::executor::Response::operator="], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8ResponseaSERK8Response", "tensorrt_llm::executor::Response::operator=::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8ResponseaSERR8Response", "tensorrt_llm::executor::Response::operator=::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8ResponseD0Ev", "tensorrt_llm::executor::Response::~Response"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor6ResultE", "tensorrt_llm::executor::Result"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result17additionalOutputsE", "tensorrt_llm::executor::Result::additionalOutputs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result23avgDecodedTokensPerIterE", "tensorrt_llm::executor::Result::avgDecodedTokensPerIter"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result13contextLogitsE", "tensorrt_llm::executor::Result::contextLogits"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result18contextPhaseParamsE", "tensorrt_llm::executor::Result::contextPhaseParams"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result11cumLogProbsE", "tensorrt_llm::executor::Result::cumLogProbs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result12decodingIterE", "tensorrt_llm::executor::Result::decodingIter"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result13encoderOutputE", "tensorrt_llm::executor::Result::encoderOutput"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result13finishReasonsE", "tensorrt_llm::executor::Result::finishReasons"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result16generationLogitsE", "tensorrt_llm::executor::Result::generationLogits"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result7isFinalE", "tensorrt_llm::executor::Result::isFinal"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result15isSequenceFinalE", "tensorrt_llm::executor::Result::isSequenceFinal"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result8logProbsE", "tensorrt_llm::executor::Result::logProbs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result14outputTokenIdsE", "tensorrt_llm::executor::Result::outputTokenIds"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result18requestPerfMetricsE", "tensorrt_llm::executor::Result::requestPerfMetrics"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result13sequenceIndexE", "tensorrt_llm::executor::Result::sequenceIndex"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result21specDecFastLogitsInfoE", "tensorrt_llm::executor::Result::specDecFastLogitsInfo"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor17RetentionPriorityE", "tensorrt_llm::executor::RetentionPriority"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor28RetentionPriorityAndDurationE", "tensorrt_llm::executor::RetentionPriorityAndDuration"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor28RetentionPriorityAndDuration28RetentionPriorityAndDurationERKNSt8optionalI17RetentionPriorityEERKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::RetentionPriorityAndDuration::RetentionPriorityAndDuration"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor28RetentionPriorityAndDuration28RetentionPriorityAndDurationERKNSt8optionalI17RetentionPriorityEERKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::RetentionPriorityAndDuration::RetentionPriorityAndDuration::durationMs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor28RetentionPriorityAndDuration28RetentionPriorityAndDurationERKNSt8optionalI17RetentionPriorityEERKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::RetentionPriorityAndDuration::RetentionPriorityAndDuration::retentionPriority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor28RetentionPriorityAndDuration10durationMsE", "tensorrt_llm::executor::RetentionPriorityAndDuration::durationMs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor28RetentionPriorityAndDuration17retentionPriorityE", "tensorrt_llm::executor::RetentionPriorityAndDuration::retentionPriority"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfigE", "tensorrt_llm::executor::SamplingConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::beamSearchDiversityRate"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::beamWidth"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::beamWidthArray"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::earlyStopping"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::frequencyPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::lengthPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::minP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::minTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::noRepeatNgramSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::numReturnSequences"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::presencePenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::repetitionPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::seed"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::temperature"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::topK"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::topP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::topPDecay"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::topPMin"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::topPResetIds"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig28checkBeamSearchDiversityRateERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkBeamSearchDiversityRate"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig28checkBeamSearchDiversityRateERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkBeamSearchDiversityRate::beamSearchDiversityRate"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14checkBeamWidthE10SizeType32", "tensorrt_llm::executor::SamplingConfig::checkBeamWidth"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14checkBeamWidthE10SizeType32", "tensorrt_llm::executor::SamplingConfig::checkBeamWidth::beamWidth"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig19checkBeamWidthArrayERKNSt8optionalINSt6vectorI10SizeType32EEEEK10SizeType32", "tensorrt_llm::executor::SamplingConfig::checkBeamWidthArray"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig19checkBeamWidthArrayERKNSt8optionalINSt6vectorI10SizeType32EEEEK10SizeType32", "tensorrt_llm::executor::SamplingConfig::checkBeamWidthArray::beamWidth"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig19checkBeamWidthArrayERKNSt8optionalINSt6vectorI10SizeType32EEEEK10SizeType32", "tensorrt_llm::executor::SamplingConfig::checkBeamWidthArray::beamWidthArray"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig18checkEarlyStoppingERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::checkEarlyStopping"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig18checkEarlyStoppingERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::checkEarlyStopping::earlyStopping"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig18checkLengthPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkLengthPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig18checkLengthPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkLengthPenalty::lengthPenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig9checkMinPERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkMinP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig9checkMinPERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkMinP::minP"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14checkMinTokensERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::checkMinTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14checkMinTokensERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::checkMinTokens::minTokens"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig22checkNoRepeatNgramSizeERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::checkNoRepeatNgramSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig22checkNoRepeatNgramSizeERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::checkNoRepeatNgramSize::noRepeatNgramSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig23checkNumReturnSequencesERKNSt8optionalI10SizeType32EE10SizeType32", "tensorrt_llm::executor::SamplingConfig::checkNumReturnSequences"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig23checkNumReturnSequencesERKNSt8optionalI10SizeType32EE10SizeType32", "tensorrt_llm::executor::SamplingConfig::checkNumReturnSequences::beamWidth"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig23checkNumReturnSequencesERKNSt8optionalI10SizeType32EE10SizeType32", "tensorrt_llm::executor::SamplingConfig::checkNumReturnSequences::numReturnSequences"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig22checkRepetitionPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkRepetitionPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig22checkRepetitionPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkRepetitionPenalty::repetitionpenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig16checkTemperatureERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTemperature"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig16checkTemperatureERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTemperature::temperature"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig9checkTopKERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopK"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig9checkTopKERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopK::topK"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig9checkTopPERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig9checkTopPERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopP::topP"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14checkTopPDecayERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopPDecay"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14checkTopPDecayERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopPDecay::topPDecay"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig12checkTopPMinERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopPMin"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig12checkTopPMinERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopPMin::topPMin"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig17checkTopPResetIdsERKNSt8optionalI11TokenIdTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopPResetIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig17checkTopPResetIdsERKNSt8optionalI11TokenIdTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopPResetIds::topPResetIds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig26getBeamSearchDiversityRateEv", "tensorrt_llm::executor::SamplingConfig::getBeamSearchDiversityRate"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig12getBeamWidthEv", "tensorrt_llm::executor::SamplingConfig::getBeamWidth"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig17getBeamWidthArrayEv", "tensorrt_llm::executor::SamplingConfig::getBeamWidthArray"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig16getEarlyStoppingEv", "tensorrt_llm::executor::SamplingConfig::getEarlyStopping"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig19getFrequencyPenaltyEv", "tensorrt_llm::executor::SamplingConfig::getFrequencyPenalty"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig16getLengthPenaltyEv", "tensorrt_llm::executor::SamplingConfig::getLengthPenalty"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig7getMinPEv", "tensorrt_llm::executor::SamplingConfig::getMinP"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig12getMinTokensEv", "tensorrt_llm::executor::SamplingConfig::getMinTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig20getNoRepeatNgramSizeEv", "tensorrt_llm::executor::SamplingConfig::getNoRepeatNgramSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig17getNumReturnBeamsEv", "tensorrt_llm::executor::SamplingConfig::getNumReturnBeams"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig21getNumReturnSequencesEv", "tensorrt_llm::executor::SamplingConfig::getNumReturnSequences"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig18getPresencePenaltyEv", "tensorrt_llm::executor::SamplingConfig::getPresencePenalty"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig20getRepetitionPenaltyEv", "tensorrt_llm::executor::SamplingConfig::getRepetitionPenalty"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig7getSeedEv", "tensorrt_llm::executor::SamplingConfig::getSeed"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig14getTemperatureEv", "tensorrt_llm::executor::SamplingConfig::getTemperature"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig7getTopKEv", "tensorrt_llm::executor::SamplingConfig::getTopK"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig7getTopPEv", "tensorrt_llm::executor::SamplingConfig::getTopP"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig12getTopPDecayEv", "tensorrt_llm::executor::SamplingConfig::getTopPDecay"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig10getTopPMinEv", "tensorrt_llm::executor::SamplingConfig::getTopPMin"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig15getTopPResetIdsEv", "tensorrt_llm::executor::SamplingConfig::getTopPResetIds"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig24mBeamSearchDiversityRateE", "tensorrt_llm::executor::SamplingConfig::mBeamSearchDiversityRate"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig10mBeamWidthE", "tensorrt_llm::executor::SamplingConfig::mBeamWidth"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig15mBeamWidthArrayE", "tensorrt_llm::executor::SamplingConfig::mBeamWidthArray"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14mEarlyStoppingE", "tensorrt_llm::executor::SamplingConfig::mEarlyStopping"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig17mFrequencyPenaltyE", "tensorrt_llm::executor::SamplingConfig::mFrequencyPenalty"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14mLengthPenaltyE", "tensorrt_llm::executor::SamplingConfig::mLengthPenalty"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig5mMinPE", "tensorrt_llm::executor::SamplingConfig::mMinP"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig10mMinTokensE", "tensorrt_llm::executor::SamplingConfig::mMinTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig18mNoRepeatNgramSizeE", "tensorrt_llm::executor::SamplingConfig::mNoRepeatNgramSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig15mNumReturnBeamsE", "tensorrt_llm::executor::SamplingConfig::mNumReturnBeams"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig19mNumReturnSequencesE", "tensorrt_llm::executor::SamplingConfig::mNumReturnSequences"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig16mPresencePenaltyE", "tensorrt_llm::executor::SamplingConfig::mPresencePenalty"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig18mRepetitionPenaltyE", "tensorrt_llm::executor::SamplingConfig::mRepetitionPenalty"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig5mSeedE", "tensorrt_llm::executor::SamplingConfig::mSeed"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig12mTemperatureE", "tensorrt_llm::executor::SamplingConfig::mTemperature"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig5mTopKE", "tensorrt_llm::executor::SamplingConfig::mTopK"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig5mTopPE", "tensorrt_llm::executor::SamplingConfig::mTopP"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig10mTopPDecayE", "tensorrt_llm::executor::SamplingConfig::mTopPDecay"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig8mTopPMinE", "tensorrt_llm::executor::SamplingConfig::mTopPMin"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig13mTopPResetIdsE", "tensorrt_llm::executor::SamplingConfig::mTopPResetIds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfigeqERK14SamplingConfig", "tensorrt_llm::executor::SamplingConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfigeqERK14SamplingConfig", "tensorrt_llm::executor::SamplingConfig::operator==::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig26setBeamSearchDiversityRateERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setBeamSearchDiversityRate"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig26setBeamSearchDiversityRateERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setBeamSearchDiversityRate::beamSearchDiversityRate"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig12setBeamWidthE10SizeType32", "tensorrt_llm::executor::SamplingConfig::setBeamWidth"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig12setBeamWidthE10SizeType32", "tensorrt_llm::executor::SamplingConfig::setBeamWidth::beamWidth"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig17setBeamWidthArrayERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::setBeamWidthArray"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig17setBeamWidthArrayERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::setBeamWidthArray::beamWidthArray"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig16setEarlyStoppingERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setEarlyStopping"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig16setEarlyStoppingERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setEarlyStopping::earlyStopping"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig19setFrequencyPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setFrequencyPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig19setFrequencyPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setFrequencyPenalty::frequencyPenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig16setLengthPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setLengthPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig16setLengthPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setLengthPenalty::lengthPenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig7setMinPERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setMinP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig7setMinPERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setMinP::minP"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig12setMinTokensERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setMinTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig12setMinTokensERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setMinTokens::minTokens"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig20setNoRepeatNgramSizeERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setNoRepeatNgramSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig20setNoRepeatNgramSizeERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setNoRepeatNgramSize::noRepeatNgramSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig21setNumReturnSequencesERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setNumReturnSequences"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig21setNumReturnSequencesERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setNumReturnSequences::numReturnSequences"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig18setPresencePenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setPresencePenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig18setPresencePenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setPresencePenalty::presencePenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig20setRepetitionPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setRepetitionPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig20setRepetitionPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setRepetitionPenalty::repetitionPenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig7setSeedERKNSt8optionalI14RandomSeedTypeEE", "tensorrt_llm::executor::SamplingConfig::setSeed"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig7setSeedERKNSt8optionalI14RandomSeedTypeEE", "tensorrt_llm::executor::SamplingConfig::setSeed::seed"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14setTemperatureERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setTemperature"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14setTemperatureERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setTemperature::temperature"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig7setTopKERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setTopK"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig7setTopKERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setTopK::topK"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig7setTopPERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setTopP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig7setTopPERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setTopP::topP"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig12setTopPDecayERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setTopPDecay"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig12setTopPDecayERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setTopPDecay::topPDecay"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig10setTopPMinERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setTopPMin"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig10setTopPMinERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setTopPMin::topPMin"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig15setTopPResetIdsERKNSt8optionalI11TokenIdTypeEE", "tensorrt_llm::executor::SamplingConfig::setTopPResetIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig15setTopPResetIdsERKNSt8optionalI11TokenIdTypeEE", "tensorrt_llm::executor::SamplingConfig::setTopPResetIds::topPResetIds"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig20updateNumReturnBeamsEv", "tensorrt_llm::executor::SamplingConfig::updateNumReturnBeams"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor15SchedulerConfigE", "tensorrt_llm::executor::SchedulerConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15SchedulerConfig15SchedulerConfigE23CapacitySchedulerPolicyNSt8optionalI21ContextChunkingPolicyEENSt8optionalI18DynamicBatchConfigEE", "tensorrt_llm::executor::SchedulerConfig::SchedulerConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15SchedulerConfig15SchedulerConfigE23CapacitySchedulerPolicyNSt8optionalI21ContextChunkingPolicyEENSt8optionalI18DynamicBatchConfigEE", "tensorrt_llm::executor::SchedulerConfig::SchedulerConfig::capacitySchedulerPolicy"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15SchedulerConfig15SchedulerConfigE23CapacitySchedulerPolicyNSt8optionalI21ContextChunkingPolicyEENSt8optionalI18DynamicBatchConfigEE", "tensorrt_llm::executor::SchedulerConfig::SchedulerConfig::contextChunkingPolicy"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15SchedulerConfig15SchedulerConfigE23CapacitySchedulerPolicyNSt8optionalI21ContextChunkingPolicyEENSt8optionalI18DynamicBatchConfigEE", "tensorrt_llm::executor::SchedulerConfig::SchedulerConfig::dynamicBatchConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15SchedulerConfig26getCapacitySchedulerPolicyEv", "tensorrt_llm::executor::SchedulerConfig::getCapacitySchedulerPolicy"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15SchedulerConfig24getContextChunkingPolicyEv", "tensorrt_llm::executor::SchedulerConfig::getContextChunkingPolicy"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15SchedulerConfig21getDynamicBatchConfigEv", "tensorrt_llm::executor::SchedulerConfig::getDynamicBatchConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15SchedulerConfig24mCapacitySchedulerPolicyE", "tensorrt_llm::executor::SchedulerConfig::mCapacitySchedulerPolicy"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15SchedulerConfig22mContextChunkingPolicyE", "tensorrt_llm::executor::SchedulerConfig::mContextChunkingPolicy"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15SchedulerConfig19mDynamicBatchConfigE", "tensorrt_llm::executor::SchedulerConfig::mDynamicBatchConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15SchedulerConfigeqERK15SchedulerConfig", "tensorrt_llm::executor::SchedulerConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor15SchedulerConfigeqERK15SchedulerConfig", "tensorrt_llm::executor::SchedulerConfig::operator==::other"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor13SerializationE", "tensorrt_llm::executor::Serialization"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization32deserializeAdditionalModelOutputERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeAdditionalModelOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization32deserializeAdditionalModelOutputERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeAdditionalModelOutput::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization27deserializeAdditionalOutputERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeAdditionalOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization27deserializeAdditionalOutputERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeAdditionalOutput::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization21deserializeAgentStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeAgentState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization21deserializeAgentStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeAgentState::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization19deserializeBlockKeyERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeBlockKey"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization19deserializeBlockKeyERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeBlockKey::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization15deserializeBoolERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeBool"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization15deserializeBoolERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeBool::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization21deserializeCacheStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeCacheState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization21deserializeCacheStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeCacheState::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization33deserializeCacheTransceiverConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeCacheTransceiverConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization33deserializeCacheTransceiverConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeCacheTransceiverConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization20deserializeCommStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeCommState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization20deserializeCommStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeCommState::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeContextPhaseParamsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeContextPhaseParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeContextPhaseParamsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeContextPhaseParams::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization31deserializeDataTransceiverStateERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeDataTransceiverState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization31deserializeDataTransceiverStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDataTransceiverState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization31deserializeDataTransceiverStateERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeDataTransceiverState::buffer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization31deserializeDataTransceiverStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDataTransceiverState::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeDebugConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDebugConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeDebugConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDebugConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeDecodingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeDecodingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDecodingConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeDecodingModeERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDecodingMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeDecodingModeERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDecodingMode::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization33deserializeDisServingRequestStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDisServingRequestStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization33deserializeDisServingRequestStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDisServingRequestStats::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeDynamicBatchConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDynamicBatchConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeDynamicBatchConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDynamicBatchConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeEagleConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeEagleConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeEagleConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeEagleConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeExecutorConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeExecutorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeExecutorConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeExecutorConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization40deserializeExtendedRuntimePerfKnobConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeExtendedRuntimePerfKnobConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization40deserializeExtendedRuntimePerfKnobConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeExtendedRuntimePerfKnobConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization36deserializeExternalDraftTokensConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeExternalDraftTokensConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization36deserializeExternalDraftTokensConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeExternalDraftTokensConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization31deserializeGuidedDecodingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeGuidedDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization31deserializeGuidedDecodingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeGuidedDecodingConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization31deserializeGuidedDecodingParamsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeGuidedDecodingParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization31deserializeGuidedDecodingParamsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeGuidedDecodingParams::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization32deserializeInflightBatchingStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeInflightBatchingStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization32deserializeInflightBatchingStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeInflightBatchingStats::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeIterationStatsERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeIterationStats"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeIterationStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeIterationStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeIterationStatsERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeIterationStats::buffer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeIterationStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeIterationStats::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization28deserializeIterationStatsVecERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeIterationStatsVec"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization28deserializeIterationStatsVecERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeIterationStatsVec::buffer"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeKVCacheCreatedDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheCreatedData"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeKVCacheCreatedDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheCreatedData::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeKVCacheEventERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheEvent"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeKVCacheEventERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheEvent::is"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization27deserializeKVCacheEventDiffE16KVCacheEventDiffI1TERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheEventDiff"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization27deserializeKVCacheEventDiffE16KVCacheEventDiffI1TERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheEventDiff::T"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization27deserializeKVCacheEventDiffE16KVCacheEventDiffI1TERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheEventDiff::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization24deserializeKVCacheEventsERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeKVCacheEvents"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization24deserializeKVCacheEventsERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeKVCacheEvents::buffer"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeKVCacheRemovedDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheRemovedData"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeKVCacheRemovedDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheRemovedData::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization33deserializeKVCacheStoredBlockDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheStoredBlockData"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization33deserializeKVCacheStoredBlockDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheStoredBlockData::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization28deserializeKVCacheStoredDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheStoredData"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization28deserializeKVCacheStoredDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheStoredData::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeKVCacheUpdatedDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheUpdatedData"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeKVCacheUpdatedDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheUpdatedData::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization24deserializeKvCacheConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKvCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization24deserializeKvCacheConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKvCacheConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization33deserializeKvCacheRetentionConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKvCacheRetentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization33deserializeKvCacheRetentionConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKvCacheRetentionConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeKvCacheStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKvCacheStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeKvCacheStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKvCacheStats::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization34deserializeLookaheadDecodingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeLookaheadDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization34deserializeLookaheadDecodingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeLookaheadDecodingConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization21deserializeLoraConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeLoraConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization21deserializeLoraConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeLoraConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization20deserializeModelTypeERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeModelType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization20deserializeModelTypeERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeModelType::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeMropeConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeMropeConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeMropeConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeMropeConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization26deserializeMultimodalInputERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeMultimodalInput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization26deserializeMultimodalInputERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeMultimodalInput::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeOrchestratorConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeOrchestratorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeOrchestratorConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeOrchestratorConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeOutputConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeOutputConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeOutputConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeOutputConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeParallelConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeParallelConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeParallelConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeParallelConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization26deserializePeftCacheConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializePeftCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization26deserializePeftCacheConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializePeftCacheConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializePromptTuningConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializePromptTuningConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializePromptTuningConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializePromptTuningConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization18deserializeRequestERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequest"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization18deserializeRequestERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequest::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeRequestPerfMetricsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequestPerfMetrics"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeRequestPerfMetricsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequestPerfMetrics::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeRequestStageERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequestStage"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeRequestStageERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequestStage::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeRequestStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequestStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeRequestStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequestStats::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization35deserializeRequestStatsPerIterationERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeRequestStatsPerIteration"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization35deserializeRequestStatsPerIterationERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequestStatsPerIteration"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization35deserializeRequestStatsPerIterationERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeRequestStatsPerIteration::buffer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization35deserializeRequestStatsPerIterationERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequestStatsPerIteration::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization38deserializeRequestStatsPerIterationVecERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeRequestStatsPerIterationVec"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization38deserializeRequestStatsPerIterationVecERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeRequestStatsPerIterationVec::buffer"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization19deserializeResponseERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeResponse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization19deserializeResponseERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeResponse::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization20deserializeResponsesERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeResponses"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization20deserializeResponsesERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeResponses::buffer"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization17deserializeResultERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeResult"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization17deserializeResultERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeResult::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeSamplingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSamplingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeSamplingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSamplingConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization26deserializeSchedulerConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSchedulerConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization26deserializeSchedulerConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSchedulerConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeSocketStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSocketState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeSocketStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSocketState::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization32deserializeSpecDecFastLogitsInfoERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSpecDecFastLogitsInfo"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization32deserializeSpecDecFastLogitsInfoERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSpecDecFastLogitsInfo::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization28deserializeSpecDecodingStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSpecDecodingStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization28deserializeSpecDecodingStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSpecDecodingStats::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization36deserializeSpeculativeDecodingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSpeculativeDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization36deserializeSpeculativeDecodingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSpeculativeDecodingConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization30deserializeStaticBatchingStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeStaticBatchingStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization30deserializeStaticBatchingStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeStaticBatchingStats::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization17deserializeStringERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeString"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization17deserializeStringERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeString::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization17deserializeTensorERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeTensor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization17deserializeTensorERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeTensor::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization20deserializeTimePointERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeTimePoint"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization20deserializeTimePointERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeTimePoint::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization36deserializeTokenRangeRetentionConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeTokenRangeRetentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization36deserializeTokenRangeRetentionConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeTokenRangeRetentionConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeUniqueTokenERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeUniqueToken"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeUniqueTokenERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeUniqueToken::is"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization9serializeEvRK16KVCacheEventDiffI1TERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK10LoraConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK11DebugConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK11EagleConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK11MropeConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12DecodingModeRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12KVCacheEventRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12KvCacheStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12OutputConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12RequestStageRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12RequestStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK13KvCacheConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14DecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14ExecutorConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14IterationStats", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14IterationStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14ParallelConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14SamplingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK15MultimodalInputRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK15PeftCacheConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK15SchedulerConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK16AdditionalOutputRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK17KVCacheStoredDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK17SpecDecodingStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18ContextPhaseParamsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18DynamicBatchConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18KVCacheCreatedDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18KVCacheRemovedDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18KVCacheUpdatedDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18OrchestratorConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18PromptTuningConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18RequestPerfMetricsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK19StaticBatchingStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20DataTransceiverState", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20DataTransceiverStateRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20GuidedDecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20GuidedDecodingParamsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK21AdditionalModelOutputRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK21InflightBatchingStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22CacheTransceiverConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22DisServingRequestStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22KVCacheStoredBlockDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22KvCacheRetentionConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK23LookaheadDecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK24RequestStatsPerIteration", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK24RequestStatsPerIterationRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK25ExternalDraftTokensConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK25SpeculativeDecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK29ExtendedRuntimePerfKnobConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK33SpeculativeDecodingFastLogitsInfoRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK6ResultRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK6TensorRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK7RequestRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK8ResponseRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN12tensorrt_llm13batch_manager16kv_cache_manager8BlockKeyERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN12tensorrt_llm7runtime11UniqueTokenERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN18RequestPerfMetrics9TimePointERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN22KvCacheRetentionConfig25TokenRangeRetentionConfigERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache10AgentStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache10CacheStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache11SocketStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache9CommStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKNSt5dequeI12KVCacheEventEE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKNSt6vectorI14IterationStatsEE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKNSt6vectorI24RequestStatsPerIterationEE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKNSt6vectorI8ResponseEE", "tensorrt_llm::executor::Serialization::serialize"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization9serializeEvRK16KVCacheEventDiffI1TERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::T"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK21AdditionalModelOutputRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::additionalModelOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK16AdditionalOutputRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::additionalOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22CacheTransceiverConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::cacheTransceiverConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK10LoraConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK11MropeConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12OutputConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14SamplingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18PromptTuningConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK25ExternalDraftTokensConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18ContextPhaseParamsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::contextPhaseParams"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization9serializeEvRK16KVCacheEventDiffI1TERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK17KVCacheStoredDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18KVCacheCreatedDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18KVCacheRemovedDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18KVCacheUpdatedDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22KVCacheStoredBlockDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20DataTransceiverState", "tensorrt_llm::executor::Serialization::serialize::dataTransceiverState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20DataTransceiverStateRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::dataTransceiverState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK11DebugConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::debugConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14DecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::decodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12DecodingModeRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::decodingMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18DynamicBatchConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::dynamicBatchConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK11EagleConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::eagleConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12KVCacheEventRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::event"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14ExecutorConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::executorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK29ExtendedRuntimePerfKnobConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::extendedRuntimePerfKnobConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20GuidedDecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::guidedDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20GuidedDecodingParamsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::guidedDecodingParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK21InflightBatchingStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::inflightBatchingStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK33SpeculativeDecodingFastLogitsInfoRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::info"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14IterationStats", "tensorrt_llm::executor::Serialization::serialize::iterStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14IterationStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::iterStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKNSt6vectorI14IterationStatsEE", "tensorrt_llm::executor::Serialization::serialize::iterStatsVec"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN12tensorrt_llm13batch_manager16kv_cache_manager8BlockKeyERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::key"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK13KvCacheConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::kvCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKNSt5dequeI12KVCacheEventEE", "tensorrt_llm::executor::Serialization::serialize::kvCacheEvents"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22KvCacheRetentionConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::kvCacheRetentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12KvCacheStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::kvCacheStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK23LookaheadDecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::lookaheadDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18RequestPerfMetricsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::metrics"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK15MultimodalInputRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::multimodalInput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18OrchestratorConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::orchestratorConfig"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization9serializeEvRK16KVCacheEventDiffI1TERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK10LoraConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK11DebugConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK11EagleConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK11MropeConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12DecodingModeRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12KVCacheEventRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12KvCacheStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12OutputConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12RequestStageRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12RequestStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK13KvCacheConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14DecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14ExecutorConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14IterationStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14ParallelConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14SamplingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK15MultimodalInputRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK15PeftCacheConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK15SchedulerConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK16AdditionalOutputRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK17KVCacheStoredDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK17SpecDecodingStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18ContextPhaseParamsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18DynamicBatchConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18KVCacheCreatedDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18KVCacheRemovedDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18KVCacheUpdatedDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18OrchestratorConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18PromptTuningConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18RequestPerfMetricsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK19StaticBatchingStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20DataTransceiverStateRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20GuidedDecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20GuidedDecodingParamsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK21AdditionalModelOutputRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK21InflightBatchingStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22CacheTransceiverConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22DisServingRequestStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22KVCacheStoredBlockDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22KvCacheRetentionConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK23LookaheadDecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK24RequestStatsPerIterationRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK25ExternalDraftTokensConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK25SpeculativeDecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK29ExtendedRuntimePerfKnobConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK33SpeculativeDecodingFastLogitsInfoRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK6ResultRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK6TensorRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK7RequestRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK8ResponseRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN12tensorrt_llm13batch_manager16kv_cache_manager8BlockKeyERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN12tensorrt_llm7runtime11UniqueTokenERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN18RequestPerfMetrics9TimePointERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN22KvCacheRetentionConfig25TokenRangeRetentionConfigERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache10AgentStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache10CacheStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache11SocketStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache9CommStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14ParallelConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::parallelConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK15PeftCacheConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::peftCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK7RequestRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::request"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12RequestStageRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::requestStage"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKNSt6vectorI24RequestStatsPerIterationEE", "tensorrt_llm::executor::Serialization::serialize::requestStatsVec"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK8ResponseRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::response"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKNSt6vectorI8ResponseEE", "tensorrt_llm::executor::Serialization::serialize::responses"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK6ResultRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::result"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK15SchedulerConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::schedulerConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK25SpeculativeDecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::specDecConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK17SpecDecodingStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::specDecodingStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12RequestStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK24RequestStatsPerIteration", "tensorrt_llm::executor::Serialization::serialize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK24RequestStatsPerIterationRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache10AgentStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache10CacheStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache11SocketStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache9CommStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK19StaticBatchingStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::staticBatchingStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22DisServingRequestStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::stats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK6TensorRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::tensor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN12tensorrt_llm7runtime11UniqueTokenERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::token"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN22KvCacheRetentionConfig25TokenRangeRetentionConfigERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::tokenRangeRetentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN18RequestPerfMetrics9TimePointERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::tp"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization14serializedSizeE6size_tRK16KVCacheEventDiffI1TE", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK10LoraConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK11DebugConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK11EagleConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK11MropeConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12DecodingMode", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12KVCacheEvent", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12KvCacheStats", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12OutputConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12RequestStage", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12RequestStats", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK13KvCacheConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14DecodingConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14ExecutorConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14IterationStats", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14ParallelConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14SamplingConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK15MultimodalInput", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK15PeftCacheConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK15SchedulerConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK16AdditionalOutput", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK17KVCacheStoredData", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK17SpecDecodingStats", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18ContextPhaseParams", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18DynamicBatchConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18KVCacheCreatedData", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18KVCacheRemovedData", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18KVCacheUpdatedData", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18OrchestratorConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18PromptTuningConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18RequestPerfMetrics", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK19StaticBatchingStats", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK20DataTransceiverState", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK20GuidedDecodingConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK20GuidedDecodingParams", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK21AdditionalModelOutput", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK21InflightBatchingStats", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK22CacheTransceiverConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK22DisServingRequestStats", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK22KVCacheStoredBlockData", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK22KvCacheRetentionConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK23LookaheadDecodingConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK24RequestStatsPerIteration", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK25ExternalDraftTokensConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK25SpeculativeDecodingConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK29ExtendedRuntimePerfKnobConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK33SpeculativeDecodingFastLogitsInfo", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK6Result", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK6Tensor", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK7Request", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK8Response", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN12tensorrt_llm13batch_manager16kv_cache_manager8BlockKeyE", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN12tensorrt_llm7runtime11UniqueTokenE", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN18RequestPerfMetrics9TimePointE", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN22KvCacheRetentionConfig25TokenRangeRetentionConfigE", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN8kv_cache10AgentStateE", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN8kv_cache10CacheStateE", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN8kv_cache11SocketStateE", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN8kv_cache9CommStateE", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization14serializedSizeE6size_tRK16KVCacheEventDiffI1TE", "tensorrt_llm::executor::Serialization::serializedSize::T"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK21AdditionalModelOutput", "tensorrt_llm::executor::Serialization::serializedSize::additionalModelOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK16AdditionalOutput", "tensorrt_llm::executor::Serialization::serializedSize::additionalOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK22CacheTransceiverConfig", "tensorrt_llm::executor::Serialization::serializedSize::cacheTransceiverConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK10LoraConfig", "tensorrt_llm::executor::Serialization::serializedSize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK11MropeConfig", "tensorrt_llm::executor::Serialization::serializedSize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12OutputConfig", "tensorrt_llm::executor::Serialization::serializedSize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14SamplingConfig", "tensorrt_llm::executor::Serialization::serializedSize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18PromptTuningConfig", "tensorrt_llm::executor::Serialization::serializedSize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK25ExternalDraftTokensConfig", "tensorrt_llm::executor::Serialization::serializedSize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18ContextPhaseParams", "tensorrt_llm::executor::Serialization::serializedSize::contextPhaseParams"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization14serializedSizeE6size_tRK16KVCacheEventDiffI1TE", "tensorrt_llm::executor::Serialization::serializedSize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK17KVCacheStoredData", "tensorrt_llm::executor::Serialization::serializedSize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18KVCacheCreatedData", "tensorrt_llm::executor::Serialization::serializedSize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18KVCacheRemovedData", "tensorrt_llm::executor::Serialization::serializedSize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18KVCacheUpdatedData", "tensorrt_llm::executor::Serialization::serializedSize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK22KVCacheStoredBlockData", "tensorrt_llm::executor::Serialization::serializedSize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK20DataTransceiverState", "tensorrt_llm::executor::Serialization::serializedSize::dataTransceiverState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK11DebugConfig", "tensorrt_llm::executor::Serialization::serializedSize::debugConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14DecodingConfig", "tensorrt_llm::executor::Serialization::serializedSize::decodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12DecodingMode", "tensorrt_llm::executor::Serialization::serializedSize::decodingMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK22DisServingRequestStats", "tensorrt_llm::executor::Serialization::serializedSize::disServingRequestStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18DynamicBatchConfig", "tensorrt_llm::executor::Serialization::serializedSize::dynamicBatchConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK11EagleConfig", "tensorrt_llm::executor::Serialization::serializedSize::eagleConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12KVCacheEvent", "tensorrt_llm::executor::Serialization::serializedSize::event"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14ExecutorConfig", "tensorrt_llm::executor::Serialization::serializedSize::executorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK29ExtendedRuntimePerfKnobConfig", "tensorrt_llm::executor::Serialization::serializedSize::extendedRuntimePerfKnobConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK20GuidedDecodingConfig", "tensorrt_llm::executor::Serialization::serializedSize::guidedDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK20GuidedDecodingParams", "tensorrt_llm::executor::Serialization::serializedSize::guidedDecodingParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK21InflightBatchingStats", "tensorrt_llm::executor::Serialization::serializedSize::inflightBatchingStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK33SpeculativeDecodingFastLogitsInfo", "tensorrt_llm::executor::Serialization::serializedSize::info"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14IterationStats", "tensorrt_llm::executor::Serialization::serializedSize::iterStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN12tensorrt_llm13batch_manager16kv_cache_manager8BlockKeyE", "tensorrt_llm::executor::Serialization::serializedSize::key"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK13KvCacheConfig", "tensorrt_llm::executor::Serialization::serializedSize::kvCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK22KvCacheRetentionConfig", "tensorrt_llm::executor::Serialization::serializedSize::kvCacheRetentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12KvCacheStats", "tensorrt_llm::executor::Serialization::serializedSize::kvCacheStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK23LookaheadDecodingConfig", "tensorrt_llm::executor::Serialization::serializedSize::lookaheadDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18RequestPerfMetrics", "tensorrt_llm::executor::Serialization::serializedSize::metrics"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK15MultimodalInput", "tensorrt_llm::executor::Serialization::serializedSize::multimodalInput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18OrchestratorConfig", "tensorrt_llm::executor::Serialization::serializedSize::orchestratorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14ParallelConfig", "tensorrt_llm::executor::Serialization::serializedSize::parallelConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK15PeftCacheConfig", "tensorrt_llm::executor::Serialization::serializedSize::peftCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK7Request", "tensorrt_llm::executor::Serialization::serializedSize::request"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12RequestStage", "tensorrt_llm::executor::Serialization::serializedSize::requestStage"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK8Response", "tensorrt_llm::executor::Serialization::serializedSize::response"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK6Result", "tensorrt_llm::executor::Serialization::serializedSize::result"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK15SchedulerConfig", "tensorrt_llm::executor::Serialization::serializedSize::schedulerConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK25SpeculativeDecodingConfig", "tensorrt_llm::executor::Serialization::serializedSize::specDecConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK17SpecDecodingStats", "tensorrt_llm::executor::Serialization::serializedSize::specDecodingStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12RequestStats", "tensorrt_llm::executor::Serialization::serializedSize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK24RequestStatsPerIteration", "tensorrt_llm::executor::Serialization::serializedSize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN8kv_cache10AgentStateE", "tensorrt_llm::executor::Serialization::serializedSize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN8kv_cache10CacheStateE", "tensorrt_llm::executor::Serialization::serializedSize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN8kv_cache11SocketStateE", "tensorrt_llm::executor::Serialization::serializedSize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN8kv_cache9CommStateE", "tensorrt_llm::executor::Serialization::serializedSize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK19StaticBatchingStats", "tensorrt_llm::executor::Serialization::serializedSize::staticBatchingStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK6Tensor", "tensorrt_llm::executor::Serialization::serializedSize::tensor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN12tensorrt_llm7runtime11UniqueTokenE", "tensorrt_llm::executor::Serialization::serializedSize::token"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN22KvCacheRetentionConfig25TokenRangeRetentionConfigE", "tensorrt_llm::executor::Serialization::serializedSize::tokenRangeRetentionConfig"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor5ShapeE", "tensorrt_llm::executor::Shape"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor5Shape4BaseE", "tensorrt_llm::executor::Shape::Base"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor5Shape9DimType64E", "tensorrt_llm::executor::Shape::DimType64"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor5Shape5ShapeENSt16initializer_listI9DimType64EE", "tensorrt_llm::executor::Shape::Shape"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor5Shape5ShapeEPK9DimType64N4Base9size_typeE", "tensorrt_llm::executor::Shape::Shape"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor5Shape5ShapeEv", "tensorrt_llm::executor::Shape::Shape"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor5Shape5ShapeEPK9DimType64N4Base9size_typeE", "tensorrt_llm::executor::Shape::Shape::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor5Shape5ShapeENSt16initializer_listI9DimType64EE", "tensorrt_llm::executor::Shape::Shape::dims"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor5Shape5ShapeEPK9DimType64N4Base9size_typeE", "tensorrt_llm::executor::Shape::Shape::size"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor10SizeType32E", "tensorrt_llm::executor::SizeType32"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor10SizeType64E", "tensorrt_llm::executor::SizeType64"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor17SpecDecodingStatsE", "tensorrt_llm::executor::SpecDecodingStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor17SpecDecodingStats16acceptanceLengthE", "tensorrt_llm::executor::SpecDecodingStats::acceptanceLength"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor17SpecDecodingStats13draftOverheadE", "tensorrt_llm::executor::SpecDecodingStats::draftOverhead"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor17SpecDecodingStats13iterLatencyMSE", "tensorrt_llm::executor::SpecDecodingStats::iterLatencyMS"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor17SpecDecodingStats17numAcceptedTokensE", "tensorrt_llm::executor::SpecDecodingStats::numAcceptedTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor17SpecDecodingStats14numDraftTokensE", "tensorrt_llm::executor::SpecDecodingStats::numDraftTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor17SpecDecodingStats26numRequestsWithDraftTokensE", "tensorrt_llm::executor::SpecDecodingStats::numRequestsWithDraftTokens"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor25SpeculativeDecodingConfigE", "tensorrt_llm::executor::SpeculativeDecodingConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor25SpeculativeDecodingConfig25SpeculativeDecodingConfigEb", "tensorrt_llm::executor::SpeculativeDecodingConfig::SpeculativeDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25SpeculativeDecodingConfig25SpeculativeDecodingConfigEb", "tensorrt_llm::executor::SpeculativeDecodingConfig::SpeculativeDecodingConfig::fastLogits"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor25SpeculativeDecodingConfig10fastLogitsE", "tensorrt_llm::executor::SpeculativeDecodingConfig::fastLogits"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor25SpeculativeDecodingConfigeqERK25SpeculativeDecodingConfig", "tensorrt_llm::executor::SpeculativeDecodingConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor25SpeculativeDecodingConfigeqERK25SpeculativeDecodingConfig", "tensorrt_llm::executor::SpeculativeDecodingConfig::operator==::other"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor33SpeculativeDecodingFastLogitsInfoE", "tensorrt_llm::executor::SpeculativeDecodingFastLogitsInfo"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor33SpeculativeDecodingFastLogitsInfo18draftParticipantIdE", "tensorrt_llm::executor::SpeculativeDecodingFastLogitsInfo::draftParticipantId"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor33SpeculativeDecodingFastLogitsInfo14draftRequestIdE", "tensorrt_llm::executor::SpeculativeDecodingFastLogitsInfo::draftRequestId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor33SpeculativeDecodingFastLogitsInfo8toTensorEv", "tensorrt_llm::executor::SpeculativeDecodingFastLogitsInfo::toTensor"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor19StaticBatchingStatsE", "tensorrt_llm::executor::StaticBatchingStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor19StaticBatchingStats13emptyGenSlotsE", "tensorrt_llm::executor::StaticBatchingStats::emptyGenSlots"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor19StaticBatchingStats18numContextRequestsE", "tensorrt_llm::executor::StaticBatchingStats::numContextRequests"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor19StaticBatchingStats12numCtxTokensE", "tensorrt_llm::executor::StaticBatchingStats::numCtxTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor19StaticBatchingStats12numGenTokensE", "tensorrt_llm::executor::StaticBatchingStats::numGenTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor19StaticBatchingStats20numScheduledRequestsE", "tensorrt_llm::executor::StaticBatchingStats::numScheduledRequests"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor9StreamPtrE", "tensorrt_llm::executor::StreamPtr"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor6TensorE", "tensorrt_llm::executor::Tensor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::CudaStreamPtr"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor6Tensor4ImplE", "tensorrt_llm::executor::Tensor::Impl"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6TensorENSt10shared_ptrIN7runtime7ITensorEEE", "tensorrt_llm::executor::Tensor::Tensor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6TensorERK6Tensor", "tensorrt_llm::executor::Tensor::Tensor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6TensorERR6Tensor", "tensorrt_llm::executor::Tensor::Tensor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6TensorEv", "tensorrt_llm::executor::Tensor::Tensor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6TensorERK6Tensor", "tensorrt_llm::executor::Tensor::Tensor::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6TensorERR6Tensor", "tensorrt_llm::executor::Tensor::Tensor::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6TensorENSt10shared_ptrIN7runtime7ITensorEEE", "tensorrt_llm::executor::Tensor::Tensor::tensor"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor6copyToENSt10shared_ptrI4ImplEE13CudaStreamPtr", "tensorrt_llm::executor::Tensor::copyTo"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor6copyToENSt10shared_ptrI4ImplEE13CudaStreamPtr", "tensorrt_llm::executor::Tensor::copyTo::stream"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor6copyToENSt10shared_ptrI4ImplEE13CudaStreamPtr", "tensorrt_llm::executor::Tensor::copyTo::tensor"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor9copyToCpuEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToCpu"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor9copyToCpuEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToCpu::stream"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor9copyToGpuEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToGpu"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor9copyToGpuEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToGpu::stream"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor13copyToManagedEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToManaged"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor13copyToManagedEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToManaged::stream"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor12copyToPinnedEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToPinned"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor12copyToPinnedEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToPinned::stream"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor18copyToPooledPinnedEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToPooledPinned"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor18copyToPooledPinnedEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToPooledPinned::stream"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor3cpuE6Tensor5Shape", "tensorrt_llm::executor::Tensor::cpu"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor3cpuE8DataType5Shape", "tensorrt_llm::executor::Tensor::cpu"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor3cpuE6Tensor5Shape", "tensorrt_llm::executor::Tensor::cpu::T"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor3cpuE8DataType5Shape", "tensorrt_llm::executor::Tensor::cpu::dataType"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor3cpuE6Tensor5Shape", "tensorrt_llm::executor::Tensor::cpu::shape"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor3cpuE8DataType5Shape", "tensorrt_llm::executor::Tensor::cpu::shape"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6detail9ofITensorENSt10shared_ptrIN7runtime7ITensorEEE", "tensorrt_llm::executor::Tensor::detail::ofITensor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6detail9ofITensorENSt10shared_ptrIN7runtime7ITensorEEE", "tensorrt_llm::executor::Tensor::detail::ofITensor::tensor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6detail9toITensorERK6Tensor", "tensorrt_llm::executor::Tensor::detail::toITensor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6detail9toITensorERK6Tensor", "tensorrt_llm::executor::Tensor::detail::toITensor::tensor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7getDataEv", "tensorrt_llm::executor::Tensor::getData"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor7getDataEv", "tensorrt_llm::executor::Tensor::getData"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor11getDataTypeEv", "tensorrt_llm::executor::Tensor::getDataType"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor13getMemoryTypeEv", "tensorrt_llm::executor::Tensor::getMemoryType"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor14getRuntimeTypeE8DataTypev", "tensorrt_llm::executor::Tensor::getRuntimeType"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor14getRuntimeTypeE8DataTypev", "tensorrt_llm::executor::Tensor::getRuntimeType::T"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor8getShapeEv", "tensorrt_llm::executor::Tensor::getShape"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor7getSizeEv", "tensorrt_llm::executor::Tensor::getSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor14getSizeInBytesEv", "tensorrt_llm::executor::Tensor::getSizeInBytes"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor3gpuE6Tensor13CudaStreamPtr5Shape", "tensorrt_llm::executor::Tensor::gpu"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor3gpuE8DataType13CudaStreamPtr5Shape", "tensorrt_llm::executor::Tensor::gpu"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor3gpuE6Tensor13CudaStreamPtr5Shape", "tensorrt_llm::executor::Tensor::gpu::T"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor3gpuE8DataType13CudaStreamPtr5Shape", "tensorrt_llm::executor::Tensor::gpu::dataType"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor3gpuE6Tensor13CudaStreamPtr5Shape", "tensorrt_llm::executor::Tensor::gpu::shape"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor3gpuE8DataType13CudaStreamPtr5Shape", "tensorrt_llm::executor::Tensor::gpu::shape"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor3gpuE6Tensor13CudaStreamPtr5Shape", "tensorrt_llm::executor::Tensor::gpu::stream"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor3gpuE8DataType13CudaStreamPtr5Shape", "tensorrt_llm::executor::Tensor::gpu::stream"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7mTensorE", "tensorrt_llm::executor::Tensor::mTensor"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor7managedE6Tensor5Shape", "tensorrt_llm::executor::Tensor::managed"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7managedE8DataType5Shape", "tensorrt_llm::executor::Tensor::managed"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor7managedE6Tensor5Shape", "tensorrt_llm::executor::Tensor::managed::T"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7managedE8DataType5Shape", "tensorrt_llm::executor::Tensor::managed::dataType"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor7managedE6Tensor5Shape", "tensorrt_llm::executor::Tensor::managed::shape"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7managedE8DataType5Shape", "tensorrt_llm::executor::Tensor::managed::shape"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor2ofE6TensorP1T5Shape", "tensorrt_llm::executor::Tensor::of"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor2ofE6TensorR1T", "tensorrt_llm::executor::Tensor::of"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor2ofE8DataTypePv5Shape", "tensorrt_llm::executor::Tensor::of"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor2ofE6TensorP1T5Shape", "tensorrt_llm::executor::Tensor::of::T"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor2ofE6TensorR1T", "tensorrt_llm::executor::Tensor::of::T"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor2ofE6TensorP1T5Shape", "tensorrt_llm::executor::Tensor::of::data"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor2ofE6TensorR1T", "tensorrt_llm::executor::Tensor::of::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor2ofE8DataTypePv5Shape", "tensorrt_llm::executor::Tensor::of::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor2ofE8DataTypePv5Shape", "tensorrt_llm::executor::Tensor::of::dataType"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor2ofE6TensorP1T5Shape", "tensorrt_llm::executor::Tensor::of::shape"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor2ofE8DataTypePv5Shape", "tensorrt_llm::executor::Tensor::of::shape"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6TensorcvbEv", "tensorrt_llm::executor::Tensor::operator bool"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6TensorneERK6Tensor", "tensorrt_llm::executor::Tensor::operator!="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor6TensorneERK6Tensor", "tensorrt_llm::executor::Tensor::operator!=::rhs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6TensoraSERK6Tensor", "tensorrt_llm::executor::Tensor::operator="], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6TensoraSERR6Tensor", "tensorrt_llm::executor::Tensor::operator="], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6TensoraSERK6Tensor", "tensorrt_llm::executor::Tensor::operator=::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6TensoraSERR6Tensor", "tensorrt_llm::executor::Tensor::operator=::other"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6TensoreqERK6Tensor", "tensorrt_llm::executor::Tensor::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor6TensoreqERK6Tensor", "tensorrt_llm::executor::Tensor::operator==::rhs"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor6pinnedE6Tensor5Shape", "tensorrt_llm::executor::Tensor::pinned"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6pinnedE8DataType5Shape", "tensorrt_llm::executor::Tensor::pinned"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor6pinnedE6Tensor5Shape", "tensorrt_llm::executor::Tensor::pinned::T"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6pinnedE8DataType5Shape", "tensorrt_llm::executor::Tensor::pinned::dataType"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor6pinnedE6Tensor5Shape", "tensorrt_llm::executor::Tensor::pinned::shape"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6pinnedE8DataType5Shape", "tensorrt_llm::executor::Tensor::pinned::shape"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor12pooledPinnedE6Tensor5Shape", "tensorrt_llm::executor::Tensor::pooledPinned"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor12pooledPinnedE8DataType5Shape", "tensorrt_llm::executor::Tensor::pooledPinned"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor12pooledPinnedE6Tensor5Shape", "tensorrt_llm::executor::Tensor::pooledPinned::T"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor12pooledPinnedE8DataType5Shape", "tensorrt_llm::executor::Tensor::pooledPinned::dataType"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor12pooledPinnedE6Tensor5Shape", "tensorrt_llm::executor::Tensor::pooledPinned::shape"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor12pooledPinnedE8DataType5Shape", "tensorrt_llm::executor::Tensor::pooledPinned::shape"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7setFromERK6Tensor13CudaStreamPtr", "tensorrt_llm::executor::Tensor::setFrom"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7setFromERK6Tensor13CudaStreamPtr", "tensorrt_llm::executor::Tensor::setFrom::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7setFromERK6Tensor13CudaStreamPtr", "tensorrt_llm::executor::Tensor::setFrom::stream"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7setZeroE13CudaStreamPtr", "tensorrt_llm::executor::Tensor::setZero"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7setZeroE13CudaStreamPtr", "tensorrt_llm::executor::Tensor::setZero::stream"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6TensorD0Ev", "tensorrt_llm::executor::Tensor::~Tensor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor9TensorPtrE", "tensorrt_llm::executor::TensorPtr"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor11TokenIdTypeE", "tensorrt_llm::executor::TokenIdType"], [0, 2, 1, "_CPPv4I0_bEN12tensorrt_llm8executor10TypeTraitsE", "tensorrt_llm::executor::TypeTraits"], [0, 8, 1, "_CPPv4I0_bEN12tensorrt_llm8executor10TypeTraitsE", "tensorrt_llm::executor::TypeTraits::T"], [0, 2, 1, "_CPPv4I0EN12tensorrt_llm8executor10TypeTraitsIP1TEE", "tensorrt_llm::executor::TypeTraits<T*>"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor10TypeTraitsIP1TEE", "tensorrt_llm::executor::TypeTraits<T*>::T"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10TypeTraitsIP1TE5valueE", "tensorrt_llm::executor::TypeTraits<T*>::value"], [0, 2, 1, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsIbEE", "tensorrt_llm::executor::TypeTraits<bool>"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10TypeTraitsIbE5valueE", "tensorrt_llm::executor::TypeTraits<bool>::value"], [0, 2, 1, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsIfEE", "tensorrt_llm::executor::TypeTraits<float>"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10TypeTraitsIfE5valueE", "tensorrt_llm::executor::TypeTraits<float>::value"], [0, 2, 1, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsI4halfEE", "tensorrt_llm::executor::TypeTraits<half>"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10TypeTraitsI4halfE5valueE", "tensorrt_llm::executor::TypeTraits<half>::value"], [0, 2, 1, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsINSt7int32_tEEE", "tensorrt_llm::executor::TypeTraits<std::int32_t>"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10TypeTraitsINSt7int32_tEE5valueE", "tensorrt_llm::executor::TypeTraits<std::int32_t>::value"], [0, 2, 1, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsINSt7int64_tEEE", "tensorrt_llm::executor::TypeTraits<std::int64_t>"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10TypeTraitsINSt7int64_tEE5valueE", "tensorrt_llm::executor::TypeTraits<std::int64_t>::value"], [0, 2, 1, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsINSt6int8_tEEE", "tensorrt_llm::executor::TypeTraits<std::int8_t>"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10TypeTraitsINSt6int8_tEE5valueE", "tensorrt_llm::executor::TypeTraits<std::int8_t>::value"], [0, 2, 1, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsINSt7uint8_tEEE", "tensorrt_llm::executor::TypeTraits<std::uint8_t>"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10TypeTraitsINSt7uint8_tEE5valueE", "tensorrt_llm::executor::TypeTraits<std::uint8_t>::value"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor11VecLogProbsE", "tensorrt_llm::executor::VecLogProbs"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor16VecTokenExtraIdsE", "tensorrt_llm::executor::VecTokenExtraIds"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor9VecTokensE", "tensorrt_llm::executor::VecTokens"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor6detailE", "tensorrt_llm::executor::detail"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor6detail9DimType64E", "tensorrt_llm::executor::detail::DimType64"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6detail9ofITensorENSt10shared_ptrIN7runtime7ITensorEEE", "tensorrt_llm::executor::detail::ofITensor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6detail9ofITensorENSt10shared_ptrIN7runtime7ITensorEEE", "tensorrt_llm::executor::detail::ofITensor::tensor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6detail9toITensorERK6Tensor", "tensorrt_llm::executor::detail::toITensor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6detail9toITensorERK6Tensor", "tensorrt_llm::executor::detail::toITensor::tensor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executorE", "tensorrt_llm::executor::disagg_executor"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestratorE", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator26DisaggExecutorOrchestratorERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorIN8executor14ExecutorConfigEEERKNSt6vectorIN8executor14ExecutorConfigEEEbb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::DisaggExecutorOrchestrator"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator26DisaggExecutorOrchestratorERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorIN8executor14ExecutorConfigEEERKNSt6vectorIN8executor14ExecutorConfigEEEbb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::DisaggExecutorOrchestrator::ctxEnginePaths"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator26DisaggExecutorOrchestratorERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorIN8executor14ExecutorConfigEEERKNSt6vectorIN8executor14ExecutorConfigEEEbb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::DisaggExecutorOrchestrator::ctxExecutorConfigs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator26DisaggExecutorOrchestratorERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorIN8executor14ExecutorConfigEEERKNSt6vectorIN8executor14ExecutorConfigEEEbb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::DisaggExecutorOrchestrator::genEnginePaths"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator26DisaggExecutorOrchestratorERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorIN8executor14ExecutorConfigEEERKNSt6vectorIN8executor14ExecutorConfigEEEbb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::DisaggExecutorOrchestrator::genExecutorConfigs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator26DisaggExecutorOrchestratorERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorIN8executor14ExecutorConfigEEERKNSt6vectorIN8executor14ExecutorConfigEEEbb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::DisaggExecutorOrchestrator::hasContextAwaitThreads"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator26DisaggExecutorOrchestratorERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorIN8executor14ExecutorConfigEEERKNSt6vectorIN8executor14ExecutorConfigEEEbb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::DisaggExecutorOrchestrator::hasGenAwaitThreads"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator21awaitContextResponsesERKNSt8optionalINSt6chrono12millisecondsEEENSt8optionalIiEE", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::awaitContextResponses"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator21awaitContextResponsesERKNSt8optionalINSt6chrono12millisecondsEEENSt8optionalIiEE", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::awaitContextResponses::contextIdx"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator21awaitContextResponsesERKNSt8optionalINSt6chrono12millisecondsEEENSt8optionalIiEE", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::awaitContextResponses::timeout"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator24awaitGenerationResponsesERKNSt8optionalINSt6chrono12millisecondsEEENSt8optionalIiEE", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::awaitGenerationResponses"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator24awaitGenerationResponsesERKNSt8optionalINSt6chrono12millisecondsEEENSt8optionalIiEE", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::awaitGenerationResponses::genIdx"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator24awaitGenerationResponsesERKNSt8optionalINSt6chrono12millisecondsEEENSt8optionalIiEE", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::awaitGenerationResponses::timeout"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator10canEnqueueEv", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::canEnqueue"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator14enqueueContextERKNSt6vectorIN5texec7RequestEEENSt8optionalIiEEb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::enqueueContext"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator14enqueueContextERKNSt6vectorIN5texec7RequestEEENSt8optionalIiEEb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::enqueueContext::batch"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator14enqueueContextERKNSt6vectorIN5texec7RequestEEENSt8optionalIiEEb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::enqueueContext::requests"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator14enqueueContextERKNSt6vectorIN5texec7RequestEEENSt8optionalIiEEb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::enqueueContext::selectContextId"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator17enqueueGenerationERKNSt6vectorIN5texec7RequestEEERKNSt6vectorI6IdTypeEENSt8optionalIiEEb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::enqueueGeneration"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator17enqueueGenerationERKNSt6vectorIN5texec7RequestEEERKNSt6vectorI6IdTypeEENSt8optionalIiEEb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::enqueueGeneration::batch"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator17enqueueGenerationERKNSt6vectorIN5texec7RequestEEERKNSt6vectorI6IdTypeEENSt8optionalIiEEb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::enqueueGeneration::globalRequestIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator17enqueueGenerationERKNSt6vectorIN5texec7RequestEEERKNSt6vectorI6IdTypeEENSt8optionalIiEEb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::enqueueGeneration::requests"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator17enqueueGenerationERKNSt6vectorIN5texec7RequestEEERKNSt6vectorI6IdTypeEENSt8optionalIiEEb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::enqueueGeneration::selectGenIdx"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator19getContextExecutorsEv", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::getContextExecutors"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator15getGenExecutorsEv", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::getGenExecutors"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator5mImplE", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::mImpl"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestratorD0Ev", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::~DisaggExecutorOrchestrator"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithIdE", "tensorrt_llm::executor::disagg_executor::ResponseWithId"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERK14ResponseWithId", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERKN12tensorrt_llm8executor8ResponseE6IdType", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERR14ResponseWithId", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERRN12tensorrt_llm8executor8ResponseE6IdType", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERKN12tensorrt_llm8executor8ResponseE6IdType", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId::gid"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERRN12tensorrt_llm8executor8ResponseE6IdType", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId::gid"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERK14ResponseWithId", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERR14ResponseWithId", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERKN12tensorrt_llm8executor8ResponseE6IdType", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId::response"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERRN12tensorrt_llm8executor8ResponseE6IdType", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId::response"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId3gidE", "tensorrt_llm::executor::disagg_executor::ResponseWithId::gid"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithIdaSERK14ResponseWithId", "tensorrt_llm::executor::disagg_executor::ResponseWithId::operator="], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithIdaSERR14ResponseWithId", "tensorrt_llm::executor::disagg_executor::ResponseWithId::operator="], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithIdaSERK14ResponseWithId", "tensorrt_llm::executor::disagg_executor::ResponseWithId::operator=::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithIdaSERR14ResponseWithId", "tensorrt_llm::executor::disagg_executor::ResponseWithId::operator=::other"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId8responseE", "tensorrt_llm::executor::disagg_executor::ResponseWithId::response"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithIdD0Ev", "tensorrt_llm::executor::disagg_executor::ResponseWithId::~ResponseWithId"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor8kv_cacheE", "tensorrt_llm::executor::kv_cache"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor8kv_cacheE", "tensorrt_llm::executor::kv_cache"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor8kv_cacheE", "tensorrt_llm::executor::kv_cache"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor8kv_cacheE", "tensorrt_llm::executor::kv_cache"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9AgentDescE", "tensorrt_llm::executor::kv_cache::AgentDesc"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9AgentDesc9AgentDescENSt6stringE", "tensorrt_llm::executor::kv_cache::AgentDesc::AgentDesc"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9AgentDesc9AgentDescENSt6stringE", "tensorrt_llm::executor::kv_cache::AgentDesc::AgentDesc::backendAgentDesc"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9AgentDesc19getBackendAgentDescEv", "tensorrt_llm::executor::kv_cache::AgentDesc::getBackendAgentDesc"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9AgentDesc17mBackendAgentDescE", "tensorrt_llm::executor::kv_cache::AgentDesc::mBackendAgentDesc"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10AgentStateE", "tensorrt_llm::executor::kv_cache::AgentState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10AgentState10AgentStateENSt6stringENSt6stringE", "tensorrt_llm::executor::kv_cache::AgentState::AgentState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10AgentState10AgentStateEv", "tensorrt_llm::executor::kv_cache::AgentState::AgentState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10AgentState10AgentStateENSt6stringENSt6stringE", "tensorrt_llm::executor::kv_cache::AgentState::AgentState::agentName"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10AgentState10AgentStateENSt6stringENSt6stringE", "tensorrt_llm::executor::kv_cache::AgentState::AgentState::connectionInfo"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10AgentState10mAgentNameE", "tensorrt_llm::executor::kv_cache::AgentState::mAgentName"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10AgentState15mConnectionInfoE", "tensorrt_llm::executor::kv_cache::AgentState::mConnectionInfo"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10AgentStateeqERK10AgentState", "tensorrt_llm::executor::kv_cache::AgentState::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10AgentStateeqERK10AgentState", "tensorrt_llm::executor::kv_cache::AgentState::operator==::other"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10AgentState8toStringEv", "tensorrt_llm::executor::kv_cache::AgentState::toString"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15BaseAgentConfigE", "tensorrt_llm::executor::kv_cache::BaseAgentConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15BaseAgentConfig5mNameE", "tensorrt_llm::executor::kv_cache::BaseAgentConfig::mName"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15BaseAgentConfig11multiThreadE", "tensorrt_llm::executor::kv_cache::BaseAgentConfig::multiThread"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15BaseAgentConfig13useProgThreadE", "tensorrt_llm::executor::kv_cache::BaseAgentConfig::useProgThread"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseLoopbackAgentE", "tensorrt_llm::executor::kv_cache::BaseLoopbackAgent"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseLoopbackAgent22executeLoopbackRequestERK11MemoryDescsRK9FileDescsb", "tensorrt_llm::executor::kv_cache::BaseLoopbackAgent::executeLoopbackRequest"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseLoopbackAgent22executeLoopbackRequestERK11MemoryDescsRK9FileDescsb", "tensorrt_llm::executor::kv_cache::BaseLoopbackAgent::executeLoopbackRequest::fileDescs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseLoopbackAgent22executeLoopbackRequestERK11MemoryDescsRK9FileDescsb", "tensorrt_llm::executor::kv_cache::BaseLoopbackAgent::executeLoopbackRequest::isOffload"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseLoopbackAgent22executeLoopbackRequestERK11MemoryDescsRK9FileDescsb", "tensorrt_llm::executor::kv_cache::BaseLoopbackAgent::executeLoopbackRequest::memoryDescs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseLoopbackAgentD0Ev", "tensorrt_llm::executor::kv_cache::BaseLoopbackAgent::~BaseLoopbackAgent"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgentE", "tensorrt_llm::executor::kv_cache::BaseTransferAgent"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent16checkRemoteDescsERKNSt6stringERK11MemoryDescs", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::checkRemoteDescs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent16checkRemoteDescsERKNSt6stringERK11MemoryDescs", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::checkRemoteDescs::memoryDescs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent16checkRemoteDescsERKNSt6stringERK11MemoryDescs", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::checkRemoteDescs::name"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent18connectRemoteAgentERKNSt6stringERK18ConnectionInfoType", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::connectRemoteAgent"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent18connectRemoteAgentERKNSt6stringERK18ConnectionInfoType", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::connectRemoteAgent::connectionInfo"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent18connectRemoteAgentERKNSt6stringERK18ConnectionInfoType", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::connectRemoteAgent::name"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent16deregisterMemoryERK13RegisterDescs", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::deregisterMemory"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent16deregisterMemoryERK13RegisterDescs", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::deregisterMemory::descs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent17getConnectionInfoEv", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::getConnectionInfo"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent17getLocalAgentDescEv", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::getLocalAgentDesc"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent23getNotifiedSyncMessagesEv", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::getNotifiedSyncMessages"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent21invalidateRemoteAgentERKNSt6stringE", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::invalidateRemoteAgent"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent21invalidateRemoteAgentERKNSt6stringE", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::invalidateRemoteAgent::name"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent15loadRemoteAgentERKNSt6stringERK9AgentDesc", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::loadRemoteAgent"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent15loadRemoteAgentERKNSt6stringERK9AgentDesc", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::loadRemoteAgent::agentDesc"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent15loadRemoteAgentERKNSt6stringERK9AgentDesc", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::loadRemoteAgent::name"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent17notifySyncMessageERKNSt6stringERK11SyncMessage", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::notifySyncMessage"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent17notifySyncMessageERKNSt6stringERK11SyncMessage", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::notifySyncMessage::name"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent17notifySyncMessageERKNSt6stringERK11SyncMessage", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::notifySyncMessage::syncMessage"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent14registerMemoryERK13RegisterDescs", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::registerMemory"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent14registerMemoryERK13RegisterDescs", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::registerMemory::descs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent22submitTransferRequestsERK15TransferRequest", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::submitTransferRequests"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent22submitTransferRequestsERK15TransferRequest", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::submitTransferRequests::request"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgentD0Ev", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::~BaseTransferAgent"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheStateE", "tensorrt_llm::executor::kv_cache::CacheState"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState15AttentionConfigE", "tensorrt_llm::executor::kv_cache::CacheState::AttentionConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState15AttentionConfig15AttentionConfigE13AttentionTypei", "tensorrt_llm::executor::kv_cache::CacheState::AttentionConfig::AttentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState15AttentionConfig15AttentionConfigE13AttentionTypei", "tensorrt_llm::executor::kv_cache::CacheState::AttentionConfig::AttentionConfig::attentionType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState15AttentionConfig15AttentionConfigE13AttentionTypei", "tensorrt_llm::executor::kv_cache::CacheState::AttentionConfig::AttentionConfig::kvFactor"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState15AttentionConfig14mAttentionTypeE", "tensorrt_llm::executor::kv_cache::CacheState::AttentionConfig::mAttentionType"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState15AttentionConfig9mKvFactorE", "tensorrt_llm::executor::kv_cache::CacheState::AttentionConfig::mKvFactor"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState15AttentionConfigeqERK15AttentionConfig", "tensorrt_llm::executor::kv_cache::CacheState::AttentionConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState15AttentionConfigeqERK15AttentionConfig", "tensorrt_llm::executor::kv_cache::CacheState::AttentionConfig::operator==::other"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState13AttentionTypeE", "tensorrt_llm::executor::kv_cache::CacheState::AttentionType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState13AttentionType8kDEFAULTE", "tensorrt_llm::executor::kv_cache::CacheState::AttentionType::kDEFAULT"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState13AttentionType4kMLAE", "tensorrt_llm::executor::kv_cache::CacheState::AttentionType::kMLA"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE11ModelConfigRKN7runtime11WorldConfigERKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::DPrank"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::DPrank"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::DPsize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::DPsize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::attentionLayerNumPerPP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE11ModelConfigRKN7runtime11WorldConfigERKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::attentionLayerNumPerPP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::attentionLayerNumPerPP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::attentionType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE11ModelConfigRKN7runtime11WorldConfigERKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::attentionType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::attentionType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::contextParallelism"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::contextParallelism"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::dataType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE11ModelConfigRKN7runtime11WorldConfigERKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::dataType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::dataType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::enableAttentionDP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::enableAttentionDP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::enableBlockReuse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE11ModelConfigRKN7runtime11WorldConfigERKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::enableBlockReuse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::enableBlockReuse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::kvFactor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE11ModelConfigRKN7runtime11WorldConfigERKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::kvFactor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::kvFactor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE11ModelConfigRKN7runtime11WorldConfigERKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::modelConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::nbAttentionLayers"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::nbKvHeadPerLayer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::nbKvHeads"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::pipelineParallelism"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::pipelineParallelism"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::sizePerHead"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::sizePerHead"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::tensorParallelism"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::tensorParallelism"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::tokensPerBlock"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::tokensPerBlock"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE11ModelConfigRKN7runtime11WorldConfigERKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::worldConfig"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState11ModelConfigE", "tensorrt_llm::executor::kv_cache::CacheState::ModelConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState11ModelConfig18mNbKvHeadsPerLayerE", "tensorrt_llm::executor::kv_cache::CacheState::ModelConfig::mNbKvHeadsPerLayer"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState11ModelConfig12mSizePerHeadE", "tensorrt_llm::executor::kv_cache::CacheState::ModelConfig::mSizePerHead"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState11ModelConfig15mTokensPerBlockE", "tensorrt_llm::executor::kv_cache::CacheState::ModelConfig::mTokensPerBlock"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState11ModelConfigeqERK11ModelConfig", "tensorrt_llm::executor::kv_cache::CacheState::ModelConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState11ModelConfigeqERK11ModelConfig", "tensorrt_llm::executor::kv_cache::CacheState::ModelConfig::operator==::other"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfigE", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfig23mAttentionLayerNumPerPPE", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig::mAttentionLayerNumPerPP"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfig19mContextParallelismE", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig::mContextParallelism"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfig7mDPrankE", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig::mDPrank"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfig7mDPsizeE", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig::mDPsize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfig18mEnableAttentionDPE", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig::mEnableAttentionDP"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfig20mPipelineParallelismE", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig::mPipelineParallelism"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfig18mTensorParallelismE", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig::mTensorParallelism"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfigeqERK14ParallelConfig", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfigeqERK14ParallelConfig", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig::operator==::other"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState18getAttentionConfigEv", "tensorrt_llm::executor::kv_cache::CacheState::getAttentionConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState11getDataTypeEv", "tensorrt_llm::executor::kv_cache::CacheState::getDataType"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState19getEnableBlockReuseEv", "tensorrt_llm::executor::kv_cache::CacheState::getEnableBlockReuse"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState14getModelConfigEv", "tensorrt_llm::executor::kv_cache::CacheState::getModelConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState17getParallelConfigEv", "tensorrt_llm::executor::kv_cache::CacheState::getParallelConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState16mAttentionConfigE", "tensorrt_llm::executor::kv_cache::CacheState::mAttentionConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState9mDataTypeE", "tensorrt_llm::executor::kv_cache::CacheState::mDataType"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState17mEnableBlockReuseE", "tensorrt_llm::executor::kv_cache::CacheState::mEnableBlockReuse"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState12mModelConfigE", "tensorrt_llm::executor::kv_cache::CacheState::mModelConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState15mParallelConfigE", "tensorrt_llm::executor::kv_cache::CacheState::mParallelConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheStateeqERKN8kv_cache10CacheStateE", "tensorrt_llm::executor::kv_cache::CacheState::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheStateeqERKN8kv_cache10CacheStateE", "tensorrt_llm::executor::kv_cache::CacheState::operator==::other"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState8toStringEv", "tensorrt_llm::executor::kv_cache::CacheState::toString"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommStateE", "tensorrt_llm::executor::kv_cache::CommState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt6vectorI10AgentStateEEi", "tensorrt_llm::executor::kv_cache::CommState::CommState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt6vectorI10SizeType32EEi", "tensorrt_llm::executor::kv_cache::CommState::CommState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt6vectorI11SocketStateEEi", "tensorrt_llm::executor::kv_cache::CommState::CommState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt8uint16_tENSt6stringE", "tensorrt_llm::executor::kv_cache::CommState::CommState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateEv", "tensorrt_llm::executor::kv_cache::CommState::CommState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt6vectorI10AgentStateEEi", "tensorrt_llm::executor::kv_cache::CommState::CommState::agentState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt8uint16_tENSt6stringE", "tensorrt_llm::executor::kv_cache::CommState::CommState::ip"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt8uint16_tENSt6stringE", "tensorrt_llm::executor::kv_cache::CommState::CommState::port"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt6vectorI10SizeType32EEi", "tensorrt_llm::executor::kv_cache::CommState::CommState::ranks"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt6vectorI10AgentStateEEi", "tensorrt_llm::executor::kv_cache::CommState::CommState::selfIdx"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt6vectorI10SizeType32EEi", "tensorrt_llm::executor::kv_cache::CommState::CommState::selfIdx"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt6vectorI11SocketStateEEi", "tensorrt_llm::executor::kv_cache::CommState::CommState::selfIdx"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt6vectorI11SocketStateEEi", "tensorrt_llm::executor::kv_cache::CommState::CommState::socketState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommState13getAgentStateEv", "tensorrt_llm::executor::kv_cache::CommState::getAgentState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommState11getMpiStateEv", "tensorrt_llm::executor::kv_cache::CommState::getMpiState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommState10getSelfIdxEv", "tensorrt_llm::executor::kv_cache::CommState::getSelfIdx"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommState14getSocketStateEv", "tensorrt_llm::executor::kv_cache::CommState::getSocketState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommState12isAgentStateEv", "tensorrt_llm::executor::kv_cache::CommState::isAgentState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommState10isMpiStateEv", "tensorrt_llm::executor::kv_cache::CommState::isMpiState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommState13isSocketStateEv", "tensorrt_llm::executor::kv_cache::CommState::isSocketState"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState8mSelfIdxE", "tensorrt_llm::executor::kv_cache::CommState::mSelfIdx"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState6mStateE", "tensorrt_llm::executor::kv_cache::CommState::mState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommStateeqERK9CommState", "tensorrt_llm::executor::kv_cache::CommState::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommStateeqERK9CommState", "tensorrt_llm::executor::kv_cache::CommState::operator==::other"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommState8toStringEv", "tensorrt_llm::executor::kv_cache::CommState::toString"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10ConnectionE", "tensorrt_llm::executor::kv_cache::Connection"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10Connection12isThreadSafeEv", "tensorrt_llm::executor::kv_cache::Connection::isThreadSafe"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10Connection4recvERK11DataContextPv6size_t", "tensorrt_llm::executor::kv_cache::Connection::recv"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10Connection4recvERK11DataContextPv6size_t", "tensorrt_llm::executor::kv_cache::Connection::recv::ctx"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10Connection4recvERK11DataContextPv6size_t", "tensorrt_llm::executor::kv_cache::Connection::recv::data"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10Connection4recvERK11DataContextPv6size_t", "tensorrt_llm::executor::kv_cache::Connection::recv::size"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10Connection4sendERK11DataContextPKv6size_t", "tensorrt_llm::executor::kv_cache::Connection::send"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10Connection4sendERK11DataContextPKv6size_t", "tensorrt_llm::executor::kv_cache::Connection::send::ctx"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10Connection4sendERK11DataContextPKv6size_t", "tensorrt_llm::executor::kv_cache::Connection::send::data"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10Connection4sendERK11DataContextPKv6size_t", "tensorrt_llm::executor::kv_cache::Connection::send::size"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10ConnectionD0Ev", "tensorrt_llm::executor::kv_cache::Connection::~Connection"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache18ConnectionInfoTypeE", "tensorrt_llm::executor::kv_cache::ConnectionInfoType"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17ConnectionManagerE", "tensorrt_llm::executor::kv_cache::ConnectionManager"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache17ConnectionManager12getCommStateEv", "tensorrt_llm::executor::kv_cache::ConnectionManager::getCommState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17ConnectionManager14getConnectionsERK9CommState", "tensorrt_llm::executor::kv_cache::ConnectionManager::getConnections"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17ConnectionManager14getConnectionsERK9CommState", "tensorrt_llm::executor::kv_cache::ConnectionManager::getConnections::state"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17ConnectionManager11recvConnectERK11DataContextPv6size_t", "tensorrt_llm::executor::kv_cache::ConnectionManager::recvConnect"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17ConnectionManager11recvConnectERK11DataContextPv6size_t", "tensorrt_llm::executor::kv_cache::ConnectionManager::recvConnect::ctx"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17ConnectionManager11recvConnectERK11DataContextPv6size_t", "tensorrt_llm::executor::kv_cache::ConnectionManager::recvConnect::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17ConnectionManager11recvConnectERK11DataContextPv6size_t", "tensorrt_llm::executor::kv_cache::ConnectionManager::recvConnect::size"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17ConnectionManagerD0Ev", "tensorrt_llm::executor::kv_cache::ConnectionManager::~ConnectionManager"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11DataContextE", "tensorrt_llm::executor::kv_cache::DataContext"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11DataContext11DataContextEi", "tensorrt_llm::executor::kv_cache::DataContext::DataContext"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11DataContext11DataContextEi", "tensorrt_llm::executor::kv_cache::DataContext::DataContext::tag"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache11DataContext6getTagEv", "tensorrt_llm::executor::kv_cache::DataContext::getTag"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11DataContext4mTagE", "tensorrt_llm::executor::kv_cache::DataContext::mTag"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoaderE", "tensorrt_llm::executor::kv_cache::DynLibLoader"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader12DynLibLoaderERK12DynLibLoader", "tensorrt_llm::executor::kv_cache::DynLibLoader::DynLibLoader"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader12DynLibLoaderEv", "tensorrt_llm::executor::kv_cache::DynLibLoader::DynLibLoader"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader5dlSymEPvPKc", "tensorrt_llm::executor::kv_cache::DynLibLoader::dlSym"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader5dlSymEPvPKc", "tensorrt_llm::executor::kv_cache::DynLibLoader::dlSym::handle"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader5dlSymEPvPKc", "tensorrt_llm::executor::kv_cache::DynLibLoader::dlSym::symbol"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor8kv_cache12DynLibLoader18getFunctionPointerE9FunctionTRKNSt6stringERKNSt6stringE", "tensorrt_llm::executor::kv_cache::DynLibLoader::getFunctionPointer"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor8kv_cache12DynLibLoader18getFunctionPointerE9FunctionTRKNSt6stringERKNSt6stringE", "tensorrt_llm::executor::kv_cache::DynLibLoader::getFunctionPointer::FunctionT"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor8kv_cache12DynLibLoader18getFunctionPointerE9FunctionTRKNSt6stringERKNSt6stringE", "tensorrt_llm::executor::kv_cache::DynLibLoader::getFunctionPointer::funcName"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor8kv_cache12DynLibLoader18getFunctionPointerE9FunctionTRKNSt6stringERKNSt6stringE", "tensorrt_llm::executor::kv_cache::DynLibLoader::getFunctionPointer::libName"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader9getHandleERKNSt6stringE", "tensorrt_llm::executor::kv_cache::DynLibLoader::getHandle"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader9getHandleERKNSt6stringE", "tensorrt_llm::executor::kv_cache::DynLibLoader::getHandle::name"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader11getInstanceEv", "tensorrt_llm::executor::kv_cache::DynLibLoader::getInstance"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader9mDllMutexE", "tensorrt_llm::executor::kv_cache::DynLibLoader::mDllMutex"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader9mHandlersE", "tensorrt_llm::executor::kv_cache::DynLibLoader::mHandlers"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoaderaSERK12DynLibLoader", "tensorrt_llm::executor::kv_cache::DynLibLoader::operator="], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoaderD0Ev", "tensorrt_llm::executor::kv_cache::DynLibLoader::~DynLibLoader"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDescE", "tensorrt_llm::executor::kv_cache::FileDesc"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc8FileDescERK8FileDesc", "tensorrt_llm::executor::kv_cache::FileDesc::FileDesc"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc8FileDescERKNSt6stringEi6mode_t6size_t", "tensorrt_llm::executor::kv_cache::FileDesc::FileDesc"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc8FileDescERR8FileDesc", "tensorrt_llm::executor::kv_cache::FileDesc::FileDesc"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc8FileDescERKNSt6stringEi6mode_t6size_t", "tensorrt_llm::executor::kv_cache::FileDesc::FileDesc::filename"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc8FileDescERKNSt6stringEi6mode_t6size_t", "tensorrt_llm::executor::kv_cache::FileDesc::FileDesc::flags"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc8FileDescERKNSt6stringEi6mode_t6size_t", "tensorrt_llm::executor::kv_cache::FileDesc::FileDesc::len"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc8FileDescERKNSt6stringEi6mode_t6size_t", "tensorrt_llm::executor::kv_cache::FileDesc::FileDesc::mode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc8FileDescERR8FileDesc", "tensorrt_llm::executor::kv_cache::FileDesc::FileDesc::other"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc2fdE", "tensorrt_llm::executor::kv_cache::FileDesc::fd"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache8FileDesc5getFdEv", "tensorrt_llm::executor::kv_cache::FileDesc::getFd"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache8FileDesc6getLenEv", "tensorrt_llm::executor::kv_cache::FileDesc::getLen"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc4mLenE", "tensorrt_llm::executor::kv_cache::FileDesc::mLen"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDescaSERK8FileDesc", "tensorrt_llm::executor::kv_cache::FileDesc::operator="], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDescaSERR8FileDesc", "tensorrt_llm::executor::kv_cache::FileDesc::operator="], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDescaSERR8FileDesc", "tensorrt_llm::executor::kv_cache::FileDesc::operator=::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDescD0Ev", "tensorrt_llm::executor::kv_cache::FileDesc::~FileDesc"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9FileDescsE", "tensorrt_llm::executor::kv_cache::FileDescs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9FileDescs9FileDescsERRNSt6vectorI8FileDescEE", "tensorrt_llm::executor::kv_cache::FileDescs::FileDescs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9FileDescs9FileDescsERRNSt6vectorI8FileDescEE", "tensorrt_llm::executor::kv_cache::FileDescs::FileDescs::descs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9FileDescs8getDescsEv", "tensorrt_llm::executor::kv_cache::FileDescs::getDescs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9FileDescs6mDescsE", "tensorrt_llm::executor::kv_cache::FileDescs::mDescs"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDescE", "tensorrt_llm::executor::kv_cache::MemoryDesc"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescE9uintptr_t6size_t8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescEPv6size_t8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescERKNSt6vectorIcEE8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescE9uintptr_t6size_t8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc::addr"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescEPv6size_t8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc::addr"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescE9uintptr_t6size_t8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc::deviceId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescEPv6size_t8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc::deviceId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescERKNSt6vectorIcEE8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc::deviceId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescE9uintptr_t6size_t8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc::len"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescEPv6size_t8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc::len"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescERKNSt6vectorIcEE8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc::vec"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc11deserializeERNSt7istreamE", "tensorrt_llm::executor::kv_cache::MemoryDesc::deserialize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc11deserializeERNSt7istreamE", "tensorrt_llm::executor::kv_cache::MemoryDesc::deserialize::is"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10MemoryDesc7getAddrEv", "tensorrt_llm::executor::kv_cache::MemoryDesc::getAddr"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10MemoryDesc11getDeviceIdEv", "tensorrt_llm::executor::kv_cache::MemoryDesc::getDeviceId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10MemoryDesc6getLenEv", "tensorrt_llm::executor::kv_cache::MemoryDesc::getLen"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc5mAddrE", "tensorrt_llm::executor::kv_cache::MemoryDesc::mAddr"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc9mDeviceIdE", "tensorrt_llm::executor::kv_cache::MemoryDesc::mDeviceId"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc4mLenE", "tensorrt_llm::executor::kv_cache::MemoryDesc::mLen"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc9serializeERK10MemoryDescRNSt7ostreamE", "tensorrt_llm::executor::kv_cache::MemoryDesc::serialize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc9serializeERK10MemoryDescRNSt7ostreamE", "tensorrt_llm::executor::kv_cache::MemoryDesc::serialize::memoryDesc"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc9serializeERK10MemoryDescRNSt7ostreamE", "tensorrt_llm::executor::kv_cache::MemoryDesc::serialize::os"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc14serializedSizeERK10MemoryDesc", "tensorrt_llm::executor::kv_cache::MemoryDesc::serializedSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc14serializedSizeERK10MemoryDesc", "tensorrt_llm::executor::kv_cache::MemoryDesc::serializedSize::memoryDesc"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11MemoryDescsE", "tensorrt_llm::executor::kv_cache::MemoryDescs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11MemoryDescs11MemoryDescsE10MemoryTypeNSt6vectorI10MemoryDescEE", "tensorrt_llm::executor::kv_cache::MemoryDescs::MemoryDescs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11MemoryDescs11MemoryDescsE10MemoryTypeNSt6vectorI10MemoryDescEE", "tensorrt_llm::executor::kv_cache::MemoryDescs::MemoryDescs::descs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11MemoryDescs11MemoryDescsE10MemoryTypeNSt6vectorI10MemoryDescEE", "tensorrt_llm::executor::kv_cache::MemoryDescs::MemoryDescs::type"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache11MemoryDescs8getDescsEv", "tensorrt_llm::executor::kv_cache::MemoryDescs::getDescs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache11MemoryDescs7getTypeEv", "tensorrt_llm::executor::kv_cache::MemoryDescs::getType"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11MemoryDescs6mDescsE", "tensorrt_llm::executor::kv_cache::MemoryDescs::mDescs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11MemoryDescs5mTypeE", "tensorrt_llm::executor::kv_cache::MemoryDescs::mType"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryTypeE", "tensorrt_llm::executor::kv_cache::MemoryType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryType4kBLKE", "tensorrt_llm::executor::kv_cache::MemoryType::kBLK"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryType5kDRAME", "tensorrt_llm::executor::kv_cache::MemoryType::kDRAM"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryType5kFILEE", "tensorrt_llm::executor::kv_cache::MemoryType::kFILE"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryType4kOBJE", "tensorrt_llm::executor::kv_cache::MemoryType::kOBJ"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryType5kVRAME", "tensorrt_llm::executor::kv_cache::MemoryType::kVRAM"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8MpiStateE", "tensorrt_llm::executor::kv_cache::MpiState"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8MpiState6mRanksE", "tensorrt_llm::executor::kv_cache::MpiState::mRanks"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache8MpiStateeqERK8MpiState", "tensorrt_llm::executor::kv_cache::MpiState::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache8MpiStateeqERK8MpiState", "tensorrt_llm::executor::kv_cache::MpiState::operator==::other"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache8MpiState8toStringEv", "tensorrt_llm::executor::kv_cache::MpiState::toString"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache13RegisterDescsE", "tensorrt_llm::executor::kv_cache::RegisterDescs"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11SocketStateE", "tensorrt_llm::executor::kv_cache::SocketState"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11SocketState3mIpE", "tensorrt_llm::executor::kv_cache::SocketState::mIp"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11SocketState5mPortE", "tensorrt_llm::executor::kv_cache::SocketState::mPort"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache11SocketStateeqERK11SocketState", "tensorrt_llm::executor::kv_cache::SocketState::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache11SocketStateeqERK11SocketState", "tensorrt_llm::executor::kv_cache::SocketState::operator==::other"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache11SocketState8toStringEv", "tensorrt_llm::executor::kv_cache::SocketState::toString"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11SyncMessageE", "tensorrt_llm::executor::kv_cache::SyncMessage"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache13TransferDescsE", "tensorrt_llm::executor::kv_cache::TransferDescs"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10TransferOpE", "tensorrt_llm::executor::kv_cache::TransferOp"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10TransferOp5kREADE", "tensorrt_llm::executor::kv_cache::TransferOp::kREAD"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10TransferOp6kWRITEE", "tensorrt_llm::executor::kv_cache::TransferOp::kWRITE"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequestE", "tensorrt_llm::executor::kv_cache::TransferRequest"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest15TransferRequestE10TransferOp13TransferDescs13TransferDescsRKNSt6stringENSt8optionalI11SyncMessageEE", "tensorrt_llm::executor::kv_cache::TransferRequest::TransferRequest"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest15TransferRequestE10TransferOp13TransferDescs13TransferDescsRKNSt6stringENSt8optionalI11SyncMessageEE", "tensorrt_llm::executor::kv_cache::TransferRequest::TransferRequest::dstDescs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest15TransferRequestE10TransferOp13TransferDescs13TransferDescsRKNSt6stringENSt8optionalI11SyncMessageEE", "tensorrt_llm::executor::kv_cache::TransferRequest::TransferRequest::op"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest15TransferRequestE10TransferOp13TransferDescs13TransferDescsRKNSt6stringENSt8optionalI11SyncMessageEE", "tensorrt_llm::executor::kv_cache::TransferRequest::TransferRequest::remoteName"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest15TransferRequestE10TransferOp13TransferDescs13TransferDescsRKNSt6stringENSt8optionalI11SyncMessageEE", "tensorrt_llm::executor::kv_cache::TransferRequest::TransferRequest::srcDescs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest15TransferRequestE10TransferOp13TransferDescs13TransferDescsRKNSt6stringENSt8optionalI11SyncMessageEE", "tensorrt_llm::executor::kv_cache::TransferRequest::TransferRequest::syncMessage"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache15TransferRequest11getDstDescsEv", "tensorrt_llm::executor::kv_cache::TransferRequest::getDstDescs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache15TransferRequest5getOpEv", "tensorrt_llm::executor::kv_cache::TransferRequest::getOp"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache15TransferRequest13getRemoteNameEv", "tensorrt_llm::executor::kv_cache::TransferRequest::getRemoteName"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache15TransferRequest11getSrcDescsEv", "tensorrt_llm::executor::kv_cache::TransferRequest::getSrcDescs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache15TransferRequest14getSyncMessageEv", "tensorrt_llm::executor::kv_cache::TransferRequest::getSyncMessage"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest9mDstDescsE", "tensorrt_llm::executor::kv_cache::TransferRequest::mDstDescs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest3mOpE", "tensorrt_llm::executor::kv_cache::TransferRequest::mOp"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest11mRemoteNameE", "tensorrt_llm::executor::kv_cache::TransferRequest::mRemoteName"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest9mSrcDescsE", "tensorrt_llm::executor::kv_cache::TransferRequest::mSrcDescs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest12mSyncMessageE", "tensorrt_llm::executor::kv_cache::TransferRequest::mSyncMessage"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache14TransferStatusE", "tensorrt_llm::executor::kv_cache::TransferStatus"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache14TransferStatus11isCompletedEv", "tensorrt_llm::executor::kv_cache::TransferStatus::isCompleted"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache14TransferStatus4waitEv", "tensorrt_llm::executor::kv_cache::TransferStatus::wait"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache14TransferStatusD0Ev", "tensorrt_llm::executor::kv_cache::TransferStatus::~TransferStatus"], [0, 3, 1, "_CPPv4IDpEN12tensorrt_llm8executor8kv_cache17makeLoopbackAgentENSt10shared_ptrI17BaseLoopbackAgentEERKNSt6stringEDpRR4Args", "tensorrt_llm::executor::kv_cache::makeLoopbackAgent"], [0, 8, 1, "_CPPv4IDpEN12tensorrt_llm8executor8kv_cache17makeLoopbackAgentENSt10shared_ptrI17BaseLoopbackAgentEERKNSt6stringEDpRR4Args", "tensorrt_llm::executor::kv_cache::makeLoopbackAgent::Args"], [0, 4, 1, "_CPPv4IDpEN12tensorrt_llm8executor8kv_cache17makeLoopbackAgentENSt10shared_ptrI17BaseLoopbackAgentEERKNSt6stringEDpRR4Args", "tensorrt_llm::executor::kv_cache::makeLoopbackAgent::args"], [0, 4, 1, "_CPPv4IDpEN12tensorrt_llm8executor8kv_cache17makeLoopbackAgentENSt10shared_ptrI17BaseLoopbackAgentEERKNSt6stringEDpRR4Args", "tensorrt_llm::executor::kv_cache::makeLoopbackAgent::backend"], [0, 3, 1, "_CPPv4IDpEN12tensorrt_llm8executor8kv_cache17makeTransferAgentENSt10unique_ptrI17BaseTransferAgentEERKNSt6stringEDpRR4Args", "tensorrt_llm::executor::kv_cache::makeTransferAgent"], [0, 8, 1, "_CPPv4IDpEN12tensorrt_llm8executor8kv_cache17makeTransferAgentENSt10unique_ptrI17BaseTransferAgentEERKNSt6stringEDpRR4Args", "tensorrt_llm::executor::kv_cache::makeTransferAgent::Args"], [0, 4, 1, "_CPPv4IDpEN12tensorrt_llm8executor8kv_cache17makeTransferAgentENSt10unique_ptrI17BaseTransferAgentEERKNSt6stringEDpRR4Args", "tensorrt_llm::executor::kv_cache::makeTransferAgent::args"], [0, 4, 1, "_CPPv4IDpEN12tensorrt_llm8executor8kv_cache17makeTransferAgentENSt10unique_ptrI17BaseTransferAgentEERKNSt6stringEDpRR4Args", "tensorrt_llm::executor::kv_cache::makeTransferAgent::backend"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executorlsERNSt7ostreamE21ContextChunkingPolicy", "tensorrt_llm::executor::operator<<"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executorlsERNSt7ostreamE23CapacitySchedulerPolicy", "tensorrt_llm::executor::operator<<"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executorlsERNSt7ostreamE21ContextChunkingPolicy", "tensorrt_llm::executor::operator<<::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executorlsERNSt7ostreamE23CapacitySchedulerPolicy", "tensorrt_llm::executor::operator<<::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executorlsERNSt7ostreamE21ContextChunkingPolicy", "tensorrt_llm::executor::operator<<::policy"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executorlsERNSt7ostreamE23CapacitySchedulerPolicy", "tensorrt_llm::executor::operator<<::policy"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7versionEv", "tensorrt_llm::executor::version"], [1, 1, 1, "_CPPv4N12tensorrt_llm6layersE", "tensorrt_llm::layers"], [0, 1, 1, "_CPPv4N12tensorrt_llm3mpiE", "tensorrt_llm::mpi"], [0, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [0, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffersE", "tensorrt_llm::runtime::AllReduceBuffers"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers16AllReduceBuffersE10SizeType3210SizeType3210SizeType3210SizeType32RK13BufferManagerRK11WorldConfigKb", "tensorrt_llm::runtime::AllReduceBuffers::AllReduceBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers16AllReduceBuffersE10SizeType3210SizeType3210SizeType3210SizeType32RK13BufferManagerRK11WorldConfigKb", "tensorrt_llm::runtime::AllReduceBuffers::AllReduceBuffers::fakeBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers16AllReduceBuffersE10SizeType3210SizeType3210SizeType3210SizeType32RK13BufferManagerRK11WorldConfigKb", "tensorrt_llm::runtime::AllReduceBuffers::AllReduceBuffers::hiddenSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers16AllReduceBuffersE10SizeType3210SizeType3210SizeType3210SizeType32RK13BufferManagerRK11WorldConfigKb", "tensorrt_llm::runtime::AllReduceBuffers::AllReduceBuffers::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers16AllReduceBuffersE10SizeType3210SizeType3210SizeType3210SizeType32RK13BufferManagerRK11WorldConfigKb", "tensorrt_llm::runtime::AllReduceBuffers::AllReduceBuffers::maxBatchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers16AllReduceBuffersE10SizeType3210SizeType3210SizeType3210SizeType32RK13BufferManagerRK11WorldConfigKb", "tensorrt_llm::runtime::AllReduceBuffers::AllReduceBuffers::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers16AllReduceBuffersE10SizeType3210SizeType3210SizeType3210SizeType32RK13BufferManagerRK11WorldConfigKb", "tensorrt_llm::runtime::AllReduceBuffers::AllReduceBuffers::maxSequenceLength"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers16AllReduceBuffersE10SizeType3210SizeType3210SizeType3210SizeType32RK13BufferManagerRK11WorldConfigKb", "tensorrt_llm::runtime::AllReduceBuffers::AllReduceBuffers::worldConfig"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers9TensorPtrE", "tensorrt_llm::runtime::AllReduceBuffers::TensorPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers18mAllReduceCommPtrsE", "tensorrt_llm::runtime::AllReduceBuffers::mAllReduceCommPtrs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers9mFlagPtrsE", "tensorrt_llm::runtime::AllReduceBuffers::mFlagPtrs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers17mIpcMemoryHandlesE", "tensorrt_llm::runtime::AllReduceBuffers::mIpcMemoryHandles"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime14BufferDataTypeE", "tensorrt_llm::runtime::BufferDataType"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType14BufferDataTypeEN8nvinfer18DataTypeEbb", "tensorrt_llm::runtime::BufferDataType::BufferDataType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType14BufferDataTypeEN8nvinfer18DataTypeEbb", "tensorrt_llm::runtime::BufferDataType::BufferDataType::_unsigned"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType14BufferDataTypeEN8nvinfer18DataTypeEbb", "tensorrt_llm::runtime::BufferDataType::BufferDataType::dataType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType14BufferDataTypeEN8nvinfer18DataTypeEbb", "tensorrt_llm::runtime::BufferDataType::BufferDataType::pointer"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType11getDataTypeEv", "tensorrt_llm::runtime::BufferDataType::getDataType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType7getSizeEv", "tensorrt_llm::runtime::BufferDataType::getSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType13getSizeInBitsEv", "tensorrt_llm::runtime::BufferDataType::getSizeInBits"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType9isPointerEv", "tensorrt_llm::runtime::BufferDataType::isPointer"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType10isUnsignedEv", "tensorrt_llm::runtime::BufferDataType::isUnsigned"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType15kTrtPointerTypeE", "tensorrt_llm::runtime::BufferDataType::kTrtPointerType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType9mDataTypeE", "tensorrt_llm::runtime::BufferDataType::mDataType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType8mPointerE", "tensorrt_llm::runtime::BufferDataType::mPointer"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType9mUnsignedE", "tensorrt_llm::runtime::BufferDataType::mUnsigned"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataTypecvN8nvinfer18DataTypeEEv", "tensorrt_llm::runtime::BufferDataType::operator nvinfer1::DataType"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManagerE", "tensorrt_llm::runtime::BufferManager"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager13BufferManagerE13CudaStreamPtrb", "tensorrt_llm::runtime::BufferManager::BufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager13BufferManagerE13CudaStreamPtrb", "tensorrt_llm::runtime::BufferManager::BufferManager::stream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager13BufferManagerE13CudaStreamPtrb", "tensorrt_llm::runtime::BufferManager::BufferManager::trimPool"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager14CudaMemPoolPtrE", "tensorrt_llm::runtime::BufferManager::CudaMemPoolPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager13CudaStreamPtrE", "tensorrt_llm::runtime::BufferManager::CudaStreamPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10IBufferPtrE", "tensorrt_llm::runtime::BufferManager::IBufferPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10ITensorPtrE", "tensorrt_llm::runtime::BufferManager::ITensorPtr"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::allocate"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeNSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::allocate"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::allocate::dims"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::allocate::memoryType"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeNSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::allocate::memoryType"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeNSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::allocate::size"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::allocate::type"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeNSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::allocate::type"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyEPKvR7IBuffer", "tensorrt_llm::runtime::BufferManager::copy"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyEPKvR7IBuffer10MemoryType", "tensorrt_llm::runtime::BufferManager::copy"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferPv", "tensorrt_llm::runtime::BufferManager::copy"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferPv10MemoryType", "tensorrt_llm::runtime::BufferManager::copy"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferR7IBuffer", "tensorrt_llm::runtime::BufferManager::copy"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyEPKvR7IBuffer", "tensorrt_llm::runtime::BufferManager::copy::dst"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyEPKvR7IBuffer10MemoryType", "tensorrt_llm::runtime::BufferManager::copy::dst"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferPv", "tensorrt_llm::runtime::BufferManager::copy::dst"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferPv10MemoryType", "tensorrt_llm::runtime::BufferManager::copy::dst"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferR7IBuffer", "tensorrt_llm::runtime::BufferManager::copy::dst"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferPv10MemoryType", "tensorrt_llm::runtime::BufferManager::copy::dstType"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyEPKvR7IBuffer", "tensorrt_llm::runtime::BufferManager::copy::src"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyEPKvR7IBuffer10MemoryType", "tensorrt_llm::runtime::BufferManager::copy::src"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferPv", "tensorrt_llm::runtime::BufferManager::copy::src"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferPv10MemoryType", "tensorrt_llm::runtime::BufferManager::copy::src"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferR7IBuffer", "tensorrt_llm::runtime::BufferManager::copy::src"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyEPKvR7IBuffer10MemoryType", "tensorrt_llm::runtime::BufferManager::copy::srcType"], [1, 3, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10IBufferPtrRKNSt6vectorI1TEE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom"], [1, 3, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrP1TN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom"], [1, 3, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrRKNSt6vectorI1TEEN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8copyFromERK7IBuffer10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8copyFromERK7ITensor10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom"], [1, 8, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10IBufferPtrRKNSt6vectorI1TEE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::T"], [1, 8, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrP1TN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::T"], [1, 8, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrRKNSt6vectorI1TEEN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::T"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrP1TN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::dims"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrRKNSt6vectorI1TEEN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::dims"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10IBufferPtrRKNSt6vectorI1TEE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::memoryType"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrP1TN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::memoryType"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrRKNSt6vectorI1TEEN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::memoryType"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8copyFromERK7IBuffer10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::memoryType"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8copyFromERK7ITensor10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::memoryType"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10IBufferPtrRKNSt6vectorI1TEE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::src"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrP1TN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::src"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrRKNSt6vectorI1TEEN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::src"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8copyFromERK7IBuffer10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::src"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8copyFromERK7ITensor10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::src"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager3cpuEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::cpu"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager3cpuENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::cpu"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager3cpuEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::cpu::dims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager3cpuENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::cpu::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager3cpuEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::cpu::type"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager3cpuENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::cpu::type"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager11emptyBufferE10MemoryTypeN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::emptyBuffer"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager11emptyBufferE10MemoryTypeN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::emptyBuffer::memoryType"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager11emptyBufferE10MemoryTypeN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::emptyBuffer::type"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager11emptyTensorE10MemoryTypeN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::emptyTensor"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager11emptyTensorE10MemoryTypeN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::emptyTensor::memoryType"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager11emptyTensorE10MemoryTypeN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::emptyTensor::type"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager9getStreamEv", "tensorrt_llm::runtime::BufferManager::getStream"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager3gpuEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpu"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager3gpuENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpu"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager3gpuEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpu::dims"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager3gpuENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpu::size"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager3gpuEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpu::type"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager3gpuENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpu::type"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7gpuSyncEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpuSync"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7gpuSyncENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpuSync"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7gpuSyncEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpuSync::dims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7gpuSyncENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpuSync::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7gpuSyncEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpuSync::type"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7gpuSyncENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpuSync::type"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7ipcNvlsENSt3setIiEEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::ipcNvls"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7ipcNvlsENSt3setIiEEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::ipcNvls::dims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7ipcNvlsENSt3setIiEEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::ipcNvls::ranks"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7ipcNvlsENSt3setIiEEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::ipcNvls::type"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10kBYTE_TYPEE", "tensorrt_llm::runtime::BufferManager::kBYTE_TYPE"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager5mPoolE", "tensorrt_llm::runtime::BufferManager::mPool"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7mStreamE", "tensorrt_llm::runtime::BufferManager::mStream"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager9mTrimPoolE", "tensorrt_llm::runtime::BufferManager::mTrimPool"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7managedEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::managed"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7managedENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::managed"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7managedEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::managed::dims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7managedENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::managed::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7managedEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::managed::type"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7managedENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::managed::type"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager14memoryPoolFreeEv", "tensorrt_llm::runtime::BufferManager::memoryPoolFree"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager18memoryPoolReservedEv", "tensorrt_llm::runtime::BufferManager::memoryPoolReserved"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager16memoryPoolTrimToENSt6size_tE", "tensorrt_llm::runtime::BufferManager::memoryPoolTrimTo"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager16memoryPoolTrimToENSt6size_tE", "tensorrt_llm::runtime::BufferManager::memoryPoolTrimTo::size"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager14memoryPoolUsedEv", "tensorrt_llm::runtime::BufferManager::memoryPoolUsed"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager6pinnedEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinned"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager6pinnedENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinned"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager6pinnedEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinned::dims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager6pinnedENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinned::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager6pinnedEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinned::type"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager6pinnedENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinned::type"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10pinnedPoolEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinnedPool"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10pinnedPoolENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinnedPool"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10pinnedPoolEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinnedPool::dims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10pinnedPoolENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinnedPool::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10pinnedPoolEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinnedPool::type"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10pinnedPoolENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinnedPool::type"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager6setMemER7IBuffer7int32_t", "tensorrt_llm::runtime::BufferManager::setMem"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager6setMemER7IBuffer7int32_t", "tensorrt_llm::runtime::BufferManager::setMem::buffer"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager6setMemER7IBuffer7int32_t", "tensorrt_llm::runtime::BufferManager::setMem::value"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager7setZeroER7IBuffer", "tensorrt_llm::runtime::BufferManager::setZero"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager7setZeroER7IBuffer", "tensorrt_llm::runtime::BufferManager::setZero::buffer"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManagerD0Ev", "tensorrt_llm::runtime::BufferManager::~BufferManager"], [1, 2, 1, "_CPPv4I0EN12tensorrt_llm7runtime11BufferRangeE", "tensorrt_llm::runtime::BufferRange"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime11BufferRange4BaseE", "tensorrt_llm::runtime::BufferRange::Base"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI1UEEbEEEN12tensorrt_llm7runtime11BufferRange11BufferRangeERK7IBuffer", "tensorrt_llm::runtime::BufferRange::BufferRange"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tIXntNSt10is_const_vI1UEEEbEEEN12tensorrt_llm7runtime11BufferRange11BufferRangeER7IBuffer", "tensorrt_llm::runtime::BufferRange::BufferRange"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11BufferRange11BufferRangeEP1T9size_type", "tensorrt_llm::runtime::BufferRange::BufferRange"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI1UEEbEEEN12tensorrt_llm7runtime11BufferRange11BufferRangeERK7IBuffer", "tensorrt_llm::runtime::BufferRange::BufferRange::U"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tIXntNSt10is_const_vI1UEEEbEEEN12tensorrt_llm7runtime11BufferRange11BufferRangeER7IBuffer", "tensorrt_llm::runtime::BufferRange::BufferRange::U"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI1UEEbEEEN12tensorrt_llm7runtime11BufferRange11BufferRangeERK7IBuffer", "tensorrt_llm::runtime::BufferRange::BufferRange::buffer"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tIXntNSt10is_const_vI1UEEEbEEEN12tensorrt_llm7runtime11BufferRange11BufferRangeER7IBuffer", "tensorrt_llm::runtime::BufferRange::BufferRange::buffer"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11BufferRange11BufferRangeEP1T9size_type", "tensorrt_llm::runtime::BufferRange::BufferRange::data"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11BufferRange11BufferRangeEP1T9size_type", "tensorrt_llm::runtime::BufferRange::BufferRange::size"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime11BufferRangeE", "tensorrt_llm::runtime::BufferRange::T"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunkE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkERK22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::CUDAVirtualMemoryChunk"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkERR10CreatorPtrRR13Configurators", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::CUDAVirtualMemoryChunk"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkERR22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::CUDAVirtualMemoryChunk"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkEv", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::CUDAVirtualMemoryChunk"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkERR10CreatorPtrRR13Configurators", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::CUDAVirtualMemoryChunk::configurators"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkERR10CreatorPtrRR13Configurators", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::CUDAVirtualMemoryChunk::creator"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkERR22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::CUDAVirtualMemoryChunk::other"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12ConfiguratorE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator12ConfiguratorERK12Configurator", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::Configurator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator12ConfiguratorERR12Configurator", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::Configurator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator12ConfiguratorEv", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::Configurator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12ConfiguratoraSERK12Configurator", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::operator="], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12ConfiguratoraSERR12Configurator", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::operator="], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator5setupE28CUmemGenericAllocationHandle", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::setup"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator5setupE28CUmemGenericAllocationHandle", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::setup::handle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator8teardownE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::teardown"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator8teardownE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::teardown::destructing"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator8teardownE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::teardown::handle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12ConfiguratorD0Ev", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::~Configurator"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk15ConfiguratorPtrE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::ConfiguratorPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk13ConfiguratorsE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurators"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7CreatorE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7CreatorERK7Creator", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::Creator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7CreatorERR7Creator", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::Creator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7CreatorEv", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::Creator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator6createEv", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::create"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7CreatoraSERK7Creator", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::operator="], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7CreatoraSERR7Creator", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::operator="], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7releaseE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::release"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7releaseE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::release::destructing"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7releaseE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::release::handle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7CreatorD0Ev", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::~Creator"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk10CreatorPtrE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::CreatorPtr"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status7ERROREDE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::ERRORED"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status7INVALIDE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::INVALID"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk13INVALID_STATEE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::INVALID_STATE"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status12MATERIALIZEDE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::MATERIALIZED"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status8RELEASEDE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::RELEASED"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6StatusE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Status"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status7ERROREDE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Status::ERRORED"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status7INVALIDE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Status::INVALID"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status12MATERIALIZEDE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Status::MATERIALIZED"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status8RELEASEDE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Status::RELEASED"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk8_releaseEb", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::_release"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk8_releaseEb", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::_release::destructing"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk14mConfiguratorsE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::mConfigurators"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk8mCreatorE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::mCreator"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7mHandleE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::mHandle"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6mStateE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::mState"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk11materializeEv", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::materialize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime22CUDAVirtualMemoryChunkcvbEv", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::operator bool"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunkaSERK22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::operator="], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunkaSERR22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::operator="], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunkaSERR22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::operator=::other"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7releaseEv", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::release"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6statusEv", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::status"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunkD0Ev", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::~CUDAVirtualMemoryChunk"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime15CacheSaltIDTypeE", "tensorrt_llm::runtime::CacheSaltIDType"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEventE", "tensorrt_llm::runtime::CudaEvent"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent9CudaEventE7pointerb", "tensorrt_llm::runtime::CudaEvent::CudaEvent"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent9CudaEventEj", "tensorrt_llm::runtime::CudaEvent::CudaEvent"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent9CudaEventE7pointerb", "tensorrt_llm::runtime::CudaEvent::CudaEvent::event"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent9CudaEventEj", "tensorrt_llm::runtime::CudaEvent::CudaEvent::flags"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent9CudaEventE7pointerb", "tensorrt_llm::runtime::CudaEvent::CudaEvent::ownsEvent"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7DeleterE", "tensorrt_llm::runtime::CudaEvent::Deleter"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7Deleter7DeleterEb", "tensorrt_llm::runtime::CudaEvent::Deleter::Deleter"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7Deleter7DeleterEv", "tensorrt_llm::runtime::CudaEvent::Deleter::Deleter"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7Deleter7DeleterEb", "tensorrt_llm::runtime::CudaEvent::Deleter::Deleter::ownsEvent"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7Deleter10mOwnsEventE", "tensorrt_llm::runtime::CudaEvent::Deleter::mOwnsEvent"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9CudaEvent7DeleterclE7pointer", "tensorrt_llm::runtime::CudaEvent::Deleter::operator()"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9CudaEvent7DeleterclE7pointer", "tensorrt_llm::runtime::CudaEvent::Deleter::operator()::event"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent8EventPtrE", "tensorrt_llm::runtime::CudaEvent::EventPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent12element_typeE", "tensorrt_llm::runtime::CudaEvent::element_type"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9CudaEvent3getEv", "tensorrt_llm::runtime::CudaEvent::get"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent6mEventE", "tensorrt_llm::runtime::CudaEvent::mEvent"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7pointerE", "tensorrt_llm::runtime::CudaEvent::pointer"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9CudaEvent11synchronizeEv", "tensorrt_llm::runtime::CudaEvent::synchronize"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStreamE", "tensorrt_llm::runtime::CudaStream"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamE12cudaStream_t", "tensorrt_llm::runtime::CudaStream::CudaStream"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamE12cudaStream_tib", "tensorrt_llm::runtime::CudaStream::CudaStream"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamEji", "tensorrt_llm::runtime::CudaStream::CudaStream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamE12cudaStream_tib", "tensorrt_llm::runtime::CudaStream::CudaStream::device"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamEji", "tensorrt_llm::runtime::CudaStream::CudaStream::flags"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamE12cudaStream_tib", "tensorrt_llm::runtime::CudaStream::CudaStream::ownsStream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamEji", "tensorrt_llm::runtime::CudaStream::CudaStream::priority"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamE12cudaStream_t", "tensorrt_llm::runtime::CudaStream::CudaStream::stream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamE12cudaStream_tib", "tensorrt_llm::runtime::CudaStream::CudaStream::stream"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7DeleterE", "tensorrt_llm::runtime::CudaStream::Deleter"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7Deleter7DeleterEb", "tensorrt_llm::runtime::CudaStream::Deleter::Deleter"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7Deleter7DeleterEv", "tensorrt_llm::runtime::CudaStream::Deleter::Deleter"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7Deleter7DeleterEb", "tensorrt_llm::runtime::CudaStream::Deleter::Deleter::ownsStream"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7Deleter11mOwnsStreamE", "tensorrt_llm::runtime::CudaStream::Deleter::mOwnsStream"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream7DeleterclE12cudaStream_t", "tensorrt_llm::runtime::CudaStream::Deleter::operator()"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream7DeleterclE12cudaStream_t", "tensorrt_llm::runtime::CudaStream::Deleter::operator()::stream"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream9StreamPtrE", "tensorrt_llm::runtime::CudaStream::StreamPtr"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream3getEv", "tensorrt_llm::runtime::CudaStream::get"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream9getDeviceEv", "tensorrt_llm::runtime::CudaStream::getDevice"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7mDeviceE", "tensorrt_llm::runtime::CudaStream::mDevice"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7mStreamE", "tensorrt_llm::runtime::CudaStream::mStream"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream6recordEN9CudaEvent7pointerE", "tensorrt_llm::runtime::CudaStream::record"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream6recordERK9CudaEvent", "tensorrt_llm::runtime::CudaStream::record"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream6recordEN9CudaEvent7pointerE", "tensorrt_llm::runtime::CudaStream::record::event"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream6recordERK9CudaEvent", "tensorrt_llm::runtime::CudaStream::record::event"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream11synchronizeEv", "tensorrt_llm::runtime::CudaStream::synchronize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream4waitEN9CudaEvent7pointerE", "tensorrt_llm::runtime::CudaStream::wait"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream4waitERK9CudaEvent", "tensorrt_llm::runtime::CudaStream::wait"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream4waitEN9CudaEvent7pointerE", "tensorrt_llm::runtime::CudaStream::wait::event"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream4waitERK9CudaEvent", "tensorrt_llm::runtime::CudaStream::wait::event"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocatorE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode3CPUE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::CPU"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13ConfigurationE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtr", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtrb", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtr", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration::backStream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtrb", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration::backStream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtrb", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration::background"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtr", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtrb", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtr", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration::mode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtrb", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration::mode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtr", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration::tag"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtrb", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration::tag"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration23backgroundConfigurationE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::backgroundConfiguration"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration11mBackStreamE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::mBackStream"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration11mBackgroundE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::mBackground"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration8mManagerE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::mManager"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration5mModeE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::mMode"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration9mPageSizeE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::mPageSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration4mTagE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::mTag"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration11pageAlignedENSt6size_tE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::pageAligned"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration11pageAlignedENSt6size_tE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::pageAligned::n"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration25setVirtualMemoryAllocatorERKNSt6stringE11RestoreModeNSt10shared_ptrI10CudaStreamEE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::setVirtualMemoryAllocator"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration25setVirtualMemoryAllocatorERKNSt6stringE11RestoreModeNSt10shared_ptrI10CudaStreamEE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::setVirtualMemoryAllocator::backStream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration25setVirtualMemoryAllocatorERKNSt6stringE11RestoreModeNSt10shared_ptrI10CudaStreamEE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::setVirtualMemoryAllocator::mode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration25setVirtualMemoryAllocatorERKNSt6stringE11RestoreModeNSt10shared_ptrI10CudaStreamEE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::setVirtualMemoryAllocator::tag"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13CudaStreamPtrE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::CudaStreamPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator26CudaVirtualMemoryAllocatorENSt10shared_ptrI13ConfigurationEE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::CudaVirtualMemoryAllocator"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator26CudaVirtualMemoryAllocatorENSt10shared_ptrI13ConfigurationEE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::CudaVirtualMemoryAllocator::config"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode6MEMSETE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::MEMSET"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode4NONEE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::NONE"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode6PINNEDE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::PINNED"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator7PointerE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Pointer"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreModeE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::RestoreMode"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode3CPUE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::RestoreMode::CPU"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode6MEMSETE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::RestoreMode::MEMSET"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode4NONEE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::RestoreMode::NONE"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode6PINNEDE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::RestoreMode::PINNED"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator8allocateEP7PointerNSt6size_tEi", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::allocate"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator8allocateEP7PointerNSt6size_tEi", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::allocate::device"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator8allocateEP7PointerNSt6size_tEi", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::allocate::n"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator8allocateEP7PointerNSt6size_tEi", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::allocate::ptr"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator10deallocateE7PointerNSt6size_tE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::deallocate"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator10deallocateE7PointerNSt6size_tE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::deallocate::n"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator10deallocateE7PointerNSt6size_tE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::deallocate::ptr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator7mConfigE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::mConfig"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocatorcvbEv", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::operator bool"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManagerE", "tensorrt_llm::runtime::CudaVirtualMemoryManager"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager5EntryE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::Entry"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager5Entry8mEntryItE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::Entry::mEntryIt"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager5Entry7mMemoryE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::Entry::mMemory"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager16PointerMemoryMapE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::PointerMemoryMap"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager11TagEntryMapE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::TagEntryMap"], [1, 3, 1, "_CPPv4IDpEN12tensorrt_llm7runtime24CudaVirtualMemoryManager3addEv9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrEDpRR13Configurators", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERR22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrERRN22CUDAVirtualMemoryChunk13ConfiguratorsE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add"], [1, 8, 1, "_CPPv4IDpEN12tensorrt_llm7runtime24CudaVirtualMemoryManager3addEv9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrEDpRR13Configurators", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::Configurators"], [1, 4, 1, "_CPPv4IDpEN12tensorrt_llm7runtime24CudaVirtualMemoryManager3addEv9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrEDpRR13Configurators", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::configurators"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrERRN22CUDAVirtualMemoryChunk13ConfiguratorsE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::configurators"], [1, 4, 1, "_CPPv4IDpEN12tensorrt_llm7runtime24CudaVirtualMemoryManager3addEv9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrEDpRR13Configurators", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::creator"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrERRN22CUDAVirtualMemoryChunk13ConfiguratorsE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::creator"], [1, 4, 1, "_CPPv4IDpEN12tensorrt_llm7runtime24CudaVirtualMemoryManager3addEv9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrEDpRR13Configurators", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::handle"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERR22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::handle"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrERRN22CUDAVirtualMemoryChunk13ConfiguratorsE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::handle"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERR22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::memory"], [1, 4, 1, "_CPPv4IDpEN12tensorrt_llm7runtime24CudaVirtualMemoryManager3addEv9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrEDpRR13Configurators", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::tag"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERR22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::tag"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrERRN22CUDAVirtualMemoryChunk13ConfiguratorsE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::tag"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager12addBadHandleE9uintptr_t", "tensorrt_llm::runtime::CudaVirtualMemoryManager::addBadHandle"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager12addBadHandleE9uintptr_t", "tensorrt_llm::runtime::CudaVirtualMemoryManager::addBadHandle::handle"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager11mBadHandlesE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::mBadHandles"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager8mEntriesE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::mEntries"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager9mMemoriesE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::mMemories"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager6mMutexE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::mMutex"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager18materializeWithTagERKNSt6stringE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::materializeWithTag"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager18materializeWithTagERKNSt6stringE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::materializeWithTag::tag"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager14releaseWithTagERKNSt6stringE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::releaseWithTag"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager14releaseWithTagERKNSt6stringE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::releaseWithTag::tag"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager6removeE9uintptr_t", "tensorrt_llm::runtime::CudaVirtualMemoryManager::remove"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager6removeE9uintptr_t", "tensorrt_llm::runtime::CudaVirtualMemoryManager::remove::handle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager18retrieveBadHandlesEv", "tensorrt_llm::runtime::CudaVirtualMemoryManager::retrieveBadHandles"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager12unsafeRemoveE9uintptr_t", "tensorrt_llm::runtime::CudaVirtualMemoryManager::unsafeRemove"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager12unsafeRemoveE9uintptr_t", "tensorrt_llm::runtime::CudaVirtualMemoryManager::unsafeRemove::handle"], [1, 2, 1, "_CPPv4I_N8nvinfer18DataTypeE_b_bEN12tensorrt_llm7runtime14DataTypeTraitsE", "tensorrt_llm::runtime::DataTypeTraits"], [1, 8, 1, "_CPPv4I_N8nvinfer18DataTypeE_b_bEN12tensorrt_llm7runtime14DataTypeTraitsE", "tensorrt_llm::runtime::DataTypeTraits::kDataType"], [1, 8, 1, "_CPPv4I_N8nvinfer18DataTypeE_b_bEN12tensorrt_llm7runtime14DataTypeTraitsE", "tensorrt_llm::runtime::DataTypeTraits::kIsPointer"], [1, 8, 1, "_CPPv4I_N8nvinfer18DataTypeE_b_bEN12tensorrt_llm7runtime14DataTypeTraitsE", "tensorrt_llm::runtime::DataTypeTraits::kIsUnsigned"], [1, 2, 1, "_CPPv4I_N8nvinfer18DataTypeE_bEN12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEEE", "tensorrt_llm::runtime::DataTypeTraits<kDataType, kUnsigned, true>"], [1, 8, 1, "_CPPv4I_N8nvinfer18DataTypeE_bEN12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEEE", "tensorrt_llm::runtime::DataTypeTraits<kDataType, kUnsigned, true>::kDataType"], [1, 8, 1, "_CPPv4I_N8nvinfer18DataTypeE_bEN12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEEE", "tensorrt_llm::runtime::DataTypeTraits<kDataType, kUnsigned, true>::kUnsigned"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEE4nameE", "tensorrt_llm::runtime::DataTypeTraits<kDataType, kUnsigned, true>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<kDataType, kUnsigned, true>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEE4typeE", "tensorrt_llm::runtime::DataTypeTraits<kDataType, kUnsigned, true>::type"], [1, 2, 1, "_CPPv4I_bEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kBOOLE9kUnsignedEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kBOOL, kUnsigned>"], [1, 8, 1, "_CPPv4I_bEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kBOOLE9kUnsignedEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kBOOL, kUnsigned>::kUnsigned"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kBOOLE9kUnsignedE4nameE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kBOOL, kUnsigned>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kBOOLE9kUnsignedE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kBOOL, kUnsigned>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kBOOLE9kUnsignedE4typeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kBOOL, kUnsigned>::type"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kFLOATEEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kFLOAT>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kFLOATEE4nameE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kFLOAT>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kFLOATEE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kFLOAT>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kFLOATEE4typeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kFLOAT>::type"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kHALFEEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kHALF>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kHALFEE4nameE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kHALF>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kHALFEE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kHALF>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kHALFEE4typeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kHALF>::type"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EXL1EEEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT32, true>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EXL1EEE4nameE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT32, true>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EXL1EEE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT32, true>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EXL1EEE4typeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT32, true>::type"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT32>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EE4nameE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT32>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT32>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EE4typeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT32>::type"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EXL1EEEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT64, true>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EXL1EEE4nameE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT64, true>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EXL1EEE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT64, true>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EXL1EEE4typeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT64, true>::type"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT64>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EE4nameE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT64>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT64>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EE4typeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT64>::type"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kINT8EEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT8>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kINT8EE4nameE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT8>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kINT8EE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT8>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kINT8EE4typeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT8>::type"], [1, 2, 1, "_CPPv4I_bEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kUINT8E9kUnsignedEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kUINT8, kUnsigned>"], [1, 8, 1, "_CPPv4I_bEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kUINT8E9kUnsignedEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kUINT8, kUnsigned>::kUnsigned"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kUINT8E9kUnsignedE4nameE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kUINT8, kUnsigned>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kUINT8E9kUnsignedE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kUINT8, kUnsigned>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kUINT8E9kUnsignedE4typeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kUINT8, kUnsigned>::type"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInputE", "tensorrt_llm::runtime::DecodingInput"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13DecodingInputEv", "tensorrt_llm::runtime::DecodingInput::DecodingInput"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputsE", "tensorrt_llm::runtime::DecodingInput::EagleInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs12acceptedLensE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::acceptedLens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs15acceptedPathIdsE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::acceptedPathIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs14acceptedTokensE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::acceptedTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs24chunkedContextNextTokensE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::chunkedContextNextTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs13lastDraftLensE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::lastDraftLens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs14lastDraftPathsE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::lastDraftPaths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs15lastDraftTokensE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::lastDraftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs13nextDraftLensE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::nextDraftLens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs14nextDraftPathsE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::nextDraftPaths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs15nextDraftTokensE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::nextDraftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs8seqSlotsE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::seqSlots"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputsE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs15bestPathIndicesE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::bestPathIndices"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs15bestPathLengthsE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::bestPathLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs16lastDraftIndicesE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::lastDraftIndices"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs15lastDraftTokensE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::lastDraftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs21lastGenerationLengthsE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::lastGenerationLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs19lastPositionIdsBaseE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::lastPositionIdsBase"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs5masksE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::masks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs18maxGenLengthDeviceE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::maxGenLengthDevice"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs16nextDraftIndicesE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::nextDraftIndices"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs14nextDraftProbsE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::nextDraftProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs15nextDraftTokensE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::nextDraftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs14nextFlatTokensE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::nextFlatTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs21nextGenerationLengthsE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::nextGenerationLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs17packedPositionIdsE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::packedPositionIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs8seqSlotsE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::seqSlots"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputsE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs17constantThresholdE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::constantThreshold"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs11draftLogitsE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::draftLogits"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs15draftLogitsHostE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::draftLogitsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs10draftProbsE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::draftProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs13draftTokenIdsE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::draftTokenIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs17draftTokenIdsHostE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::draftTokenIdsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs14numDraftTokensE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::numDraftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs18numDraftTokensHostE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::numDraftTokensHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs4stepE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::step"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs11targetProbsE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::targetProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs14useDraftLogitsE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::useDraftLogits"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs18useDraftLogitsHostE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::useDraftLogitsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs28useRandomAcceptanceThresholdE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::useRandomAcceptanceThreshold"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15LookaheadInputsE", "tensorrt_llm::runtime::DecodingInput::LookaheadInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15LookaheadInputs13tokensPerStepE", "tensorrt_llm::runtime::DecodingInput::LookaheadInputs::tokensPerStep"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputsE", "tensorrt_llm::runtime::DecodingInput::MedusaInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs22medusaCurTokensPerStepE", "tensorrt_llm::runtime::DecodingInput::MedusaInputs::medusaCurTokensPerStep"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs12medusaLogitsE", "tensorrt_llm::runtime::DecodingInput::MedusaInputs::medusaLogits"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs11medusaPathsE", "tensorrt_llm::runtime::DecodingInput::MedusaInputs::medusaPaths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs25medusaTargetTokensPerStepE", "tensorrt_llm::runtime::DecodingInput::MedusaInputs::medusaTargetTokensPerStep"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs13medusaTreeIdsE", "tensorrt_llm::runtime::DecodingInput::MedusaInputs::medusaTreeIds"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput14TensorConstPtrE", "tensorrt_llm::runtime::DecodingInput::TensorConstPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput9TensorPtrE", "tensorrt_llm::runtime::DecodingInput::TensorPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12badWordsLensE", "tensorrt_llm::runtime::DecodingInput::badWordsLens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13badWordsListsE", "tensorrt_llm::runtime::DecodingInput::badWordsLists"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12badWordsPtrsE", "tensorrt_llm::runtime::DecodingInput::badWordsPtrs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput9batchSizeE", "tensorrt_llm::runtime::DecodingInput::batchSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput10batchSlotsE", "tensorrt_llm::runtime::DecodingInput::batchSlots"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput10beamWidthsE", "tensorrt_llm::runtime::DecodingInput::beamWidths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput16cacheIndirectionE", "tensorrt_llm::runtime::DecodingInput::cacheIndirection"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11eagleInputsE", "tensorrt_llm::runtime::DecodingInput::eagleInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13embeddingBiasE", "tensorrt_llm::runtime::DecodingInput::embeddingBias"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput6endIdsE", "tensorrt_llm::runtime::DecodingInput::endIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25explicitDraftTokensInputsE", "tensorrt_llm::runtime::DecodingInput::explicitDraftTokensInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25externalDraftTokensInputsE", "tensorrt_llm::runtime::DecodingInput::externalDraftTokensInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13finishReasonsE", "tensorrt_llm::runtime::DecodingInput::finishReasons"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15generationStepsE", "tensorrt_llm::runtime::DecodingInput::generationSteps"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput7lengthsE", "tensorrt_llm::runtime::DecodingInput::lengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput9logitsVecE", "tensorrt_llm::runtime::DecodingInput::logitsVec"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15lookaheadInputsE", "tensorrt_llm::runtime::DecodingInput::lookaheadInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput18maxAttentionWindowE", "tensorrt_llm::runtime::DecodingInput::maxAttentionWindow"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput14maxBadWordsLenE", "tensorrt_llm::runtime::DecodingInput::maxBadWordsLen"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput9maxLengthE", "tensorrt_llm::runtime::DecodingInput::maxLength"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15maxStopWordsLenE", "tensorrt_llm::runtime::DecodingInput::maxStopWordsLen"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12medusaInputsE", "tensorrt_llm::runtime::DecodingInput::medusaInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput17noRepeatNgramSizeE", "tensorrt_llm::runtime::DecodingInput::noRepeatNgramSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput19sequenceLimitLengthE", "tensorrt_llm::runtime::DecodingInput::sequenceLimitLength"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15sinkTokenLengthE", "tensorrt_llm::runtime::DecodingInput::sinkTokenLength"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput4stepE", "tensorrt_llm::runtime::DecodingInput::step"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13stopWordsLensE", "tensorrt_llm::runtime::DecodingInput::stopWordsLens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput14stopWordsListsE", "tensorrt_llm::runtime::DecodingInput::stopWordsLists"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13stopWordsPtrsE", "tensorrt_llm::runtime::DecodingInput::stopWordsPtrs"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutputE", "tensorrt_llm::runtime::DecodingOutput"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypothesesE", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses10batchDonesE", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::batchDones"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses14cumLogProbsCBAE", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::cumLogProbsCBA"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses5emptyERK13BufferManager", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::empty"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses5emptyERK13BufferManager", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::empty::manager"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses4initERK13BufferManager11TokenIdType", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::init"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses4initERK13BufferManager11TokenIdType", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::init::endId"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses4initERK13BufferManager11TokenIdType", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::init::manager"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses11logProbsCBAE", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::logProbsCBA"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses18minNormedScoresCBAE", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::minNormedScoresCBA"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses15normedScoresCBAE", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::normedScoresCBA"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses11numBeamsCBAE", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::numBeamsCBA"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses12outputIdsCBAE", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::outputIdsCBA"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses7releaseEv", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::release"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses7reshapeE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::reshape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses7reshapeE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::reshape::batchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses7reshapeE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::reshape::beamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses7reshapeE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::reshape::maxSequenceLength"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses18sequenceLengthsCBAE", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::sequenceLengthsCBA"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses5sliceE10SizeType3210SizeType32", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::slice"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses5sliceE10SizeType3210SizeType32", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::slice::batchIndex"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses5sliceE10SizeType3210SizeType32", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::slice::size"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14DecodingOutputEv", "tensorrt_llm::runtime::DecodingOutput::DecodingOutput"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputsE", "tensorrt_llm::runtime::DecodingOutput::SpeculativeDecodingOutputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs21acceptedLengthsCumSumE", "tensorrt_llm::runtime::DecodingOutput::SpeculativeDecodingOutputs::acceptedLengthsCumSum"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs17acceptedTokensLenE", "tensorrt_llm::runtime::DecodingOutput::SpeculativeDecodingOutputs::acceptedTokensLen"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs15nextDraftTokensE", "tensorrt_llm::runtime::DecodingOutput::SpeculativeDecodingOutputs::nextDraftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs18nextDraftTokensLenE", "tensorrt_llm::runtime::DecodingOutput::SpeculativeDecodingOutputs::nextDraftTokensLen"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs12pathsOffsetsE", "tensorrt_llm::runtime::DecodingOutput::SpeculativeDecodingOutputs::pathsOffsets"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs18prevDraftTokensLenE", "tensorrt_llm::runtime::DecodingOutput::SpeculativeDecodingOutputs::prevDraftTokensLen"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput9TensorPtrE", "tensorrt_llm::runtime::DecodingOutput::TensorPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14beamHypothesesE", "tensorrt_llm::runtime::DecodingOutput::beamHypotheses"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput16cacheIndirectionE", "tensorrt_llm::runtime::DecodingOutput::cacheIndirection"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput11cumLogProbsE", "tensorrt_llm::runtime::DecodingOutput::cumLogProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput12eagleBuffersE", "tensorrt_llm::runtime::DecodingOutput::eagleBuffers"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26explicitDraftTokensBuffersE", "tensorrt_llm::runtime::DecodingOutput::explicitDraftTokensBuffers"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput13finishReasonsE", "tensorrt_llm::runtime::DecodingOutput::finishReasons"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput11finishedSumE", "tensorrt_llm::runtime::DecodingOutput::finishedSum"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput11gatheredIdsE", "tensorrt_llm::runtime::DecodingOutput::gatheredIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput3idsE", "tensorrt_llm::runtime::DecodingOutput::ids"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput17kNegativeInfinityE", "tensorrt_llm::runtime::DecodingOutput::kNegativeInfinity"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput7lengthsE", "tensorrt_llm::runtime::DecodingOutput::lengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput8logProbsE", "tensorrt_llm::runtime::DecodingOutput::logProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput13logProbsTiledE", "tensorrt_llm::runtime::DecodingOutput::logProbsTiled"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput16lookaheadOutputsE", "tensorrt_llm::runtime::DecodingOutput::lookaheadOutputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput9newTokensE", "tensorrt_llm::runtime::DecodingOutput::newTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14newTokensStepsE", "tensorrt_llm::runtime::DecodingOutput::newTokensSteps"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput12newTokensVecE", "tensorrt_llm::runtime::DecodingOutput::newTokensVec"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput9parentIdsE", "tensorrt_llm::runtime::DecodingOutput::parentIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26speculativeDecodingOutputsE", "tensorrt_llm::runtime::DecodingOutput::speculativeDecodingOutputs"], [1, 2, 1, "_CPPv4I0EN12tensorrt_llm7runtime20DeviceAllocationNvlsE", "tensorrt_llm::runtime::DeviceAllocationNvls"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls20DeviceAllocationNvlsEv", "tensorrt_llm::runtime::DeviceAllocationNvls::DeviceAllocationNvls"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime20DeviceAllocationNvlsE", "tensorrt_llm::runtime::DeviceAllocationNvls::T"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls9_capacityE", "tensorrt_llm::runtime::DeviceAllocationNvls::_capacity"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls7_handleE", "tensorrt_llm::runtime::DeviceAllocationNvls::_handle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls4freeEv", "tensorrt_llm::runtime::DeviceAllocationNvls::free"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime20DeviceAllocationNvls11getCapacityEv", "tensorrt_llm::runtime::DeviceAllocationNvls::getCapacity"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls21getIpcUnicastPointersEv", "tensorrt_llm::runtime::DeviceAllocationNvls::getIpcUnicastPointers"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime20DeviceAllocationNvls19getMulticastPointerEv", "tensorrt_llm::runtime::DeviceAllocationNvls::getMulticastPointer"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime20DeviceAllocationNvls17getUnicastPointerEv", "tensorrt_llm::runtime::DeviceAllocationNvls::getUnicastPointer"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls5resetE6size_tNSt3setIiEE", "tensorrt_llm::runtime::DeviceAllocationNvls::reset"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls5resetE6size_tNSt3setIiEE", "tensorrt_llm::runtime::DeviceAllocationNvls::reset::ranks"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls5resetE6size_tNSt3setIiEE", "tensorrt_llm::runtime::DeviceAllocationNvls::reset::size"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvlsD0Ev", "tensorrt_llm::runtime::DeviceAllocationNvls::~DeviceAllocationNvls"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffersE", "tensorrt_llm::runtime::EagleBuffers"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers9BufferPtrE", "tensorrt_llm::runtime::EagleBuffers::BufferPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12EagleBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN8executor14DecodingConfigE", "tensorrt_llm::runtime::EagleBuffers::EagleBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12EagleBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN8executor14DecodingConfigE", "tensorrt_llm::runtime::EagleBuffers::EagleBuffers::decodingConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12EagleBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN8executor14DecodingConfigE", "tensorrt_llm::runtime::EagleBuffers::EagleBuffers::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12EagleBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN8executor14DecodingConfigE", "tensorrt_llm::runtime::EagleBuffers::EagleBuffers::maxBatchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12EagleBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN8executor14DecodingConfigE", "tensorrt_llm::runtime::EagleBuffers::EagleBuffers::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12EagleBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN8executor14DecodingConfigE", "tensorrt_llm::runtime::EagleBuffers::EagleBuffers::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12EagleBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN8executor14DecodingConfigE", "tensorrt_llm::runtime::EagleBuffers::EagleBuffers::worldConfig"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputsE", "tensorrt_llm::runtime::EagleBuffers::EngineOutputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs12acceptedLensE", "tensorrt_llm::runtime::EagleBuffers::EngineOutputs::acceptedLens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs13acceptedPathsE", "tensorrt_llm::runtime::EagleBuffers::EngineOutputs::acceptedPaths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs14acceptedTokensE", "tensorrt_llm::runtime::EagleBuffers::EngineOutputs::acceptedTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs24chunkedContextNextTokensE", "tensorrt_llm::runtime::EagleBuffers::EngineOutputs::chunkedContextNextTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs13nextDraftLensE", "tensorrt_llm::runtime::EagleBuffers::EngineOutputs::nextDraftLens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs14nextDraftPathsE", "tensorrt_llm::runtime::EagleBuffers::EngineOutputs::nextDraftPaths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs15nextDraftTokensE", "tensorrt_llm::runtime::EagleBuffers::EngineOutputs::nextDraftTokens"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers7ITensorE", "tensorrt_llm::runtime::EagleBuffers::ITensor"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6InputsE", "tensorrt_llm::runtime::EagleBuffers::Inputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs22allLayersDraftTokenIdsE", "tensorrt_llm::runtime::EagleBuffers::Inputs::allLayersDraftTokenIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs33allLayersDraftTokenIdsPredecessorE", "tensorrt_llm::runtime::EagleBuffers::Inputs::allLayersDraftTokenIdsPredecessor"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs15allLayersScoresE", "tensorrt_llm::runtime::EagleBuffers::Inputs::allLayersScores"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs24chunkedContextNextTokensE", "tensorrt_llm::runtime::EagleBuffers::Inputs::chunkedContextNextTokens"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs6createE10SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::EagleBuffers::Inputs::create"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs6createE10SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::EagleBuffers::Inputs::create::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs6createE10SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::EagleBuffers::Inputs::create::maxNumSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs6createE10SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::EagleBuffers::Inputs::create::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs6createE10SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::EagleBuffers::Inputs::create::worldConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs20currentExpandIndicesE", "tensorrt_llm::runtime::EagleBuffers::Inputs::currentExpandIndices"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs9draftLensE", "tensorrt_llm::runtime::EagleBuffers::Inputs::draftLens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs10draftPathsE", "tensorrt_llm::runtime::EagleBuffers::Inputs::draftPaths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs14draftPathsHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::draftPathsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs11draftTokensE", "tensorrt_llm::runtime::EagleBuffers::Inputs::draftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs22dynamicTreeMaxTopKHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::dynamicTreeMaxTopKHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs29eagleNetCtxContextLengthsHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::eagleNetCtxContextLengthsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs34eagleNetCtxPastKeyValueLengthsHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::eagleNetCtxPastKeyValueLengthsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs27eagleNetCtxRequestTypesHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::eagleNetCtxRequestTypesHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs29eagleNetGenContextLengthsHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::eagleNetGenContextLengthsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs34eagleNetGenPastKeyValueLengthsHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::eagleNetGenPastKeyValueLengthsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs27eagleNetGenRequestTypesHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::eagleNetGenRequestTypesHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs18inputGenTokensHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::inputGenTokensHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs14posteriorAlphaE", "tensorrt_llm::runtime::EagleBuffers::Inputs::posteriorAlpha"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs18posteriorThresholdE", "tensorrt_llm::runtime::EagleBuffers::Inputs::posteriorThreshold"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs10prevScoresE", "tensorrt_llm::runtime::EagleBuffers::Inputs::prevScores"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs16randomDataSampleE", "tensorrt_llm::runtime::EagleBuffers::Inputs::randomDataSample"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs20randomDataValidationE", "tensorrt_llm::runtime::EagleBuffers::Inputs::randomDataValidation"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs29specDecodingGenerationLengthsE", "tensorrt_llm::runtime::EagleBuffers::Inputs::specDecodingGenerationLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs33specDecodingGenerationLengthsHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::specDecodingGenerationLengthsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs23specDecodingPackedMasksE", "tensorrt_llm::runtime::EagleBuffers::Inputs::specDecodingPackedMasks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs27specDecodingPositionOffsetsE", "tensorrt_llm::runtime::EagleBuffers::Inputs::specDecodingPositionOffsets"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs12temperaturesE", "tensorrt_llm::runtime::EagleBuffers::Inputs::temperatures"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs18useDynamicTreeHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::useDynamicTreeHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs15useSpecDecodingE", "tensorrt_llm::runtime::EagleBuffers::Inputs::useSpecDecoding"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13LlmRequestPtrE", "tensorrt_llm::runtime::EagleBuffers::LlmRequestPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13RequestVectorE", "tensorrt_llm::runtime::EagleBuffers::RequestVector"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers10SizeType32E", "tensorrt_llm::runtime::EagleBuffers::SizeType32"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers9TensorMapE", "tensorrt_llm::runtime::EagleBuffers::TensorMap"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers9TensorPtrE", "tensorrt_llm::runtime::EagleBuffers::TensorPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers28chunkedContextNextTokensHostE", "tensorrt_llm::runtime::EagleBuffers::chunkedContextNextTokensHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers23cumSumGenerationLengthsE", "tensorrt_llm::runtime::EagleBuffers::cumSumGenerationLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12engineInputsE", "tensorrt_llm::runtime::EagleBuffers::engineInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13engineOutputsE", "tensorrt_llm::runtime::EagleBuffers::engineOutputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers18greedySamplingHostE", "tensorrt_llm::runtime::EagleBuffers::greedySamplingHost"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::insertInputTensors"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::insertInputTensors::inputBuffers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::insertInputTensors::outputBuffers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::insertInputTensors::worldConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers26mDefaultPosteriorThresholdE", "tensorrt_llm::runtime::EagleBuffers::mDefaultPosteriorThreshold"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers17mDoGreedySamplingE", "tensorrt_llm::runtime::EagleBuffers::mDoGreedySampling"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers19maxGenerationLengthE", "tensorrt_llm::runtime::EagleBuffers::maxGenerationLength"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers18posteriorAlphaHostE", "tensorrt_llm::runtime::EagleBuffers::posteriorAlphaHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers22posteriorThresholdHostE", "tensorrt_llm::runtime::EagleBuffers::posteriorThresholdHost"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", "tensorrt_llm::runtime::EagleBuffers::reshape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", "tensorrt_llm::runtime::EagleBuffers::reshape::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", "tensorrt_llm::runtime::EagleBuffers::reshape::numCtxSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", "tensorrt_llm::runtime::EagleBuffers::reshape::numGenSequences"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers21scanReduceTempStorageE", "tensorrt_llm::runtime::EagleBuffers::scanReduceTempStorage"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers26scanReduceTempStorageBytesE", "tensorrt_llm::runtime::EagleBuffers::scanReduceTempStorageBytes"], [1, 3, 1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs"], [1, 8, 1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::T"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::contextRequests"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::contextRequests"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::decoderBuffers"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::draftBuffers"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::eagleModule"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::genRequests"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::genRequests"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::manager"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::manager"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::modelConfig"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::requestTypes"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::seqSlots"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::seqSlots"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::vocabSizePadded"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::worldConfig"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModuleE", "tensorrt_llm::runtime::EagleModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModule11EagleModuleE10SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::EagleModule::EagleModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModule11EagleModuleEv", "tensorrt_llm::runtime::EagleModule::EagleModule"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModule11EagleModuleE10SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::EagleModule::EagleModule::maxDecodingDraftTokens"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModule11EagleModuleE10SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::EagleModule::EagleModule::maxDraftPathLen"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModule11EagleModuleE10SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::EagleModule::EagleModule::maxNonLeafNodesPerLayer"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModule11EagleModuleE10SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::EagleModule::EagleModule::numTransformersLayer"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11EagleModule22getDefaultEagleChoicesEv", "tensorrt_llm::runtime::EagleModule::getDefaultEagleChoices"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11EagleModule26getMaxNonLeafNodesPerLayerEv", "tensorrt_llm::runtime::EagleModule::getMaxNonLeafNodesPerLayer"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11EagleModule23getNumTransformerLayersEv", "tensorrt_llm::runtime::EagleModule::getNumTransformerLayers"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModule20mDefaultEagleChoicesE", "tensorrt_llm::runtime::EagleModule::mDefaultEagleChoices"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModule24mMaxNonLeafNodesPerLayerE", "tensorrt_llm::runtime::EagleModule::mMaxNonLeafNodesPerLayer"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModule21mNumTransformersLayerE", "tensorrt_llm::runtime::EagleModule::mNumTransformersLayer"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffersE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers9BufferPtrE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::BufferPtr"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers12EngineInputsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers12EngineInputs15positionOffsetsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineInputs::positionOffsets"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers12EngineInputs18requestTypesDeviceE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineInputs::requestTypesDevice"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs15bestPathIndicesE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::bestPathIndices"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs15bestPathLengthsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::bestPathLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs5masksE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::masks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs11maxGenTokenE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::maxGenToken"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs16nextDraftIndicesE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::nextDraftIndices"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs14nextDraftProbsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::nextDraftProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs15nextDraftTokensE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::nextDraftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs14nextFlatTokensE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::nextFlatTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs21nextGenerationLengthsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::nextGenerationLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs19nextPositionOffsetsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::nextPositionOffsets"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs17packedPositionIdsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::packedPositionIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs13totalGenTokenE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::totalGenToken"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers26ExplicitDraftTokensBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::ExplicitDraftTokensBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers26ExplicitDraftTokensBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::ExplicitDraftTokensBuffers::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers26ExplicitDraftTokensBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::ExplicitDraftTokensBuffers::maxBatchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers26ExplicitDraftTokensBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::ExplicitDraftTokensBuffers::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers26ExplicitDraftTokensBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::ExplicitDraftTokensBuffers::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers26ExplicitDraftTokensBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::ExplicitDraftTokensBuffers::worldConfig"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers7ITensorE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::ITensor"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6InputsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs6createE10SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::create"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs6createE10SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::create::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs6createE10SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::create::maxNumSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs6createE10SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::create::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs6createE10SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::create::worldConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs12draftIndicesE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::draftIndices"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs10draftProbsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::draftProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs11draftTokensE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::draftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs17generationLengthsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::generationLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs21generationLengthsHostE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::generationLengthsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs16maxGenLengthHostE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::maxGenLengthHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs11packedMasksE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::packedMasks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs11positionIdsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::positionIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs15positionIdsBaseE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::positionIdsBase"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs16randomDataSampleE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::randomDataSample"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs20randomDataValidationE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::randomDataValidation"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs12temperaturesE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::temperatures"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs15useSpecDecodingE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::useSpecDecoding"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers10SizeType32E", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::SizeType32"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers9TensorMapE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::TensorMap"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers9TensorPtrE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::TensorPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers23cumSumGenerationLengthsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::cumSumGenerationLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers12engineInputsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::engineInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13engineOutputsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::engineOutputs"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::insertInputTensors"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::insertInputTensors::inputBuffers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::insertInputTensors::outputBuffers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::insertInputTensors::worldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::reshape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::reshape::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::reshape::numCtxSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::reshape::numGenSequences"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers15scanTempStorageE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::scanTempStorage"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers20scanTempStorageBytesE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::scanTempStorageBytes"], [1, 3, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs"], [1, 8, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::T"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::contextPositionIds"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::contextPositionIds"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::decoderBuffers"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::draftBuffers"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::explicitDraftTokensModule"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::manager"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::modelConfig"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::numCtxSequences"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::numCtxSequences"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::numGenSequences"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::numGenSequences"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::requestTypes"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::seqSlots"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::seqSlots"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::stream"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::stream"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::vocabSizePadded"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::worldConfig"], [1, 2, 1, "_CPPv4I0EN12tensorrt_llm7runtime25GenericPromptTuningParamsE", "tensorrt_llm::runtime::GenericPromptTuningParams"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams25GenericPromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", "tensorrt_llm::runtime::GenericPromptTuningParams::GenericPromptTuningParams"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams25GenericPromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", "tensorrt_llm::runtime::GenericPromptTuningParams::GenericPromptTuningParams::embeddingTable"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams25GenericPromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", "tensorrt_llm::runtime::GenericPromptTuningParams::GenericPromptTuningParams::tasks"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams25GenericPromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", "tensorrt_llm::runtime::GenericPromptTuningParams::GenericPromptTuningParams::vocabSize"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams10SizeType32E", "tensorrt_llm::runtime::GenericPromptTuningParams::SizeType32"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime25GenericPromptTuningParamsE", "tensorrt_llm::runtime::GenericPromptTuningParams::TTensor"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams9TensorPtrE", "tensorrt_llm::runtime::GenericPromptTuningParams::TensorPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams14embeddingTableE", "tensorrt_llm::runtime::GenericPromptTuningParams::embeddingTable"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams19promptTuningEnabledE", "tensorrt_llm::runtime::GenericPromptTuningParams::promptTuningEnabled"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams5tasksE", "tensorrt_llm::runtime::GenericPromptTuningParams::tasks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams9vocabSizeE", "tensorrt_llm::runtime::GenericPromptTuningParams::vocabSize"], [1, 2, 1, "_CPPv4I0EN12tensorrt_llm7runtime10GptDecoderE", "tensorrt_llm::runtime::GptDecoder"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder13CudaStreamPtrE", "tensorrt_llm::runtime::GptDecoder::CudaStreamPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10GptDecoderERKN8executor12DecodingModeE6size_t6size_t6size_t6size_tRK13CudaStreamPtrNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::GptDecoder::GptDecoder"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10GptDecoderERKN8executor12DecodingModeE6size_t6size_t6size_t6size_tRK13CudaStreamPtrNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::GptDecoder::GptDecoder::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10GptDecoderERKN8executor12DecodingModeE6size_t6size_t6size_t6size_tRK13CudaStreamPtrNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::GptDecoder::GptDecoder::maxNumSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10GptDecoderERKN8executor12DecodingModeE6size_t6size_t6size_t6size_tRK13CudaStreamPtrNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::GptDecoder::GptDecoder::mode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10GptDecoderERKN8executor12DecodingModeE6size_t6size_t6size_t6size_tRK13CudaStreamPtrNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::GptDecoder::GptDecoder::speculativeDecodingModule"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10GptDecoderERKN8executor12DecodingModeE6size_t6size_t6size_t6size_tRK13CudaStreamPtrNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::GptDecoder::GptDecoder::stream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10GptDecoderERKN8executor12DecodingModeE6size_t6size_t6size_t6size_tRK13CudaStreamPtrNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::GptDecoder::GptDecoder::vocabSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10GptDecoderERKN8executor12DecodingModeE6size_t6size_t6size_t6size_tRK13CudaStreamPtrNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::GptDecoder::GptDecoder::vocabSizePadded"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime10GptDecoderE", "tensorrt_llm::runtime::GptDecoder::T"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder9TensorPtrE", "tensorrt_llm::runtime::GptDecoder::TensorPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", "tensorrt_llm::runtime::GptDecoder::disableLookahead"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", "tensorrt_llm::runtime::GptDecoder::disableLookahead::batchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", "tensorrt_llm::runtime::GptDecoder::disableLookahead::batchSlots"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", "tensorrt_llm::runtime::GptDecoder::disableLookahead::samplingConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder12forwardAsyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::GptDecoder::forwardAsync"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder12forwardAsyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::GptDecoder::forwardAsync::input"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder12forwardAsyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::GptDecoder::forwardAsync::output"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder11forwardSyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::GptDecoder::forwardSync"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder11forwardSyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::GptDecoder::forwardSync::input"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder11forwardSyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::GptDecoder::forwardSync::output"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder17getSamplingConfigEv", "tensorrt_llm::runtime::GptDecoder::getSamplingConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder23mDecodingLayerWorkspaceE", "tensorrt_llm::runtime::GptDecoder::mDecodingLayerWorkspace"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder13mDecodingModeE", "tensorrt_llm::runtime::GptDecoder::mDecodingMode"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder19mDynamicDecodeLayerE", "tensorrt_llm::runtime::GptDecoder::mDynamicDecodeLayer"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder8mManagerE", "tensorrt_llm::runtime::GptDecoder::mManager"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder16mMaxNumSequencesE", "tensorrt_llm::runtime::GptDecoder::mMaxNumSequences"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder15mSamplingConfigE", "tensorrt_llm::runtime::GptDecoder::mSamplingConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10mVocabSizeE", "tensorrt_llm::runtime::GptDecoder::mVocabSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder16mVocabSizePaddedE", "tensorrt_llm::runtime::GptDecoder::mVocabSizePadded"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::GptDecoder::setup"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::GptDecoder::setup::batchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::GptDecoder::setup::batchSlots"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::GptDecoder::setup::explicitDraftTokensDType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::GptDecoder::setup::lookaheadAlgoConfigs"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::GptDecoder::setup::lookaheadPrompt"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::GptDecoder::setup::output"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::GptDecoder::setup::samplingConfig"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatchedE", "tensorrt_llm::runtime::GptDecoderBatched"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched13CudaStreamPtrE", "tensorrt_llm::runtime::GptDecoderBatched::CudaStreamPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched17GptDecoderBatchedE13CudaStreamPtr", "tensorrt_llm::runtime::GptDecoderBatched::GptDecoderBatched"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched17GptDecoderBatchedE13CudaStreamPtr", "tensorrt_llm::runtime::GptDecoderBatched::GptDecoderBatched::stream"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched13GptDecoderPtrE", "tensorrt_llm::runtime::GptDecoderBatched::GptDecoderPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched13LlmRequestPtrE", "tensorrt_llm::runtime::GptDecoderBatched::LlmRequestPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched13RequestVectorE", "tensorrt_llm::runtime::GptDecoderBatched::RequestVector"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched9TensorPtrE", "tensorrt_llm::runtime::GptDecoderBatched::TensorPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched16disableLookaheadERK13RequestVectorRK9TensorPtr", "tensorrt_llm::runtime::GptDecoderBatched::disableLookahead"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched16disableLookaheadERK13RequestVectorRK9TensorPtr", "tensorrt_llm::runtime::GptDecoderBatched::disableLookahead::batchSlots"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched16disableLookaheadERK13RequestVectorRK9TensorPtr", "tensorrt_llm::runtime::GptDecoderBatched::disableLookahead::genRequests"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::GptDecoderBatched::finalize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::GptDecoderBatched::finalize::batchSlot"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::GptDecoderBatched::finalize::decoderState"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::GptDecoderBatched::finalize::samplingConfig"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::GptDecoderBatched::finalize::streaming"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched7forwardERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::GptDecoderBatched::forward"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched7forwardERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::GptDecoderBatched::forward::decoderState"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched7forwardERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::GptDecoderBatched::forward::input"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched12forwardAsyncERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::GptDecoderBatched::forwardAsync"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched12forwardAsyncERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::GptDecoderBatched::forwardAsync::decoderState"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched12forwardAsyncERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::GptDecoderBatched::forwardAsync::input"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched15forwardDispatchERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::GptDecoderBatched::forwardDispatch"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched15forwardDispatchERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::GptDecoderBatched::forwardDispatch::decoderState"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched15forwardDispatchERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::GptDecoderBatched::forwardDispatch::input"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched16getBufferManagerEv", "tensorrt_llm::runtime::GptDecoderBatched::getBufferManager"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched16getDecoderStreamEv", "tensorrt_llm::runtime::GptDecoderBatched::getDecoderStream"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched20getUnderlyingDecoderEv", "tensorrt_llm::runtime::GptDecoderBatched::getUnderlyingDecoder"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched14mBufferManagerE", "tensorrt_llm::runtime::GptDecoderBatched::mBufferManager"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched8mDecoderE", "tensorrt_llm::runtime::GptDecoderBatched::mDecoder"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched14mDecoderStreamE", "tensorrt_llm::runtime::GptDecoderBatched::mDecoderStream"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched14mRuntimeStreamE", "tensorrt_llm::runtime::GptDecoderBatched::mRuntimeStream"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::GptDecoderBatched::setup"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::GptDecoderBatched::setup::dtype"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::GptDecoderBatched::setup::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::GptDecoderBatched::setup::maxNumSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::GptDecoderBatched::setup::mode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::GptDecoderBatched::setup::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::GptDecoderBatched::setup::worldConfig"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfigE", "tensorrt_llm::runtime::GptJsonConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig::contextParallelism"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig::gpusPerNode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig::name"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig::pipelineParallelism"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig::precision"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig::runtimeDefaults"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig::tensorParallelism"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig::version"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14engineFilenameERK11WorldConfig", "tensorrt_llm::runtime::GptJsonConfig::engineFilename"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14engineFilenameERK11WorldConfigRKNSt6stringE", "tensorrt_llm::runtime::GptJsonConfig::engineFilename"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14engineFilenameERK11WorldConfigRKNSt6stringE", "tensorrt_llm::runtime::GptJsonConfig::engineFilename::model"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14engineFilenameERK11WorldConfig", "tensorrt_llm::runtime::GptJsonConfig::engineFilename::worldConfig"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14engineFilenameERK11WorldConfigRKNSt6stringE", "tensorrt_llm::runtime::GptJsonConfig::engineFilename::worldConfig"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig21getContextParallelismEv", "tensorrt_llm::runtime::GptJsonConfig::getContextParallelism"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14getGpusPerNodeEv", "tensorrt_llm::runtime::GptJsonConfig::getGpusPerNode"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14getModelConfigEv", "tensorrt_llm::runtime::GptJsonConfig::getModelConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig21getModelConfigMutableEv", "tensorrt_llm::runtime::GptJsonConfig::getModelConfigMutable"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig7getNameEv", "tensorrt_llm::runtime::GptJsonConfig::getName"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig22getPipelineParallelismEv", "tensorrt_llm::runtime::GptJsonConfig::getPipelineParallelism"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig12getPrecisionEv", "tensorrt_llm::runtime::GptJsonConfig::getPrecision"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig18getRuntimeDefaultsEv", "tensorrt_llm::runtime::GptJsonConfig::getRuntimeDefaults"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig20getTensorParallelismEv", "tensorrt_llm::runtime::GptJsonConfig::getTensorParallelism"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig10getVersionEv", "tensorrt_llm::runtime::GptJsonConfig::getVersion"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig12getWorldSizeEv", "tensorrt_llm::runtime::GptJsonConfig::getWorldSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig19mContextParallelismE", "tensorrt_llm::runtime::GptJsonConfig::mContextParallelism"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig12mGpusPerNodeE", "tensorrt_llm::runtime::GptJsonConfig::mGpusPerNode"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig12mModelConfigE", "tensorrt_llm::runtime::GptJsonConfig::mModelConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5mNameE", "tensorrt_llm::runtime::GptJsonConfig::mName"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig20mPipelineParallelismE", "tensorrt_llm::runtime::GptJsonConfig::mPipelineParallelism"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig10mPrecisionE", "tensorrt_llm::runtime::GptJsonConfig::mPrecision"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig16mRuntimeDefaultsE", "tensorrt_llm::runtime::GptJsonConfig::mRuntimeDefaults"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig18mTensorParallelismE", "tensorrt_llm::runtime::GptJsonConfig::mTensorParallelism"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig8mVersionE", "tensorrt_llm::runtime::GptJsonConfig::mVersion"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5parseERKNSt10filesystem4pathE", "tensorrt_llm::runtime::GptJsonConfig::parse"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5parseERKNSt6stringE", "tensorrt_llm::runtime::GptJsonConfig::parse"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5parseERNSt7istreamE", "tensorrt_llm::runtime::GptJsonConfig::parse"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5parseERKNSt6stringE", "tensorrt_llm::runtime::GptJsonConfig::parse::json"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5parseERNSt7istreamE", "tensorrt_llm::runtime::GptJsonConfig::parse::json"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5parseERKNSt10filesystem4pathE", "tensorrt_llm::runtime::GptJsonConfig::parse::path"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime7IBufferE", "tensorrt_llm::runtime::IBuffer"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer8DataTypeE", "tensorrt_llm::runtime::IBuffer::DataType"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer7IBufferERK7IBuffer", "tensorrt_llm::runtime::IBuffer::IBuffer"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer7IBufferEv", "tensorrt_llm::runtime::IBuffer::IBuffer"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer14SharedConstPtrE", "tensorrt_llm::runtime::IBuffer::SharedConstPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer9SharedPtrE", "tensorrt_llm::runtime::IBuffer::SharedPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer14UniqueConstPtrE", "tensorrt_llm::runtime::IBuffer::UniqueConstPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer9UniquePtrE", "tensorrt_llm::runtime::IBuffer::UniquePtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4dataENSt6size_tE", "tensorrt_llm::runtime::IBuffer::data"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4dataEv", "tensorrt_llm::runtime::IBuffer::data"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer4dataENSt6size_tE", "tensorrt_llm::runtime::IBuffer::data"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer4dataEv", "tensorrt_llm::runtime::IBuffer::data"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4dataENSt6size_tE", "tensorrt_llm::runtime::IBuffer::data::index"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer4dataENSt6size_tE", "tensorrt_llm::runtime::IBuffer::data::index"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer11getCapacityEv", "tensorrt_llm::runtime::IBuffer::getCapacity"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer11getDataTypeEv", "tensorrt_llm::runtime::IBuffer::getDataType"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer15getDataTypeNameE8DataType", "tensorrt_llm::runtime::IBuffer::getDataTypeName"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer15getDataTypeNameEv", "tensorrt_llm::runtime::IBuffer::getDataTypeName"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer15getDataTypeNameE8DataType", "tensorrt_llm::runtime::IBuffer::getDataTypeName::dataType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer13getMemoryTypeEv", "tensorrt_llm::runtime::IBuffer::getMemoryType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer17getMemoryTypeNameEv", "tensorrt_llm::runtime::IBuffer::getMemoryTypeName"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer7getSizeEv", "tensorrt_llm::runtime::IBuffer::getSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer14getSizeInBytesEv", "tensorrt_llm::runtime::IBuffer::getSizeInBytes"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer10memoryTypeEPKv", "tensorrt_llm::runtime::IBuffer::memoryType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer10memoryTypeEPKv", "tensorrt_llm::runtime::IBuffer::memoryType::data"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBufferaSERK7IBuffer", "tensorrt_llm::runtime::IBuffer::operator="], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer7releaseEv", "tensorrt_llm::runtime::IBuffer::release"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer6resizeENSt6size_tE", "tensorrt_llm::runtime::IBuffer::resize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer6resizeENSt6size_tE", "tensorrt_llm::runtime::IBuffer::resize::newSize"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer5sliceE9SharedPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer5sliceE9SharedPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::TConstPtr"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::TConstPtr"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer5sliceE9SharedPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::buffer"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer5sliceE9SharedPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::buffer"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::offset"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::offset"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer5sliceE9SharedPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::offset"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer5sliceE9SharedPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::offset"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer5sliceE9SharedPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::size"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::tensor"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::tensor"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer7toBytesENSt6size_tE", "tensorrt_llm::runtime::IBuffer::toBytes"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer7toBytesENSt6size_tE", "tensorrt_llm::runtime::IBuffer::toBytes::size"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer4viewE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::view"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4viewE9SharedPtr", "tensorrt_llm::runtime::IBuffer::view"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4viewE9SharedPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::view"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer4viewE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::view::TConstPtr"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer4viewE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::view::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4viewE9SharedPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::view::size"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer4viewE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::view::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4viewE9SharedPtr", "tensorrt_llm::runtime::IBuffer::view::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4viewE9SharedPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::view::tensor"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrRNSt6vectorI1TEE", "tensorrt_llm::runtime::IBuffer::wrap"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrRNSt6vectorI1TEE", "tensorrt_llm::runtime::IBuffer::wrap::T"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::capacity"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::capacity"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::data"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::data"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::data"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::data"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::size"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::type"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::type"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrRNSt6vectorI1TEE", "tensorrt_llm::runtime::IBuffer::wrap::v"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBufferD0Ev", "tensorrt_llm::runtime::IBuffer::~IBuffer"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoderE", "tensorrt_llm::runtime::IGptDecoder"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder14TensorConstPtrE", "tensorrt_llm::runtime::IGptDecoder::TensorConstPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder9TensorPtrE", "tensorrt_llm::runtime::IGptDecoder::TensorPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::IGptDecoder::create"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::IGptDecoder::create::dtype"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::IGptDecoder::create::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::IGptDecoder::create::maxNumSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::IGptDecoder::create::mode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::IGptDecoder::create::speculativeDecodingModule"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::IGptDecoder::create::stream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::IGptDecoder::create::vocabSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::IGptDecoder::create::vocabSizePadded"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", "tensorrt_llm::runtime::IGptDecoder::disableLookahead"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", "tensorrt_llm::runtime::IGptDecoder::disableLookahead::batchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", "tensorrt_llm::runtime::IGptDecoder::disableLookahead::batchSlots"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", "tensorrt_llm::runtime::IGptDecoder::disableLookahead::samplingConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder12forwardAsyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::IGptDecoder::forwardAsync"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder12forwardAsyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::IGptDecoder::forwardAsync::input"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder12forwardAsyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::IGptDecoder::forwardAsync::output"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder11forwardSyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::IGptDecoder::forwardSync"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder11forwardSyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::IGptDecoder::forwardSync::input"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder11forwardSyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::IGptDecoder::forwardSync::output"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder17getSamplingConfigEv", "tensorrt_llm::runtime::IGptDecoder::getSamplingConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::IGptDecoder::setup"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::IGptDecoder::setup::batchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::IGptDecoder::setup::batchSlots"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::IGptDecoder::setup::explicitDraftTokensDType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::IGptDecoder::setup::lookaheadAlgoConfigs"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::IGptDecoder::setup::lookaheadPrompt"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::IGptDecoder::setup::output"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::IGptDecoder::setup::samplingConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoderD0Ev", "tensorrt_llm::runtime::IGptDecoder::~IGptDecoder"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatchedE", "tensorrt_llm::runtime::IGptDecoderBatched"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched13CudaStreamPtrE", "tensorrt_llm::runtime::IGptDecoderBatched::CudaStreamPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched18IGptDecoderBatchedEv", "tensorrt_llm::runtime::IGptDecoderBatched::IGptDecoderBatched"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched13LlmRequestPtrE", "tensorrt_llm::runtime::IGptDecoderBatched::LlmRequestPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched13RequestVectorE", "tensorrt_llm::runtime::IGptDecoderBatched::RequestVector"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched9TensorPtrE", "tensorrt_llm::runtime::IGptDecoderBatched::TensorPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched16disableLookaheadERK13RequestVectorRK9TensorPtr", "tensorrt_llm::runtime::IGptDecoderBatched::disableLookahead"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched16disableLookaheadERK13RequestVectorRK9TensorPtr", "tensorrt_llm::runtime::IGptDecoderBatched::disableLookahead::batchSlots"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched16disableLookaheadERK13RequestVectorRK9TensorPtr", "tensorrt_llm::runtime::IGptDecoderBatched::disableLookahead::genRequests"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime18IGptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::IGptDecoderBatched::finalize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime18IGptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::IGptDecoderBatched::finalize::batchSlot"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime18IGptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::IGptDecoderBatched::finalize::decoderState"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime18IGptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::IGptDecoderBatched::finalize::samplingConfig"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime18IGptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::IGptDecoderBatched::finalize::streaming"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched7forwardERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::IGptDecoderBatched::forward"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched7forwardERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::IGptDecoderBatched::forward::decoderState"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched7forwardERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::IGptDecoderBatched::forward::input"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched12forwardAsyncERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::IGptDecoderBatched::forwardAsync"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched12forwardAsyncERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::IGptDecoderBatched::forwardAsync::decoderState"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched12forwardAsyncERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::IGptDecoderBatched::forwardAsync::input"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::IGptDecoderBatched::setup"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::IGptDecoderBatched::setup::dtype"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::IGptDecoderBatched::setup::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::IGptDecoderBatched::setup::maxNumSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::IGptDecoderBatched::setup::mode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::IGptDecoderBatched::setup::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::IGptDecoderBatched::setup::worldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatchedD0Ev", "tensorrt_llm::runtime::IGptDecoderBatched::~IGptDecoderBatched"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime7ITensorE", "tensorrt_llm::runtime::ITensor"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9DimType64E", "tensorrt_llm::runtime::ITensor::DimType64"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7ITensorERK7ITensor", "tensorrt_llm::runtime::ITensor::ITensor"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7ITensorEv", "tensorrt_llm::runtime::ITensor::ITensor"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5ShapeE", "tensorrt_llm::runtime::ITensor::Shape"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor14SharedConstPtrE", "tensorrt_llm::runtime::ITensor::SharedConstPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9SharedPtrE", "tensorrt_llm::runtime::ITensor::SharedPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9TensorMapE", "tensorrt_llm::runtime::ITensor::TensorMap"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor14UniqueConstPtrE", "tensorrt_llm::runtime::ITensor::UniqueConstPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9UniquePtrE", "tensorrt_llm::runtime::ITensor::UniquePtr"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::at"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atEN7ITensor14UniqueConstPtrERR9TConstPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::at"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor2atE9SharedPtrRK5Shape", "tensorrt_llm::runtime::ITensor::at"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor2atE9SharedPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::at"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::at::TConstPtr"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atEN7ITensor14UniqueConstPtrERR9TConstPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::at::TConstPtr"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::at::offsetDims"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atEN7ITensor14UniqueConstPtrERR9TConstPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::at::offsetDims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor2atE9SharedPtrRK5Shape", "tensorrt_llm::runtime::ITensor::at::offsetDims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor2atE9SharedPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::at::offsetDims"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::at::tensor"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atEN7ITensor14UniqueConstPtrERR9TConstPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::at::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor2atE9SharedPtrRK5Shape", "tensorrt_llm::runtime::ITensor::at::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor2atE9SharedPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::at::tensor"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor8castSizeE6size_t", "tensorrt_llm::runtime::ITensor::castSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor8castSizeE6size_t", "tensorrt_llm::runtime::ITensor::castSize::newSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor8flattenNE9SharedPtrNSt7int64_tE", "tensorrt_llm::runtime::ITensor::flattenN"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor8flattenNE9SharedPtrNSt7int64_tE", "tensorrt_llm::runtime::ITensor::flattenN::sliceN"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor8flattenNE9SharedPtrNSt7int64_tE", "tensorrt_llm::runtime::ITensor::flattenN::tensor"], [1, 3, 1, "_CPPv4I_10SizeType32ENK12tensorrt_llm7runtime7ITensor12getDimensionE9DimType64v", "tensorrt_llm::runtime::ITensor::getDimension"], [1, 8, 1, "_CPPv4I_10SizeType32ENK12tensorrt_llm7runtime7ITensor12getDimensionE9DimType64v", "tensorrt_llm::runtime::ITensor::getDimension::n"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7ITensor8getShapeEv", "tensorrt_llm::runtime::ITensor::getShape"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9makeShapeERKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::makeShape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9makeShapeERKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::makeShape::dims"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensoraSERK7ITensor", "tensorrt_llm::runtime::ITensor::operator="], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7reshapeERK5Shape", "tensorrt_llm::runtime::ITensor::reshape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7reshapeERK5Shape", "tensorrt_llm::runtime::ITensor::reshape::dims"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor6resizeENSt6size_tE", "tensorrt_llm::runtime::ITensor::resize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor6resizeENSt6size_tE", "tensorrt_llm::runtime::ITensor::resize::newSize"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor11shapeEqualsEbRK5ShapePK1T10SizeType32", "tensorrt_llm::runtime::ITensor::shapeEquals"], [1, 3, 1, "_CPPv4I0ENK12tensorrt_llm7runtime7ITensor11shapeEqualsEbPK1T10SizeType32", "tensorrt_llm::runtime::ITensor::shapeEquals"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor11shapeEqualsERK5ShapeRK5Shape", "tensorrt_llm::runtime::ITensor::shapeEquals"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7ITensor11shapeEqualsERK5Shape", "tensorrt_llm::runtime::ITensor::shapeEquals"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7ITensor11shapeEqualsERKNSt16initializer_listI10SizeType32EE", "tensorrt_llm::runtime::ITensor::shapeEquals"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor11shapeEqualsEbRK5ShapePK1T10SizeType32", "tensorrt_llm::runtime::ITensor::shapeEquals::T"], [1, 8, 1, "_CPPv4I0ENK12tensorrt_llm7runtime7ITensor11shapeEqualsEbPK1T10SizeType32", "tensorrt_llm::runtime::ITensor::shapeEquals::T"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor11shapeEqualsEbRK5ShapePK1T10SizeType32", "tensorrt_llm::runtime::ITensor::shapeEquals::count"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime7ITensor11shapeEqualsEbPK1T10SizeType32", "tensorrt_llm::runtime::ITensor::shapeEquals::count"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor11shapeEqualsEbRK5ShapePK1T10SizeType32", "tensorrt_llm::runtime::ITensor::shapeEquals::dims"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime7ITensor11shapeEqualsEbPK1T10SizeType32", "tensorrt_llm::runtime::ITensor::shapeEquals::dims"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor11shapeEqualsEbRK5ShapePK1T10SizeType32", "tensorrt_llm::runtime::ITensor::shapeEquals::lhs"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor11shapeEqualsERK5ShapeRK5Shape", "tensorrt_llm::runtime::ITensor::shapeEquals::lhs"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7ITensor11shapeEqualsERK5Shape", "tensorrt_llm::runtime::ITensor::shapeEquals::other"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7ITensor11shapeEqualsERKNSt16initializer_listI10SizeType32EE", "tensorrt_llm::runtime::ITensor::shapeEquals::other"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor11shapeEqualsERK5ShapeRK5Shape", "tensorrt_llm::runtime::ITensor::shapeEquals::rhs"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRK5Shape", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRK5Shape9DimType64", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRKNSt16initializer_listI9DimType64EE9DimType64", "tensorrt_llm::runtime::ITensor::slice"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::TConstPtr"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::TConstPtr"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::slice::TConstPtr"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::TConstPtr"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::slice::TConstPtr"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::TConstPtr"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::offset"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::offset"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::offset"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::offset"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::slice::offsetDims"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::offsetDims"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::slice::offsetDims"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::offsetDims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRK5Shape", "tensorrt_llm::runtime::ITensor::slice::offsetDims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRK5Shape9DimType64", "tensorrt_llm::runtime::ITensor::slice::offsetDims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::slice::offsetDims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRKNSt16initializer_listI9DimType64EE9DimType64", "tensorrt_llm::runtime::ITensor::slice::offsetDims"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::size"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::size"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRK5Shape9DimType64", "tensorrt_llm::runtime::ITensor::slice::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRKNSt16initializer_listI9DimType64EE9DimType64", "tensorrt_llm::runtime::ITensor::slice::size"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRK5Shape", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRK5Shape9DimType64", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRKNSt16initializer_listI9DimType64EE9DimType64", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7squeezeE10SizeType32", "tensorrt_llm::runtime::ITensor::squeeze"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7squeezeERK5Shape10SizeType32", "tensorrt_llm::runtime::ITensor::squeeze"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7squeezeE10SizeType32", "tensorrt_llm::runtime::ITensor::squeeze::dim"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7squeezeERK5Shape10SizeType32", "tensorrt_llm::runtime::ITensor::squeeze::dim"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7squeezeERK5Shape10SizeType32", "tensorrt_llm::runtime::ITensor::squeeze::shape"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7stridesERK5Shape", "tensorrt_llm::runtime::ITensor::strides"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7stridesERK5Shape", "tensorrt_llm::runtime::ITensor::strides::dims"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor8toStringERK5Shape", "tensorrt_llm::runtime::ITensor::toString"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor8toStringERK5Shape", "tensorrt_llm::runtime::ITensor::toString::dims"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9unsqueezeE10SizeType32", "tensorrt_llm::runtime::ITensor::unsqueeze"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9unsqueezeERK5Shape10SizeType32", "tensorrt_llm::runtime::ITensor::unsqueeze"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9unsqueezeE10SizeType32", "tensorrt_llm::runtime::ITensor::unsqueeze::dim"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9unsqueezeERK5Shape10SizeType32", "tensorrt_llm::runtime::ITensor::unsqueeze::dim"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9unsqueezeERK5Shape10SizeType32", "tensorrt_llm::runtime::ITensor::unsqueeze::shape"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor4viewE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::view"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4viewE9SharedPtr", "tensorrt_llm::runtime::ITensor::view"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4viewEN7IBuffer9SharedPtrERK5Shape", "tensorrt_llm::runtime::ITensor::view"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor4viewE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::view::TConstPtr"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4viewEN7IBuffer9SharedPtrERK5Shape", "tensorrt_llm::runtime::ITensor::view::buffer"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor4viewE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::view::dims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4viewEN7IBuffer9SharedPtrERK5Shape", "tensorrt_llm::runtime::ITensor::view::dims"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor4viewE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::view::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4viewE9SharedPtr", "tensorrt_llm::runtime::ITensor::view::tensor"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor6volumeERK5Shape", "tensorrt_llm::runtime::ITensor::volume"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor6volumeERK5Shape", "tensorrt_llm::runtime::ITensor::volume::dims"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor17volumeNonNegativeERK5Shape", "tensorrt_llm::runtime::ITensor::volumeNonNegative"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor17volumeNonNegativeERK5Shape", "tensorrt_llm::runtime::ITensor::volumeNonNegative::shape"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5Shape", "tensorrt_llm::runtime::ITensor::wrap"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrRNSt6vectorI1TEERK5Shape", "tensorrt_llm::runtime::ITensor::wrap"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5Shape", "tensorrt_llm::runtime::ITensor::wrap"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5Shape", "tensorrt_llm::runtime::ITensor::wrap::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrRNSt6vectorI1TEERK5Shape", "tensorrt_llm::runtime::ITensor::wrap::T"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap::capacity"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap::capacity"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5Shape", "tensorrt_llm::runtime::ITensor::wrap::data"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap::data"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5Shape", "tensorrt_llm::runtime::ITensor::wrap::data"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap::data"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5Shape", "tensorrt_llm::runtime::ITensor::wrap::shape"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap::shape"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrRNSt6vectorI1TEERK5Shape", "tensorrt_llm::runtime::ITensor::wrap::shape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5Shape", "tensorrt_llm::runtime::ITensor::wrap::shape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap::shape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5Shape", "tensorrt_llm::runtime::ITensor::wrap::type"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap::type"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrRNSt6vectorI1TEERK5Shape", "tensorrt_llm::runtime::ITensor::wrap::v"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensorD0Ev", "tensorrt_llm::runtime::ITensor::~ITensor"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemoryE", "tensorrt_llm::runtime::IpcMemory"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9BufferPtrE", "tensorrt_llm::runtime::IpcMemory::BufferPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory10FLAGS_SIZEE", "tensorrt_llm::runtime::IpcMemory::FLAGS_SIZE"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfigb", "tensorrt_llm::runtime::IpcMemory::IpcMemory"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryERK9IpcMemory", "tensorrt_llm::runtime::IpcMemory::IpcMemory"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryERR9IpcMemory", "tensorrt_llm::runtime::IpcMemory::IpcMemory"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfigb", "tensorrt_llm::runtime::IpcMemory::IpcMemory::bufferSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfigb", "tensorrt_llm::runtime::IpcMemory::IpcMemory::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfigb", "tensorrt_llm::runtime::IpcMemory::IpcMemory::openIpc"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfigb", "tensorrt_llm::runtime::IpcMemory::IpcMemory::worldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory17allocateIpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfig", "tensorrt_llm::runtime::IpcMemory::allocateIpcMemory"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory17allocateIpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfig", "tensorrt_llm::runtime::IpcMemory::allocateIpcMemory::bufferSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory17allocateIpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfig", "tensorrt_llm::runtime::IpcMemory::allocateIpcMemory::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory17allocateIpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfig", "tensorrt_llm::runtime::IpcMemory::allocateIpcMemory::worldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory16destroyIpcMemoryEv", "tensorrt_llm::runtime::IpcMemory::destroyIpcMemory"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9IpcMemory11getCommPtrsEv", "tensorrt_llm::runtime::IpcMemory::getCommPtrs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory7mBufferE", "tensorrt_llm::runtime::IpcMemory::mBuffer"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9mCommPtrsE", "tensorrt_llm::runtime::IpcMemory::mCommPtrs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory8mOpenIpcE", "tensorrt_llm::runtime::IpcMemory::mOpenIpc"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory7mTpRankE", "tensorrt_llm::runtime::IpcMemory::mTpRank"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemoryaSERK9IpcMemory", "tensorrt_llm::runtime::IpcMemory::operator="], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemoryaSERR9IpcMemory", "tensorrt_llm::runtime::IpcMemory::operator="], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemoryD0Ev", "tensorrt_llm::runtime::IpcMemory::~IpcMemory"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandleE", "tensorrt_llm::runtime::IpcNvlsHandle"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle14ipc_uc_handlesE", "tensorrt_llm::runtime::IpcNvlsHandle::ipc_uc_handles"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle11ipc_uc_ptrsE", "tensorrt_llm::runtime::IpcNvlsHandle::ipc_uc_ptrs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle10ipc_uc_vasE", "tensorrt_llm::runtime::IpcNvlsHandle::ipc_uc_vas"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle9mc_handleE", "tensorrt_llm::runtime::IpcNvlsHandle::mc_handle"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle6mc_ptrE", "tensorrt_llm::runtime::IpcNvlsHandle::mc_ptr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle5mc_vaE", "tensorrt_llm::runtime::IpcNvlsHandle::mc_va"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle4sizeE", "tensorrt_llm::runtime::IpcNvlsHandle::size"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle9uc_handleE", "tensorrt_llm::runtime::IpcNvlsHandle::uc_handle"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle6uc_ptrE", "tensorrt_llm::runtime::IpcNvlsHandle::uc_ptr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle5uc_vaE", "tensorrt_llm::runtime::IpcNvlsHandle::uc_va"], [1, 2, 1, "_CPPv4I_bEN12tensorrt_llm7runtime12LocalCreatorE", "tensorrt_llm::runtime::LocalCreator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator12LocalCreatorERK19CUmemAllocationProp6size_t", "tensorrt_llm::runtime::LocalCreator::LocalCreator"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator12LocalCreatorERK19CUmemAllocationProp6size_t", "tensorrt_llm::runtime::LocalCreator::LocalCreator::prop"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator12LocalCreatorERK19CUmemAllocationProp6size_t", "tensorrt_llm::runtime::LocalCreator::LocalCreator::size"], [1, 8, 1, "_CPPv4I_bEN12tensorrt_llm7runtime12LocalCreatorE", "tensorrt_llm::runtime::LocalCreator::count"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator6createEv", "tensorrt_llm::runtime::LocalCreator::create"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator5mPropE", "tensorrt_llm::runtime::LocalCreator::mProp"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator5mSizeE", "tensorrt_llm::runtime::LocalCreator::mSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator7releaseE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::LocalCreator::release"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator7releaseE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::LocalCreator::release::destructing"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator7releaseE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::LocalCreator::release::handle"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffersE", "tensorrt_llm::runtime::LookaheadDecodingBuffers"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers24LookaheadDecodingBuffersE10SizeType3210SizeType32RK13BufferManager", "tensorrt_llm::runtime::LookaheadDecodingBuffers::LookaheadDecodingBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers24LookaheadDecodingBuffersE10SizeType3210SizeType32RK13BufferManager", "tensorrt_llm::runtime::LookaheadDecodingBuffers::LookaheadDecodingBuffers::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers24LookaheadDecodingBuffersE10SizeType3210SizeType32RK13BufferManager", "tensorrt_llm::runtime::LookaheadDecodingBuffers::LookaheadDecodingBuffers::maxNumSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers24LookaheadDecodingBuffersE10SizeType3210SizeType32RK13BufferManager", "tensorrt_llm::runtime::LookaheadDecodingBuffers::LookaheadDecodingBuffers::maxTokensPerStep"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers9TensorPtrE", "tensorrt_llm::runtime::LookaheadDecodingBuffers::TensorPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers17generationLengthsE", "tensorrt_llm::runtime::LookaheadDecodingBuffers::generationLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers11packedMasksE", "tensorrt_llm::runtime::LookaheadDecodingBuffers::packedMasks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers11positionIdsE", "tensorrt_llm::runtime::LookaheadDecodingBuffers::positionIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers15positionOffsetsE", "tensorrt_llm::runtime::LookaheadDecodingBuffers::positionOffsets"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModuleE", "tensorrt_llm::runtime::LookaheadModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule15LookaheadModuleE10SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadModule::LookaheadModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule15LookaheadModuleEv", "tensorrt_llm::runtime::LookaheadModule::LookaheadModule"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule15LookaheadModuleE10SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadModule::LookaheadModule::maxDecodingDraftTokens"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule15LookaheadModuleE10SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadModule::LookaheadModule::maxDraftPathLen"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime15LookaheadModule18getExecutionConfigEv", "tensorrt_llm::runtime::LookaheadModule::getExecutionConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule16mExecutionConfigE", "tensorrt_llm::runtime::LookaheadModule::mExecutionConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule18setExecutionConfigERKN8executor23LookaheadDecodingConfigE", "tensorrt_llm::runtime::LookaheadModule::setExecutionConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule18setExecutionConfigERKN8executor23LookaheadDecodingConfigE", "tensorrt_llm::runtime::LookaheadModule::setExecutionConfig::config"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffersE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23LookaheadRuntimeBuffersE10SizeType3210SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfigRKN8executor14DecodingConfigERK11TllmRuntime", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::LookaheadRuntimeBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23LookaheadRuntimeBuffersE10SizeType3210SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfigRKN8executor14DecodingConfigERK11TllmRuntime", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::LookaheadRuntimeBuffers::decodingConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23LookaheadRuntimeBuffersE10SizeType3210SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfigRKN8executor14DecodingConfigERK11TllmRuntime", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::LookaheadRuntimeBuffers::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23LookaheadRuntimeBuffersE10SizeType3210SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfigRKN8executor14DecodingConfigERK11TllmRuntime", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::LookaheadRuntimeBuffers::maxBatchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23LookaheadRuntimeBuffersE10SizeType3210SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfigRKN8executor14DecodingConfigERK11TllmRuntime", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::LookaheadRuntimeBuffers::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23LookaheadRuntimeBuffersE10SizeType3210SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfigRKN8executor14DecodingConfigERK11TllmRuntime", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::LookaheadRuntimeBuffers::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23LookaheadRuntimeBuffersE10SizeType3210SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfigRKN8executor14DecodingConfigERK11TllmRuntime", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::LookaheadRuntimeBuffers::runtime"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23LookaheadRuntimeBuffersE10SizeType3210SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfigRKN8executor14DecodingConfigERK11TllmRuntime", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::LookaheadRuntimeBuffers::worldConfig"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers9TensorMapE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::TensorMap"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers9TensorPtrE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::TensorPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers18batchSlotsHostCopyE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::batchSlotsHostCopy"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers12cumSumLengthE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::cumSumLength"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers24disableLookaheadDecodingEv", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::disableLookaheadDecoding"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23enableLookaheadDecodingE10SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::enableLookaheadDecoding"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23enableLookaheadDecodingE10SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::enableLookaheadDecoding::maxBatchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23enableLookaheadDecodingE10SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::enableLookaheadDecoding::tokensPerStep"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23generationLengthsDeviceE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::generationLengthsDevice"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers21generationLengthsHostE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::generationLengthsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers25generationLengthsHostCopyE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::generationLengthsHostCopy"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers18insertInputTensorsER9TensorMapR9TensorMapRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::insertInputTensors"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers18insertInputTensorsER9TensorMapR9TensorMapRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::insertInputTensors::inputBuffers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers18insertInputTensorsER9TensorMapR9TensorMapRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::insertInputTensors::outputBuffers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers18insertInputTensorsER9TensorMapR9TensorMapRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::insertInputTensors::worldConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers14packedMaskHostE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::packedMaskHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers18packedMaskHostCopyE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::packedMaskHostCopy"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers17packedMasksDeviceE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::packedMasksDevice"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers17positionIdsDeviceE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::positionIdsDevice"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers15positionIdsHostE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::positionIdsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers19positionIdsHostCopyE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::positionIdsHostCopy"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers21positionOffsetsDeviceE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::positionOffsetsDevice"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers19positionOffsetsHostE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::positionOffsetsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23positionOffsetsHostCopyE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::positionOffsetsHostCopy"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers7reshapeE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::reshape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers7reshapeE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::reshape::numCtxSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers7reshapeE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::reshape::numGenSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers7reshapeE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::reshape::tokensPerStep"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::setFromInputs"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::setFromInputs::decoderLookaheadBuffers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::setFromInputs::modelConfig"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::setFromInputs::numCtxSequences"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::setFromInputs::numGenSequences"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::setFromInputs::requestTypes"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::setFromInputs::runtime"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::setFromInputs::seqSlots"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::setFromInputs::worldConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers15useSpecDecodingE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::useSpecDecoding"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCacheE", "tensorrt_llm::runtime::LoraCache"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9LoraCacheERK26LoraCachePageManagerConfigRK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::LoraCache::LoraCache"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9LoraCacheERK26LoraCachePageManagerConfigRK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::LoraCache::LoraCache::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9LoraCacheERK26LoraCachePageManagerConfigRK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::LoraCache::LoraCache::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9LoraCacheERK26LoraCachePageManagerConfigRK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::LoraCache::LoraCache::pageManagerConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9LoraCacheERK26LoraCachePageManagerConfigRK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::LoraCache::LoraCache::worldConfig"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache10TaskIdTypeE", "tensorrt_llm::runtime::LoraCache::TaskIdType"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfigE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig11adapterSizeE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::adapterSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig6inSizeE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::inSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig7layerIdE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::layerId"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig8moduleIdE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::moduleId"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig8numSlotsE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::numSlots"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfigeqERKN9LoraCache21TaskLayerModuleConfigE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::operator=="], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfigeqERKN9LoraCache21TaskLayerModuleConfigE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::operator==::o"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig7outSizeE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::outSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig6pageIdE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::pageId"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig17scalingVecPointerE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::scalingVecPointer"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig7slotIdxE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::slotIdx"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig8toStringEv", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::toString"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig16weightsInPointerE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::weightsInPointer"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig17weightsOutPointerE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::weightsOutPointer"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache28TaskLayerModuleConfigListPtrE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfigListPtr"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValueE", "tensorrt_llm::runtime::LoraCache::TaskValue"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERKNSt6vectorINSt6size_tEEERK28TaskLayerModuleConfigListPtrNSt4listI10TaskIdTypeE8iteratorEbbbb", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERR9TaskValue", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueEv", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERKNSt6vectorINSt6size_tEEERK28TaskLayerModuleConfigListPtrNSt4listI10TaskIdTypeE8iteratorEbbbb", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue::configs"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERKNSt6vectorINSt6size_tEEERK28TaskLayerModuleConfigListPtrNSt4listI10TaskIdTypeE8iteratorEbbbb", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue::done"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERKNSt6vectorINSt6size_tEEERK28TaskLayerModuleConfigListPtrNSt4listI10TaskIdTypeE8iteratorEbbbb", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue::inProgress"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERKNSt6vectorINSt6size_tEEERK28TaskLayerModuleConfigListPtrNSt4listI10TaskIdTypeE8iteratorEbbbb", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue::it"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERKNSt6vectorINSt6size_tEEERK28TaskLayerModuleConfigListPtrNSt4listI10TaskIdTypeE8iteratorEbbbb", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue::loadInProgress"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERKNSt6vectorINSt6size_tEEERK28TaskLayerModuleConfigListPtrNSt4listI10TaskIdTypeE8iteratorEbbbb", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue::loaded"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERR9TaskValue", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue::o"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERKNSt6vectorINSt6size_tEEERK28TaskLayerModuleConfigListPtrNSt4listI10TaskIdTypeE8iteratorEbbbb", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue::pageIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue7configsE", "tensorrt_llm::runtime::LoraCache::TaskValue::configs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue4doneE", "tensorrt_llm::runtime::LoraCache::TaskValue::done"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue10inProgressE", "tensorrt_llm::runtime::LoraCache::TaskValue::inProgress"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue2itE", "tensorrt_llm::runtime::LoraCache::TaskValue::it"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue14loadInProgressE", "tensorrt_llm::runtime::LoraCache::TaskValue::loadInProgress"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue6loadedE", "tensorrt_llm::runtime::LoraCache::TaskValue::loaded"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValueaSERR9TaskValue", "tensorrt_llm::runtime::LoraCache::TaskValue::operator="], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValueaSERR9TaskValue", "tensorrt_llm::runtime::LoraCache::TaskValue::operator=::o"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue7pageIdsE", "tensorrt_llm::runtime::LoraCache::TaskValue::pageIds"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValueD0Ev", "tensorrt_llm::runtime::LoraCache::TaskValue::~TaskValue"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache12TaskValuePtrE", "tensorrt_llm::runtime::LoraCache::TaskValuePtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TensorPtrE", "tensorrt_llm::runtime::LoraCache::TensorPtr"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatusE", "tensorrt_llm::runtime::LoraCache::ValueStatus"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatus20kVALUE_STATUS_LOADEDE", "tensorrt_llm::runtime::LoraCache::ValueStatus::kVALUE_STATUS_LOADED"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatus21kVALUE_STATUS_MISSINGE", "tensorrt_llm::runtime::LoraCache::ValueStatus::kVALUE_STATUS_MISSING"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatus24kVALUE_STATUS_PROCESSINGE", "tensorrt_llm::runtime::LoraCache::ValueStatus::kVALUE_STATUS_PROCESSING"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache4bumpE10TaskIdType", "tensorrt_llm::runtime::LoraCache::bump"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache4bumpE10TaskIdType", "tensorrt_llm::runtime::LoraCache::bump::taskId"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache18bumpTaskInProgressE10TaskIdType", "tensorrt_llm::runtime::LoraCache::bumpTaskInProgress"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache18bumpTaskInProgressE10TaskIdType", "tensorrt_llm::runtime::LoraCache::bumpTaskInProgress::taskId"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache19claimPagesWithEvictE10SizeType32", "tensorrt_llm::runtime::LoraCache::claimPagesWithEvict"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache19claimPagesWithEvictE10SizeType32", "tensorrt_llm::runtime::LoraCache::claimPagesWithEvict::numPages"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache8copyTaskE10TaskIdTypeR9LoraCacheb", "tensorrt_llm::runtime::LoraCache::copyTask"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache8copyTaskE10TaskIdTypeR9LoraCacheb", "tensorrt_llm::runtime::LoraCache::copyTask::deviceCache"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache8copyTaskE10TaskIdTypeR9LoraCacheb", "tensorrt_llm::runtime::LoraCache::copyTask::markDone"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache8copyTaskE10TaskIdTypeR9LoraCacheb", "tensorrt_llm::runtime::LoraCache::copyTask::taskId"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache16copyTaskMapPagesER9TaskValueRK9TaskValueRKNSt6vectorI6size_tEERK9LoraCache", "tensorrt_llm::runtime::LoraCache::copyTaskMapPages"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache16copyTaskMapPagesER9TaskValueRK9TaskValueRKNSt6vectorI6size_tEERK9LoraCache", "tensorrt_llm::runtime::LoraCache::copyTaskMapPages::sourceTaskValue"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache16copyTaskMapPagesER9TaskValueRK9TaskValueRKNSt6vectorI6size_tEERK9LoraCache", "tensorrt_llm::runtime::LoraCache::copyTaskMapPages::targetCache"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache16copyTaskMapPagesER9TaskValueRK9TaskValueRKNSt6vectorI6size_tEERK9LoraCache", "tensorrt_llm::runtime::LoraCache::copyTaskMapPages::targetPageIds"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache16copyTaskMapPagesER9TaskValueRK9TaskValueRKNSt6vectorI6size_tEERK9LoraCache", "tensorrt_llm::runtime::LoraCache::copyTaskMapPages::targetTaskValue"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCache::copyToPages"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCache::copyToPages::config"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCache::copyToPages::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCache::copyToPages::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCache::copyToPages::moduleIdToModel"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCache::copyToPages::pageIds"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCache::copyToPages::pages"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCache::copyToPages::weights"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCache::copyToPages::worldConfig"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache17determineNumPagesE10TaskIdType", "tensorrt_llm::runtime::LoraCache::determineNumPages"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache17determineNumPagesE9TensorPtr", "tensorrt_llm::runtime::LoraCache::determineNumPages"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache17determineNumPagesE9TensorPtr", "tensorrt_llm::runtime::LoraCache::determineNumPages::config"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache17determineNumPagesE10TaskIdType", "tensorrt_llm::runtime::LoraCache::determineNumPages::taskId"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache4fitsE9TensorPtr", "tensorrt_llm::runtime::LoraCache::fits"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache4fitsE9TensorPtr", "tensorrt_llm::runtime::LoraCache::fits::config"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache3getE10TaskIdType", "tensorrt_llm::runtime::LoraCache::get"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache3getE10TaskIdType", "tensorrt_llm::runtime::LoraCache::get::taskId"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache11getNumPagesEv", "tensorrt_llm::runtime::LoraCache::getNumPages"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache10getPagePtrE6size_t", "tensorrt_llm::runtime::LoraCache::getPagePtr"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache10getPagePtrE6size_t", "tensorrt_llm::runtime::LoraCache::getPagePtr::pageId"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache9getStatusE10TaskIdType", "tensorrt_llm::runtime::LoraCache::getStatus"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache9getStatusE10TaskIdType", "tensorrt_llm::runtime::LoraCache::getStatus::taskId"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache3hasE10TaskIdType", "tensorrt_llm::runtime::LoraCache::has"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache3hasE10TaskIdType", "tensorrt_llm::runtime::LoraCache::has::taskId"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache6isDoneE10TaskIdType", "tensorrt_llm::runtime::LoraCache::isDone"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache6isDoneE10TaskIdType", "tensorrt_llm::runtime::LoraCache::isDone::taskId"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache8isLoadedE10TaskIdType", "tensorrt_llm::runtime::LoraCache::isLoaded"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache8isLoadedE10TaskIdType", "tensorrt_llm::runtime::LoraCache::isLoaded::taskId"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatus20kVALUE_STATUS_LOADEDE", "tensorrt_llm::runtime::LoraCache::kVALUE_STATUS_LOADED"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatus21kVALUE_STATUS_MISSINGE", "tensorrt_llm::runtime::LoraCache::kVALUE_STATUS_MISSING"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatus24kVALUE_STATUS_PROCESSINGE", "tensorrt_llm::runtime::LoraCache::kVALUE_STATUS_PROCESSING"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsE10TaskIdType9TensorPtr9TensorPtr", "tensorrt_llm::runtime::LoraCache::loadWeights"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsER9TaskValue9TensorPtr9TensorPtr", "tensorrt_llm::runtime::LoraCache::loadWeights"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsER9TaskValue9TensorPtr9TensorPtr", "tensorrt_llm::runtime::LoraCache::loadWeights::cacheValue"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsE10TaskIdType9TensorPtr9TensorPtr", "tensorrt_llm::runtime::LoraCache::loadWeights::config"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsER9TaskValue9TensorPtr9TensorPtr", "tensorrt_llm::runtime::LoraCache::loadWeights::config"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsE10TaskIdType9TensorPtr9TensorPtr", "tensorrt_llm::runtime::LoraCache::loadWeights::taskId"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsE10TaskIdType9TensorPtr9TensorPtr", "tensorrt_llm::runtime::LoraCache::loadWeights::weights"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsER9TaskValue9TensorPtr9TensorPtr", "tensorrt_llm::runtime::LoraCache::loadWeights::weights"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache14mBufferManagerE", "tensorrt_llm::runtime::LoraCache::mBufferManager"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9mCacheMapE", "tensorrt_llm::runtime::LoraCache::mCacheMap"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11mCacheMutexE", "tensorrt_llm::runtime::LoraCache::mCacheMutex"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17mCachePageManagerE", "tensorrt_llm::runtime::LoraCache::mCachePageManager"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21mDeviceBufferManagersE", "tensorrt_llm::runtime::LoraCache::mDeviceBufferManagers"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache10mDoneTasksE", "tensorrt_llm::runtime::LoraCache::mDoneTasks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache16mInProgressTasksE", "tensorrt_llm::runtime::LoraCache::mInProgressTasks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache12mModelConfigE", "tensorrt_llm::runtime::LoraCache::mModelConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17mModuleIdToModuleE", "tensorrt_llm::runtime::LoraCache::mModuleIdToModule"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache18mPageManagerConfigE", "tensorrt_llm::runtime::LoraCache::mPageManagerConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11mPagesMutexE", "tensorrt_llm::runtime::LoraCache::mPagesMutex"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache12mWorldConfigE", "tensorrt_llm::runtime::LoraCache::mWorldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11markAllDoneEv", "tensorrt_llm::runtime::LoraCache::markAllDone"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache12markTaskDoneE10TaskIdType", "tensorrt_llm::runtime::LoraCache::markTaskDone"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache12markTaskDoneE10TaskIdType", "tensorrt_llm::runtime::LoraCache::markTaskDone::taskId"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache3putE10TaskIdType9TensorPtr9TensorPtrb", "tensorrt_llm::runtime::LoraCache::put"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache3putE10TaskIdType9TensorPtr9TensorPtrb", "tensorrt_llm::runtime::LoraCache::put::config"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache3putE10TaskIdType9TensorPtr9TensorPtrb", "tensorrt_llm::runtime::LoraCache::put::load"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache3putE10TaskIdType9TensorPtr9TensorPtrb", "tensorrt_llm::runtime::LoraCache::put::taskId"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache3putE10TaskIdType9TensorPtr9TensorPtrb", "tensorrt_llm::runtime::LoraCache::put::weights"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17splitTransposeCpuER7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpu"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17splitTransposeCpuER7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpu::input"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17splitTransposeCpuER7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpu::output"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17splitTransposeCpuER7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpu::tpRank"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17splitTransposeCpuER7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpu::tpSize"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime9LoraCache22splitTransposeCpuInnerEvR7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpuInner"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime9LoraCache22splitTransposeCpuInnerEvR7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpuInner::T"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime9LoraCache22splitTransposeCpuInnerEvR7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpuInner::input"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime9LoraCache22splitTransposeCpuInnerEvR7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpuInner::output"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime9LoraCache22splitTransposeCpuInnerEvR7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpuInner::tpRank"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime9LoraCache22splitTransposeCpuInnerEvR7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpuInner::tpSize"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime22LoraCacheFullExceptionE", "tensorrt_llm::runtime::LoraCacheFullException"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22LoraCacheFullException22LoraCacheFullExceptionERKNSt6stringE", "tensorrt_llm::runtime::LoraCacheFullException::LoraCacheFullException"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22LoraCacheFullException22LoraCacheFullExceptionERKNSt6stringE", "tensorrt_llm::runtime::LoraCacheFullException::LoraCacheFullException::msg"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22LoraCacheFullExceptionD0Ev", "tensorrt_llm::runtime::LoraCacheFullException::~LoraCacheFullException"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManagerE", "tensorrt_llm::runtime::LoraCachePageManager"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager20LoraCachePageManagerERK26LoraCachePageManagerConfigRK13BufferManager", "tensorrt_llm::runtime::LoraCachePageManager::LoraCachePageManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager20LoraCachePageManagerERK26LoraCachePageManagerConfigRK13BufferManager", "tensorrt_llm::runtime::LoraCachePageManager::LoraCachePageManager::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager20LoraCachePageManagerERK26LoraCachePageManagerConfigRK13BufferManager", "tensorrt_llm::runtime::LoraCachePageManager::LoraCachePageManager::config"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager9TensorPtrE", "tensorrt_llm::runtime::LoraCachePageManager::TensorPtr"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime20LoraCachePageManager8blockPtrE10SizeType32", "tensorrt_llm::runtime::LoraCachePageManager::blockPtr"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime20LoraCachePageManager8blockPtrE10SizeType32", "tensorrt_llm::runtime::LoraCachePageManager::blockPtr::blockIdx"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager10claimPagesE10SizeType32", "tensorrt_llm::runtime::LoraCachePageManager::claimPages"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager10claimPagesE10SizeType32", "tensorrt_llm::runtime::LoraCachePageManager::claimPages::numPages"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager10initializeERK13BufferManager", "tensorrt_llm::runtime::LoraCachePageManager::initialize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager10initializeERK13BufferManager", "tensorrt_llm::runtime::LoraCachePageManager::initialize::bufferManager"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager7mConfigE", "tensorrt_llm::runtime::LoraCachePageManager::mConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager12mFreePageIdsE", "tensorrt_llm::runtime::LoraCachePageManager::mFreePageIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager11mIsPageFreeE", "tensorrt_llm::runtime::LoraCachePageManager::mIsPageFree"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager11mPageBlocksE", "tensorrt_llm::runtime::LoraCachePageManager::mPageBlocks"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager14mutablePagePtrENSt6size_tE", "tensorrt_llm::runtime::LoraCachePageManager::mutablePagePtr"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager14mutablePagePtrENSt6size_tE", "tensorrt_llm::runtime::LoraCachePageManager::mutablePagePtr::pageIdx"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime20LoraCachePageManager17numAvailablePagesEv", "tensorrt_llm::runtime::LoraCachePageManager::numAvailablePages"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime20LoraCachePageManager7pagePtrENSt6size_tE", "tensorrt_llm::runtime::LoraCachePageManager::pagePtr"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime20LoraCachePageManager7pagePtrENSt6size_tE", "tensorrt_llm::runtime::LoraCachePageManager::pagePtr::pageIdx"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager12releasePagesERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCachePageManager::releasePages"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager12releasePagesERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCachePageManager::releasePages::pages"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfigE", "tensorrt_llm::runtime::LoraCachePageManagerConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig26LoraCachePageManagerConfigEN7runtime10MemoryTypeEN8nvinfer18DataTypeE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::LoraCachePageManagerConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig26LoraCachePageManagerConfigEN7runtime10MemoryTypeEN8nvinfer18DataTypeE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::LoraCachePageManagerConfig::dType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig26LoraCachePageManagerConfigEN7runtime10MemoryTypeEN8nvinfer18DataTypeE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::LoraCachePageManagerConfig::maxPagesPerBlock"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig26LoraCachePageManagerConfigEN7runtime10MemoryTypeEN8nvinfer18DataTypeE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::LoraCachePageManagerConfig::memType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig26LoraCachePageManagerConfigEN7runtime10MemoryTypeEN8nvinfer18DataTypeE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::LoraCachePageManagerConfig::numCopyStreams"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig26LoraCachePageManagerConfigEN7runtime10MemoryTypeEN8nvinfer18DataTypeE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::LoraCachePageManagerConfig::pageWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig26LoraCachePageManagerConfigEN7runtime10MemoryTypeEN8nvinfer18DataTypeE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::LoraCachePageManagerConfig::slotsPerPage"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig26LoraCachePageManagerConfigEN7runtime10MemoryTypeEN8nvinfer18DataTypeE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::LoraCachePageManagerConfig::totalNumPages"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig11getDataTypeEv", "tensorrt_llm::runtime::LoraCachePageManagerConfig::getDataType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig13getInitToZeroEv", "tensorrt_llm::runtime::LoraCachePageManagerConfig::getInitToZero"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig19getMaxPagesPerBlockEv", "tensorrt_llm::runtime::LoraCachePageManagerConfig::getMaxPagesPerBlock"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig13getMemoryTypeEv", "tensorrt_llm::runtime::LoraCachePageManagerConfig::getMemoryType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig17getNumCopyStreamsEv", "tensorrt_llm::runtime::LoraCachePageManagerConfig::getNumCopyStreams"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig12getPageWidthEv", "tensorrt_llm::runtime::LoraCachePageManagerConfig::getPageWidth"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig15getSlotsPerPageEv", "tensorrt_llm::runtime::LoraCachePageManagerConfig::getSlotsPerPage"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig16getTotalNumPagesEv", "tensorrt_llm::runtime::LoraCachePageManagerConfig::getTotalNumPages"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig9mDataTypeE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::mDataType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig11mInitToZeroE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::mInitToZero"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig17mMaxPagesPerBlockE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::mMaxPagesPerBlock"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig11mMemoryTypeE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::mMemoryType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig15mNumCopyStreamsE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::mNumCopyStreams"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig10mPageWidthE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::mPageWidth"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig13mSlotsPerPageE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::mSlotsPerPage"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig14mTotalNumPagesE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::mTotalNumPages"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig11setDataTypeERKN8nvinfer18DataTypeE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setDataType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig11setDataTypeERKN8nvinfer18DataTypeE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setDataType::dtype"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig13setInitToZeroEb", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setInitToZero"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig13setInitToZeroEb", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setInitToZero::initToZero"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig19setMaxPagesPerBlockERK10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setMaxPagesPerBlock"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig19setMaxPagesPerBlockERK10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setMaxPagesPerBlock::maxPagesPerBlock"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig13setMemoryTypeERKN7runtime10MemoryTypeE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setMemoryType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig13setMemoryTypeERKN7runtime10MemoryTypeE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setMemoryType::memoryType"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig17setNumCopyStreamsE10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setNumCopyStreams"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig17setNumCopyStreamsE10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setNumCopyStreams::numCopyStreams"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig12setPageWidthERK10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setPageWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig12setPageWidthERK10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setPageWidth::pageWidth"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig15setSlotsPerPageERK10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setSlotsPerPage"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig15setSlotsPerPageERK10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setSlotsPerPage::slotsPerPage"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig15setTotalNumPageERK10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setTotalNumPage"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig15setTotalNumPageERK10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setTotalNumPage::totalNumPages"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime21LoraExpectedExceptionE", "tensorrt_llm::runtime::LoraExpectedException"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime21LoraExpectedException21LoraExpectedExceptionERKNSt6stringE", "tensorrt_llm::runtime::LoraExpectedException::LoraExpectedException"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime21LoraExpectedException21LoraExpectedExceptionERKNSt6stringE", "tensorrt_llm::runtime::LoraExpectedException::LoraExpectedException::msg"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime21LoraExpectedExceptionD0Ev", "tensorrt_llm::runtime::LoraExpectedException::~LoraExpectedException"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModuleE", "tensorrt_llm::runtime::LoraModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10LoraModule", "tensorrt_llm::runtime::LoraModule::LoraModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10ModuleType10SizeType3210SizeType32bb10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::LoraModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleEv", "tensorrt_llm::runtime::LoraModule::LoraModule"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10ModuleType10SizeType3210SizeType32bb10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::LoraModule::inDim"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10ModuleType10SizeType3210SizeType32bb10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::LoraModule::inDimFirst"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10ModuleType10SizeType3210SizeType32bb10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::LoraModule::inTpSplitDim"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10LoraModule", "tensorrt_llm::runtime::LoraModule::LoraModule::o"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10ModuleType10SizeType3210SizeType32bb10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::LoraModule::outDim"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10ModuleType10SizeType3210SizeType32bb10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::LoraModule::outDimFirst"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10ModuleType10SizeType3210SizeType32bb10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::LoraModule::outTpSplitDim"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10ModuleType10SizeType3210SizeType32bb10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::LoraModule::t"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleTypeE", "tensorrt_llm::runtime::LoraModule::ModuleType"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType11kATTN_DENSEE", "tensorrt_llm::runtime::LoraModule::ModuleType::kATTN_DENSE"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType7kATTN_KE", "tensorrt_llm::runtime::LoraModule::ModuleType::kATTN_K"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType7kATTN_QE", "tensorrt_llm::runtime::LoraModule::ModuleType::kATTN_Q"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType9kATTN_QKVE", "tensorrt_llm::runtime::LoraModule::ModuleType::kATTN_QKV"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType7kATTN_VE", "tensorrt_llm::runtime::LoraModule::ModuleType::kATTN_V"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType17kCROSS_ATTN_DENSEE", "tensorrt_llm::runtime::LoraModule::ModuleType::kCROSS_ATTN_DENSE"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType13kCROSS_ATTN_KE", "tensorrt_llm::runtime::LoraModule::ModuleType::kCROSS_ATTN_K"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType13kCROSS_ATTN_QE", "tensorrt_llm::runtime::LoraModule::ModuleType::kCROSS_ATTN_Q"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType15kCROSS_ATTN_QKVE", "tensorrt_llm::runtime::LoraModule::ModuleType::kCROSS_ATTN_QKV"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType13kCROSS_ATTN_VE", "tensorrt_llm::runtime::LoraModule::ModuleType::kCROSS_ATTN_V"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType8kINVALIDE", "tensorrt_llm::runtime::LoraModule::ModuleType::kINVALID"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMLP_4H_TO_HE", "tensorrt_llm::runtime::LoraModule::ModuleType::kMLP_4H_TO_H"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType9kMLP_GATEE", "tensorrt_llm::runtime::LoraModule::ModuleType::kMLP_GATE"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMLP_GATE_UPE", "tensorrt_llm::runtime::LoraModule::ModuleType::kMLP_GATE_UP"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMLP_H_TO_4HE", "tensorrt_llm::runtime::LoraModule::ModuleType::kMLP_H_TO_4H"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType11kMLP_ROUTERE", "tensorrt_llm::runtime::LoraModule::ModuleType::kMLP_ROUTER"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMOE_4H_TO_HE", "tensorrt_llm::runtime::LoraModule::ModuleType::kMOE_4H_TO_H"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType9kMOE_GATEE", "tensorrt_llm::runtime::LoraModule::ModuleType::kMOE_GATE"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMOE_H_TO_4HE", "tensorrt_llm::runtime::LoraModule::ModuleType::kMOE_H_TO_4H"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType11kMOE_ROUTERE", "tensorrt_llm::runtime::LoraModule::ModuleType::kMOE_ROUTER"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule9TensorPtrE", "tensorrt_llm::runtime::LoraModule::TensorPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::createLoraModules"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::createLoraModules::attentionHeadSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::createLoraModules::hiddenSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::createLoraModules::loraModuleNames"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::createLoraModules::mlpHiddenSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::createLoraModules::numAttentionHeads"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::createLoraModules::numExperts"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::createLoraModules::numKvAttentionHeads"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::createLoraModules::tpSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule18flattenedInOutSizeE10SizeType32b", "tensorrt_llm::runtime::LoraModule::flattenedInOutSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule18flattenedInOutSizeE10SizeType32b", "tensorrt_llm::runtime::LoraModule::flattenedInOutSize::adapterSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule18flattenedInOutSizeE10SizeType32b", "tensorrt_llm::runtime::LoraModule::flattenedInOutSize::isDora"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule5inDimEv", "tensorrt_llm::runtime::LoraModule::inDim"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule10inDimFirstEv", "tensorrt_llm::runtime::LoraModule::inDimFirst"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule6inSizeE10SizeType32", "tensorrt_llm::runtime::LoraModule::inSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule6inSizeE10SizeType32", "tensorrt_llm::runtime::LoraModule::inSize::adapterSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule12inTpSplitDimEv", "tensorrt_llm::runtime::LoraModule::inTpSplitDim"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule18localInAdapterSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localInAdapterSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule18localInAdapterSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localInAdapterSize::adapterSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule18localInAdapterSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localInAdapterSize::tpSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule10localInDimE10SizeType32", "tensorrt_llm::runtime::LoraModule::localInDim"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule10localInDimE10SizeType32", "tensorrt_llm::runtime::LoraModule::localInDim::tpSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule14localInOutSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localInOutSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule14localInOutSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localInOutSize::adapterSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule14localInOutSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localInOutSize::tpSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule11localInSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localInSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule11localInSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localInSize::adapterSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule11localInSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localInSize::tpSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule19localOutAdapterSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localOutAdapterSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule19localOutAdapterSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localOutAdapterSize::adapterSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule19localOutAdapterSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localOutAdapterSize::tpSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule11localOutDimE10SizeType32", "tensorrt_llm::runtime::LoraModule::localOutDim"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule11localOutDimE10SizeType32", "tensorrt_llm::runtime::LoraModule::localOutDim::tpSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule12localOutSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localOutSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule12localOutSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localOutSize::adapterSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule12localOutSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localOutSize::tpSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule15localScalesSizeE10SizeType32b", "tensorrt_llm::runtime::LoraModule::localScalesSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule15localScalesSizeE10SizeType32b", "tensorrt_llm::runtime::LoraModule::localScalesSize::isDora"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule15localScalesSizeE10SizeType32b", "tensorrt_llm::runtime::LoraModule::localScalesSize::tpSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule14localTotalSizeE10SizeType3210SizeType32b", "tensorrt_llm::runtime::LoraModule::localTotalSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule14localTotalSizeE10SizeType3210SizeType32b", "tensorrt_llm::runtime::LoraModule::localTotalSize::adapterSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule14localTotalSizeE10SizeType3210SizeType32b", "tensorrt_llm::runtime::LoraModule::localTotalSize::isDora"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule14localTotalSizeE10SizeType3210SizeType32b", "tensorrt_llm::runtime::LoraModule::localTotalSize::tpSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule6mInDimE", "tensorrt_llm::runtime::LoraModule::mInDim"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule11mInDimFirstE", "tensorrt_llm::runtime::LoraModule::mInDimFirst"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule13mInTpSplitDimE", "tensorrt_llm::runtime::LoraModule::mInTpSplitDim"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule7mOutDimE", "tensorrt_llm::runtime::LoraModule::mOutDim"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12mOutDimFirstE", "tensorrt_llm::runtime::LoraModule::mOutDimFirst"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule14mOutTpSplitDimE", "tensorrt_llm::runtime::LoraModule::mOutTpSplitDim"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule5mTypeE", "tensorrt_llm::runtime::LoraModule::mType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule4nameEv", "tensorrt_llm::runtime::LoraModule::name"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModuleaSERK10LoraModule", "tensorrt_llm::runtime::LoraModule::operator="], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModuleaSERK10LoraModule", "tensorrt_llm::runtime::LoraModule::operator=::o"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule6outDimEv", "tensorrt_llm::runtime::LoraModule::outDim"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule11outDimFirstEv", "tensorrt_llm::runtime::LoraModule::outDimFirst"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule7outSizeE10SizeType32", "tensorrt_llm::runtime::LoraModule::outSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule7outSizeE10SizeType32", "tensorrt_llm::runtime::LoraModule::outSize::adapterSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule13outTpSplitDimEv", "tensorrt_llm::runtime::LoraModule::outTpSplitDim"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12toModuleNameE10ModuleType", "tensorrt_llm::runtime::LoraModule::toModuleName"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12toModuleNameE10SizeType32", "tensorrt_llm::runtime::LoraModule::toModuleName"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12toModuleNameE10SizeType32", "tensorrt_llm::runtime::LoraModule::toModuleName::id"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12toModuleNameE10ModuleType", "tensorrt_llm::runtime::LoraModule::toModuleName::t"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12toModuleTypeERKNSt11string_viewE", "tensorrt_llm::runtime::LoraModule::toModuleType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12toModuleTypeERKNSt11string_viewE", "tensorrt_llm::runtime::LoraModule::toModuleType::name"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule5valueEv", "tensorrt_llm::runtime::LoraModule::value"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14LoraTaskIdTypeE", "tensorrt_llm::runtime::LoraTaskIdType"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime17MPI_group_barrierENSt3setIiEE", "tensorrt_llm::runtime::MPI_group_barrier"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17MPI_group_barrierENSt3setIiEE", "tensorrt_llm::runtime::MPI_group_barrier::ranks"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime12MedusaModuleE", "tensorrt_llm::runtime::MedusaModule"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule13MedusaChoicesE", "tensorrt_llm::runtime::MedusaModule::MedusaChoices"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule12MedusaModuleE10SizeType3210SizeType32", "tensorrt_llm::runtime::MedusaModule::MedusaModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule12MedusaModuleEv", "tensorrt_llm::runtime::MedusaModule::MedusaModule"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule12MedusaModuleE10SizeType3210SizeType32", "tensorrt_llm::runtime::MedusaModule::MedusaModule::maxAcceptedTokens"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule12MedusaModuleE10SizeType3210SizeType32", "tensorrt_llm::runtime::MedusaModule::MedusaModule::maxDraftTokens"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule9TensorPtrE", "tensorrt_llm::runtime::MedusaModule::TensorPtr"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime12MedusaModule16getMedusaChoicesEv", "tensorrt_llm::runtime::MedusaModule::getMedusaChoices"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule21mDefaultMedusaChoicesE", "tensorrt_llm::runtime::MedusaModule::mDefaultMedusaChoices"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCountersE", "tensorrt_llm::runtime::MemoryCounters"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8DiffTypeE", "tensorrt_llm::runtime::MemoryCounters::DiffType"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters14MemoryCountersEv", "tensorrt_llm::runtime::MemoryCounters::MemoryCounters"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters10SizeType32E", "tensorrt_llm::runtime::MemoryCounters::SizeType32"], [1, 3, 1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime14MemoryCounters8allocateEv10SizeType32", "tensorrt_llm::runtime::MemoryCounters::allocate"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8allocateE10MemoryType10SizeType32", "tensorrt_llm::runtime::MemoryCounters::allocate"], [1, 8, 1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime14MemoryCounters8allocateEv10SizeType32", "tensorrt_llm::runtime::MemoryCounters::allocate::T"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8allocateE10MemoryType10SizeType32", "tensorrt_llm::runtime::MemoryCounters::allocate::memoryType"], [1, 4, 1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime14MemoryCounters8allocateEv10SizeType32", "tensorrt_llm::runtime::MemoryCounters::allocate::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8allocateE10MemoryType10SizeType32", "tensorrt_llm::runtime::MemoryCounters::allocate::size"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters13bytesToStringE10SizeType32i", "tensorrt_llm::runtime::MemoryCounters::bytesToString"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters13bytesToStringE8DiffTypei", "tensorrt_llm::runtime::MemoryCounters::bytesToString"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters13bytesToStringE10SizeType32i", "tensorrt_llm::runtime::MemoryCounters::bytesToString::bytes"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters13bytesToStringE8DiffTypei", "tensorrt_llm::runtime::MemoryCounters::bytesToString::bytes"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters13bytesToStringE10SizeType32i", "tensorrt_llm::runtime::MemoryCounters::bytesToString::precision"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters13bytesToStringE8DiffTypei", "tensorrt_llm::runtime::MemoryCounters::bytesToString::precision"], [1, 3, 1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime14MemoryCounters10deallocateEv10SizeType32", "tensorrt_llm::runtime::MemoryCounters::deallocate"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters10deallocateE10MemoryType10SizeType32", "tensorrt_llm::runtime::MemoryCounters::deallocate"], [1, 8, 1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime14MemoryCounters10deallocateEv10SizeType32", "tensorrt_llm::runtime::MemoryCounters::deallocate::T"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters10deallocateE10MemoryType10SizeType32", "tensorrt_llm::runtime::MemoryCounters::deallocate::memoryType"], [1, 4, 1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime14MemoryCounters10deallocateEv10SizeType32", "tensorrt_llm::runtime::MemoryCounters::deallocate::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters10deallocateE10MemoryType10SizeType32", "tensorrt_llm::runtime::MemoryCounters::deallocate::size"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters6getCpuEv", "tensorrt_llm::runtime::MemoryCounters::getCpu"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters10getCpuDiffEv", "tensorrt_llm::runtime::MemoryCounters::getCpuDiff"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters6getGpuEv", "tensorrt_llm::runtime::MemoryCounters::getGpu"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters10getGpuDiffEv", "tensorrt_llm::runtime::MemoryCounters::getGpuDiff"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters11getInstanceEv", "tensorrt_llm::runtime::MemoryCounters::getInstance"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters9getPinnedEv", "tensorrt_llm::runtime::MemoryCounters::getPinned"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters13getPinnedDiffEv", "tensorrt_llm::runtime::MemoryCounters::getPinnedDiff"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters13getPinnedPoolEv", "tensorrt_llm::runtime::MemoryCounters::getPinnedPool"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters17getPinnedPoolDiffEv", "tensorrt_llm::runtime::MemoryCounters::getPinnedPoolDiff"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters6getUVMEv", "tensorrt_llm::runtime::MemoryCounters::getUVM"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters10getUVMDiffEv", "tensorrt_llm::runtime::MemoryCounters::getUVMDiff"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters4mCpuE", "tensorrt_llm::runtime::MemoryCounters::mCpu"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8mCpuDiffE", "tensorrt_llm::runtime::MemoryCounters::mCpuDiff"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters4mGpuE", "tensorrt_llm::runtime::MemoryCounters::mGpu"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8mGpuDiffE", "tensorrt_llm::runtime::MemoryCounters::mGpuDiff"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters7mPinnedE", "tensorrt_llm::runtime::MemoryCounters::mPinned"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters11mPinnedDiffE", "tensorrt_llm::runtime::MemoryCounters::mPinnedDiff"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters11mPinnedPoolE", "tensorrt_llm::runtime::MemoryCounters::mPinnedPool"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters15mPinnedPoolDiffE", "tensorrt_llm::runtime::MemoryCounters::mPinnedPoolDiff"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters4mUVME", "tensorrt_llm::runtime::MemoryCounters::mUVM"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8mUVMDiffE", "tensorrt_llm::runtime::MemoryCounters::mUVMDiff"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters8toStringEv", "tensorrt_llm::runtime::MemoryCounters::toString"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime10MemoryTypeE", "tensorrt_llm::runtime::MemoryType"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10MemoryType4kCPUE", "tensorrt_llm::runtime::MemoryType::kCPU"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10MemoryType4kGPUE", "tensorrt_llm::runtime::MemoryType::kGPU"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10MemoryType7kPINNEDE", "tensorrt_llm::runtime::MemoryType::kPINNED"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10MemoryType11kPINNEDPOOLE", "tensorrt_llm::runtime::MemoryType::kPINNEDPOOL"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10MemoryType4kUVME", "tensorrt_llm::runtime::MemoryType::kUVM"], [1, 2, 1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime16MemoryTypeStringE", "tensorrt_llm::runtime::MemoryTypeString"], [1, 8, 1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime16MemoryTypeStringE", "tensorrt_llm::runtime::MemoryTypeString::T"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kCPUEEE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kCPU>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kCPUEE5valueE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kCPU>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kGPUEEE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kGPU>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kGPUEE5valueE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kGPU>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType7kPINNEDEEE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kPINNED>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType7kPINNEDEE5valueE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kPINNED>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType11kPINNEDPOOLEEE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kPINNEDPOOL>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType11kPINNEDPOOLEE5valueE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kPINNEDPOOL>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kUVMEEE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kUVM>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kUVMEE5valueE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kUVM>::value"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfiguratorE", "tensorrt_llm::runtime::MemsetConfigurator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator18MemsetConfiguratorE11CUdeviceptr6size_t7uint8_t8CUstream", "tensorrt_llm::runtime::MemsetConfigurator::MemsetConfigurator"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator18MemsetConfiguratorE11CUdeviceptr6size_t7uint8_t8CUstream", "tensorrt_llm::runtime::MemsetConfigurator::MemsetConfigurator::address"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator18MemsetConfiguratorE11CUdeviceptr6size_t7uint8_t8CUstream", "tensorrt_llm::runtime::MemsetConfigurator::MemsetConfigurator::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator18MemsetConfiguratorE11CUdeviceptr6size_t7uint8_t8CUstream", "tensorrt_llm::runtime::MemsetConfigurator::MemsetConfigurator::stream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator18MemsetConfiguratorE11CUdeviceptr6size_t7uint8_t8CUstream", "tensorrt_llm::runtime::MemsetConfigurator::MemsetConfigurator::value"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator8mAddressE", "tensorrt_llm::runtime::MemsetConfigurator::mAddress"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator10mFirstTimeE", "tensorrt_llm::runtime::MemsetConfigurator::mFirstTime"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator5mSizeE", "tensorrt_llm::runtime::MemsetConfigurator::mSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator7mStreamE", "tensorrt_llm::runtime::MemsetConfigurator::mStream"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator6mValueE", "tensorrt_llm::runtime::MemsetConfigurator::mValue"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator5setupE28CUmemGenericAllocationHandle", "tensorrt_llm::runtime::MemsetConfigurator::setup"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator8teardownE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::MemsetConfigurator::teardown"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfigE", "tensorrt_llm::runtime::ModelConfig"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11KVCacheTypeE", "tensorrt_llm::runtime::ModelConfig::KVCacheType"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11KVCacheType11kCONTINUOUSE", "tensorrt_llm::runtime::ModelConfig::KVCacheType::kCONTINUOUS"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11KVCacheType9kDISABLEDE", "tensorrt_llm::runtime::ModelConfig::KVCacheType::kDISABLED"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11KVCacheType6kPAGEDE", "tensorrt_llm::runtime::ModelConfig::KVCacheType::kPAGED"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21KVCacheTypeFromStringENSt6stringE", "tensorrt_llm::runtime::ModelConfig::KVCacheTypeFromString"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21KVCacheTypeFromStringENSt6stringE", "tensorrt_llm::runtime::ModelConfig::KVCacheTypeFromString::value"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerTypeE", "tensorrt_llm::runtime::ModelConfig::LayerType"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerType10kATTENTIONE", "tensorrt_llm::runtime::ModelConfig::LayerType::kATTENTION"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerType7kLINEARE", "tensorrt_llm::runtime::ModelConfig::LayerType::kLINEAR"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerType5kNOOPE", "tensorrt_llm::runtime::ModelConfig::LayerType::kNOOP"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerType10kRECURRENTE", "tensorrt_llm::runtime::ModelConfig::LayerType::kRECURRENT"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17ManageWeightsTypeE", "tensorrt_llm::runtime::ModelConfig::ManageWeightsType"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17ManageWeightsType9kDisabledE", "tensorrt_llm::runtime::ModelConfig::ManageWeightsType::kDisabled"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17ManageWeightsType8kEnabledE", "tensorrt_llm::runtime::ModelConfig::ManageWeightsType::kEnabled"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11ModelConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::ModelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11ModelConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::ModelConfig::dtype"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11ModelConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::ModelConfig::hiddenSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11ModelConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::ModelConfig::nbAttentionLayers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11ModelConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::ModelConfig::nbHeads"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11ModelConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::ModelConfig::nbLayers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11ModelConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::ModelConfig::nbRnnLayers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11ModelConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::ModelConfig::vocabSize"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariantE", "tensorrt_llm::runtime::ModelConfig::ModelVariant"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant8kChatGlmE", "tensorrt_llm::runtime::ModelConfig::ModelVariant::kChatGlm"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant7kEncDecE", "tensorrt_llm::runtime::ModelConfig::ModelVariant::kEncDec"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant4kGlmE", "tensorrt_llm::runtime::ModelConfig::ModelVariant::kGlm"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant4kGptE", "tensorrt_llm::runtime::ModelConfig::ModelVariant::kGpt"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant6kMambaE", "tensorrt_llm::runtime::ModelConfig::ModelVariant::kMamba"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant15kRecurrentGemmaE", "tensorrt_llm::runtime::ModelConfig::ModelVariant::kRecurrentGemma"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfigE", "tensorrt_llm::runtime::ModelConfig::RnnConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig10convKernelE", "tensorrt_llm::runtime::ModelConfig::RnnConfig::convKernel"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig14rnnConvDimSizeE", "tensorrt_llm::runtime::ModelConfig::RnnConfig::rnnConvDimSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig11rnnHeadSizeE", "tensorrt_llm::runtime::ModelConfig::RnnConfig::rnnHeadSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig13rnnHiddenSizeE", "tensorrt_llm::runtime::ModelConfig::RnnConfig::rnnHiddenSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig9stateSizeE", "tensorrt_llm::runtime::ModelConfig::RnnConfig::stateSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20computeContextLogitsEb", "tensorrt_llm::runtime::ModelConfig::computeContextLogits"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20computeContextLogitsEv", "tensorrt_llm::runtime::ModelConfig::computeContextLogits"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20computeContextLogitsEb", "tensorrt_llm::runtime::ModelConfig::computeContextLogits::computeContextLogits"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig23computeGenerationLogitsEb", "tensorrt_llm::runtime::ModelConfig::computeGenerationLogits"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig23computeGenerationLogitsEv", "tensorrt_llm::runtime::ModelConfig::computeGenerationLogits"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig23computeGenerationLogitsEb", "tensorrt_llm::runtime::ModelConfig::computeGenerationLogits::computeGenerationLogits"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16countLocalLayersE9LayerType10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::countLocalLayers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16countLocalLayersE9LayerType10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::countLocalLayers::layerType"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16countLocalLayersE9LayerType10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::countLocalLayers::pipelineParallelism"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16countLocalLayersE9LayerType10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::countLocalLayers::pipelineParallelismRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20countLowerRankLayersE9LayerType10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::countLowerRankLayers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20countLowerRankLayersE9LayerType10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::countLowerRankLayers::layerType"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20countLowerRankLayersE9LayerType10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::countLowerRankLayers::pipelineParallelism"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20countLowerRankLayersE9LayerType10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::countLowerRankLayers::pipelineParallelismRank"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig32disableSeamlessLookaheadDecodingEv", "tensorrt_llm::runtime::ModelConfig::disableSeamlessLookaheadDecoding"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig31enableSeamlessLookaheadDecodingE10SizeType32", "tensorrt_llm::runtime::ModelConfig::enableSeamlessLookaheadDecoding"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig31enableSeamlessLookaheadDecodingE10SizeType32", "tensorrt_llm::runtime::ModelConfig::enableSeamlessLookaheadDecoding::maxDraftTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getContextFMHAEv", "tensorrt_llm::runtime::ModelConfig::getContextFMHA"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig11getDataTypeEv", "tensorrt_llm::runtime::ModelConfig::getDataType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getEncoderHiddenSizeEv", "tensorrt_llm::runtime::ModelConfig::getEncoderHiddenSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18getFirstLocalLayerE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getFirstLocalLayer"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18getFirstLocalLayerE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getFirstLocalLayer::pipelineParallelism"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18getFirstLocalLayerE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getFirstLocalLayer::pipelineParallelismRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21getGemmAllReduceDtypeEv", "tensorrt_llm::runtime::ModelConfig::getGemmAllReduceDtype"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13getHiddenSizeEv", "tensorrt_llm::runtime::ModelConfig::getHiddenSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getKVCacheTypeEv", "tensorrt_llm::runtime::ModelConfig::getKVCacheType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13getKvDataTypeEv", "tensorrt_llm::runtime::ModelConfig::getKvDataType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13getLayerTypesEv", "tensorrt_llm::runtime::ModelConfig::getLayerTypes"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getLogitsDtypeEv", "tensorrt_llm::runtime::ModelConfig::getLogitsDtype"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getLoraModulesEv", "tensorrt_llm::runtime::ModelConfig::getLoraModules"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getManageWeightsTypeEv", "tensorrt_llm::runtime::ModelConfig::getManageWeightsType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getMaxBatchSizeEv", "tensorrt_llm::runtime::ModelConfig::getMaxBatchSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getMaxBeamWidthEv", "tensorrt_llm::runtime::ModelConfig::getMaxBeamWidth"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig25getMaxDecodingDraftTokensEv", "tensorrt_llm::runtime::ModelConfig::getMaxDecodingDraftTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getMaxDecodingTokensEv", "tensorrt_llm::runtime::ModelConfig::getMaxDecodingTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16getMaxEncoderLenEv", "tensorrt_llm::runtime::ModelConfig::getMaxEncoderLen"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getMaxInputLenEv", "tensorrt_llm::runtime::ModelConfig::getMaxInputLen"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getMaxLoraRankEv", "tensorrt_llm::runtime::ModelConfig::getMaxLoraRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getMaxNumTokensEv", "tensorrt_llm::runtime::ModelConfig::getMaxNumTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig24getMaxPositionEmbeddingsEv", "tensorrt_llm::runtime::ModelConfig::getMaxPositionEmbeddings"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig30getMaxPromptEmbeddingTableSizeEv", "tensorrt_llm::runtime::ModelConfig::getMaxPromptEmbeddingTableSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig17getMaxSequenceLenEv", "tensorrt_llm::runtime::ModelConfig::getMaxSequenceLen"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16getMlpHiddenSizeEv", "tensorrt_llm::runtime::ModelConfig::getMlpHiddenSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getModelNameEv", "tensorrt_llm::runtime::ModelConfig::getModelName"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getModelVariantEv", "tensorrt_llm::runtime::ModelConfig::getModelVariant"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getNbAttentionLayersE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbAttentionLayers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getNbAttentionLayersE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbAttentionLayers::pipelineParallelism"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getNbAttentionLayersE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbAttentionLayers::pipelineParallelismRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig10getNbHeadsEv", "tensorrt_llm::runtime::ModelConfig::getNbHeads"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getNbKvHeadsE10SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbKvHeads"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getNbKvHeadsE10SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbKvHeads::layerIdx"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig11getNbLayersE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbLayers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig11getNbLayersE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbLayers::pipelineParallelism"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig11getNbLayersE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbLayers::pipelineParallelismRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getNbRnnLayersE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbRnnLayers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getNbRnnLayersE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbRnnLayers::pipelineParallelism"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getNbRnnLayersE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbRnnLayers::pipelineParallelismRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig27getNumKvHeadsForGivenLayersERKNSt6vectorI10SizeType32EEb", "tensorrt_llm::runtime::ModelConfig::getNumKvHeadsForGivenLayers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig27getNumKvHeadsForGivenLayersERKNSt6vectorI10SizeType32EEb", "tensorrt_llm::runtime::ModelConfig::getNumKvHeadsForGivenLayers::isCrossAttention"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig27getNumKvHeadsForGivenLayersERKNSt6vectorI10SizeType32EEb", "tensorrt_llm::runtime::ModelConfig::getNumKvHeadsForGivenLayers::layers"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21getNumKvHeadsPerLayerEv", "tensorrt_llm::runtime::ModelConfig::getNumKvHeadsPerLayer"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig31getNumKvHeadsPerLayerLocalRangeE10SizeType3210SizeType32b", "tensorrt_llm::runtime::ModelConfig::getNumKvHeadsPerLayerLocalRange"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig31getNumKvHeadsPerLayerLocalRangeE10SizeType3210SizeType32b", "tensorrt_llm::runtime::ModelConfig::getNumKvHeadsPerLayerLocalRange::isCrossAttention"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig31getNumKvHeadsPerLayerLocalRangeE10SizeType3210SizeType32b", "tensorrt_llm::runtime::ModelConfig::getNumKvHeadsPerLayerLocalRange::pipelineParallelism"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig31getNumKvHeadsPerLayerLocalRangeE10SizeType3210SizeType32b", "tensorrt_llm::runtime::ModelConfig::getNumKvHeadsPerLayerLocalRange::pipelineParallelismRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getNumLanguagesEv", "tensorrt_llm::runtime::ModelConfig::getNumLanguages"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig25getOptProfilesSplitPointsEv", "tensorrt_llm::runtime::ModelConfig::getOptProfilesSplitPoints"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig19getPagedContextFMHAEv", "tensorrt_llm::runtime::ModelConfig::getPagedContextFMHA"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18getPpReduceScatterEv", "tensorrt_llm::runtime::ModelConfig::getPpReduceScatter"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getQuantModeEv", "tensorrt_llm::runtime::ModelConfig::getQuantMode"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getRnnConfigEv", "tensorrt_llm::runtime::ModelConfig::getRnnConfig"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21getRotaryEmbeddingDimEv", "tensorrt_llm::runtime::ModelConfig::getRotaryEmbeddingDim"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getSizePerHeadEv", "tensorrt_llm::runtime::ModelConfig::getSizePerHead"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig26getSpeculativeDecodingModeEv", "tensorrt_llm::runtime::ModelConfig::getSpeculativeDecodingMode"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig28getSpeculativeDecodingModuleEv", "tensorrt_llm::runtime::ModelConfig::getSpeculativeDecodingModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig31getSpeculativeDecodingModulePtrEv", "tensorrt_llm::runtime::ModelConfig::getSpeculativeDecodingModulePtr"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig31getSpeculativeDecodingModulePtrEv", "tensorrt_llm::runtime::ModelConfig::getSpeculativeDecodingModulePtr"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig17getTokensPerBlockEv", "tensorrt_llm::runtime::ModelConfig::getTokensPerBlock"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getVocabSizeEv", "tensorrt_llm::runtime::ModelConfig::getVocabSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18getVocabSizePaddedE10SizeType32", "tensorrt_llm::runtime::ModelConfig::getVocabSizePadded"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18getVocabSizePaddedE10SizeType32", "tensorrt_llm::runtime::ModelConfig::getVocabSizePadded::worldSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12hasRnnConfigEv", "tensorrt_llm::runtime::ModelConfig::hasRnnConfig"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig28hasSpeculativeDecodingModuleEv", "tensorrt_llm::runtime::ModelConfig::hasSpeculativeDecodingModule"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig19isContinuousKVCacheEv", "tensorrt_llm::runtime::ModelConfig::isContinuousKVCache"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16isKVCacheEnabledEv", "tensorrt_llm::runtime::ModelConfig::isKVCacheEnabled"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12isMultiModalEv", "tensorrt_llm::runtime::ModelConfig::isMultiModal"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14isPagedKVCacheEv", "tensorrt_llm::runtime::ModelConfig::isPagedKVCache"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig10isRnnBasedEv", "tensorrt_llm::runtime::ModelConfig::isRnnBased"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18isTransformerBasedEv", "tensorrt_llm::runtime::ModelConfig::isTransformerBased"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig9isWhisperEv", "tensorrt_llm::runtime::ModelConfig::isWhisper"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig29kDEFAULT_NUM_TOKENS_PER_BLOCKE", "tensorrt_llm::runtime::ModelConfig::kDEFAULT_NUM_TOKENS_PER_BLOCK"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26kOPT_PROFILES_SPLIT_POINTSE", "tensorrt_llm::runtime::ModelConfig::kOPT_PROFILES_SPLIT_POINTS"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21mComputeContextLogitsE", "tensorrt_llm::runtime::ModelConfig::mComputeContextLogits"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24mComputeGenerationLogitsE", "tensorrt_llm::runtime::ModelConfig::mComputeGenerationLogits"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mContextFMHAE", "tensorrt_llm::runtime::ModelConfig::mContextFMHA"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9mDataTypeE", "tensorrt_llm::runtime::ModelConfig::mDataType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mEncoderHiddenSizeE", "tensorrt_llm::runtime::ModelConfig::mEncoderHiddenSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig19mGemmAllReduceDtypeE", "tensorrt_llm::runtime::ModelConfig::mGemmAllReduceDtype"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11mHiddenSizeE", "tensorrt_llm::runtime::ModelConfig::mHiddenSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mInputPackedE", "tensorrt_llm::runtime::ModelConfig::mInputPacked"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mKVCacheTypeE", "tensorrt_llm::runtime::ModelConfig::mKVCacheType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11mLayerTypesE", "tensorrt_llm::runtime::ModelConfig::mLayerTypes"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mLogitsDtypeE", "tensorrt_llm::runtime::ModelConfig::mLogitsDtype"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mLoraModulesE", "tensorrt_llm::runtime::ModelConfig::mLoraModules"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mManageWeightsTypeE", "tensorrt_llm::runtime::ModelConfig::mManageWeightsType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mMaxBatchSizeE", "tensorrt_llm::runtime::ModelConfig::mMaxBatchSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mMaxBeamWidthE", "tensorrt_llm::runtime::ModelConfig::mMaxBeamWidth"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14mMaxEncoderLenE", "tensorrt_llm::runtime::ModelConfig::mMaxEncoderLen"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mMaxInputLenE", "tensorrt_llm::runtime::ModelConfig::mMaxInputLen"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mMaxLoraRankE", "tensorrt_llm::runtime::ModelConfig::mMaxLoraRank"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mMaxNumTokensE", "tensorrt_llm::runtime::ModelConfig::mMaxNumTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22mMaxPositionEmbeddingsE", "tensorrt_llm::runtime::ModelConfig::mMaxPositionEmbeddings"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig28mMaxPromptEmbeddingTableSizeE", "tensorrt_llm::runtime::ModelConfig::mMaxPromptEmbeddingTableSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15mMaxSequenceLenE", "tensorrt_llm::runtime::ModelConfig::mMaxSequenceLen"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14mMlpHiddenSizeE", "tensorrt_llm::runtime::ModelConfig::mMlpHiddenSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig10mModelNameE", "tensorrt_llm::runtime::ModelConfig::mModelName"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mModelVariantE", "tensorrt_llm::runtime::ModelConfig::mModelVariant"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mNbAttentionLayersE", "tensorrt_llm::runtime::ModelConfig::mNbAttentionLayers"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig8mNbHeadsE", "tensorrt_llm::runtime::ModelConfig::mNbHeads"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9mNbLayersE", "tensorrt_llm::runtime::ModelConfig::mNbLayers"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mNbRnnLayersE", "tensorrt_llm::runtime::ModelConfig::mNbRnnLayers"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig28mNumKvHeadsPerAttentionLayerE", "tensorrt_llm::runtime::ModelConfig::mNumKvHeadsPerAttentionLayer"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig33mNumKvHeadsPerCrossAttentionLayerE", "tensorrt_llm::runtime::ModelConfig::mNumKvHeadsPerCrossAttentionLayer"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mNumLanguagesE", "tensorrt_llm::runtime::ModelConfig::mNumLanguages"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17mPagedContextFMHAE", "tensorrt_llm::runtime::ModelConfig::mPagedContextFMHA"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11mPagedStateE", "tensorrt_llm::runtime::ModelConfig::mPagedState"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig16mPpReduceScatterE", "tensorrt_llm::runtime::ModelConfig::mPpReduceScatter"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig10mQuantModeE", "tensorrt_llm::runtime::ModelConfig::mQuantMode"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig10mRnnConfigE", "tensorrt_llm::runtime::ModelConfig::mRnnConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig19mRotaryEmbeddingDimE", "tensorrt_llm::runtime::ModelConfig::mRotaryEmbeddingDim"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mSizePerHeadE", "tensorrt_llm::runtime::ModelConfig::mSizePerHead"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20mSkipCrossAttnBlocksE", "tensorrt_llm::runtime::ModelConfig::mSkipCrossAttnBlocks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24mSpeculativeDecodingModeE", "tensorrt_llm::runtime::ModelConfig::mSpeculativeDecodingMode"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26mSpeculativeDecodingModuleE", "tensorrt_llm::runtime::ModelConfig::mSpeculativeDecodingModule"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15mTokensPerBlockE", "tensorrt_llm::runtime::ModelConfig::mTokensPerBlock"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mUseCrossAttentionE", "tensorrt_llm::runtime::ModelConfig::mUseCrossAttention"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig23mUseGemmAllReducePluginE", "tensorrt_llm::runtime::ModelConfig::mUseGemmAllReducePlugin"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22mUseGptAttentionPluginE", "tensorrt_llm::runtime::ModelConfig::mUseGptAttentionPlugin"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14mUseLoraPluginE", "tensorrt_llm::runtime::ModelConfig::mUseLoraPlugin"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21mUseMambaConv1dPluginE", "tensorrt_llm::runtime::ModelConfig::mUseMambaConv1dPlugin"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9mUseMropeE", "tensorrt_llm::runtime::ModelConfig::mUseMrope"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21mUsePositionEmbeddingE", "tensorrt_llm::runtime::ModelConfig::mUsePositionEmbedding"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mUseShapeInferenceE", "tensorrt_llm::runtime::ModelConfig::mUseShapeInference"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22mUseTokenTypeEmbeddingE", "tensorrt_llm::runtime::ModelConfig::mUseTokenTypeEmbedding"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig10mVocabSizeE", "tensorrt_llm::runtime::ModelConfig::mVocabSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig30resetSpeculativeDecodingModuleEv", "tensorrt_llm::runtime::ModelConfig::resetSpeculativeDecodingModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setContextFMHAEb", "tensorrt_llm::runtime::ModelConfig::setContextFMHA"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setContextFMHAEb", "tensorrt_llm::runtime::ModelConfig::setContextFMHA::contextFMHA"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setEncoderHiddenSizeE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setEncoderHiddenSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setEncoderHiddenSizeE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setEncoderHiddenSize::encoderHiddenSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21setGemmAllReduceDtypeEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::setGemmAllReduceDtype"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21setGemmAllReduceDtypeEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::setGemmAllReduceDtype::inputDtype"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setKVCacheTypeE11KVCacheType", "tensorrt_llm::runtime::ModelConfig::setKVCacheType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setKVCacheTypeE11KVCacheType", "tensorrt_llm::runtime::ModelConfig::setKVCacheType::kvCacheType"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13setLayerTypesERKNSt6vectorI9LayerTypeEE", "tensorrt_llm::runtime::ModelConfig::setLayerTypes"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13setLayerTypesERKNSt6vectorI9LayerTypeEE", "tensorrt_llm::runtime::ModelConfig::setLayerTypes::layerTypes"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setLogitsDtypeEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::setLogitsDtype"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setLogitsDtypeEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::setLogitsDtype::inputDtype"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setLoraModulesERKNSt6vectorI10LoraModuleEE", "tensorrt_llm::runtime::ModelConfig::setLoraModules"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setLoraModulesERKNSt6vectorI10LoraModuleEE", "tensorrt_llm::runtime::ModelConfig::setLoraModules::loraModules"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setManageWeightsTypeEK17ManageWeightsType", "tensorrt_llm::runtime::ModelConfig::setManageWeightsType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setManageWeightsTypeEK17ManageWeightsType", "tensorrt_llm::runtime::ModelConfig::setManageWeightsType::manageWeightType"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setMaxBatchSizeE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxBatchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setMaxBatchSizeE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxBatchSize::maxBatchSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setMaxBeamWidthE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setMaxBeamWidthE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxBeamWidth::maxBeamWidth"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig16setMaxEncoderLenE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxEncoderLen"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig16setMaxEncoderLenE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxEncoderLen::maxEncoderLen"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setMaxInputLenE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxInputLen"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setMaxInputLenE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxInputLen::maxInputLen"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setMaxLoraRankE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxLoraRank"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setMaxLoraRankE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxLoraRank::maxLoraRank"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setMaxNumTokensENSt8optionalI10SizeType32EE", "tensorrt_llm::runtime::ModelConfig::setMaxNumTokens"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setMaxNumTokensENSt8optionalI10SizeType32EE", "tensorrt_llm::runtime::ModelConfig::setMaxNumTokens::maxNumTokens"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24setMaxPositionEmbeddingsE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxPositionEmbeddings"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24setMaxPositionEmbeddingsE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxPositionEmbeddings::maxPositionEmbeddings"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig30setMaxPromptEmbeddingTableSizeE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxPromptEmbeddingTableSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig30setMaxPromptEmbeddingTableSizeE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxPromptEmbeddingTableSize::maxPromptEmbeddingTableSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17setMaxSequenceLenE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxSequenceLen"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17setMaxSequenceLenE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxSequenceLen::maxSequenceLen"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig16setMlpHiddenSizeE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMlpHiddenSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig16setMlpHiddenSizeE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMlpHiddenSize::mlpHiddenSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setModelNameERKNSt6stringE", "tensorrt_llm::runtime::ModelConfig::setModelName"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setModelNameERKNSt6stringE", "tensorrt_llm::runtime::ModelConfig::setModelName::modelName"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setModelVariantE12ModelVariant", "tensorrt_llm::runtime::ModelConfig::setModelVariant"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setModelVariantE12ModelVariant", "tensorrt_llm::runtime::ModelConfig::setModelVariant::modelVariant"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17setNbCrossKvHeadsE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setNbCrossKvHeads"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17setNbCrossKvHeadsE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setNbCrossKvHeads::nbKvHeads"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setNbKvHeadsE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setNbKvHeads"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setNbKvHeadsE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setNbKvHeads::nbKvHeads"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26setNumKvHeadsPerCrossLayerERKNSt6vectorI10SizeType32EE", "tensorrt_llm::runtime::ModelConfig::setNumKvHeadsPerCrossLayer"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26setNumKvHeadsPerCrossLayerERKNSt6vectorI10SizeType32EE", "tensorrt_llm::runtime::ModelConfig::setNumKvHeadsPerCrossLayer::headsPerLayer"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21setNumKvHeadsPerLayerERKNSt6vectorI10SizeType32EE", "tensorrt_llm::runtime::ModelConfig::setNumKvHeadsPerLayer"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21setNumKvHeadsPerLayerERKNSt6vectorI10SizeType32EE", "tensorrt_llm::runtime::ModelConfig::setNumKvHeadsPerLayer::headsPerLayer"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setNumLanguagesENSt8optionalI10SizeType32EE", "tensorrt_llm::runtime::ModelConfig::setNumLanguages"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setNumLanguagesENSt8optionalI10SizeType32EE", "tensorrt_llm::runtime::ModelConfig::setNumLanguages::numLanguages"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig19setPagedContextFMHAEb", "tensorrt_llm::runtime::ModelConfig::setPagedContextFMHA"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig19setPagedContextFMHAEb", "tensorrt_llm::runtime::ModelConfig::setPagedContextFMHA::pagedContextFMHA"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18setPpReduceScatterEb", "tensorrt_llm::runtime::ModelConfig::setPpReduceScatter"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18setPpReduceScatterEb", "tensorrt_llm::runtime::ModelConfig::setPpReduceScatter::ppReduceScatter"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setQuantModeEN6common9QuantModeE", "tensorrt_llm::runtime::ModelConfig::setQuantMode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setQuantModeEN6common9QuantModeE", "tensorrt_llm::runtime::ModelConfig::setQuantMode::QuantMode"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setRnnConfigERK9RnnConfig", "tensorrt_llm::runtime::ModelConfig::setRnnConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setRnnConfigERK9RnnConfig", "tensorrt_llm::runtime::ModelConfig::setRnnConfig::rnnConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21setRotaryEmbeddingDimE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setRotaryEmbeddingDim"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21setRotaryEmbeddingDimE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setRotaryEmbeddingDim::rotaryEmbeddingDim"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setSizePerHeadE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setSizePerHead"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setSizePerHeadE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setSizePerHead::sizePerHead"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22setSkipCrossAttnBlocksEb", "tensorrt_llm::runtime::ModelConfig::setSkipCrossAttnBlocks"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22setSkipCrossAttnBlocksEb", "tensorrt_llm::runtime::ModelConfig::setSkipCrossAttnBlocks::skipCrossAttnBlocks"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26setSpeculativeDecodingModeE23SpeculativeDecodingMode", "tensorrt_llm::runtime::ModelConfig::setSpeculativeDecodingMode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26setSpeculativeDecodingModeE23SpeculativeDecodingMode", "tensorrt_llm::runtime::ModelConfig::setSpeculativeDecodingMode::mode"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig28setSpeculativeDecodingModuleERKNSt10shared_ptrI25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::ModelConfig::setSpeculativeDecodingModule"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig28setSpeculativeDecodingModuleERKNSt10shared_ptrI25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::ModelConfig::setSpeculativeDecodingModule::speculativeDecodingModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17setTokensPerBlockE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setTokensPerBlock"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17setTokensPerBlockE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setTokensPerBlock::TokensPerBlock"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setUseCrossAttentionEb", "tensorrt_llm::runtime::ModelConfig::setUseCrossAttention"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setUseCrossAttentionEb", "tensorrt_llm::runtime::ModelConfig::setUseCrossAttention::useCrossAttention"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11setUseMropeEb", "tensorrt_llm::runtime::ModelConfig::setUseMrope"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11setUseMropeEb", "tensorrt_llm::runtime::ModelConfig::setUseMrope::useMrope"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig23setUsePositionEmbeddingEb", "tensorrt_llm::runtime::ModelConfig::setUsePositionEmbedding"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig23setUsePositionEmbeddingEb", "tensorrt_llm::runtime::ModelConfig::setUsePositionEmbedding::usePositionEmbedding"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setUseShapeInferenceEb", "tensorrt_llm::runtime::ModelConfig::setUseShapeInference"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setUseShapeInferenceEb", "tensorrt_llm::runtime::ModelConfig::setUseShapeInference::useShapeInference"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24setUseTokenTypeEmbeddingEb", "tensorrt_llm::runtime::ModelConfig::setUseTokenTypeEmbedding"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24setUseTokenTypeEmbeddingEb", "tensorrt_llm::runtime::ModelConfig::setUseTokenTypeEmbedding::useTokenTypeEmbedding"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig19skipCrossAttnBlocksEv", "tensorrt_llm::runtime::ModelConfig::skipCrossAttnBlocks"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig24supportsInflightBatchingEv", "tensorrt_llm::runtime::ModelConfig::supportsInflightBatching"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig17useCrossAttentionEv", "tensorrt_llm::runtime::ModelConfig::useCrossAttention"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22useGemmAllReducePluginEb", "tensorrt_llm::runtime::ModelConfig::useGemmAllReducePlugin"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig22useGemmAllReducePluginEv", "tensorrt_llm::runtime::ModelConfig::useGemmAllReducePlugin"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22useGemmAllReducePluginEb", "tensorrt_llm::runtime::ModelConfig::useGemmAllReducePlugin::useGemmAllReducePlugin"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21useGptAttentionPluginEb", "tensorrt_llm::runtime::ModelConfig::useGptAttentionPlugin"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21useGptAttentionPluginEv", "tensorrt_llm::runtime::ModelConfig::useGptAttentionPlugin"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21useGptAttentionPluginEb", "tensorrt_llm::runtime::ModelConfig::useGptAttentionPlugin::useGptAttentionPlugin"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18useLanguageAdapterEv", "tensorrt_llm::runtime::ModelConfig::useLanguageAdapter"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13useLoraPluginEb", "tensorrt_llm::runtime::ModelConfig::useLoraPlugin"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13useLoraPluginEv", "tensorrt_llm::runtime::ModelConfig::useLoraPlugin"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13useLoraPluginEb", "tensorrt_llm::runtime::ModelConfig::useLoraPlugin::useLoraPlugin"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20useMambaConv1dPluginEb", "tensorrt_llm::runtime::ModelConfig::useMambaConv1dPlugin"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20useMambaConv1dPluginEv", "tensorrt_llm::runtime::ModelConfig::useMambaConv1dPlugin"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20useMambaConv1dPluginEb", "tensorrt_llm::runtime::ModelConfig::useMambaConv1dPlugin::useMambaConv1dPlugin"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig8useMropeEv", "tensorrt_llm::runtime::ModelConfig::useMrope"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14usePackedInputEb", "tensorrt_llm::runtime::ModelConfig::usePackedInput"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14usePackedInputEv", "tensorrt_llm::runtime::ModelConfig::usePackedInput"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14usePackedInputEb", "tensorrt_llm::runtime::ModelConfig::usePackedInput::inputPacked"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13usePagedStateEb", "tensorrt_llm::runtime::ModelConfig::usePagedState"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13usePagedStateEv", "tensorrt_llm::runtime::ModelConfig::usePagedState"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13usePagedStateEb", "tensorrt_llm::runtime::ModelConfig::usePagedState::pagedState"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20usePositionEmbeddingEv", "tensorrt_llm::runtime::ModelConfig::usePositionEmbedding"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15usePromptTuningEv", "tensorrt_llm::runtime::ModelConfig::usePromptTuning"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig17useShapeInferenceEv", "tensorrt_llm::runtime::ModelConfig::useShapeInference"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21useTokenTypeEmbeddingEv", "tensorrt_llm::runtime::ModelConfig::useTokenTypeEmbedding"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfiguratorE", "tensorrt_llm::runtime::MulticastConfigurator"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator11mBindOffsetE", "tensorrt_llm::runtime::MulticastConfigurator::mBindOffset"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator7mDeviceE", "tensorrt_llm::runtime::MulticastConfigurator::mDevice"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator10mMulticastE", "tensorrt_llm::runtime::MulticastConfigurator::mMulticast"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator5mSizeE", "tensorrt_llm::runtime::MulticastConfigurator::mSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator5setupE28CUmemGenericAllocationHandle", "tensorrt_llm::runtime::MulticastConfigurator::setup"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator5setupE28CUmemGenericAllocationHandle", "tensorrt_llm::runtime::MulticastConfigurator::setup::handle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator8teardownE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::MulticastConfigurator::teardown"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfiguratorE", "tensorrt_llm::runtime::OffloadConfigurator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator19OffloadConfiguratorE11CUdeviceptr6size_t10MemoryType8CUstreamb", "tensorrt_llm::runtime::OffloadConfigurator::OffloadConfigurator"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator19OffloadConfiguratorE11CUdeviceptr6size_t10MemoryType8CUstreamb", "tensorrt_llm::runtime::OffloadConfigurator::OffloadConfigurator::address"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator19OffloadConfiguratorE11CUdeviceptr6size_t10MemoryType8CUstreamb", "tensorrt_llm::runtime::OffloadConfigurator::OffloadConfigurator::backType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator19OffloadConfiguratorE11CUdeviceptr6size_t10MemoryType8CUstreamb", "tensorrt_llm::runtime::OffloadConfigurator::OffloadConfigurator::ondemand"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator19OffloadConfiguratorE11CUdeviceptr6size_t10MemoryType8CUstreamb", "tensorrt_llm::runtime::OffloadConfigurator::OffloadConfigurator::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator19OffloadConfiguratorE11CUdeviceptr6size_t10MemoryType8CUstreamb", "tensorrt_llm::runtime::OffloadConfigurator::OffloadConfigurator::stream"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator8mAddressE", "tensorrt_llm::runtime::OffloadConfigurator::mAddress"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator9mBackTypeE", "tensorrt_llm::runtime::OffloadConfigurator::mBackType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator14mBackedStorageE", "tensorrt_llm::runtime::OffloadConfigurator::mBackedStorage"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator9mOndemandE", "tensorrt_llm::runtime::OffloadConfigurator::mOndemand"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator5mSizeE", "tensorrt_llm::runtime::OffloadConfigurator::mSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator7mStreamE", "tensorrt_llm::runtime::OffloadConfigurator::mStream"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator5setupE28CUmemGenericAllocationHandle", "tensorrt_llm::runtime::OffloadConfigurator::setup"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator5setupE28CUmemGenericAllocationHandle", "tensorrt_llm::runtime::OffloadConfigurator::setup::handle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator8teardownE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::OffloadConfigurator::teardown"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator8teardownE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::OffloadConfigurator::teardown::destructing"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator8teardownE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::OffloadConfigurator::teardown::handle"], [1, 1, 1, "_CPPv4I0EN12tensorrt_llm7runtime18PointerElementTypeE", "tensorrt_llm::runtime::PointerElementType"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime18PointerElementTypeE", "tensorrt_llm::runtime::PointerElementType::T"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParamsE", "tensorrt_llm::runtime::PromptTuningParams"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams18PromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", "tensorrt_llm::runtime::PromptTuningParams::PromptTuningParams"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams18PromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", "tensorrt_llm::runtime::PromptTuningParams::PromptTuningParams::embeddingTable"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams18PromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", "tensorrt_llm::runtime::PromptTuningParams::PromptTuningParams::tasks"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams18PromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", "tensorrt_llm::runtime::PromptTuningParams::PromptTuningParams::vocabSize"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams10SizeType32E", "tensorrt_llm::runtime::PromptTuningParams::SizeType32"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams9TensorPtrE", "tensorrt_llm::runtime::PromptTuningParams::TensorPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams15fillTasksTensorE9TensorPtr10SizeType3210SizeType32RKNSt6vectorI10SizeType32EERKNSt6vectorI10SizeType32EERK13BufferManagerb", "tensorrt_llm::runtime::PromptTuningParams::fillTasksTensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams15fillTasksTensorE9TensorPtr10SizeType3210SizeType32RKNSt6vectorI10SizeType32EERKNSt6vectorI10SizeType32EERK13BufferManagerb", "tensorrt_llm::runtime::PromptTuningParams::fillTasksTensor::batchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams15fillTasksTensorE9TensorPtr10SizeType3210SizeType32RKNSt6vectorI10SizeType32EERKNSt6vectorI10SizeType32EERK13BufferManagerb", "tensorrt_llm::runtime::PromptTuningParams::fillTasksTensor::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams15fillTasksTensorE9TensorPtr10SizeType3210SizeType32RKNSt6vectorI10SizeType32EERKNSt6vectorI10SizeType32EERK13BufferManagerb", "tensorrt_llm::runtime::PromptTuningParams::fillTasksTensor::numContextRequests"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams15fillTasksTensorE9TensorPtr10SizeType3210SizeType32RKNSt6vectorI10SizeType32EERKNSt6vectorI10SizeType32EERK13BufferManagerb", "tensorrt_llm::runtime::PromptTuningParams::fillTasksTensor::packedInput"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams15fillTasksTensorE9TensorPtr10SizeType3210SizeType32RKNSt6vectorI10SizeType32EERKNSt6vectorI10SizeType32EERK13BufferManagerb", "tensorrt_llm::runtime::PromptTuningParams::fillTasksTensor::reqBeamWidths"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams15fillTasksTensorE9TensorPtr10SizeType3210SizeType32RKNSt6vectorI10SizeType32EERKNSt6vectorI10SizeType32EERK13BufferManagerb", "tensorrt_llm::runtime::PromptTuningParams::fillTasksTensor::reqPromptLengths"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams15fillTasksTensorE9TensorPtr10SizeType3210SizeType32RKNSt6vectorI10SizeType32EERKNSt6vectorI10SizeType32EERK13BufferManagerb", "tensorrt_llm::runtime::PromptTuningParams::fillTasksTensor::tasksHost"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngineE", "tensorrt_llm::runtime::RawEngine"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4Type15AddressWithSizeE", "tensorrt_llm::runtime::RawEngine::AddressWithSize"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4Type8FilePathE", "tensorrt_llm::runtime::RawEngine::FilePath"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4Type10HostMemoryE", "tensorrt_llm::runtime::RawEngine::HostMemory"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineENSt10filesystem4pathE", "tensorrt_llm::runtime::RawEngine::RawEngine"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineEPKN8nvinfer111IHostMemoryE", "tensorrt_llm::runtime::RawEngine::RawEngine"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineEPKvNSt6size_tE", "tensorrt_llm::runtime::RawEngine::RawEngine"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineEPKvNSt6size_tE", "tensorrt_llm::runtime::RawEngine::RawEngine::engineAddr"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineEPKN8nvinfer111IHostMemoryE", "tensorrt_llm::runtime::RawEngine::RawEngine::engineBuffer"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineENSt10filesystem4pathE", "tensorrt_llm::runtime::RawEngine::RawEngine::enginePath"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineEPKvNSt6size_tE", "tensorrt_llm::runtime::RawEngine::RawEngine::engineSize"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4TypeE", "tensorrt_llm::runtime::RawEngine::Type"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4Type15AddressWithSizeE", "tensorrt_llm::runtime::RawEngine::Type::AddressWithSize"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4Type8FilePathE", "tensorrt_llm::runtime::RawEngine::Type::FilePath"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4Type10HostMemoryE", "tensorrt_llm::runtime::RawEngine::Type::HostMemory"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine10getAddressEv", "tensorrt_llm::runtime::RawEngine::getAddress"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine13getHostMemoryEv", "tensorrt_llm::runtime::RawEngine::getHostMemory"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine23getManagedWeightsMapOptEv", "tensorrt_llm::runtime::RawEngine::getManagedWeightsMapOpt"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine7getPathEv", "tensorrt_llm::runtime::RawEngine::getPath"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine10getPathOptEv", "tensorrt_llm::runtime::RawEngine::getPathOpt"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine7getSizeEv", "tensorrt_llm::runtime::RawEngine::getSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine7getTypeEv", "tensorrt_llm::runtime::RawEngine::getType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine11mEngineAddrE", "tensorrt_llm::runtime::RawEngine::mEngineAddr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine13mEngineBufferE", "tensorrt_llm::runtime::RawEngine::mEngineBuffer"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine11mEnginePathE", "tensorrt_llm::runtime::RawEngine::mEnginePath"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine11mEngineSizeE", "tensorrt_llm::runtime::RawEngine::mEngineSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine18mManagedWeightsMapE", "tensorrt_llm::runtime::RawEngine::mManagedWeightsMap"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine5mTypeE", "tensorrt_llm::runtime::RawEngine::mType"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine20setManagedWeightsMapENSt3mapINSt6stringEN12tensorrt_llm8executor6TensorEEE", "tensorrt_llm::runtime::RawEngine::setManagedWeightsMap"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine20setManagedWeightsMapENSt3mapINSt6stringEN12tensorrt_llm8executor6TensorEEE", "tensorrt_llm::runtime::RawEngine::setManagedWeightsMap::managedWeightsMap"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine7setPathENSt10filesystem4pathE", "tensorrt_llm::runtime::RawEngine::setPath"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine7setPathENSt10filesystem4pathE", "tensorrt_llm::runtime::RawEngine::setPath::enginePath"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime11RequestTypeE", "tensorrt_llm::runtime::RequestType"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11RequestType8kCONTEXTE", "tensorrt_llm::runtime::RequestType::kCONTEXT"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11RequestType11kGENERATIONE", "tensorrt_llm::runtime::RequestType::kGENERATION"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaultsE", "tensorrt_llm::runtime::RuntimeDefaults"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults15RuntimeDefaultsENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalI10SizeType32EE", "tensorrt_llm::runtime::RuntimeDefaults::RuntimeDefaults"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults15RuntimeDefaultsEv", "tensorrt_llm::runtime::RuntimeDefaults::RuntimeDefaults"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults15RuntimeDefaultsENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalI10SizeType32EE", "tensorrt_llm::runtime::RuntimeDefaults::RuntimeDefaults::maxAttentionWindowVec"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults15RuntimeDefaultsENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalI10SizeType32EE", "tensorrt_llm::runtime::RuntimeDefaults::RuntimeDefaults::sinkTokenLength"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults21maxAttentionWindowVecE", "tensorrt_llm::runtime::RuntimeDefaults::maxAttentionWindowVec"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults15sinkTokenLengthE", "tensorrt_llm::runtime::RuntimeDefaults::sinkTokenLength"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfigE", "tensorrt_llm::runtime::SamplingConfig"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig9FloatTypeE", "tensorrt_llm::runtime::SamplingConfig::FloatType"], [1, 1, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig6OptVecE", "tensorrt_llm::runtime::SamplingConfig::OptVec"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig6OptVecE", "tensorrt_llm::runtime::SamplingConfig::OptVec::T"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigE10SizeType32", "tensorrt_llm::runtime::SamplingConfig::SamplingConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigERKN8executor14SamplingConfigERKNSt8optionalIN8executor25ExternalDraftTokensConfigEEE", "tensorrt_llm::runtime::SamplingConfig::SamplingConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigERKNSt6vectorI14SamplingConfigEE", "tensorrt_llm::runtime::SamplingConfig::SamplingConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigE10SizeType32", "tensorrt_llm::runtime::SamplingConfig::SamplingConfig::beamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigERKNSt6vectorI14SamplingConfigEE", "tensorrt_llm::runtime::SamplingConfig::SamplingConfig::configs"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigERKN8executor14SamplingConfigERKNSt8optionalIN8executor25ExternalDraftTokensConfigEEE", "tensorrt_llm::runtime::SamplingConfig::SamplingConfig::externalDraftTokensConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigERKN8executor14SamplingConfigERKNSt8optionalIN8executor25ExternalDraftTokensConfigEEE", "tensorrt_llm::runtime::SamplingConfig::SamplingConfig::samplingConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig23beamSearchDiversityRateE", "tensorrt_llm::runtime::SamplingConfig::beamSearchDiversityRate"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig9beamWidthE", "tensorrt_llm::runtime::SamplingConfig::beamWidth"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14beamWidthArrayE", "tensorrt_llm::runtime::SamplingConfig::beamWidthArray"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig11cumLogProbsE", "tensorrt_llm::runtime::SamplingConfig::cumLogProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig24draftAcceptanceThresholdE", "tensorrt_llm::runtime::SamplingConfig::draftAcceptanceThreshold"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig13earlyStoppingE", "tensorrt_llm::runtime::SamplingConfig::earlyStopping"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig16frequencyPenaltyE", "tensorrt_llm::runtime::SamplingConfig::frequencyPenalty"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig10fuseValuesE6OptVecI1TERKNSt6vectorI14SamplingConfigEENSt8functionIF6OptVecI1TE6size_tEEE1T", "tensorrt_llm::runtime::SamplingConfig::fuseValues"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig10fuseValuesE6OptVecI1TERKNSt6vectorI14SamplingConfigEENSt8functionIF6OptVecI1TE6size_tEEE1T", "tensorrt_llm::runtime::SamplingConfig::fuseValues::T"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig10fuseValuesE6OptVecI1TERKNSt6vectorI14SamplingConfigEENSt8functionIF6OptVecI1TE6size_tEEE1T", "tensorrt_llm::runtime::SamplingConfig::fuseValues::accessor"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig10fuseValuesE6OptVecI1TERKNSt6vectorI14SamplingConfigEENSt8functionIF6OptVecI1TE6size_tEEE1T", "tensorrt_llm::runtime::SamplingConfig::fuseValues::configs"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig10fuseValuesE6OptVecI1TERKNSt6vectorI14SamplingConfigEENSt8functionIF6OptVecI1TE6size_tEEE1T", "tensorrt_llm::runtime::SamplingConfig::fuseValues::defaultValue"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14SamplingConfig15getMaxBeamWidthEv", "tensorrt_llm::runtime::SamplingConfig::getMaxBeamWidth"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14SamplingConfig17getNumReturnBeamsEv", "tensorrt_llm::runtime::SamplingConfig::getNumReturnBeams"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig13lengthPenaltyE", "tensorrt_llm::runtime::SamplingConfig::lengthPenalty"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig9minLengthE", "tensorrt_llm::runtime::SamplingConfig::minLength"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig4minPE", "tensorrt_llm::runtime::SamplingConfig::minP"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig17noRepeatNgramSizeE", "tensorrt_llm::runtime::SamplingConfig::noRepeatNgramSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig17normalizeLogProbsE", "tensorrt_llm::runtime::SamplingConfig::normalizeLogProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig18numReturnSequencesE", "tensorrt_llm::runtime::SamplingConfig::numReturnSequences"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14SamplingConfigeqERK14SamplingConfig", "tensorrt_llm::runtime::SamplingConfig::operator=="], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime14SamplingConfigeqERK14SamplingConfig", "tensorrt_llm::runtime::SamplingConfig::operator==::other"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig19originalTemperatureE", "tensorrt_llm::runtime::SamplingConfig::originalTemperature"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14outputLogProbsE", "tensorrt_llm::runtime::SamplingConfig::outputLogProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig15presencePenaltyE", "tensorrt_llm::runtime::SamplingConfig::presencePenalty"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig10randomSeedE", "tensorrt_llm::runtime::SamplingConfig::randomSeed"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig17repetitionPenaltyE", "tensorrt_llm::runtime::SamplingConfig::repetitionPenalty"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig11temperatureE", "tensorrt_llm::runtime::SamplingConfig::temperature"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig4topKE", "tensorrt_llm::runtime::SamplingConfig::topK"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig15topKMedusaHeadsE", "tensorrt_llm::runtime::SamplingConfig::topKMedusaHeads"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig4topPE", "tensorrt_llm::runtime::SamplingConfig::topP"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig9topPDecayE", "tensorrt_llm::runtime::SamplingConfig::topPDecay"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig7topPMinE", "tensorrt_llm::runtime::SamplingConfig::topPMin"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig12topPResetIdsE", "tensorrt_llm::runtime::SamplingConfig::topPResetIds"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig16useDefaultValuesEbRK6OptVecI1TE1T", "tensorrt_llm::runtime::SamplingConfig::useDefaultValues"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig16useDefaultValuesEbRK6OptVecI1TE1T", "tensorrt_llm::runtime::SamplingConfig::useDefaultValues::T"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig16useDefaultValuesEbRK6OptVecI1TE1T", "tensorrt_llm::runtime::SamplingConfig::useDefaultValues::defaultValue"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig16useDefaultValuesEbRK6OptVecI1TE1T", "tensorrt_llm::runtime::SamplingConfig::useDefaultValues::vec"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig8validateEv", "tensorrt_llm::runtime::SamplingConfig::validate"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig11validateVecEbNSt6stringERK6OptVecI1TE1TNSt8optionalI1TEE", "tensorrt_llm::runtime::SamplingConfig::validateVec"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig11validateVecEbNSt6stringERK6OptVecI1TE1TNSt8optionalI1TEE", "tensorrt_llm::runtime::SamplingConfig::validateVec::T"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig11validateVecEbNSt6stringERK6OptVecI1TE1TNSt8optionalI1TEE", "tensorrt_llm::runtime::SamplingConfig::validateVec::max"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig11validateVecEbNSt6stringERK6OptVecI1TE1TNSt8optionalI1TEE", "tensorrt_llm::runtime::SamplingConfig::validateVec::min"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig11validateVecEbNSt6stringERK6OptVecI1TE1TNSt8optionalI1TEE", "tensorrt_llm::runtime::SamplingConfig::validateVec::name"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig11validateVecEbNSt6stringERK6OptVecI1TE1TNSt8optionalI1TEE", "tensorrt_llm::runtime::SamplingConfig::validateVec::vec"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime10SizeType32E", "tensorrt_llm::runtime::SizeType32"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime10SizeType64E", "tensorrt_llm::runtime::SizeType64"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingModeE", "tensorrt_llm::runtime::SpeculativeDecodingMode"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode19DraftTokensExternalEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::DraftTokensExternal"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode5EagleEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::Eagle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode19ExplicitDraftTokensEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::ExplicitDraftTokens"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode17LookaheadDecodingEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::LookaheadDecoding"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode6MedusaEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::Medusa"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode4NoneEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::None"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode23SpeculativeDecodingModeE14UnderlyingType", "tensorrt_llm::runtime::SpeculativeDecodingMode::SpeculativeDecodingMode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode23SpeculativeDecodingModeE14UnderlyingType", "tensorrt_llm::runtime::SpeculativeDecodingMode::SpeculativeDecodingMode::state"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode14UnderlyingTypeE", "tensorrt_llm::runtime::SpeculativeDecodingMode::UnderlyingType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode9allBitSetE14UnderlyingType", "tensorrt_llm::runtime::SpeculativeDecodingMode::allBitSet"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode9allBitSetE14UnderlyingType", "tensorrt_llm::runtime::SpeculativeDecodingMode::allBitSet::bits"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode9anyBitSetE14UnderlyingType", "tensorrt_llm::runtime::SpeculativeDecodingMode::anyBitSet"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode9anyBitSetE14UnderlyingType", "tensorrt_llm::runtime::SpeculativeDecodingMode::anyBitSet::bits"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode14hasDraftLogitsEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::hasDraftLogits"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode21isDraftTokensExternalEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::isDraftTokensExternal"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode7isEagleEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::isEagle"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode21isExplicitDraftTokensEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::isExplicitDraftTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode19isLookaheadDecodingEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::isLookaheadDecoding"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode8isMedusaEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::isMedusa"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode6isNoneEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::isNone"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode20kDraftTokensExternalE", "tensorrt_llm::runtime::SpeculativeDecodingMode::kDraftTokensExternal"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode6kEagleE", "tensorrt_llm::runtime::SpeculativeDecodingMode::kEagle"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode20kExplicitDraftTokensE", "tensorrt_llm::runtime::SpeculativeDecodingMode::kExplicitDraftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode18kLookaheadDecodingE", "tensorrt_llm::runtime::SpeculativeDecodingMode::kLookaheadDecoding"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode7kMedusaE", "tensorrt_llm::runtime::SpeculativeDecodingMode::kMedusa"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode5kNoneE", "tensorrt_llm::runtime::SpeculativeDecodingMode::kNone"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode6mStateE", "tensorrt_llm::runtime::SpeculativeDecodingMode::mState"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode20needsDecoderPrologueEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::needsDecoderPrologue"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode18needsKVCacheRewindEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::needsKVCacheRewind"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingModeeqERK23SpeculativeDecodingMode", "tensorrt_llm::runtime::SpeculativeDecodingMode::operator=="], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingModeeqERK23SpeculativeDecodingMode", "tensorrt_llm::runtime::SpeculativeDecodingMode::operator==::other"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode19predictsDraftTokensEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::predictsDraftTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode21requiresAttentionMaskEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::requiresAttentionMask"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode18updatesPositionIdsEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::updatesPositionIds"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode19variableDraftLengthEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::variableDraftLength"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModuleE", "tensorrt_llm::runtime::SpeculativeDecodingModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::SpeculativeDecodingModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleERK25SpeculativeDecodingModule", "tensorrt_llm::runtime::SpeculativeDecodingModule::SpeculativeDecodingModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleEv", "tensorrt_llm::runtime::SpeculativeDecodingModule::SpeculativeDecodingModule"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::SpeculativeDecodingModule::maxDecodingDraftTokens"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::SpeculativeDecodingModule::maxDraftPathLen"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::SpeculativeDecodingModule::maxNumPaths"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleERK25SpeculativeDecodingModule", "tensorrt_llm::runtime::SpeculativeDecodingModule::SpeculativeDecodingModule::o"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule21computeNumPackedMasksEv", "tensorrt_llm::runtime::SpeculativeDecodingModule::computeNumPackedMasks"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule25getMaxDecodingDraftTokensEv", "tensorrt_llm::runtime::SpeculativeDecodingModule::getMaxDecodingDraftTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule20getMaxDecodingTokensEv", "tensorrt_llm::runtime::SpeculativeDecodingModule::getMaxDecodingTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule18getMaxDraftPathLenEv", "tensorrt_llm::runtime::SpeculativeDecodingModule::getMaxDraftPathLen"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule14getMaxNumPathsEv", "tensorrt_llm::runtime::SpeculativeDecodingModule::getMaxNumPaths"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule13getMaxPathLenEv", "tensorrt_llm::runtime::SpeculativeDecodingModule::getMaxPathLen"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule17getNumPackedMasksEv", "tensorrt_llm::runtime::SpeculativeDecodingModule::getNumPackedMasks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule23mMaxDecodingDraftTokensE", "tensorrt_llm::runtime::SpeculativeDecodingModule::mMaxDecodingDraftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule16mMaxDraftPathLenE", "tensorrt_llm::runtime::SpeculativeDecodingModule::mMaxDraftPathLen"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule18mMaxNumPackedMasksE", "tensorrt_llm::runtime::SpeculativeDecodingModule::mMaxNumPackedMasks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule12mMaxNumPathsE", "tensorrt_llm::runtime::SpeculativeDecodingModule::mMaxNumPaths"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModuleaSERK25SpeculativeDecodingModule", "tensorrt_llm::runtime::SpeculativeDecodingModule::operator="], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModuleaSERK25SpeculativeDecodingModule", "tensorrt_llm::runtime::SpeculativeDecodingModule::operator=::o"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule18setMaxDraftPathLenE10SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::setMaxDraftPathLen"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule18setMaxDraftPathLenE10SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::setMaxDraftPathLen::maxDraftPathLen"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule17setMaxDraftTokensE10SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::setMaxDraftTokens"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule17setMaxDraftTokensE10SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::setMaxDraftTokens::maxDraftTokens"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule14setMaxNumPathsE10SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::setMaxNumPaths"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule14setMaxNumPathsE10SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::setMaxNumPaths::maxNumPaths"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModuleD0Ev", "tensorrt_llm::runtime::SpeculativeDecodingModule::~SpeculativeDecodingModule"], [1, 1, 1, "_CPPv4I0EN12tensorrt_llm7runtime12StringPtrMapE", "tensorrt_llm::runtime::StringPtrMap"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime12StringPtrMapE", "tensorrt_llm::runtime::StringPtrMap::T"], [1, 2, 1, "_CPPv4I0_bEN12tensorrt_llm7runtime11TRTDataTypeE", "tensorrt_llm::runtime::TRTDataType"], [1, 8, 1, "_CPPv4I0_bEN12tensorrt_llm7runtime11TRTDataTypeE", "tensorrt_llm::runtime::TRTDataType::T"], [1, 2, 1, "_CPPv4I0EN12tensorrt_llm7runtime11TRTDataTypeIP1TEE", "tensorrt_llm::runtime::TRTDataType<T*>"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime11TRTDataTypeIP1TEE", "tensorrt_llm::runtime::TRTDataType<T*>::T"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIP1TE15kUnderlyingTypeE", "tensorrt_llm::runtime::TRTDataType<T*>::kUnderlyingType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIP1TE5valueE", "tensorrt_llm::runtime::TRTDataType<T*>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIbEE", "tensorrt_llm::runtime::TRTDataType<bool>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIbE5valueE", "tensorrt_llm::runtime::TRTDataType<bool>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIfEE", "tensorrt_llm::runtime::TRTDataType<float>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIfE5valueE", "tensorrt_llm::runtime::TRTDataType<float>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeI4halfEE", "tensorrt_llm::runtime::TRTDataType<half>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeI4halfE5valueE", "tensorrt_llm::runtime::TRTDataType<half>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIN7kernels13FinishedStateEEE", "tensorrt_llm::runtime::TRTDataType<kernels::FinishedState>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIN7kernels13FinishedStateEE5valueE", "tensorrt_llm::runtime::TRTDataType<kernels::FinishedState>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIN7kernels12KVCacheIndexEEE", "tensorrt_llm::runtime::TRTDataType<kernels::KVCacheIndex>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIN7kernels12KVCacheIndexEE5valueE", "tensorrt_llm::runtime::TRTDataType<kernels::KVCacheIndex>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIN7runtime11RequestTypeEEE", "tensorrt_llm::runtime::TRTDataType<runtime::RequestType>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIN7runtime11RequestTypeEE5valueE", "tensorrt_llm::runtime::TRTDataType<runtime::RequestType>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt7int32_tEEE", "tensorrt_llm::runtime::TRTDataType<std::int32_t>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt7int32_tEE5valueE", "tensorrt_llm::runtime::TRTDataType<std::int32_t>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt7int64_tEEE", "tensorrt_llm::runtime::TRTDataType<std::int64_t>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt7int64_tEE5valueE", "tensorrt_llm::runtime::TRTDataType<std::int64_t>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt6int8_tEEE", "tensorrt_llm::runtime::TRTDataType<std::int8_t>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt6int8_tEE5valueE", "tensorrt_llm::runtime::TRTDataType<std::int8_t>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt8uint32_tEEE", "tensorrt_llm::runtime::TRTDataType<std::uint32_t>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt8uint32_tEE5valueE", "tensorrt_llm::runtime::TRTDataType<std::uint32_t>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt8uint64_tEEE", "tensorrt_llm::runtime::TRTDataType<std::uint64_t>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt8uint64_tEE5valueE", "tensorrt_llm::runtime::TRTDataType<std::uint64_t>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt7uint8_tEEE", "tensorrt_llm::runtime::TRTDataType<std::uint8_t>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt7uint8_tEE5valueE", "tensorrt_llm::runtime::TRTDataType<std::uint8_t>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIPvEE", "tensorrt_llm::runtime::TRTDataType<void*>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIPvE5valueE", "tensorrt_llm::runtime::TRTDataType<void*>::value"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime10TllmLoggerE", "tensorrt_llm::runtime::TllmLogger"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10TllmLogger8getLevelEv", "tensorrt_llm::runtime::TllmLogger::getLevel"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10TllmLogger3logE8SeverityPKN8nvinfer19AsciiCharE", "tensorrt_llm::runtime::TllmLogger::log"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10TllmLogger3logE8SeverityPKN8nvinfer19AsciiCharE", "tensorrt_llm::runtime::TllmLogger::log::msg"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10TllmLogger3logE8SeverityPKN8nvinfer19AsciiCharE", "tensorrt_llm::runtime::TllmLogger::log::severity"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10TllmLogger8setLevelE8Severity", "tensorrt_llm::runtime::TllmLogger::setLevel"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10TllmLogger8setLevelE8Severity", "tensorrt_llm::runtime::TllmLogger::setLevel::level"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime16TokenExtraIdTypeE", "tensorrt_llm::runtime::TokenExtraIdType"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime11TokenIdTypeE", "tensorrt_llm::runtime::TokenIdType"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfiguratorE", "tensorrt_llm::runtime::UnicastConfigurator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator19UnicastConfiguratorE11CUdeviceptr6size_tRK15CUmemAccessDesc", "tensorrt_llm::runtime::UnicastConfigurator::UnicastConfigurator"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator19UnicastConfiguratorE11CUdeviceptr6size_tRK15CUmemAccessDesc", "tensorrt_llm::runtime::UnicastConfigurator::UnicastConfigurator::address"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator19UnicastConfiguratorE11CUdeviceptr6size_tRK15CUmemAccessDesc", "tensorrt_llm::runtime::UnicastConfigurator::UnicastConfigurator::desc"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator19UnicastConfiguratorE11CUdeviceptr6size_tRK15CUmemAccessDesc", "tensorrt_llm::runtime::UnicastConfigurator::UnicastConfigurator::size"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator8mAddressE", "tensorrt_llm::runtime::UnicastConfigurator::mAddress"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator5mDescE", "tensorrt_llm::runtime::UnicastConfigurator::mDesc"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator5mSizeE", "tensorrt_llm::runtime::UnicastConfigurator::mSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator5setupE28CUmemGenericAllocationHandle", "tensorrt_llm::runtime::UnicastConfigurator::setup"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator5setupE28CUmemGenericAllocationHandle", "tensorrt_llm::runtime::UnicastConfigurator::setup::handle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator8teardownE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::UnicastConfigurator::teardown"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime11UniqueTokenE", "tensorrt_llm::runtime::UniqueToken"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11UniqueTokeneqERK11UniqueToken", "tensorrt_llm::runtime::UniqueToken::operator=="], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11UniqueTokeneqERK11UniqueToken", "tensorrt_llm::runtime::UniqueToken::operator==::other"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11UniqueToken12tokenExtraIdE", "tensorrt_llm::runtime::UniqueToken::tokenExtraId"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11UniqueToken7tokenIdE", "tensorrt_llm::runtime::UniqueToken::tokenId"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime16VecTokenExtraIdsE", "tensorrt_llm::runtime::VecTokenExtraIds"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime15VecUniqueTokensE", "tensorrt_llm::runtime::VecUniqueTokens"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfigE", "tensorrt_llm::runtime::WorldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig11WorldConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::WorldConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig11WorldConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::WorldConfig::contextParallelism"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig11WorldConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::WorldConfig::deviceIds"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig11WorldConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::WorldConfig::enableAttentionDP"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig11WorldConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::WorldConfig::gpusPerNode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig11WorldConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::WorldConfig::pipelineParallelism"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig11WorldConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::WorldConfig::rank"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig11WorldConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::WorldConfig::tensorParallelism"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig17enableAttentionDPEv", "tensorrt_llm::runtime::WorldConfig::enableAttentionDP"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig23getContextParallelGroupEv", "tensorrt_llm::runtime::WorldConfig::getContextParallelGroup"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig22getContextParallelRankEv", "tensorrt_llm::runtime::WorldConfig::getContextParallelRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig21getContextParallelismEv", "tensorrt_llm::runtime::WorldConfig::getContextParallelism"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig9getDeviceEv", "tensorrt_llm::runtime::WorldConfig::getDevice"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig11getDeviceOfE10SizeType32", "tensorrt_llm::runtime::WorldConfig::getDeviceOf"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig11getDeviceOfE10SizeType32", "tensorrt_llm::runtime::WorldConfig::getDeviceOf::rank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig15getGpusPerGroupEv", "tensorrt_llm::runtime::WorldConfig::getGpusPerGroup"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig14getGpusPerNodeEv", "tensorrt_llm::runtime::WorldConfig::getGpusPerNode"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig11getLastRankEv", "tensorrt_llm::runtime::WorldConfig::getLastRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig12getLocalRankEv", "tensorrt_llm::runtime::WorldConfig::getLocalRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig11getNodeRankEv", "tensorrt_llm::runtime::WorldConfig::getNodeRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig13getNodeRankOfE10SizeType32", "tensorrt_llm::runtime::WorldConfig::getNodeRankOf"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig13getNodeRankOfE10SizeType32", "tensorrt_llm::runtime::WorldConfig::getNodeRankOf::rank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig24getPipelineParallelGroupEv", "tensorrt_llm::runtime::WorldConfig::getPipelineParallelGroup"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig23getPipelineParallelRankEv", "tensorrt_llm::runtime::WorldConfig::getPipelineParallelRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig22getPipelineParallelismEv", "tensorrt_llm::runtime::WorldConfig::getPipelineParallelism"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig7getRankEv", "tensorrt_llm::runtime::WorldConfig::getRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig7getSizeEv", "tensorrt_llm::runtime::WorldConfig::getSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig22getTensorParallelGroupEv", "tensorrt_llm::runtime::WorldConfig::getTensorParallelGroup"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig21getTensorParallelRankEv", "tensorrt_llm::runtime::WorldConfig::getTensorParallelRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig20getTensorParallelismEv", "tensorrt_llm::runtime::WorldConfig::getTensorParallelism"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig17isContextParallelEv", "tensorrt_llm::runtime::WorldConfig::isContextParallel"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig26isFirstContextParallelRankEv", "tensorrt_llm::runtime::WorldConfig::isFirstContextParallelRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig27isFirstPipelineParallelRankEv", "tensorrt_llm::runtime::WorldConfig::isFirstPipelineParallelRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig25isFirstTensorParallelRankEv", "tensorrt_llm::runtime::WorldConfig::isFirstTensorParallelRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig26isLastPipelineParallelRankEv", "tensorrt_llm::runtime::WorldConfig::isLastPipelineParallelRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig18isPipelineParallelEv", "tensorrt_llm::runtime::WorldConfig::isPipelineParallel"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig16isTensorParallelEv", "tensorrt_llm::runtime::WorldConfig::isTensorParallel"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig19kDefaultGpusPerNodeE", "tensorrt_llm::runtime::WorldConfig::kDefaultGpusPerNode"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig19mContextParallelismE", "tensorrt_llm::runtime::WorldConfig::mContextParallelism"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig10mDeviceIdsE", "tensorrt_llm::runtime::WorldConfig::mDeviceIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig18mEnableAttentionDPE", "tensorrt_llm::runtime::WorldConfig::mEnableAttentionDP"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig12mGpusPerNodeE", "tensorrt_llm::runtime::WorldConfig::mGpusPerNode"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig20mPipelineParallelismE", "tensorrt_llm::runtime::WorldConfig::mPipelineParallelism"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig5mRankE", "tensorrt_llm::runtime::WorldConfig::mRank"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig18mTensorParallelismE", "tensorrt_llm::runtime::WorldConfig::mTensorParallelism"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig3mpiE10SizeType32NSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::mpi"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig3mpiE10SizeType32NSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::mpi::contextParallelism"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig3mpiE10SizeType32NSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::mpi::deviceIds"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig3mpiE10SizeType32NSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::mpi::enableAttentionDP"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig3mpiE10SizeType32NSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::mpi::gpusPerNode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig3mpiE10SizeType32NSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::mpi::pipelineParallelism"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig3mpiE10SizeType32NSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::mpi::tensorParallelism"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig14validMpiConfigEv", "tensorrt_llm::runtime::WorldConfig::validMpiConfig"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime10bufferCastEP1TR7IBuffer", "tensorrt_llm::runtime::bufferCast"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime10bufferCastEPK1TRK7IBuffer", "tensorrt_llm::runtime::bufferCast"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime10bufferCastEP1TR7IBuffer", "tensorrt_llm::runtime::bufferCast::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime10bufferCastEPK1TRK7IBuffer", "tensorrt_llm::runtime::bufferCast::T"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime10bufferCastEP1TR7IBuffer", "tensorrt_llm::runtime::bufferCast::buffer"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime10bufferCastEPK1TRK7IBuffer", "tensorrt_llm::runtime::bufferCast::buffer"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKN7IBuffer9SharedPtrE", "tensorrt_llm::runtime::bufferCastOrNull"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKN7ITensor9SharedPtrE", "tensorrt_llm::runtime::bufferCastOrNull"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKNSt8optionalIN7IBuffer9SharedPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKNSt8optionalIN7ITensor9SharedPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKN7IBuffer14SharedConstPtrE", "tensorrt_llm::runtime::bufferCastOrNull"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKN7ITensor14SharedConstPtrE", "tensorrt_llm::runtime::bufferCastOrNull"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKNSt8optionalIN7IBuffer14SharedConstPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKNSt8optionalIN7ITensor14SharedConstPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKN7IBuffer9SharedPtrE", "tensorrt_llm::runtime::bufferCastOrNull::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKN7ITensor9SharedPtrE", "tensorrt_llm::runtime::bufferCastOrNull::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKNSt8optionalIN7IBuffer9SharedPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKNSt8optionalIN7ITensor9SharedPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKN7IBuffer14SharedConstPtrE", "tensorrt_llm::runtime::bufferCastOrNull::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKN7ITensor14SharedConstPtrE", "tensorrt_llm::runtime::bufferCastOrNull::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKNSt8optionalIN7IBuffer14SharedConstPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKNSt8optionalIN7ITensor14SharedConstPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull::T"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKN7IBuffer9SharedPtrE", "tensorrt_llm::runtime::bufferCastOrNull::bufferPtr"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKN7IBuffer14SharedConstPtrE", "tensorrt_llm::runtime::bufferCastOrNull::bufferPtr"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKNSt8optionalIN7IBuffer9SharedPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull::optionalBufferPtr"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKNSt8optionalIN7IBuffer14SharedConstPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull::optionalBufferPtr"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKNSt8optionalIN7ITensor9SharedPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull::optionalTensorPtr"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKNSt8optionalIN7ITensor14SharedConstPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull::optionalTensorPtr"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKN7ITensor9SharedPtrE", "tensorrt_llm::runtime::bufferCastOrNull::tensorPtr"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKN7ITensor14SharedConstPtrE", "tensorrt_llm::runtime::bufferCastOrNull::tensorPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13canAccessPeerERK11WorldConfig", "tensorrt_llm::runtime::canAccessPeer"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13canAccessPeerERK11WorldConfig", "tensorrt_llm::runtime::canAccessPeer::worldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime27clearVirtualMemoryAllocatorEv", "tensorrt_llm::runtime::clearVirtualMemoryAllocator"], [1, 3, 1, "_CPPv4I00EN12tensorrt_llm7runtime16constPointerCastENSt10shared_ptrINSt14remove_const_tI1TEEEERRNSt10unique_ptrI1T1DEE", "tensorrt_llm::runtime::constPointerCast"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime16constPointerCastENSt10shared_ptrINSt14remove_const_tI1TEEEERKNSt10shared_ptrI1TEE", "tensorrt_llm::runtime::constPointerCast"], [1, 8, 1, "_CPPv4I00EN12tensorrt_llm7runtime16constPointerCastENSt10shared_ptrINSt14remove_const_tI1TEEEERRNSt10unique_ptrI1T1DEE", "tensorrt_llm::runtime::constPointerCast::D"], [1, 8, 1, "_CPPv4I00EN12tensorrt_llm7runtime16constPointerCastENSt10shared_ptrINSt14remove_const_tI1TEEEERRNSt10unique_ptrI1T1DEE", "tensorrt_llm::runtime::constPointerCast::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime16constPointerCastENSt10shared_ptrINSt14remove_const_tI1TEEEERKNSt10shared_ptrI1TEE", "tensorrt_llm::runtime::constPointerCast::T"], [1, 4, 1, "_CPPv4I00EN12tensorrt_llm7runtime16constPointerCastENSt10shared_ptrINSt14remove_const_tI1TEEEERRNSt10unique_ptrI1T1DEE", "tensorrt_llm::runtime::constPointerCast::ptr"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime16constPointerCastENSt10shared_ptrINSt14remove_const_tI1TEEEERKNSt10shared_ptrI1TEE", "tensorrt_llm::runtime::constPointerCast::ptr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7decoderE", "tensorrt_llm::runtime::decoder"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7decoderE", "tensorrt_llm::runtime::decoder"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffersE", "tensorrt_llm::runtime::decoder::BeamSearchBuffers"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers17BeamSearchBuffersERK13BufferManager", "tensorrt_llm::runtime::decoder::BeamSearchBuffers::BeamSearchBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers17BeamSearchBuffersERK13BufferManager", "tensorrt_llm::runtime::decoder::BeamSearchBuffers::BeamSearchBuffers::bufferManager"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers15mCumLogProbsTmpE", "tensorrt_llm::runtime::decoder::BeamSearchBuffers::mCumLogProbsTmp"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers7mNumSMsE", "tensorrt_llm::runtime::decoder::BeamSearchBuffers::mNumSMs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers21mOutputBeamHypothesesE", "tensorrt_llm::runtime::decoder::BeamSearchBuffers::mOutputBeamHypotheses"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers7reshapeE10SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::BeamSearchBuffers::reshape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers7reshapeE10SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::BeamSearchBuffers::reshape::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers7reshapeE10SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::BeamSearchBuffers::reshape::maxSequenceLength"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderStateE", "tensorrt_llm::runtime::decoder::DecoderState"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12DecoderStateEv", "tensorrt_llm::runtime::decoder::DecoderState::DecoderState"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState16DecodingInputPtrE", "tensorrt_llm::runtime::decoder::DecoderState::DecodingInputPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState17DecodingOutputPtrE", "tensorrt_llm::runtime::decoder::DecoderState::DecodingOutputPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState13LlmRequestPtrE", "tensorrt_llm::runtime::decoder::DecoderState::LlmRequestPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState13RequestVectorE", "tensorrt_llm::runtime::decoder::DecoderState::RequestVector"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState9TensorPtrE", "tensorrt_llm::runtime::decoder::DecoderState::TensorPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState16disableLookaheadERK13RequestVector", "tensorrt_llm::runtime::decoder::DecoderState::disableLookahead"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState16disableLookaheadERK13RequestVector", "tensorrt_llm::runtime::decoder::DecoderState::disableLookahead::genRequests"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState24getAcceptedLengthsCumSumEv", "tensorrt_llm::runtime::decoder::DecoderState::getAcceptedLengthsCumSum"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState22getAcceptedPackedPathsEv", "tensorrt_llm::runtime::decoder::DecoderState::getAcceptedPackedPaths"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState15getAllNewTokensEv", "tensorrt_llm::runtime::decoder::DecoderState::getAllNewTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState20getBeamSearchBuffersEv", "tensorrt_llm::runtime::decoder::DecoderState::getBeamSearchBuffers"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState24getCacheIndirectionInputEv", "tensorrt_llm::runtime::decoder::DecoderState::getCacheIndirectionInput"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState25getCacheIndirectionOutputEv", "tensorrt_llm::runtime::decoder::DecoderState::getCacheIndirectionOutput"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getCumLogProbsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getCumLogProbs"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getCumLogProbsEv", "tensorrt_llm::runtime::decoder::DecoderState::getCumLogProbs"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getCumLogProbsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getCumLogProbs::batchIdx"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState15getEagleBuffersEv", "tensorrt_llm::runtime::decoder::DecoderState::getEagleBuffers"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState29getExplicitDraftTokensBuffersEv", "tensorrt_llm::runtime::decoder::DecoderState::getExplicitDraftTokensBuffers"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState16getFinishReasonsEv", "tensorrt_llm::runtime::decoder::DecoderState::getFinishReasons"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getFinishedSumEv", "tensorrt_llm::runtime::decoder::DecoderState::getFinishedSum"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getGatheredIdsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getGatheredIds"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getGatheredIdsEv", "tensorrt_llm::runtime::decoder::DecoderState::getGatheredIds"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getGatheredIdsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getGatheredIds::batchIdx"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getGenerationStepsEv", "tensorrt_llm::runtime::decoder::DecoderState::getGenerationSteps"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState6getIdsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getIds"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState6getIdsEv", "tensorrt_llm::runtime::decoder::DecoderState::getIds"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState6getIdsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getIds::batchIdx"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState21getJointDecodingInputEv", "tensorrt_llm::runtime::decoder::DecoderState::getJointDecodingInput"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState22getJointDecodingOutputEv", "tensorrt_llm::runtime::decoder::DecoderState::getJointDecodingOutput"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState11getLogProbsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getLogProbs"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState11getLogProbsEv", "tensorrt_llm::runtime::decoder::DecoderState::getLogProbs"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState11getLogProbsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getLogProbs::batchIdx"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState19getLookaheadBuffersEv", "tensorrt_llm::runtime::decoder::DecoderState::getLookaheadBuffers"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState15getMaxBeamWidthEv", "tensorrt_llm::runtime::decoder::DecoderState::getMaxBeamWidth"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState27getMaxDecodingDecoderTokensEv", "tensorrt_llm::runtime::decoder::DecoderState::getMaxDecodingDecoderTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState26getMaxDecodingEngineTokensEv", "tensorrt_llm::runtime::decoder::DecoderState::getMaxDecodingEngineTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getMaxNumSequencesEv", "tensorrt_llm::runtime::decoder::DecoderState::getMaxNumSequences"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState20getMaxSequenceLengthEv", "tensorrt_llm::runtime::decoder::DecoderState::getMaxSequenceLength"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getNextDraftTokensEv", "tensorrt_llm::runtime::decoder::DecoderState::getNextDraftTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState25getNextDraftTokensLengthsEv", "tensorrt_llm::runtime::decoder::DecoderState::getNextDraftTokensLengths"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState26getNumDecodingEngineTokensE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getNumDecodingEngineTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState26getNumDecodingEngineTokensEv", "tensorrt_llm::runtime::decoder::DecoderState::getNumDecodingEngineTokens"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState26getNumDecodingEngineTokensE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getNumDecodingEngineTokens::batchIdx"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState12getParentIdsEv", "tensorrt_llm::runtime::decoder::DecoderState::getParentIds"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState25getPrevDraftTokensLengthsEv", "tensorrt_llm::runtime::decoder::DecoderState::getPrevDraftTokensLengths"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getSequenceLengthsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getSequenceLengths"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getSequenceLengthsEv", "tensorrt_llm::runtime::decoder::DecoderState::getSequenceLengths"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getSequenceLengthsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getSequenceLengths::batchIdx"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState26getSpeculativeDecodingModeEv", "tensorrt_llm::runtime::decoder::DecoderState::getSpeculativeDecodingMode"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState18mBeamSearchBuffersE", "tensorrt_llm::runtime::decoder::DecoderState::mBeamSearchBuffers"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState19mJointDecodingInputE", "tensorrt_llm::runtime::decoder::DecoderState::mJointDecodingInput"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState20mJointDecodingOutputE", "tensorrt_llm::runtime::decoder::DecoderState::mJointDecodingOutput"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState13mMaxBeamWidthE", "tensorrt_llm::runtime::decoder::DecoderState::mMaxBeamWidth"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState25mMaxDecodingDecoderTokensE", "tensorrt_llm::runtime::decoder::DecoderState::mMaxDecodingDecoderTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24mMaxDecodingEngineTokensE", "tensorrt_llm::runtime::decoder::DecoderState::mMaxDecodingEngineTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState16mMaxNumSequencesE", "tensorrt_llm::runtime::decoder::DecoderState::mMaxNumSequences"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState18mMaxSequenceLengthE", "tensorrt_llm::runtime::decoder::DecoderState::mMaxSequenceLength"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24mNumDecodingEngineTokensE", "tensorrt_llm::runtime::decoder::DecoderState::mNumDecodingEngineTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24mSpeculativeDecodingModeE", "tensorrt_llm::runtime::decoder::DecoderState::mSpeculativeDecodingMode"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeBuffers::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeBuffers::maxAttentionWindow"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeBuffers::maxBatchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeBuffers::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeBuffers::maxSequenceLength"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeBuffers::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeBuffers::sinkTokenLength"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeBuffers::worldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState30reshapeCacheIndirectionBuffersE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::reshapeCacheIndirectionBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState30reshapeCacheIndirectionBuffersE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::reshapeCacheIndirectionBuffers::maxAttentionWindow"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState30reshapeCacheIndirectionBuffersE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::reshapeCacheIndirectionBuffers::maxBatchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState30reshapeCacheIndirectionBuffersE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::reshapeCacheIndirectionBuffers::maxBeamWidth"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState33reshapeSpeculativeDecodingBuffersERK23SpeculativeDecodingMode10SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeSpeculativeDecodingBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState33reshapeSpeculativeDecodingBuffersERK23SpeculativeDecodingMode10SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeSpeculativeDecodingBuffers::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState33reshapeSpeculativeDecodingBuffersERK23SpeculativeDecodingMode10SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeSpeculativeDecodingBuffers::maxTokensPerEngineStep"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState33reshapeSpeculativeDecodingBuffersERK23SpeculativeDecodingMode10SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeSpeculativeDecodingBuffers::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState33reshapeSpeculativeDecodingBuffersERK23SpeculativeDecodingMode10SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeSpeculativeDecodingBuffers::speculativeDecodingMode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState33reshapeSpeculativeDecodingBuffersERK23SpeculativeDecodingMode10SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeSpeculativeDecodingBuffers::worldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12setBeamWidthE10SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::setBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12setBeamWidthE10SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::setBeamWidth::batchIdx"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12setBeamWidthE10SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::setBeamWidth::beamWidth"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState18setGenerationStepsERKNSt6vectorI10SizeType32EE", "tensorrt_llm::runtime::decoder::DecoderState::setGenerationSteps"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState18setGenerationStepsERKNSt6vectorI10SizeType32EE", "tensorrt_llm::runtime::decoder::DecoderState::setGenerationSteps::generationSteps"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState26setNumDecodingEngineTokensE10SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::setNumDecodingEngineTokens"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState26setNumDecodingEngineTokensE10SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::setNumDecodingEngineTokens::batchIdx"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState26setNumDecodingEngineTokensE10SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::setNumDecodingEngineTokens::numTokens"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup::dtype"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup::maxAttentionWindow"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup::maxNumSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup::maxSequenceLength"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup::sinkTokenLength"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup::worldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12setupBuffersEN8nvinfer18DataTypeERK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12setupBuffersEN8nvinfer18DataTypeERK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupBuffers::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12setupBuffersEN8nvinfer18DataTypeERK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupBuffers::dtype"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState21setupCacheIndirectionE10SizeType3210SizeType3210SizeType32RK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupCacheIndirection"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState21setupCacheIndirectionE10SizeType3210SizeType3210SizeType32RK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupCacheIndirection::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState21setupCacheIndirectionE10SizeType3210SizeType3210SizeType32RK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupCacheIndirection::maxAttentionWindow"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState21setupCacheIndirectionE10SizeType3210SizeType3210SizeType32RK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupCacheIndirection::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState21setupCacheIndirectionE10SizeType3210SizeType3210SizeType32RK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupCacheIndirection::maxNumSequences"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState28setupCacheIndirectionBuffersERK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupCacheIndirectionBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState28setupCacheIndirectionBuffersERK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupCacheIndirectionBuffers::bufferManager"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24setupSpeculativeDecodingERK23SpeculativeDecodingMode10SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecoding"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24setupSpeculativeDecodingERK23SpeculativeDecodingMode10SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecoding::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24setupSpeculativeDecodingERK23SpeculativeDecodingMode10SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecoding::dtype"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24setupSpeculativeDecodingERK23SpeculativeDecodingMode10SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecoding::maxTokensPerEngineStep"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24setupSpeculativeDecodingERK23SpeculativeDecodingMode10SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecoding::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24setupSpeculativeDecodingERK23SpeculativeDecodingMode10SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecoding::speculativeDecodingMode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24setupSpeculativeDecodingERK23SpeculativeDecodingMode10SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecoding::worldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState31setupSpeculativeDecodingBuffersE23SpeculativeDecodingModeN8nvinfer18DataTypeERK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecodingBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState31setupSpeculativeDecodingBuffersE23SpeculativeDecodingModeN8nvinfer18DataTypeERK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecodingBuffers::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState31setupSpeculativeDecodingBuffersE23SpeculativeDecodingModeN8nvinfer18DataTypeERK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecodingBuffers::dtype"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState31setupSpeculativeDecodingBuffersE23SpeculativeDecodingModeN8nvinfer18DataTypeERK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecodingBuffers::speculativeDecodingMode"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batchE", "tensorrt_llm::runtime::decoder_batch"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5InputE", "tensorrt_llm::runtime::decoder_batch::Input"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input5InputERKNSt6vectorI14TensorConstPtrEE", "tensorrt_llm::runtime::decoder_batch::Input::Input"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input5InputERKNSt6vectorINSt6vectorI14TensorConstPtrEEEE10SizeType32", "tensorrt_llm::runtime::decoder_batch::Input::Input"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input5InputERKNSt6vectorI14TensorConstPtrEE", "tensorrt_llm::runtime::decoder_batch::Input::Input::logits"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input5InputERKNSt6vectorINSt6vectorI14TensorConstPtrEEEE10SizeType32", "tensorrt_llm::runtime::decoder_batch::Input::Input::logits"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input5InputERKNSt6vectorINSt6vectorI14TensorConstPtrEEEE10SizeType32", "tensorrt_llm::runtime::decoder_batch::Input::Input::maxDecoderSteps"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input14TensorConstPtrE", "tensorrt_llm::runtime::decoder_batch::Input::TensorConstPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input9TensorPtrE", "tensorrt_llm::runtime::decoder_batch::Input::TensorPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input10batchSlotsE", "tensorrt_llm::runtime::decoder_batch::Input::batchSlots"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input6logitsE", "tensorrt_llm::runtime::decoder_batch::Input::logits"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input15maxDecoderStepsE", "tensorrt_llm::runtime::decoder_batch::Input::maxDecoderSteps"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20getDefaultBatchSlotsEN7runtime10SizeType32E", "tensorrt_llm::runtime::getDefaultBatchSlots"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20getDefaultBatchSlotsEN7runtime10SizeType32E", "tensorrt_llm::runtime::getDefaultBatchSlots::batchSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25getVirtualMemoryAllocatorEv", "tensorrt_llm::runtime::getVirtualMemoryAllocator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23getVirtualMemoryManagerEv", "tensorrt_llm::runtime::getVirtualMemoryManager"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime15ipcNvlsAllocateE6size_tNSt3setIiEE", "tensorrt_llm::runtime::ipcNvlsAllocate"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime15ipcNvlsAllocateE6size_tNSt3setIiEE", "tensorrt_llm::runtime::ipcNvlsAllocate::ranks"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime15ipcNvlsAllocateE6size_tNSt3setIiEE", "tensorrt_llm::runtime::ipcNvlsAllocate::size"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ipcNvlsFreeEP13IpcNvlsHandle", "tensorrt_llm::runtime::ipcNvlsFree"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ipcNvlsFreeEP13IpcNvlsHandle", "tensorrt_llm::runtime::ipcNvlsFree::handle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime16ipcNvlsSupportedEv", "tensorrt_llm::runtime::ipcNvlsSupported"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20lamportInitializeAllEPvPvPv6size_t", "tensorrt_llm::runtime::lamportInitializeAll"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20lamportInitializeAllEPvPvPv6size_t", "tensorrt_llm::runtime::lamportInitializeAll::buffer_0"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20lamportInitializeAllEPvPvPv6size_t", "tensorrt_llm::runtime::lamportInitializeAll::buffer_1"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20lamportInitializeAllEPvPvPv6size_t", "tensorrt_llm::runtime::lamportInitializeAll::buffer_2"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20lamportInitializeAllEPvPvPv6size_t", "tensorrt_llm::runtime::lamportInitializeAll::size"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK10LoraModule", "tensorrt_llm::runtime::operator<<"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK26LoraCachePageManagerConfig", "tensorrt_llm::runtime::operator<<"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK7IBuffer", "tensorrt_llm::runtime::operator<<"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK7ITensor", "tensorrt_llm::runtime::operator<<"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERKN7ITensor5ShapeE", "tensorrt_llm::runtime::operator<<"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERKN9LoraCache21TaskLayerModuleConfigE", "tensorrt_llm::runtime::operator<<"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK7IBuffer", "tensorrt_llm::runtime::operator<<::buffer"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK26LoraCachePageManagerConfig", "tensorrt_llm::runtime::operator<<::c"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERKN7ITensor5ShapeE", "tensorrt_llm::runtime::operator<<::dims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK10LoraModule", "tensorrt_llm::runtime::operator<<::module"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK26LoraCachePageManagerConfig", "tensorrt_llm::runtime::operator<<::os"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERKN9LoraCache21TaskLayerModuleConfigE", "tensorrt_llm::runtime::operator<<::os"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK10LoraModule", "tensorrt_llm::runtime::operator<<::output"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK7IBuffer", "tensorrt_llm::runtime::operator<<::output"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK7ITensor", "tensorrt_llm::runtime::operator<<::output"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERKN7ITensor5ShapeE", "tensorrt_llm::runtime::operator<<::output"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK7ITensor", "tensorrt_llm::runtime::operator<<::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERKN9LoraCache21TaskLayerModuleConfigE", "tensorrt_llm::runtime::operator<<::v"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25setVirtualMemoryAllocatorERKNSt6stringEN26CudaVirtualMemoryAllocator11RestoreModeENSt10shared_ptrI10CudaStreamEE", "tensorrt_llm::runtime::setVirtualMemoryAllocator"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25setVirtualMemoryAllocatorERKNSt6stringEN26CudaVirtualMemoryAllocator11RestoreModeENSt10shared_ptrI10CudaStreamEE", "tensorrt_llm::runtime::setVirtualMemoryAllocator::backStream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25setVirtualMemoryAllocatorERKNSt6stringEN26CudaVirtualMemoryAllocator11RestoreModeENSt10shared_ptrI10CudaStreamEE", "tensorrt_llm::runtime::setVirtualMemoryAllocator::mode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25setVirtualMemoryAllocatorERKNSt6stringEN26CudaVirtualMemoryAllocator11RestoreModeENSt10shared_ptrI10CudaStreamEE", "tensorrt_llm::runtime::setVirtualMemoryAllocator::tag"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9to_stringERK26LoraCachePageManagerConfig", "tensorrt_llm::runtime::to_string"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9to_stringERKN9LoraCache21TaskLayerModuleConfigE", "tensorrt_llm::runtime::to_string"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9to_stringERK26LoraCachePageManagerConfig", "tensorrt_llm::runtime::to_string::c"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9to_stringERKN9LoraCache21TaskLayerModuleConfigE", "tensorrt_llm::runtime::to_string::v"], [134, 9, 0, "-", "tensorrt_llm"]], "tensorrt_llm": [[129, 9, 0, "-", "functional"], [131, 9, 0, "-", "models"], [132, 9, 0, "-", "plugin"], [133, 9, 0, "-", "quantization"], [134, 9, 0, "-", "runtime"]], "tensorrt_llm.functional": [[129, 10, 1, "", "AllReduceFusionOp"], [129, 10, 1, "", "AllReduceParams"], [129, 10, 1, "", "AllReduceStrategy"], [129, 10, 1, "", "AttentionMaskType"], [129, 10, 1, "", "Conditional"], [129, 10, 1, "", "DimRange"], [129, 10, 1, "", "LayerNormPositionType"], [129, 10, 1, "", "LayerNormType"], [129, 10, 1, "", "MLPType"], [129, 10, 1, "", "MoEAllReduceParams"], [129, 10, 1, "", "PositionEmbeddingType"], [129, 10, 1, "", "RopeEmbeddingUtils"], [129, 10, 1, "", "RotaryScalingType"], [129, 10, 1, "", "SideStreamIDType"], [129, 10, 1, "", "SliceInputType"], [129, 10, 1, "", "Tensor"], [129, 14, 1, "", "abs"], [129, 14, 1, "", "activation"], [129, 14, 1, "", "add"], [129, 14, 1, "", "allgather"], [129, 14, 1, "", "allreduce"], [129, 14, 1, "", "arange"], [129, 14, 1, "", "argmax"], [129, 14, 1, "", "assertion"], [129, 14, 1, "", "avg_pool2d"], [129, 14, 1, "", "bert_attention"], [129, 14, 1, "", "broadcast_helper"], [129, 14, 1, "", "cast"], [129, 14, 1, "", "categorical_sample"], [129, 14, 1, "", "chunk"], [129, 14, 1, "", "clip"], [129, 14, 1, "", "concat"], [129, 14, 1, "", "constant"], [129, 14, 1, "", "constant_to_tensor_"], [129, 14, 1, "", "constants_to_tensors_"], [129, 14, 1, "", "conv1d"], [129, 14, 1, "", "conv2d"], [129, 14, 1, "", "conv3d"], [129, 14, 1, "", "conv_transpose2d"], [129, 14, 1, "", "cos"], [129, 14, 1, "", "cp_split_plugin"], [129, 14, 1, "", "create_allreduce_plugin"], [129, 14, 1, "", "cuda_stream_sync"], [129, 14, 1, "", "cumsum"], [129, 14, 1, "", "div"], [129, 14, 1, "", "dora_plugin"], [129, 14, 1, "", "einsum"], [129, 14, 1, "", "elementwise_binary"], [129, 14, 1, "", "embedding"], [129, 14, 1, "", "eq"], [129, 14, 1, "", "exp"], [129, 14, 1, "", "expand"], [129, 14, 1, "", "expand_dims"], [129, 14, 1, "", "expand_dims_like"], [129, 14, 1, "", "expand_mask"], [129, 14, 1, "", "flatten"], [129, 14, 1, "", "flip"], [129, 14, 1, "", "floordiv"], [129, 14, 1, "", "gather"], [129, 14, 1, "", "gather_last_token_logits"], [129, 14, 1, "", "gather_nd"], [129, 14, 1, "", "gegelu"], [129, 14, 1, "", "geglu"], [129, 14, 1, "", "gelu"], [129, 14, 1, "", "gemm_allreduce"], [129, 14, 1, "", "gemm_swiglu"], [129, 14, 1, "", "generate_alibi_biases"], [129, 14, 1, "", "generate_alibi_slopes"], [129, 14, 1, "", "generate_logn_scaling"], [129, 14, 1, "", "gpt_attention"], [129, 14, 1, "", "group_norm"], [129, 14, 1, "", "gt"], [129, 14, 1, "", "identity"], [129, 14, 1, "", "index_select"], [129, 14, 1, "", "int_clip"], [129, 14, 1, "", "interpolate"], [129, 14, 1, "", "is_gated_activation"], [129, 14, 1, "", "layer_norm"], [129, 14, 1, "", "log"], [129, 14, 1, "", "log_softmax"], [129, 14, 1, "", "lora_plugin"], [129, 14, 1, "", "low_latency_gemm"], [129, 14, 1, "", "low_latency_gemm_swiglu"], [129, 14, 1, "", "lt"], [129, 14, 1, "", "mamba_conv1d"], [129, 14, 1, "", "masked_scatter"], [129, 14, 1, "", "masked_select"], [129, 14, 1, "", "matmul"], [129, 14, 1, "", "max"], [129, 14, 1, "", "maximum"], [129, 14, 1, "", "mean"], [129, 14, 1, "", "meshgrid2d"], [129, 14, 1, "", "min"], [129, 14, 1, "", "minimum"], [129, 14, 1, "", "modulo"], [129, 14, 1, "", "mul"], [129, 14, 1, "", "non_gated_version"], [129, 14, 1, "", "nonzero"], [129, 14, 1, "", "not_op"], [129, 14, 1, "", "op_and"], [129, 14, 1, "", "op_or"], [129, 14, 1, "", "op_xor"], [129, 14, 1, "", "outer"], [129, 14, 1, "", "pad"], [129, 14, 1, "", "permute"], [129, 14, 1, "", "pow"], [129, 14, 1, "", "prod"], [129, 14, 1, "", "quick_gelu"], [129, 14, 1, "", "rand"], [129, 14, 1, "", "rearrange"], [129, 14, 1, "", "recv"], [129, 14, 1, "", "reduce"], [129, 14, 1, "", "reduce_scatter"], [129, 14, 1, "", "relu"], [129, 14, 1, "", "repeat"], [129, 14, 1, "", "repeat_interleave"], [129, 14, 1, "", "rg_lru"], [129, 14, 1, "", "rms_norm"], [129, 14, 1, "", "round"], [129, 14, 1, "", "scatter"], [129, 14, 1, "", "scatter_nd"], [129, 14, 1, "", "select"], [129, 14, 1, "", "selective_scan"], [129, 14, 1, "", "send"], [129, 14, 1, "", "shape"], [129, 14, 1, "", "sigmoid"], [129, 14, 1, "", "silu"], [129, 14, 1, "", "sin"], [129, 14, 1, "", "slice"], [129, 14, 1, "", "softmax"], [129, 14, 1, "", "softplus"], [129, 14, 1, "", "split"], [129, 14, 1, "", "sqrt"], [129, 14, 1, "", "squared_relu"], [129, 14, 1, "", "squeeze"], [129, 14, 1, "", "stack"], [129, 14, 1, "", "sub"], [129, 14, 1, "", "sum"], [129, 14, 1, "", "swiglu"], [129, 14, 1, "", "tanh"], [129, 14, 1, "", "topk"], [129, 14, 1, "", "transpose"], [129, 14, 1, "", "unary"], [129, 14, 1, "", "unbind"], [129, 14, 1, "", "unsqueeze"], [129, 14, 1, "", "view"], [129, 14, 1, "", "where"]], "tensorrt_llm.functional.AllReduceFusionOp": [[129, 11, 1, "", "LAST_PROCESS_FOR_UB"], [129, 11, 1, "", "MOE_FINALIZE_ALLREDUCE_RESIDUAL_RMS_NORM"], [129, 11, 1, "", "NONE"], [129, 11, 1, "", "RESIDUAL_RMS_NORM"], [129, 11, 1, "", "RESIDUAL_RMS_NORM_OUT_QUANT_FP8"], [129, 11, 1, "", "RESIDUAL_RMS_NORM_OUT_QUANT_NVFP4"], [129, 11, 1, "", "RESIDUAL_RMS_NORM_QUANT_FP8"], [129, 11, 1, "", "RESIDUAL_RMS_NORM_QUANT_NVFP4"], [129, 11, 1, "", "RESIDUAL_RMS_PREPOST_NORM"]], "tensorrt_llm.functional.AllReduceParams": [[129, 12, 1, "", "has_affine"], [129, 12, 1, "", "has_bias"], [129, 12, 1, "", "has_scale"], [129, 12, 1, "", "update_strategy"]], "tensorrt_llm.functional.AllReduceStrategy": [[129, 11, 1, "", "AUTO"], [129, 11, 1, "", "LOWPRECISION"], [129, 11, 1, "", "MIN_LATENCY"], [129, 11, 1, "", "MNNVL"], [129, 11, 1, "", "NCCL"], [129, 11, 1, "", "NCCL_SYMMETRIC"], [129, 11, 1, "", "ONESHOT"], [129, 11, 1, "", "TWOSHOT"], [129, 11, 1, "", "UB"]], "tensorrt_llm.functional.AttentionMaskType": [[129, 11, 1, "", "bidirectional"], [129, 11, 1, "", "bidirectionalglm"], [129, 11, 1, "", "blocksparse"], [129, 11, 1, "", "causal"], [129, 11, 1, "", "custom_mask"], [129, 11, 1, "", "padding"], [129, 11, 1, "", "sliding_window_causal"]], "tensorrt_llm.functional.Conditional": [[129, 12, 1, "", "add_input"], [129, 12, 1, "", "add_output"]], "tensorrt_llm.functional.LayerNormPositionType": [[129, 11, 1, "", "post_layernorm"], [129, 11, 1, "", "pre_layernorm"]], "tensorrt_llm.functional.LayerNormType": [[129, 11, 1, "", "GroupNorm"], [129, 11, 1, "", "LayerNorm"], [129, 11, 1, "", "RmsNorm"]], "tensorrt_llm.functional.MLPType": [[129, 11, 1, "", "FusedGatedMLP"], [129, 11, 1, "", "GatedMLP"], [129, 11, 1, "", "MLP"]], "tensorrt_llm.functional.MoEAllReduceParams": [[129, 12, 1, "", "is_valid"]], "tensorrt_llm.functional.PositionEmbeddingType": [[129, 11, 1, "", "alibi"], [129, 11, 1, "", "alibi_with_scale"], [129, 11, 1, "", "chatglm"], [129, 12, 1, "", "choices"], [129, 11, 1, "", "deferred"], [129, 12, 1, "", "from_string"], [129, 12, 1, "", "is_alibi"], [129, 12, 1, "", "is_deferred"], [129, 12, 1, "", "is_mrope"], [129, 12, 1, "", "is_rope"], [129, 11, 1, "", "learned_absolute"], [129, 11, 1, "", "long_rope"], [129, 11, 1, "", "mrope"], [129, 11, 1, "", "relative"], [129, 11, 1, "", "rope_gpt_neox"], [129, 11, 1, "", "rope_gptj"], [129, 11, 1, "", "yarn"]], "tensorrt_llm.functional.RopeEmbeddingUtils": [[129, 12, 1, "", "apply_llama3_scaling"], [129, 12, 1, "", "apply_rotary_pos_emb"], [129, 12, 1, "", "apply_rotary_pos_emb_chatglm"], [129, 12, 1, "", "apply_rotary_pos_emb_cogvlm"], [129, 12, 1, "", "create_fake_weight"], [129, 12, 1, "", "create_sinusoidal_positions"], [129, 12, 1, "", "create_sinusoidal_positions_for_attention_plugin"], [129, 12, 1, "", "create_sinusoidal_positions_for_cogvlm_attention_plugin"], [129, 12, 1, "", "create_sinusoidal_positions_long_rope"], [129, 12, 1, "", "create_sinusoidal_positions_long_rope_for_attention_plugin"], [129, 12, 1, "", "create_sinusoidal_positions_yarn"], [129, 12, 1, "", "rotate_every_two"], [129, 12, 1, "", "rotate_half"]], "tensorrt_llm.functional.RotaryScalingType": [[129, 11, 1, "", "dynamic"], [129, 12, 1, "", "from_string"], [129, 11, 1, "", "linear"], [129, 11, 1, "", "llama3"], [129, 11, 1, "", "longrope"], [129, 11, 1, "", "mrope"], [129, 11, 1, "", "none"], [129, 11, 1, "", "yarn"]], "tensorrt_llm.functional.SideStreamIDType": [[129, 11, 1, "", "disable"], [129, 11, 1, "", "moe"]], "tensorrt_llm.functional.SliceInputType": [[129, 11, 1, "", "axes"], [129, 11, 1, "", "data"], [129, 11, 1, "", "fill_value"], [129, 11, 1, "", "size"], [129, 11, 1, "", "start"], [129, 11, 1, "", "stride"]], "tensorrt_llm.functional.Tensor": [[129, 12, 1, "", "abs"], [129, 12, 1, "", "cast"], [129, 13, 1, "", "dtype"], [129, 12, 1, "", "flatten"], [129, 12, 1, "", "get_parent"], [129, 12, 1, "", "get_users"], [129, 12, 1, "", "is_dynamic"], [129, 12, 1, "", "is_trt_wrapper"], [129, 13, 1, "", "location"], [129, 12, 1, "", "log"], [129, 12, 1, "", "mark_output"], [129, 12, 1, "", "max"], [129, 12, 1, "", "mean"], [129, 13, 1, "", "name"], [129, 12, 1, "", "ndim"], [129, 13, 1, "", "network"], [129, 12, 1, "", "permute"], [129, 12, 1, "", "rank"], [129, 12, 1, "", "repeat"], [129, 12, 1, "", "replace_all_uses_with"], [129, 12, 1, "", "select"], [129, 13, 1, "", "shape"], [129, 12, 1, "", "size"], [129, 12, 1, "", "split"], [129, 12, 1, "", "sqrt"], [129, 12, 1, "", "squeeze"], [129, 12, 1, "", "transpose"], [129, 12, 1, "", "unbind"], [129, 12, 1, "", "unsqueeze"], [129, 12, 1, "", "view"]], "tensorrt_llm.layers": [[130, 9, 0, "-", "activation"], [130, 9, 0, "-", "attention"], [130, 9, 0, "-", "cast"], [130, 9, 0, "-", "conv"], [130, 9, 0, "-", "embedding"], [130, 9, 0, "-", "linear"], [130, 9, 0, "-", "mlp"], [130, 9, 0, "-", "normalization"], [130, 9, 0, "-", "pooling"]], "tensorrt_llm.layers.activation": [[130, 10, 1, "", "Mish"]], "tensorrt_llm.layers.activation.Mish": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.attention": [[130, 10, 1, "", "Attention"], [130, 10, 1, "", "AttentionMaskParams"], [130, 10, 1, "", "AttentionParams"], [130, 10, 1, "", "BertAttention"], [130, 10, 1, "", "BlockSparseAttnParams"], [130, 10, 1, "", "CogVLMAttention"], [130, 10, 1, "", "DeepseekV2Attention"], [130, 10, 1, "", "DiffusersAttention"], [130, 10, 1, "", "KeyValueCacheParams"], [130, 10, 1, "", "MropeParams"], [130, 10, 1, "", "SpecDecodingParams"], [130, 14, 1, "", "compute_relative_bias"], [130, 14, 1, "", "make_causal_mask"]], "tensorrt_llm.layers.attention.Attention": [[130, 12, 1, "", "create_attention_const_params"], [130, 12, 1, "", "fill_attention_params"], [130, 12, 1, "", "forward"], [130, 12, 1, "", "postprocess"], [130, 12, 1, "", "set_rel_attn_table"]], "tensorrt_llm.layers.attention.AttentionParams": [[130, 12, 1, "", "fill_attention_const_params_for_long_rope"], [130, 12, 1, "", "fill_attention_const_params_for_rope"], [130, 12, 1, "", "is_valid"], [130, 12, 1, "", "is_valid_cross_attn"]], "tensorrt_llm.layers.attention.BertAttention": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.attention.CogVLMAttention": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.attention.DeepseekV2Attention": [[130, 12, 1, "", "forward"], [130, 12, 1, "", "postprocess"], [130, 12, 1, "", "weight_loader"]], "tensorrt_llm.layers.attention.DiffusersAttention": [[130, 12, 1, "", "forward"], [130, 12, 1, "", "joint_attn_forward"]], "tensorrt_llm.layers.attention.KeyValueCacheParams": [[130, 12, 1, "", "fill_none_tensor_list"], [130, 12, 1, "", "get_first_past_key_value"], [130, 12, 1, "", "is_valid"]], "tensorrt_llm.layers.cast": [[130, 10, 1, "", "Cast"]], "tensorrt_llm.layers.cast.Cast": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.conv": [[130, 10, 1, "", "Conv1d"], [130, 10, 1, "", "Conv2d"], [130, 10, 1, "", "Conv3d"], [130, 10, 1, "", "ConvTranspose2d"]], "tensorrt_llm.layers.conv.Conv1d": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.conv.Conv2d": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.conv.Conv3d": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.conv.ConvTranspose2d": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.embedding": [[130, 10, 1, "", "CombinedTimestepLabelEmbeddings"], [130, 10, 1, "", "CombinedTimestepTextProjEmbeddings"], [130, 10, 1, "", "Embedding"], [130, 10, 1, "", "LabelEmbedding"], [130, 10, 1, "", "PixArtAlphaTextProjection"], [130, 10, 1, "", "PromptTuningEmbedding"], [130, 10, 1, "", "SD3PatchEmbed"], [130, 10, 1, "", "TimestepEmbedding"], [130, 10, 1, "", "Timesteps"], [130, 14, 1, "", "get_1d_sincos_pos_embed_from_grid"], [130, 14, 1, "", "get_2d_sincos_pos_embed"], [130, 14, 1, "", "get_2d_sincos_pos_embed_from_grid"], [130, 14, 1, "", "get_timestep_embedding"]], "tensorrt_llm.layers.embedding.CombinedTimestepLabelEmbeddings": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.embedding.CombinedTimestepTextProjEmbeddings": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.embedding.Embedding": [[130, 12, 1, "", "forward"], [130, 12, 1, "", "postprocess"], [130, 12, 1, "", "weight_loader"]], "tensorrt_llm.layers.embedding.LabelEmbedding": [[130, 12, 1, "", "forward"], [130, 12, 1, "", "token_drop"]], "tensorrt_llm.layers.embedding.PixArtAlphaTextProjection": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.embedding.PromptTuningEmbedding": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.embedding.SD3PatchEmbed": [[130, 12, 1, "", "cropped_pos_embed"], [130, 12, 1, "", "forward"]], "tensorrt_llm.layers.embedding.TimestepEmbedding": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.embedding.Timesteps": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.linear": [[130, 11, 1, "", "ColumnLinear"], [130, 10, 1, "", "Linear"], [130, 10, 1, "", "LinearBase"], [130, 10, 1, "", "RowLinear"]], "tensorrt_llm.layers.linear.Linear": [[130, 12, 1, "", "collect_and_bias"], [130, 12, 1, "", "postprocess"], [130, 12, 1, "", "tp_split_dim"]], "tensorrt_llm.layers.linear.LinearBase": [[130, 12, 1, "", "collect_and_bias"], [130, 12, 1, "", "forward"], [130, 12, 1, "", "get_weight"], [130, 12, 1, "", "multiply_and_lora"], [130, 12, 1, "", "multiply_collect"], [130, 12, 1, "", "tp_split_dim"], [130, 12, 1, "", "weight_loader"]], "tensorrt_llm.layers.linear.RowLinear": [[130, 12, 1, "", "collect_and_bias"], [130, 12, 1, "", "multiply_collect"], [130, 12, 1, "", "tp_split_dim"]], "tensorrt_llm.layers.mlp": [[130, 10, 1, "", "FusedGatedMLP"], [130, 10, 1, "", "GatedMLP"], [130, 10, 1, "", "LinearActivation"], [130, 10, 1, "", "LinearApproximateGELU"], [130, 10, 1, "", "LinearGEGLU"], [130, 10, 1, "", "LinearGELU"], [130, 10, 1, "", "LinearSwiGLU"], [130, 10, 1, "", "MLP"], [130, 14, 1, "", "fc_gate_dora"], [130, 14, 1, "", "fc_gate_lora"]], "tensorrt_llm.layers.mlp.FusedGatedMLP": [[130, 12, 1, "", "fc_gate"], [130, 12, 1, "", "fc_gate_plugin"], [130, 12, 1, "", "forward"]], "tensorrt_llm.layers.mlp.GatedMLP": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.mlp.LinearActivation": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.mlp.LinearApproximateGELU": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.mlp.LinearGEGLU": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.mlp.LinearGELU": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.mlp.LinearSwiGLU": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.mlp.MLP": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.normalization": [[130, 10, 1, "", "AdaLayerNorm"], [130, 10, 1, "", "AdaLayerNormContinuous"], [130, 10, 1, "", "AdaLayerNormZero"], [130, 10, 1, "", "AdaLayerNormZeroSingle"], [130, 10, 1, "", "GroupNorm"], [130, 10, 1, "", "LayerNorm"], [130, 10, 1, "", "RmsNorm"], [130, 10, 1, "", "SD35AdaLayerNormZeroX"]], "tensorrt_llm.layers.normalization.AdaLayerNorm": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.normalization.AdaLayerNormContinuous": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.normalization.AdaLayerNormZero": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.normalization.AdaLayerNormZeroSingle": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.normalization.GroupNorm": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.normalization.LayerNorm": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.normalization.RmsNorm": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.normalization.SD35AdaLayerNormZeroX": [[130, 12, 1, "", "forward"]], "tensorrt_llm.layers.pooling": [[130, 10, 1, "", "AvgPool2d"]], "tensorrt_llm.layers.pooling.AvgPool2d": [[130, 12, 1, "", "forward"]], "tensorrt_llm.llmapi": [[143, 10, 1, "", "AttentionDpConfig"], [143, 10, 1, "", "AutoDecodingConfig"], [143, 10, 1, "", "BatchingType"], [143, 10, 1, "", "BuildCacheConfig"], [143, 10, 1, "", "BuildConfig"], [143, 10, 1, "", "CacheTransceiverConfig"], [143, 10, 1, "", "CalibConfig"], [143, 10, 1, "", "CapacitySchedulerPolicy"], [143, 10, 1, "", "CompletionOutput"], [143, 10, 1, "", "ContextChunkingPolicy"], [143, 10, 1, "", "CudaGraphConfig"], [143, 10, 1, "", "DisaggregatedParams"], [143, 10, 1, "", "DraftTargetDecodingConfig"], [143, 10, 1, "", "DynamicBatchConfig"], [143, 10, 1, "", "EagleDecodingConfig"], [143, 10, 1, "", "ExtendedRuntimePerfKnobConfig"], [143, 10, 1, "", "GuidedDecodingParams"], [143, 10, 1, "", "KvCacheConfig"], [143, 10, 1, "", "KvCacheRetentionConfig"], [143, 10, 1, "", "LLM"], [143, 11, 1, "", "LlmArgs"], [143, 10, 1, "", "LoRARequest"], [143, 10, 1, "", "LookaheadDecodingConfig"], [143, 10, 1, "", "MTPDecodingConfig"], [143, 10, 1, "", "MedusaDecodingConfig"], [143, 10, 1, "", "MoeConfig"], [143, 10, 1, "", "MpiCommSession"], [143, 10, 1, "", "MultimodalEncoder"], [143, 10, 1, "", "NGramDecodingConfig"], [143, 10, 1, "", "QuantAlgo"], [143, 10, 1, "", "QuantConfig"], [143, 10, 1, "", "RequestError"], [143, 10, 1, "", "RequestOutput"], [143, 10, 1, "", "SamplingParams"], [143, 10, 1, "", "SchedulerConfig"], [143, 10, 1, "", "TorchCompileConfig"], [143, 10, 1, "", "TorchLlmArgs"], [143, 10, 1, "", "TrtLlmArgs"], [143, 10, 1, "", "UserProvidedDecodingConfig"]], "tensorrt_llm.llmapi.AttentionDpConfig": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 15, 1, "", "batching_wait_iters"], [143, 12, 1, "", "construct"], [143, 12, 1, "", "copy"], [143, 12, 1, "", "dict"], [143, 15, 1, "", "enable_balance"], [143, 12, 1, "", "from_dict"], [143, 12, 1, "", "from_orm"], [143, 12, 1, "", "json"], [143, 11, 1, "", "model_computed_fields"], [143, 11, 1, "", "model_config"], [143, 12, 1, "", "model_construct"], [143, 12, 1, "", "model_copy"], [143, 12, 1, "", "model_dump"], [143, 12, 1, "", "model_dump_json"], [143, 13, 1, "", "model_extra"], [143, 11, 1, "", "model_fields"], [143, 13, 1, "", "model_fields_set"], [143, 12, 1, "", "model_json_schema"], [143, 12, 1, "", "model_parametrized_name"], [143, 12, 1, "", "model_post_init"], [143, 12, 1, "", "model_rebuild"], [143, 12, 1, "", "model_validate"], [143, 12, 1, "", "model_validate_json"], [143, 12, 1, "", "model_validate_strings"], [143, 12, 1, "", "parse_file"], [143, 12, 1, "", "parse_obj"], [143, 12, 1, "", "parse_raw"], [143, 12, 1, "", "schema"], [143, 12, 1, "", "schema_json"], [143, 15, 1, "", "timeout_iters"], [143, 12, 1, "", "update_forward_refs"], [143, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.AttentionDpConfig.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.AutoDecodingConfig": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 12, 1, "", "construct"], [143, 12, 1, "", "copy"], [143, 11, 1, "", "decoding_type"], [143, 12, 1, "", "dict"], [143, 12, 1, "", "from_dict"], [143, 12, 1, "", "from_orm"], [143, 12, 1, "", "json"], [143, 15, 1, "", "load_format"], [143, 15, 1, "", "max_concurrency"], [143, 15, 1, "", "max_draft_len"], [143, 11, 1, "", "model_computed_fields"], [143, 11, 1, "", "model_config"], [143, 12, 1, "", "model_construct"], [143, 12, 1, "", "model_copy"], [143, 12, 1, "", "model_dump"], [143, 12, 1, "", "model_dump_json"], [143, 13, 1, "", "model_extra"], [143, 11, 1, "", "model_fields"], [143, 13, 1, "", "model_fields_set"], [143, 12, 1, "", "model_json_schema"], [143, 12, 1, "", "model_parametrized_name"], [143, 12, 1, "", "model_post_init"], [143, 12, 1, "", "model_rebuild"], [143, 12, 1, "", "model_validate"], [143, 12, 1, "", "model_validate_json"], [143, 12, 1, "", "model_validate_strings"], [143, 12, 1, "", "parse_file"], [143, 12, 1, "", "parse_obj"], [143, 12, 1, "", "parse_raw"], [143, 12, 1, "", "schema"], [143, 12, 1, "", "schema_json"], [143, 13, 1, "", "spec_dec_mode"], [143, 15, 1, "", "speculative_model_dir"], [143, 12, 1, "", "supports_backend"], [143, 12, 1, "", "update_forward_refs"], [143, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.AutoDecodingConfig.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.BatchingType": [[143, 11, 1, "", "INFLIGHT"], [143, 11, 1, "", "STATIC"], [143, 12, 1, "", "__init__"], [143, 12, 1, "", "capitalize"], [143, 12, 1, "", "casefold"], [143, 12, 1, "", "center"], [143, 12, 1, "", "count"], [143, 12, 1, "", "encode"], [143, 12, 1, "", "endswith"], [143, 12, 1, "", "expandtabs"], [143, 12, 1, "", "find"], [143, 12, 1, "", "format"], [143, 12, 1, "", "format_map"], [143, 12, 1, "", "index"], [143, 12, 1, "", "isalnum"], [143, 12, 1, "", "isalpha"], [143, 12, 1, "", "isascii"], [143, 12, 1, "", "isdecimal"], [143, 12, 1, "", "isdigit"], [143, 12, 1, "", "isidentifier"], [143, 12, 1, "", "islower"], [143, 12, 1, "", "isnumeric"], [143, 12, 1, "", "isprintable"], [143, 12, 1, "", "isspace"], [143, 12, 1, "", "istitle"], [143, 12, 1, "", "isupper"], [143, 12, 1, "", "join"], [143, 12, 1, "", "ljust"], [143, 12, 1, "", "lower"], [143, 12, 1, "", "lstrip"], [143, 12, 1, "", "maketrans"], [143, 12, 1, "", "partition"], [143, 12, 1, "", "removeprefix"], [143, 12, 1, "", "removesuffix"], [143, 12, 1, "", "replace"], [143, 12, 1, "", "rfind"], [143, 12, 1, "", "rindex"], [143, 12, 1, "", "rjust"], [143, 12, 1, "", "rpartition"], [143, 12, 1, "", "rsplit"], [143, 12, 1, "", "rstrip"], [143, 12, 1, "", "split"], [143, 12, 1, "", "splitlines"], [143, 12, 1, "", "startswith"], [143, 12, 1, "", "strip"], [143, 12, 1, "", "swapcase"], [143, 12, 1, "", "title"], [143, 12, 1, "", "translate"], [143, 12, 1, "", "upper"], [143, 12, 1, "", "zfill"]], "tensorrt_llm.llmapi.BuildCacheConfig": [[143, 12, 1, "", "__init__"], [143, 13, 1, "id13", "cache_root"], [143, 13, 1, "id14", "max_cache_storage_gb"], [143, 13, 1, "id15", "max_records"]], "tensorrt_llm.llmapi.BuildConfig": [[143, 12, 1, "", "__init__"], [143, 11, 1, "", "auto_parallel_config"], [143, 11, 1, "", "dry_run"], [143, 11, 1, "", "enable_debug_output"], [143, 11, 1, "", "force_num_profiles"], [143, 12, 1, "", "from_dict"], [143, 12, 1, "", "from_json_file"], [143, 11, 1, "", "gather_context_logits"], [143, 11, 1, "", "gather_generation_logits"], [143, 12, 1, "", "get_build_config_defaults"], [143, 11, 1, "", "input_timing_cache"], [143, 11, 1, "", "kv_cache_type"], [143, 11, 1, "", "lora_config"], [143, 11, 1, "", "max_batch_size"], [143, 11, 1, "", "max_beam_width"], [143, 11, 1, "", "max_draft_len"], [143, 11, 1, "", "max_encoder_input_len"], [143, 11, 1, "", "max_input_len"], [143, 11, 1, "", "max_num_tokens"], [143, 11, 1, "", "max_prompt_embedding_table_size"], [143, 11, 1, "", "max_seq_len"], [143, 11, 1, "", "monitor_memory"], [143, 11, 1, "", "opt_batch_size"], [143, 11, 1, "", "opt_num_tokens"], [143, 11, 1, "", "output_timing_cache"], [143, 11, 1, "", "plugin_config"], [143, 11, 1, "", "profiling_verbosity"], [143, 11, 1, "", "speculative_decoding_mode"], [143, 11, 1, "", "strongly_typed"], [143, 12, 1, "", "to_dict"], [143, 12, 1, "", "update"], [143, 12, 1, "", "update_from_dict"], [143, 12, 1, "", "update_kv_cache_type"], [143, 11, 1, "", "use_mrope"], [143, 11, 1, "", "use_refit"], [143, 11, 1, "", "use_strip_plan"], [143, 11, 1, "", "visualize_network"], [143, 11, 1, "", "weight_sparsity"], [143, 11, 1, "", "weight_streaming"]], "tensorrt_llm.llmapi.CacheTransceiverConfig": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 15, 1, "", "backend"], [143, 12, 1, "", "construct"], [143, 12, 1, "", "copy"], [143, 12, 1, "", "dict"], [143, 12, 1, "", "from_orm"], [143, 12, 1, "", "from_pybind"], [143, 12, 1, "", "get_pybind_enum_fields"], [143, 12, 1, "", "get_pybind_variable_fields"], [143, 12, 1, "", "json"], [143, 15, 1, "", "max_tokens_in_buffer"], [143, 12, 1, "", "maybe_to_pybind"], [143, 12, 1, "", "mirror_pybind_enum"], [143, 12, 1, "", "mirror_pybind_fields"], [143, 11, 1, "", "model_computed_fields"], [143, 11, 1, "", "model_config"], [143, 12, 1, "", "model_construct"], [143, 12, 1, "", "model_copy"], [143, 12, 1, "", "model_dump"], [143, 12, 1, "", "model_dump_json"], [143, 13, 1, "", "model_extra"], [143, 11, 1, "", "model_fields"], [143, 13, 1, "", "model_fields_set"], [143, 12, 1, "", "model_json_schema"], [143, 12, 1, "", "model_parametrized_name"], [143, 12, 1, "", "model_post_init"], [143, 12, 1, "", "model_rebuild"], [143, 12, 1, "", "model_validate"], [143, 12, 1, "", "model_validate_json"], [143, 12, 1, "", "model_validate_strings"], [143, 12, 1, "", "parse_file"], [143, 12, 1, "", "parse_obj"], [143, 12, 1, "", "parse_raw"], [143, 12, 1, "", "pybind_equals"], [143, 12, 1, "", "schema"], [143, 12, 1, "", "schema_json"], [143, 12, 1, "", "update_forward_refs"], [143, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.CacheTransceiverConfig.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.CalibConfig": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 15, 1, "", "calib_batch_size"], [143, 15, 1, "", "calib_batches"], [143, 15, 1, "", "calib_dataset"], [143, 15, 1, "", "calib_max_seq_length"], [143, 12, 1, "", "construct"], [143, 12, 1, "", "copy"], [143, 15, 1, "", "device"], [143, 12, 1, "", "dict"], [143, 12, 1, "", "from_dict"], [143, 12, 1, "", "from_orm"], [143, 12, 1, "", "json"], [143, 11, 1, "", "model_computed_fields"], [143, 11, 1, "", "model_config"], [143, 12, 1, "", "model_construct"], [143, 12, 1, "", "model_copy"], [143, 12, 1, "", "model_dump"], [143, 12, 1, "", "model_dump_json"], [143, 13, 1, "", "model_extra"], [143, 11, 1, "", "model_fields"], [143, 13, 1, "", "model_fields_set"], [143, 12, 1, "", "model_json_schema"], [143, 12, 1, "", "model_parametrized_name"], [143, 12, 1, "", "model_post_init"], [143, 12, 1, "", "model_rebuild"], [143, 12, 1, "", "model_validate"], [143, 12, 1, "", "model_validate_json"], [143, 12, 1, "", "model_validate_strings"], [143, 12, 1, "", "parse_file"], [143, 12, 1, "", "parse_obj"], [143, 12, 1, "", "parse_raw"], [143, 15, 1, "", "random_seed"], [143, 12, 1, "", "schema"], [143, 12, 1, "", "schema_json"], [143, 12, 1, "", "to_dict"], [143, 15, 1, "", "tokenizer_max_seq_length"], [143, 12, 1, "", "update_forward_refs"], [143, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.CalibConfig.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.CapacitySchedulerPolicy": [[143, 11, 1, "", "GUARANTEED_NO_EVICT"], [143, 11, 1, "", "MAX_UTILIZATION"], [143, 11, 1, "", "STATIC_BATCH"], [143, 12, 1, "", "__init__"], [143, 12, 1, "", "capitalize"], [143, 12, 1, "", "casefold"], [143, 12, 1, "", "center"], [143, 12, 1, "", "count"], [143, 12, 1, "", "encode"], [143, 12, 1, "", "endswith"], [143, 12, 1, "", "expandtabs"], [143, 12, 1, "", "find"], [143, 12, 1, "", "format"], [143, 12, 1, "", "format_map"], [143, 12, 1, "", "index"], [143, 12, 1, "", "isalnum"], [143, 12, 1, "", "isalpha"], [143, 12, 1, "", "isascii"], [143, 12, 1, "", "isdecimal"], [143, 12, 1, "", "isdigit"], [143, 12, 1, "", "isidentifier"], [143, 12, 1, "", "islower"], [143, 12, 1, "", "isnumeric"], [143, 12, 1, "", "isprintable"], [143, 12, 1, "", "isspace"], [143, 12, 1, "", "istitle"], [143, 12, 1, "", "isupper"], [143, 12, 1, "", "join"], [143, 12, 1, "", "ljust"], [143, 12, 1, "", "lower"], [143, 12, 1, "", "lstrip"], [143, 12, 1, "", "maketrans"], [143, 12, 1, "", "partition"], [143, 12, 1, "", "removeprefix"], [143, 12, 1, "", "removesuffix"], [143, 12, 1, "", "replace"], [143, 12, 1, "", "rfind"], [143, 12, 1, "", "rindex"], [143, 12, 1, "", "rjust"], [143, 12, 1, "", "rpartition"], [143, 12, 1, "", "rsplit"], [143, 12, 1, "", "rstrip"], [143, 12, 1, "", "split"], [143, 12, 1, "", "splitlines"], [143, 12, 1, "", "startswith"], [143, 12, 1, "", "strip"], [143, 12, 1, "", "swapcase"], [143, 12, 1, "", "title"], [143, 12, 1, "", "translate"], [143, 12, 1, "", "upper"], [143, 12, 1, "", "zfill"]], "tensorrt_llm.llmapi.CompletionOutput": [[143, 12, 1, "", "__init__"], [143, 11, 1, "", "cumulative_logprob"], [143, 11, 1, "", "disaggregated_params"], [143, 11, 1, "", "finish_reason"], [143, 11, 1, "", "generation_logits"], [143, 11, 1, "", "index"], [143, 13, 1, "id2", "length"], [143, 11, 1, "", "logprobs"], [143, 13, 1, "id3", "logprobs_diff"], [143, 11, 1, "", "prompt_logprobs"], [143, 11, 1, "", "request_perf_metrics"], [143, 11, 1, "", "stop_reason"], [143, 11, 1, "", "text"], [143, 13, 1, "id4", "text_diff"], [143, 11, 1, "", "token_ids"], [143, 13, 1, "id5", "token_ids_diff"]], "tensorrt_llm.llmapi.ContextChunkingPolicy": [[143, 11, 1, "", "EQUAL_PROGRESS"], [143, 11, 1, "", "FIRST_COME_FIRST_SERVED"], [143, 12, 1, "", "__init__"], [143, 12, 1, "", "capitalize"], [143, 12, 1, "", "casefold"], [143, 12, 1, "", "center"], [143, 12, 1, "", "count"], [143, 12, 1, "", "encode"], [143, 12, 1, "", "endswith"], [143, 12, 1, "", "expandtabs"], [143, 12, 1, "", "find"], [143, 12, 1, "", "format"], [143, 12, 1, "", "format_map"], [143, 12, 1, "", "index"], [143, 12, 1, "", "isalnum"], [143, 12, 1, "", "isalpha"], [143, 12, 1, "", "isascii"], [143, 12, 1, "", "isdecimal"], [143, 12, 1, "", "isdigit"], [143, 12, 1, "", "isidentifier"], [143, 12, 1, "", "islower"], [143, 12, 1, "", "isnumeric"], [143, 12, 1, "", "isprintable"], [143, 12, 1, "", "isspace"], [143, 12, 1, "", "istitle"], [143, 12, 1, "", "isupper"], [143, 12, 1, "", "join"], [143, 12, 1, "", "ljust"], [143, 12, 1, "", "lower"], [143, 12, 1, "", "lstrip"], [143, 12, 1, "", "maketrans"], [143, 12, 1, "", "partition"], [143, 12, 1, "", "removeprefix"], [143, 12, 1, "", "removesuffix"], [143, 12, 1, "", "replace"], [143, 12, 1, "", "rfind"], [143, 12, 1, "", "rindex"], [143, 12, 1, "", "rjust"], [143, 12, 1, "", "rpartition"], [143, 12, 1, "", "rsplit"], [143, 12, 1, "", "rstrip"], [143, 12, 1, "", "split"], [143, 12, 1, "", "splitlines"], [143, 12, 1, "", "startswith"], [143, 12, 1, "", "strip"], [143, 12, 1, "", "swapcase"], [143, 12, 1, "", "title"], [143, 12, 1, "", "translate"], [143, 12, 1, "", "upper"], [143, 12, 1, "", "zfill"]], "tensorrt_llm.llmapi.CudaGraphConfig": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 15, 1, "", "batch_sizes"], [143, 12, 1, "", "construct"], [143, 12, 1, "", "copy"], [143, 12, 1, "", "dict"], [143, 15, 1, "", "enable_padding"], [143, 12, 1, "", "from_orm"], [143, 12, 1, "", "json"], [143, 15, 1, "", "max_batch_size"], [143, 11, 1, "", "model_computed_fields"], [143, 11, 1, "", "model_config"], [143, 12, 1, "", "model_construct"], [143, 12, 1, "", "model_copy"], [143, 12, 1, "", "model_dump"], [143, 12, 1, "", "model_dump_json"], [143, 13, 1, "", "model_extra"], [143, 11, 1, "", "model_fields"], [143, 13, 1, "", "model_fields_set"], [143, 12, 1, "", "model_json_schema"], [143, 12, 1, "", "model_parametrized_name"], [143, 12, 1, "", "model_post_init"], [143, 12, 1, "", "model_rebuild"], [143, 12, 1, "", "model_validate"], [143, 12, 1, "", "model_validate_json"], [143, 12, 1, "", "model_validate_strings"], [143, 12, 1, "", "parse_file"], [143, 12, 1, "", "parse_obj"], [143, 12, 1, "", "parse_raw"], [143, 12, 1, "", "schema"], [143, 12, 1, "", "schema_json"], [143, 12, 1, "", "update_forward_refs"], [143, 12, 1, "", "validate"], [143, 16, 1, "", "validate_cuda_graph_max_batch_size"]], "tensorrt_llm.llmapi.CudaGraphConfig.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.DisaggregatedParams": [[143, 12, 1, "", "__init__"], [143, 11, 1, "", "ctx_request_id"], [143, 11, 1, "", "draft_tokens"], [143, 11, 1, "", "first_gen_tokens"], [143, 12, 1, "", "get_context_phase_params"], [143, 12, 1, "", "get_request_type"], [143, 11, 1, "", "multimodal_embedding_handles"], [143, 11, 1, "", "multimodal_hashes"], [143, 11, 1, "", "opaque_state"], [143, 11, 1, "", "request_type"]], "tensorrt_llm.llmapi.DraftTargetDecodingConfig": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 12, 1, "", "construct"], [143, 12, 1, "", "copy"], [143, 11, 1, "", "decoding_type"], [143, 12, 1, "", "dict"], [143, 12, 1, "", "from_dict"], [143, 12, 1, "", "from_orm"], [143, 12, 1, "", "json"], [143, 15, 1, "", "load_format"], [143, 15, 1, "", "max_concurrency"], [143, 15, 1, "", "max_draft_len"], [143, 11, 1, "", "model_computed_fields"], [143, 11, 1, "", "model_config"], [143, 12, 1, "", "model_construct"], [143, 12, 1, "", "model_copy"], [143, 12, 1, "", "model_dump"], [143, 12, 1, "", "model_dump_json"], [143, 13, 1, "", "model_extra"], [143, 11, 1, "", "model_fields"], [143, 13, 1, "", "model_fields_set"], [143, 12, 1, "", "model_json_schema"], [143, 12, 1, "", "model_parametrized_name"], [143, 12, 1, "", "model_post_init"], [143, 12, 1, "", "model_rebuild"], [143, 12, 1, "", "model_validate"], [143, 12, 1, "", "model_validate_json"], [143, 12, 1, "", "model_validate_strings"], [143, 12, 1, "", "parse_file"], [143, 12, 1, "", "parse_obj"], [143, 12, 1, "", "parse_raw"], [143, 12, 1, "", "schema"], [143, 12, 1, "", "schema_json"], [143, 13, 1, "", "spec_dec_mode"], [143, 15, 1, "", "speculative_model_dir"], [143, 12, 1, "", "supports_backend"], [143, 12, 1, "", "update_forward_refs"], [143, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.DraftTargetDecodingConfig.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.DynamicBatchConfig": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 12, 1, "", "construct"], [143, 12, 1, "", "copy"], [143, 12, 1, "", "dict"], [143, 15, 1, "", "dynamic_batch_moving_average_window"], [143, 15, 1, "", "enable_batch_size_tuning"], [143, 15, 1, "", "enable_max_num_tokens_tuning"], [143, 12, 1, "", "from_orm"], [143, 12, 1, "", "from_pybind"], [143, 12, 1, "", "get_pybind_enum_fields"], [143, 12, 1, "", "get_pybind_variable_fields"], [143, 12, 1, "", "json"], [143, 12, 1, "", "maybe_to_pybind"], [143, 12, 1, "", "mirror_pybind_enum"], [143, 12, 1, "", "mirror_pybind_fields"], [143, 11, 1, "", "model_computed_fields"], [143, 11, 1, "", "model_config"], [143, 12, 1, "", "model_construct"], [143, 12, 1, "", "model_copy"], [143, 12, 1, "", "model_dump"], [143, 12, 1, "", "model_dump_json"], [143, 13, 1, "", "model_extra"], [143, 11, 1, "", "model_fields"], [143, 13, 1, "", "model_fields_set"], [143, 12, 1, "", "model_json_schema"], [143, 12, 1, "", "model_parametrized_name"], [143, 12, 1, "", "model_post_init"], [143, 12, 1, "", "model_rebuild"], [143, 12, 1, "", "model_validate"], [143, 12, 1, "", "model_validate_json"], [143, 12, 1, "", "model_validate_strings"], [143, 12, 1, "", "parse_file"], [143, 12, 1, "", "parse_obj"], [143, 12, 1, "", "parse_raw"], [143, 12, 1, "", "pybind_equals"], [143, 12, 1, "", "schema"], [143, 12, 1, "", "schema_json"], [143, 12, 1, "", "update_forward_refs"], [143, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.DynamicBatchConfig.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.EagleDecodingConfig": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 12, 1, "", "check_eagle_choices"], [143, 12, 1, "", "construct"], [143, 12, 1, "", "copy"], [143, 11, 1, "", "decoding_type"], [143, 12, 1, "", "dict"], [143, 15, 1, "", "dynamic_tree_max_topK"], [143, 15, 1, "", "eagle3_layers_to_capture"], [143, 15, 1, "", "eagle3_one_model"], [143, 15, 1, "", "eagle_choices"], [143, 12, 1, "", "from_dict"], [143, 12, 1, "", "from_orm"], [143, 15, 1, "", "greedy_sampling"], [143, 12, 1, "", "json"], [143, 15, 1, "", "load_format"], [143, 15, 1, "", "max_concurrency"], [143, 15, 1, "", "max_draft_len"], [143, 15, 1, "", "max_non_leaves_per_layer"], [143, 15, 1, "", "max_total_draft_tokens"], [143, 11, 1, "", "model_computed_fields"], [143, 11, 1, "", "model_config"], [143, 12, 1, "", "model_construct"], [143, 12, 1, "", "model_copy"], [143, 12, 1, "", "model_dump"], [143, 12, 1, "", "model_dump_json"], [143, 13, 1, "", "model_extra"], [143, 11, 1, "", "model_fields"], [143, 13, 1, "", "model_fields_set"], [143, 12, 1, "", "model_json_schema"], [143, 12, 1, "", "model_parametrized_name"], [143, 12, 1, "", "model_post_init"], [143, 12, 1, "", "model_rebuild"], [143, 12, 1, "", "model_validate"], [143, 12, 1, "", "model_validate_json"], [143, 12, 1, "", "model_validate_strings"], [143, 13, 1, "", "num_capture_layers"], [143, 15, 1, "", "num_eagle_layers"], [143, 12, 1, "", "parse_file"], [143, 12, 1, "", "parse_obj"], [143, 12, 1, "", "parse_raw"], [143, 15, 1, "", "posterior_threshold"], [143, 12, 1, "", "schema"], [143, 12, 1, "", "schema_json"], [143, 13, 1, "", "spec_dec_mode"], [143, 15, 1, "", "speculative_model_dir"], [143, 12, 1, "", "supports_backend"], [143, 12, 1, "", "update_forward_refs"], [143, 15, 1, "", "use_dynamic_tree"], [143, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.EagleDecodingConfig.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.ExtendedRuntimePerfKnobConfig": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 12, 1, "", "construct"], [143, 12, 1, "", "copy"], [143, 15, 1, "", "cuda_graph_cache_size"], [143, 15, 1, "", "cuda_graph_mode"], [143, 12, 1, "", "dict"], [143, 15, 1, "", "enable_context_fmha_fp32_acc"], [143, 12, 1, "", "from_orm"], [143, 12, 1, "", "from_pybind"], [143, 12, 1, "", "get_pybind_enum_fields"], [143, 12, 1, "", "get_pybind_variable_fields"], [143, 12, 1, "", "json"], [143, 12, 1, "", "maybe_to_pybind"], [143, 12, 1, "", "mirror_pybind_enum"], [143, 12, 1, "", "mirror_pybind_fields"], [143, 11, 1, "", "model_computed_fields"], [143, 11, 1, "", "model_config"], [143, 12, 1, "", "model_construct"], [143, 12, 1, "", "model_copy"], [143, 12, 1, "", "model_dump"], [143, 12, 1, "", "model_dump_json"], [143, 13, 1, "", "model_extra"], [143, 11, 1, "", "model_fields"], [143, 13, 1, "", "model_fields_set"], [143, 12, 1, "", "model_json_schema"], [143, 12, 1, "", "model_parametrized_name"], [143, 12, 1, "", "model_post_init"], [143, 12, 1, "", "model_rebuild"], [143, 12, 1, "", "model_validate"], [143, 12, 1, "", "model_validate_json"], [143, 12, 1, "", "model_validate_strings"], [143, 15, 1, "", "multi_block_mode"], [143, 12, 1, "", "parse_file"], [143, 12, 1, "", "parse_obj"], [143, 12, 1, "", "parse_raw"], [143, 12, 1, "", "pybind_equals"], [143, 12, 1, "", "schema"], [143, 12, 1, "", "schema_json"], [143, 12, 1, "", "update_forward_refs"], [143, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.ExtendedRuntimePerfKnobConfig.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.GuidedDecodingParams": [[143, 12, 1, "", "__init__"], [143, 11, 1, "", "grammar"], [143, 11, 1, "", "json"], [143, 11, 1, "", "json_object"], [143, 11, 1, "", "regex"], [143, 11, 1, "", "structural_tag"]], "tensorrt_llm.llmapi.KvCacheConfig": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 15, 1, "", "attention_dp_events_gather_period_ms"], [143, 12, 1, "", "construct"], [143, 12, 1, "", "copy"], [143, 15, 1, "", "copy_on_partial_reuse"], [143, 15, 1, "", "cross_kv_cache_fraction"], [143, 12, 1, "", "dict"], [143, 15, 1, "", "dtype"], [143, 15, 1, "", "enable_block_reuse"], [143, 15, 1, "", "enable_partial_reuse"], [143, 15, 1, "", "event_buffer_max_size"], [143, 15, 1, "", "free_gpu_memory_fraction"], [143, 12, 1, "", "from_orm"], [143, 12, 1, "", "from_pybind"], [143, 12, 1, "", "get_pybind_enum_fields"], [143, 12, 1, "", "get_pybind_variable_fields"], [143, 15, 1, "", "host_cache_size"], [143, 12, 1, "", "json"], [143, 15, 1, "", "mamba_ssm_cache_dtype"], [143, 15, 1, "", "max_attention_window"], [143, 15, 1, "", "max_gpu_total_bytes"], [143, 15, 1, "", "max_tokens"], [143, 12, 1, "", "maybe_to_pybind"], [143, 12, 1, "", "mirror_pybind_enum"], [143, 12, 1, "", "mirror_pybind_fields"], [143, 11, 1, "", "model_computed_fields"], [143, 11, 1, "", "model_config"], [143, 12, 1, "", "model_construct"], [143, 12, 1, "", "model_copy"], [143, 12, 1, "", "model_dump"], [143, 12, 1, "", "model_dump_json"], [143, 13, 1, "", "model_extra"], [143, 11, 1, "", "model_fields"], [143, 13, 1, "", "model_fields_set"], [143, 12, 1, "", "model_json_schema"], [143, 12, 1, "", "model_parametrized_name"], [143, 12, 1, "", "model_post_init"], [143, 12, 1, "", "model_rebuild"], [143, 12, 1, "", "model_validate"], [143, 12, 1, "", "model_validate_json"], [143, 12, 1, "", "model_validate_strings"], [143, 15, 1, "", "onboard_blocks"], [143, 12, 1, "", "parse_file"], [143, 12, 1, "", "parse_obj"], [143, 12, 1, "", "parse_raw"], [143, 12, 1, "", "pybind_equals"], [143, 12, 1, "", "schema"], [143, 12, 1, "", "schema_json"], [143, 15, 1, "", "secondary_offload_min_priority"], [143, 15, 1, "", "sink_token_length"], [143, 15, 1, "", "tokens_per_block"], [143, 12, 1, "", "update_forward_refs"], [143, 15, 1, "", "use_uvm"], [143, 12, 1, "", "validate"], [143, 16, 1, "", "validate_max_attention_window"], [143, 16, 1, "", "validate_max_gpu_total_bytes"]], "tensorrt_llm.llmapi.KvCacheConfig.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.KvCacheRetentionConfig": [[143, 10, 1, "", "TokenRangeRetentionConfig"], [143, 11, 1, "", "__init__"], [143, 13, 1, "", "decode_duration_ms"], [143, 13, 1, "", "decode_retention_priority"], [143, 13, 1, "", "directory"], [143, 13, 1, "", "token_range_retention_configs"], [143, 13, 1, "", "transfer_mode"]], "tensorrt_llm.llmapi.KvCacheRetentionConfig.TokenRangeRetentionConfig": [[143, 11, 1, "", "__init__"], [143, 13, 1, "", "duration_ms"], [143, 13, 1, "", "priority"], [143, 13, 1, "", "token_end"], [143, 13, 1, "", "token_start"]], "tensorrt_llm.llmapi.LLM": [[143, 12, 1, "", "__init__"], [143, 12, 1, "", "generate"], [143, 12, 1, "", "generate_async"], [143, 12, 1, "", "get_kv_cache_events"], [143, 12, 1, "", "get_kv_cache_events_async"], [143, 12, 1, "", "get_stats"], [143, 12, 1, "", "get_stats_async"], [143, 13, 1, "id0", "llm_id"], [143, 12, 1, "", "shutdown"], [143, 13, 1, "id1", "tokenizer"]], "tensorrt_llm.llmapi.LoRARequest": [[143, 12, 1, "", "__init__"], [143, 13, 1, "", "adapter_id"], [143, 13, 1, "", "ckpt_source"], [143, 11, 1, "", "lora_ckpt_source"], [143, 11, 1, "", "lora_int_id"], [143, 11, 1, "", "lora_name"], [143, 11, 1, "", "lora_path"], [143, 13, 1, "", "name"], [143, 13, 1, "", "path"]], "tensorrt_llm.llmapi.LookaheadDecodingConfig": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 12, 1, "", "calculate_speculative_resource"], [143, 12, 1, "", "construct"], [143, 12, 1, "", "copy"], [143, 11, 1, "", "decoding_type"], [143, 12, 1, "", "dict"], [143, 12, 1, "", "from_dict"], [143, 12, 1, "", "from_orm"], [143, 12, 1, "", "from_pybind"], [143, 12, 1, "", "get_pybind_enum_fields"], [143, 12, 1, "", "get_pybind_variable_fields"], [143, 12, 1, "", "json"], [143, 15, 1, "", "load_format"], [143, 15, 1, "", "max_concurrency"], [143, 15, 1, "", "max_draft_len"], [143, 15, 1, "", "max_ngram_size"], [143, 15, 1, "", "max_verification_set_size"], [143, 15, 1, "", "max_window_size"], [143, 12, 1, "", "maybe_to_pybind"], [143, 12, 1, "", "mirror_pybind_enum"], [143, 12, 1, "", "mirror_pybind_fields"], [143, 11, 1, "", "model_computed_fields"], [143, 11, 1, "", "model_config"], [143, 12, 1, "", "model_construct"], [143, 12, 1, "", "model_copy"], [143, 12, 1, "", "model_dump"], [143, 12, 1, "", "model_dump_json"], [143, 13, 1, "", "model_extra"], [143, 11, 1, "", "model_fields"], [143, 13, 1, "", "model_fields_set"], [143, 12, 1, "", "model_json_schema"], [143, 12, 1, "", "model_parametrized_name"], [143, 12, 1, "", "model_post_init"], [143, 12, 1, "", "model_rebuild"], [143, 12, 1, "", "model_validate"], [143, 12, 1, "", "model_validate_json"], [143, 12, 1, "", "model_validate_strings"], [143, 12, 1, "", "parse_file"], [143, 12, 1, "", "parse_obj"], [143, 12, 1, "", "parse_raw"], [143, 12, 1, "", "pybind_equals"], [143, 12, 1, "", "schema"], [143, 12, 1, "", "schema_json"], [143, 13, 1, "", "spec_dec_mode"], [143, 15, 1, "", "speculative_model_dir"], [143, 12, 1, "", "supports_backend"], [143, 12, 1, "", "update_forward_refs"], [143, 12, 1, "", "validate"], [143, 16, 1, "", "validate_positive_values"]], "tensorrt_llm.llmapi.LookaheadDecodingConfig.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.MTPDecodingConfig": [[143, 15, 1, "", "BEGIN_THINKING_PHASE_TOKEN"], [143, 10, 1, "", "Config"], [143, 15, 1, "", "END_THINKING_PHASE_TOKEN"], [143, 12, 1, "", "__init__"], [143, 12, 1, "", "construct"], [143, 12, 1, "", "copy"], [143, 11, 1, "", "decoding_type"], [143, 12, 1, "", "dict"], [143, 12, 1, "", "from_dict"], [143, 12, 1, "", "from_orm"], [143, 12, 1, "", "json"], [143, 15, 1, "", "load_format"], [143, 15, 1, "", "max_concurrency"], [143, 15, 1, "", "max_draft_len"], [143, 11, 1, "", "model_computed_fields"], [143, 11, 1, "", "model_config"], [143, 12, 1, "", "model_construct"], [143, 12, 1, "", "model_copy"], [143, 12, 1, "", "model_dump"], [143, 12, 1, "", "model_dump_json"], [143, 13, 1, "", "model_extra"], [143, 11, 1, "", "model_fields"], [143, 13, 1, "", "model_fields_set"], [143, 12, 1, "", "model_json_schema"], [143, 12, 1, "", "model_parametrized_name"], [143, 12, 1, "", "model_post_init"], [143, 12, 1, "", "model_rebuild"], [143, 12, 1, "", "model_validate"], [143, 12, 1, "", "model_validate_json"], [143, 12, 1, "", "model_validate_strings"], [143, 15, 1, "", "mtp_eagle_one_model"], [143, 13, 1, "", "num_capture_layers"], [143, 15, 1, "", "num_nextn_predict_layers"], [143, 15, 1, "", "num_nextn_predict_layers_from_model_config"], [143, 12, 1, "", "parse_file"], [143, 12, 1, "", "parse_obj"], [143, 12, 1, "", "parse_raw"], [143, 15, 1, "", "relaxed_delta"], [143, 15, 1, "", "relaxed_topk"], [143, 12, 1, "", "schema"], [143, 12, 1, "", "schema_json"], [143, 13, 1, "", "spec_dec_mode"], [143, 15, 1, "", "speculative_model_dir"], [143, 12, 1, "", "supports_backend"], [143, 12, 1, "", "update_forward_refs"], [143, 15, 1, "", "use_mtp_vanilla"], [143, 15, 1, "", "use_relaxed_acceptance_for_thinking"], [143, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.MTPDecodingConfig.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.MedusaDecodingConfig": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 12, 1, "", "construct"], [143, 12, 1, "", "copy"], [143, 11, 1, "", "decoding_type"], [143, 12, 1, "", "dict"], [143, 12, 1, "", "from_dict"], [143, 12, 1, "", "from_orm"], [143, 12, 1, "", "json"], [143, 15, 1, "", "load_format"], [143, 15, 1, "", "max_concurrency"], [143, 15, 1, "", "max_draft_len"], [143, 15, 1, "", "medusa_choices"], [143, 11, 1, "", "model_computed_fields"], [143, 11, 1, "", "model_config"], [143, 12, 1, "", "model_construct"], [143, 12, 1, "", "model_copy"], [143, 12, 1, "", "model_dump"], [143, 12, 1, "", "model_dump_json"], [143, 13, 1, "", "model_extra"], [143, 11, 1, "", "model_fields"], [143, 13, 1, "", "model_fields_set"], [143, 12, 1, "", "model_json_schema"], [143, 12, 1, "", "model_parametrized_name"], [143, 12, 1, "", "model_post_init"], [143, 12, 1, "", "model_rebuild"], [143, 12, 1, "", "model_validate"], [143, 12, 1, "", "model_validate_json"], [143, 12, 1, "", "model_validate_strings"], [143, 15, 1, "", "num_medusa_heads"], [143, 12, 1, "", "parse_file"], [143, 12, 1, "", "parse_obj"], [143, 12, 1, "", "parse_raw"], [143, 12, 1, "", "schema"], [143, 12, 1, "", "schema_json"], [143, 13, 1, "", "spec_dec_mode"], [143, 15, 1, "", "speculative_model_dir"], [143, 12, 1, "", "supports_backend"], [143, 12, 1, "", "update_forward_refs"], [143, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.MedusaDecodingConfig.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.MoeConfig": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 15, 1, "", "backend"], [143, 12, 1, "", "construct"], [143, 12, 1, "", "copy"], [143, 12, 1, "", "dict"], [143, 15, 1, "", "disable_finalize_fusion"], [143, 12, 1, "", "from_dict"], [143, 12, 1, "", "from_orm"], [143, 12, 1, "", "json"], [143, 15, 1, "", "load_balancer"], [143, 15, 1, "", "max_num_tokens"], [143, 11, 1, "", "model_computed_fields"], [143, 11, 1, "", "model_config"], [143, 12, 1, "", "model_construct"], [143, 12, 1, "", "model_copy"], [143, 12, 1, "", "model_dump"], [143, 12, 1, "", "model_dump_json"], [143, 13, 1, "", "model_extra"], [143, 11, 1, "", "model_fields"], [143, 13, 1, "", "model_fields_set"], [143, 12, 1, "", "model_json_schema"], [143, 12, 1, "", "model_parametrized_name"], [143, 12, 1, "", "model_post_init"], [143, 12, 1, "", "model_rebuild"], [143, 12, 1, "", "model_validate"], [143, 12, 1, "", "model_validate_json"], [143, 12, 1, "", "model_validate_strings"], [143, 12, 1, "", "parse_file"], [143, 12, 1, "", "parse_obj"], [143, 12, 1, "", "parse_raw"], [143, 12, 1, "", "schema"], [143, 12, 1, "", "schema_json"], [143, 12, 1, "", "update_forward_refs"], [143, 15, 1, "", "use_low_precision_moe_combine"], [143, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.MoeConfig.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.MpiCommSession": [[143, 12, 1, "", "__init__"], [143, 12, 1, "", "abort"], [143, 12, 1, "", "get_comm"], [143, 12, 1, "", "is_comm_session"], [143, 12, 1, "", "shutdown"], [143, 12, 1, "", "shutdown_abort"], [143, 12, 1, "", "submit"], [143, 12, 1, "", "submit_sync"]], "tensorrt_llm.llmapi.MultimodalEncoder": [[143, 12, 1, "", "__init__"], [143, 12, 1, "", "generate"], [143, 12, 1, "", "generate_async"], [143, 12, 1, "", "get_kv_cache_events"], [143, 12, 1, "", "get_kv_cache_events_async"], [143, 12, 1, "", "get_stats"], [143, 12, 1, "", "get_stats_async"], [143, 13, 1, "", "llm_id"], [143, 12, 1, "", "shutdown"], [143, 13, 1, "", "tokenizer"]], "tensorrt_llm.llmapi.NGramDecodingConfig": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 12, 1, "", "construct"], [143, 12, 1, "", "copy"], [143, 11, 1, "", "decoding_type"], [143, 12, 1, "", "dict"], [143, 12, 1, "", "from_dict"], [143, 12, 1, "", "from_orm"], [143, 15, 1, "", "is_keep_all"], [143, 15, 1, "", "is_public_pool"], [143, 15, 1, "", "is_use_oldest"], [143, 12, 1, "", "json"], [143, 15, 1, "", "load_format"], [143, 15, 1, "", "max_concurrency"], [143, 15, 1, "", "max_draft_len"], [143, 15, 1, "", "max_matching_ngram_size"], [143, 11, 1, "", "model_computed_fields"], [143, 11, 1, "", "model_config"], [143, 12, 1, "", "model_construct"], [143, 12, 1, "", "model_copy"], [143, 12, 1, "", "model_dump"], [143, 12, 1, "", "model_dump_json"], [143, 13, 1, "", "model_extra"], [143, 11, 1, "", "model_fields"], [143, 13, 1, "", "model_fields_set"], [143, 12, 1, "", "model_json_schema"], [143, 12, 1, "", "model_parametrized_name"], [143, 12, 1, "", "model_post_init"], [143, 12, 1, "", "model_rebuild"], [143, 12, 1, "", "model_validate"], [143, 12, 1, "", "model_validate_json"], [143, 12, 1, "", "model_validate_strings"], [143, 12, 1, "", "parse_file"], [143, 12, 1, "", "parse_obj"], [143, 12, 1, "", "parse_raw"], [143, 12, 1, "", "schema"], [143, 12, 1, "", "schema_json"], [143, 13, 1, "", "spec_dec_mode"], [143, 15, 1, "", "speculative_model_dir"], [143, 12, 1, "", "supports_backend"], [143, 12, 1, "", "update_forward_refs"], [143, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.NGramDecodingConfig.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.QuantAlgo": [[143, 11, 1, "", "FP8"], [143, 11, 1, "", "FP8_BLOCK_SCALES"], [143, 11, 1, "", "FP8_PER_CHANNEL_PER_TOKEN"], [143, 11, 1, "", "INT8"], [143, 11, 1, "", "MIXED_PRECISION"], [143, 11, 1, "", "NO_QUANT"], [143, 11, 1, "", "NVFP4"], [143, 11, 1, "", "W4A16"], [143, 11, 1, "", "W4A16_AWQ"], [143, 11, 1, "", "W4A16_GPTQ"], [143, 11, 1, "", "W4A16_MXFP4"], [143, 11, 1, "", "W4A8_AWQ"], [143, 11, 1, "", "W4A8_MXFP4_FP8"], [143, 11, 1, "", "W4A8_MXFP4_MXFP8"], [143, 11, 1, "", "W4A8_NVFP4_FP8"], [143, 11, 1, "", "W4A8_QSERVE_PER_CHANNEL"], [143, 11, 1, "", "W4A8_QSERVE_PER_GROUP"], [143, 11, 1, "", "W8A16"], [143, 11, 1, "", "W8A16_GPTQ"], [143, 11, 1, "", "W8A8_SQ_PER_CHANNEL"], [143, 11, 1, "", "W8A8_SQ_PER_CHANNEL_PER_TENSOR_PLUGIN"], [143, 11, 1, "", "W8A8_SQ_PER_CHANNEL_PER_TOKEN_PLUGIN"], [143, 11, 1, "", "W8A8_SQ_PER_TENSOR_PER_TOKEN_PLUGIN"], [143, 11, 1, "", "W8A8_SQ_PER_TENSOR_PLUGIN"], [143, 12, 1, "", "__init__"], [143, 12, 1, "", "capitalize"], [143, 12, 1, "", "casefold"], [143, 12, 1, "", "center"], [143, 12, 1, "", "count"], [143, 12, 1, "", "encode"], [143, 12, 1, "", "endswith"], [143, 12, 1, "", "expandtabs"], [143, 12, 1, "", "find"], [143, 12, 1, "", "format"], [143, 12, 1, "", "format_map"], [143, 12, 1, "", "index"], [143, 12, 1, "", "isalnum"], [143, 12, 1, "", "isalpha"], [143, 12, 1, "", "isascii"], [143, 12, 1, "", "isdecimal"], [143, 12, 1, "", "isdigit"], [143, 12, 1, "", "isidentifier"], [143, 12, 1, "", "islower"], [143, 12, 1, "", "isnumeric"], [143, 12, 1, "", "isprintable"], [143, 12, 1, "", "isspace"], [143, 12, 1, "", "istitle"], [143, 12, 1, "", "isupper"], [143, 12, 1, "", "join"], [143, 12, 1, "", "ljust"], [143, 12, 1, "", "lower"], [143, 12, 1, "", "lstrip"], [143, 12, 1, "", "maketrans"], [143, 12, 1, "", "partition"], [143, 12, 1, "", "removeprefix"], [143, 12, 1, "", "removesuffix"], [143, 12, 1, "", "replace"], [143, 12, 1, "", "rfind"], [143, 12, 1, "", "rindex"], [143, 12, 1, "", "rjust"], [143, 12, 1, "", "rpartition"], [143, 12, 1, "", "rsplit"], [143, 12, 1, "", "rstrip"], [143, 12, 1, "", "split"], [143, 12, 1, "", "splitlines"], [143, 12, 1, "", "startswith"], [143, 12, 1, "", "strip"], [143, 12, 1, "", "swapcase"], [143, 12, 1, "", "title"], [143, 12, 1, "", "translate"], [143, 12, 1, "", "upper"], [143, 12, 1, "", "zfill"]], "tensorrt_llm.llmapi.QuantConfig": [[143, 12, 1, "", "__init__"], [143, 11, 1, "", "clamp_val"], [143, 11, 1, "", "exclude_modules"], [143, 12, 1, "", "from_dict"], [143, 11, 1, "", "group_size"], [143, 11, 1, "", "has_zero_point"], [143, 12, 1, "", "is_module_excluded_from_quantization"], [143, 11, 1, "", "kv_cache_quant_algo"], [143, 13, 1, "", "layer_quant_mode"], [143, 11, 1, "", "mamba_ssm_cache_dtype"], [143, 11, 1, "", "pre_quant_scale"], [143, 11, 1, "", "quant_algo"], [143, 13, 1, "", "quant_mode"], [143, 11, 1, "", "smoothquant_val"], [143, 12, 1, "", "to_dict"], [143, 11, 1, "", "use_meta_recipe"]], "tensorrt_llm.llmapi.RequestError": [[143, 12, 1, "", "__init__"], [143, 12, 1, "", "add_note"], [143, 11, 1, "", "args"], [143, 12, 1, "", "with_traceback"]], "tensorrt_llm.llmapi.RequestOutput": [[143, 10, 1, "", "PostprocWorker"], [143, 12, 1, "", "__init__"], [143, 12, 1, "", "abort"], [143, 12, 1, "", "aborted"], [143, 12, 1, "", "aresult"], [143, 12, 1, "", "clear_logprob_params"], [143, 13, 1, "id6", "context_logits"], [143, 13, 1, "id7", "finished"], [143, 13, 1, "id8", "mm_embedding_handle"], [143, 13, 1, "id9", "outputs"], [143, 13, 1, "id10", "prompt"], [143, 13, 1, "id11", "prompt_token_ids"], [143, 12, 1, "", "record_stats"], [143, 13, 1, "id12", "request_id"], [143, 12, 1, "", "result"]], "tensorrt_llm.llmapi.RequestOutput.PostprocWorker": [[143, 10, 1, "", "Input"], [143, 10, 1, "", "Output"], [143, 12, 1, "", "__init__"], [143, 12, 1, "", "default_record_creator"], [143, 12, 1, "", "start"]], "tensorrt_llm.llmapi.RequestOutput.PostprocWorker.Input": [[143, 12, 1, "", "__init__"], [143, 11, 1, "", "postproc_params"], [143, 11, 1, "", "rsp"], [143, 11, 1, "", "sampling_params"], [143, 11, 1, "", "streaming"]], "tensorrt_llm.llmapi.RequestOutput.PostprocWorker.Output": [[143, 11, 1, "", "client_id"], [143, 12, 1, "", "count"], [143, 11, 1, "", "error"], [143, 12, 1, "", "index"], [143, 11, 1, "", "is_final"], [143, 11, 1, "", "metrics"], [143, 11, 1, "", "res"]], "tensorrt_llm.llmapi.SamplingParams": [[143, 12, 1, "", "__init__"], [143, 11, 1, "", "add_special_tokens"], [143, 11, 1, "", "additional_model_outputs"], [143, 11, 1, "", "apply_batched_logits_processor"], [143, 11, 1, "", "bad"], [143, 11, 1, "", "bad_token_ids"], [143, 11, 1, "", "beam_search_diversity_rate"], [143, 11, 1, "", "beam_width_array"], [143, 11, 1, "", "best_of"], [143, 11, 1, "", "detokenize"], [143, 11, 1, "", "early_stopping"], [143, 11, 1, "", "embedding_bias"], [143, 11, 1, "", "end_id"], [143, 11, 1, "", "exclude_input_from_output"], [143, 11, 1, "", "frequency_penalty"], [143, 11, 1, "", "guided_decoding"], [143, 11, 1, "", "ignore_eos"], [143, 11, 1, "", "include_stop_str_in_output"], [143, 11, 1, "", "length_penalty"], [143, 11, 1, "", "logits_processor"], [143, 11, 1, "", "logprobs"], [143, 11, 1, "", "lookahead_config"], [143, 11, 1, "", "max_tokens"], [143, 11, 1, "", "min_p"], [143, 11, 1, "", "min_tokens"], [143, 11, 1, "", "n"], [143, 11, 1, "", "no_repeat_ngram_size"], [143, 11, 1, "", "pad_id"], [143, 11, 1, "", "presence_penalty"], [143, 11, 1, "", "prompt_logprobs"], [143, 11, 1, "", "repetition_penalty"], [143, 11, 1, "", "return_context_logits"], [143, 11, 1, "", "return_encoder_output"], [143, 11, 1, "", "return_generation_logits"], [143, 11, 1, "", "return_perf_metrics"], [143, 11, 1, "", "seed"], [143, 11, 1, "", "skip_special_tokens"], [143, 11, 1, "", "spaces_between_special_tokens"], [143, 11, 1, "", "stop"], [143, 11, 1, "", "stop_token_ids"], [143, 11, 1, "", "temperature"], [143, 11, 1, "", "top_k"], [143, 11, 1, "", "top_p"], [143, 11, 1, "", "top_p_decay"], [143, 11, 1, "", "top_p_min"], [143, 11, 1, "", "top_p_reset_ids"], [143, 11, 1, "", "truncate_prompt_tokens"], [143, 11, 1, "", "use_beam_search"]], "tensorrt_llm.llmapi.SchedulerConfig": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 15, 1, "", "capacity_scheduler_policy"], [143, 12, 1, "", "construct"], [143, 15, 1, "", "context_chunking_policy"], [143, 12, 1, "", "copy"], [143, 12, 1, "", "dict"], [143, 15, 1, "", "dynamic_batch_config"], [143, 12, 1, "", "from_orm"], [143, 12, 1, "", "from_pybind"], [143, 12, 1, "", "get_pybind_enum_fields"], [143, 12, 1, "", "get_pybind_variable_fields"], [143, 12, 1, "", "json"], [143, 12, 1, "", "maybe_to_pybind"], [143, 12, 1, "", "mirror_pybind_enum"], [143, 12, 1, "", "mirror_pybind_fields"], [143, 11, 1, "", "model_computed_fields"], [143, 11, 1, "", "model_config"], [143, 12, 1, "", "model_construct"], [143, 12, 1, "", "model_copy"], [143, 12, 1, "", "model_dump"], [143, 12, 1, "", "model_dump_json"], [143, 13, 1, "", "model_extra"], [143, 11, 1, "", "model_fields"], [143, 13, 1, "", "model_fields_set"], [143, 12, 1, "", "model_json_schema"], [143, 12, 1, "", "model_parametrized_name"], [143, 12, 1, "", "model_post_init"], [143, 12, 1, "", "model_rebuild"], [143, 12, 1, "", "model_validate"], [143, 12, 1, "", "model_validate_json"], [143, 12, 1, "", "model_validate_strings"], [143, 12, 1, "", "parse_file"], [143, 12, 1, "", "parse_obj"], [143, 12, 1, "", "parse_raw"], [143, 12, 1, "", "pybind_equals"], [143, 12, 1, "", "schema"], [143, 12, 1, "", "schema_json"], [143, 12, 1, "", "update_forward_refs"], [143, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.SchedulerConfig.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.TorchCompileConfig": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 15, 1, "", "capture_num_tokens"], [143, 12, 1, "", "construct"], [143, 12, 1, "", "copy"], [143, 12, 1, "", "dict"], [143, 15, 1, "", "enable_fullgraph"], [143, 15, 1, "", "enable_inductor"], [143, 15, 1, "", "enable_piecewise_cuda_graph"], [143, 15, 1, "", "enable_userbuffers"], [143, 12, 1, "", "from_orm"], [143, 12, 1, "", "json"], [143, 15, 1, "", "max_num_streams"], [143, 11, 1, "", "model_computed_fields"], [143, 11, 1, "", "model_config"], [143, 12, 1, "", "model_construct"], [143, 12, 1, "", "model_copy"], [143, 12, 1, "", "model_dump"], [143, 12, 1, "", "model_dump_json"], [143, 13, 1, "", "model_extra"], [143, 11, 1, "", "model_fields"], [143, 13, 1, "", "model_fields_set"], [143, 12, 1, "", "model_json_schema"], [143, 12, 1, "", "model_parametrized_name"], [143, 12, 1, "", "model_post_init"], [143, 12, 1, "", "model_rebuild"], [143, 12, 1, "", "model_validate"], [143, 12, 1, "", "model_validate_json"], [143, 12, 1, "", "model_validate_strings"], [143, 12, 1, "", "parse_file"], [143, 12, 1, "", "parse_obj"], [143, 12, 1, "", "parse_raw"], [143, 12, 1, "", "schema"], [143, 12, 1, "", "schema_json"], [143, 12, 1, "", "update_forward_refs"], [143, 12, 1, "", "validate"], [143, 16, 1, "", "validate_capture_num_tokens"], [143, 16, 1, "", "validate_torch_compile_max_num_streams"]], "tensorrt_llm.llmapi.TorchCompileConfig.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.TorchLlmArgs": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 15, 1, "", "allreduce_strategy"], [143, 15, 1, "", "attention_dp_config"], [143, 15, 1, "", "attn_backend"], [143, 15, 1, "", "backend"], [143, 15, 1, "", "batch_wait_max_tokens_ratio"], [143, 15, 1, "", "batch_wait_timeout_iters"], [143, 15, 1, "", "batch_wait_timeout_ms"], [143, 15, 1, "", "batched_logits_processor"], [143, 15, 1, "", "build_config"], [143, 15, 1, "", "cache_transceiver_config"], [143, 15, 1, "", "checkpoint_format"], [143, 15, 1, "", "checkpoint_loader"], [143, 15, 1, "", "context_parallel_size"], [143, 16, 1, "", "convert_load_format"], [143, 15, 1, "", "cp_config"], [143, 15, 1, "", "cuda_graph_config"], [143, 11, 1, "", "decoding_config"], [143, 15, 1, "", "disable_overlap_scheduler"], [143, 15, 1, "", "dtype"], [143, 15, 1, "", "enable_attention_dp"], [143, 15, 1, "", "enable_autotuner"], [143, 15, 1, "", "enable_chunked_prefill"], [143, 15, 1, "", "enable_iter_perf_stats"], [143, 15, 1, "", "enable_iter_req_stats"], [143, 15, 1, "", "enable_layerwise_nvtx_marker"], [143, 15, 1, "", "enable_lm_head_tp_in_adp"], [143, 15, 1, "", "enable_lora"], [143, 15, 1, "", "enable_min_latency"], [143, 13, 1, "", "extra_resource_managers"], [143, 15, 1, "", "fail_fast_on_attention_window_too_large"], [143, 11, 1, "", "field_name"], [143, 15, 1, "", "force_dynamic_quantization"], [143, 12, 1, "", "from_kwargs"], [143, 15, 1, "", "garbage_collection_gen0_threshold"], [143, 15, 1, "", "gather_generation_logits"], [143, 12, 1, "", "get_executor_config"], [143, 12, 1, "", "get_pytorch_backend_config"], [143, 12, 1, "", "get_runtime_sizes"], [143, 15, 1, "", "gpus_per_node"], [143, 15, 1, "", "guided_decoding_backend"], [143, 16, 1, "", "init_backend"], [143, 16, 1, "", "init_build_config"], [143, 15, 1, "", "iter_stats_max_iterations"], [143, 15, 1, "", "kv_cache_config"], [143, 15, 1, "", "kv_connector_config"], [143, 15, 1, "", "load_format"], [143, 15, 1, "", "lora_config"], [143, 15, 1, "", "max_batch_size"], [143, 15, 1, "", "max_beam_width"], [143, 15, 1, "", "max_input_len"], [143, 15, 1, "", "max_num_tokens"], [143, 15, 1, "", "max_seq_len"], [143, 15, 1, "", "mm_encoder_only"], [143, 15, 1, "", "model"], [143, 13, 1, "", "model_format"], [143, 15, 1, "", "moe_cluster_parallel_size"], [143, 15, 1, "", "moe_config"], [143, 15, 1, "", "moe_expert_parallel_size"], [143, 15, 1, "", "moe_tensor_parallel_size"], [143, 15, 1, "", "mpi_session"], [143, 11, 1, "", "msg"], [143, 15, 1, "", "num_postprocess_workers"], [143, 13, 1, "", "parallel_config"], [143, 15, 1, "", "peft_cache_config"], [143, 15, 1, "", "perf_metrics_max_requests"], [143, 15, 1, "", "pipeline_parallel_size"], [143, 15, 1, "", "postprocess_tokenizer_dir"], [143, 15, 1, "", "print_iter_log"], [143, 13, 1, "", "quant_config"], [143, 15, 1, "", "reasoning_parser"], [143, 15, 1, "", "request_stats_max_iterations"], [143, 15, 1, "", "return_perf_metrics"], [143, 15, 1, "", "revision"], [143, 15, 1, "", "sampler_type"], [143, 15, 1, "", "scheduler_config"], [143, 16, 1, "", "set_default_max_input_len"], [143, 16, 1, "", "set_runtime_knobs_from_build_config"], [143, 15, 1, "", "skip_tokenizer_init"], [143, 15, 1, "", "speculative_config"], [143, 13, 1, "", "speculative_model_dir"], [143, 13, 1, "", "speculative_model_format"], [143, 15, 1, "", "stream_interval"], [143, 16, 1, "", "sync_quant_config_with_kv_cache_config_dtype"], [143, 15, 1, "", "tensor_parallel_size"], [143, 12, 1, "", "to_dict"], [143, 15, 1, "", "tokenizer"], [143, 15, 1, "", "tokenizer_mode"], [143, 15, 1, "", "tokenizer_revision"], [143, 15, 1, "", "torch_compile_config"], [143, 15, 1, "", "trust_remote_code"], [143, 16, 1, "", "validate_and_init_tokenizer"], [143, 16, 1, "", "validate_attention_dp_config"], [143, 16, 1, "", "validate_batch_wait_max_tokens_ratio"], [143, 16, 1, "", "validate_batch_wait_timeout_iters"], [143, 16, 1, "", "validate_batch_wait_timeout_ms"], [143, 16, 1, "", "validate_build_config_remaining"], [143, 16, 1, "", "validate_build_config_with_runtime_params"], [143, 16, 1, "", "validate_checkpoint_format"], [143, 16, 1, "", "validate_cuda_graph_config"], [143, 16, 1, "", "validate_dtype"], [143, 16, 1, "", "validate_gpus_per_node"], [143, 16, 1, "", "validate_load_balancer"], [143, 16, 1, "", "validate_lora_config_consistency"], [143, 16, 1, "", "validate_model"], [143, 16, 1, "", "validate_model_format_misc"], [143, 16, 1, "", "validate_parallel_config"], [143, 16, 1, "", "validate_peft_cache_config"], [143, 16, 1, "", "validate_runtime_args"], [143, 16, 1, "", "validate_speculative_config"], [143, 16, 1, "", "validate_stream_interval"], [143, 12, 1, "", "warn_on_unstable_feature_usage"], [143, 11, 1, "", "wrapped_property"]], "tensorrt_llm.llmapi.TorchLlmArgs.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.TrtLlmArgs": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 11, 1, "", "auto_parallel"], [143, 13, 1, "", "auto_parallel_config"], [143, 11, 1, "", "auto_parallel_world_size"], [143, 15, 1, "", "backend"], [143, 15, 1, "", "batched_logits_processor"], [143, 15, 1, "", "batching_type"], [143, 15, 1, "", "build_config"], [143, 15, 1, "", "cache_transceiver_config"], [143, 15, 1, "", "calib_config"], [143, 15, 1, "", "context_parallel_size"], [143, 15, 1, "", "cp_config"], [143, 11, 1, "", "decoding_config"], [143, 15, 1, "", "dtype"], [143, 15, 1, "", "embedding_parallel_mode"], [143, 15, 1, "", "enable_attention_dp"], [143, 15, 1, "", "enable_build_cache"], [143, 15, 1, "", "enable_chunked_prefill"], [143, 15, 1, "", "enable_lm_head_tp_in_adp"], [143, 15, 1, "", "enable_lora"], [143, 15, 1, "", "enable_prompt_adapter"], [143, 15, 1, "", "enable_tqdm"], [143, 15, 1, "", "extended_runtime_perf_knob_config"], [143, 15, 1, "", "fail_fast_on_attention_window_too_large"], [143, 15, 1, "", "fast_build"], [143, 11, 1, "id21", "field_name"], [143, 12, 1, "", "from_kwargs"], [143, 15, 1, "", "gather_generation_logits"], [143, 12, 1, "", "get_runtime_sizes"], [143, 15, 1, "", "gpus_per_node"], [143, 15, 1, "", "guided_decoding_backend"], [143, 16, 1, "", "init_build_config"], [143, 16, 1, "", "init_calib_config"], [143, 15, 1, "", "iter_stats_max_iterations"], [143, 15, 1, "", "kv_cache_config"], [143, 15, 1, "", "load_format"], [143, 15, 1, "", "lora_config"], [143, 15, 1, "", "max_batch_size"], [143, 15, 1, "", "max_beam_width"], [143, 15, 1, "", "max_input_len"], [143, 15, 1, "", "max_num_tokens"], [143, 15, 1, "", "max_prompt_adapter_token"], [143, 15, 1, "", "max_seq_len"], [143, 15, 1, "", "model"], [143, 13, 1, "", "model_format"], [143, 15, 1, "", "moe_cluster_parallel_size"], [143, 15, 1, "", "moe_expert_parallel_size"], [143, 15, 1, "", "moe_tensor_parallel_size"], [143, 15, 1, "", "mpi_session"], [143, 11, 1, "id19", "msg"], [143, 15, 1, "", "normalize_log_probs"], [143, 15, 1, "", "num_postprocess_workers"], [143, 13, 1, "", "parallel_config"], [143, 15, 1, "", "peft_cache_config"], [143, 15, 1, "", "pipeline_parallel_size"], [143, 15, 1, "", "postprocess_tokenizer_dir"], [143, 15, 1, "", "quant_config"], [143, 15, 1, "", "reasoning_parser"], [143, 15, 1, "", "request_stats_max_iterations"], [143, 15, 1, "", "return_perf_metrics"], [143, 15, 1, "", "revision"], [143, 15, 1, "", "scheduler_config"], [143, 16, 1, "", "set_default_max_input_len"], [143, 16, 1, "", "set_runtime_knobs_from_build_config"], [143, 16, 1, "", "setup_embedding_parallel_mode"], [143, 15, 1, "", "skip_tokenizer_init"], [143, 15, 1, "", "speculative_config"], [143, 13, 1, "", "speculative_model_dir"], [143, 13, 1, "", "speculative_model_format"], [143, 15, 1, "", "tensor_parallel_size"], [143, 12, 1, "", "to_dict"], [143, 15, 1, "", "tokenizer"], [143, 15, 1, "", "tokenizer_mode"], [143, 15, 1, "", "tokenizer_revision"], [143, 15, 1, "", "trust_remote_code"], [143, 16, 1, "", "validate_and_init_tokenizer"], [143, 16, 1, "", "validate_auto_parallel"], [143, 16, 1, "", "validate_build_config_remaining"], [143, 16, 1, "", "validate_build_config_with_runtime_params"], [143, 16, 1, "", "validate_dtype"], [143, 16, 1, "", "validate_enable_build_cache"], [143, 16, 1, "", "validate_gpus_per_node"], [143, 16, 1, "", "validate_kv_cache_dtype"], [143, 16, 1, "", "validate_lora_config_consistency"], [143, 16, 1, "", "validate_model"], [143, 16, 1, "", "validate_model_format_misc"], [143, 16, 1, "", "validate_parallel_config"], [143, 16, 1, "", "validate_peft_cache_config"], [143, 16, 1, "", "validate_quant_config"], [143, 16, 1, "", "validate_runtime_args"], [143, 16, 1, "", "validate_speculative_config"], [143, 15, 1, "", "workspace"], [143, 11, 1, "id20", "wrapped_property"]], "tensorrt_llm.llmapi.TrtLlmArgs.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.UserProvidedDecodingConfig": [[143, 10, 1, "", "Config"], [143, 12, 1, "", "__init__"], [143, 12, 1, "", "construct"], [143, 12, 1, "", "copy"], [143, 11, 1, "", "decoding_type"], [143, 12, 1, "", "dict"], [143, 15, 1, "", "drafter"], [143, 12, 1, "", "from_dict"], [143, 12, 1, "", "from_orm"], [143, 12, 1, "", "json"], [143, 15, 1, "", "load_format"], [143, 15, 1, "", "max_concurrency"], [143, 15, 1, "", "max_draft_len"], [143, 11, 1, "", "model_computed_fields"], [143, 11, 1, "", "model_config"], [143, 12, 1, "", "model_construct"], [143, 12, 1, "", "model_copy"], [143, 12, 1, "", "model_dump"], [143, 12, 1, "", "model_dump_json"], [143, 13, 1, "", "model_extra"], [143, 11, 1, "", "model_fields"], [143, 13, 1, "", "model_fields_set"], [143, 12, 1, "", "model_json_schema"], [143, 12, 1, "", "model_parametrized_name"], [143, 12, 1, "", "model_post_init"], [143, 12, 1, "", "model_rebuild"], [143, 12, 1, "", "model_validate"], [143, 12, 1, "", "model_validate_json"], [143, 12, 1, "", "model_validate_strings"], [143, 12, 1, "", "parse_file"], [143, 12, 1, "", "parse_obj"], [143, 12, 1, "", "parse_raw"], [143, 15, 1, "", "resource_manager"], [143, 12, 1, "", "schema"], [143, 12, 1, "", "schema_json"], [143, 13, 1, "", "spec_dec_mode"], [143, 15, 1, "", "speculative_model_dir"], [143, 12, 1, "", "supports_backend"], [143, 12, 1, "", "update_forward_refs"], [143, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.UserProvidedDecodingConfig.Config": [[143, 11, 1, "", "extra"]], "tensorrt_llm.models": [[131, 10, 1, "", "BaichuanForCausalLM"], [131, 10, 1, "", "BertForQuestionAnswering"], [131, 10, 1, "", "BertForSequenceClassification"], [131, 10, 1, "", "BertModel"], [131, 10, 1, "", "BloomForCausalLM"], [131, 10, 1, "", "BloomModel"], [131, 10, 1, "", "CLIPVisionTransformer"], [131, 10, 1, "", "ChatGLMConfig"], [131, 10, 1, "", "ChatGLMForCausalLM"], [131, 10, 1, "", "ChatGLMModel"], [131, 10, 1, "", "CogVLMConfig"], [131, 10, 1, "", "CogVLMForCausalLM"], [131, 10, 1, "", "CohereForCausalLM"], [131, 10, 1, "", "DbrxConfig"], [131, 10, 1, "", "DbrxForCausalLM"], [131, 10, 1, "", "DecoderModel"], [131, 10, 1, "", "DeepseekForCausalLM"], [131, 10, 1, "", "DeepseekV2ForCausalLM"], [131, 10, 1, "", "DiT"], [131, 10, 1, "", "EagleForCausalLM"], [131, 10, 1, "", "EncoderModel"], [131, 10, 1, "", "FalconConfig"], [131, 10, 1, "", "FalconForCausalLM"], [131, 10, 1, "", "FalconModel"], [131, 10, 1, "", "GPTConfig"], [131, 10, 1, "", "GPTForCausalLM"], [131, 10, 1, "", "GPTJConfig"], [131, 10, 1, "", "GPTJForCausalLM"], [131, 10, 1, "", "GPTJModel"], [131, 10, 1, "", "GPTModel"], [131, 10, 1, "", "GPTNeoXForCausalLM"], [131, 10, 1, "", "GPTNeoXModel"], [131, 10, 1, "", "GemmaConfig"], [131, 10, 1, "", "GemmaForCausalLM"], [131, 10, 1, "", "LLaMAConfig"], [131, 10, 1, "", "LLaMAForCausalLM"], [131, 10, 1, "", "LLaMAModel"], [131, 10, 1, "", "LlavaNextVisionConfig"], [131, 10, 1, "", "LlavaNextVisionWrapper"], [131, 10, 1, "", "MLLaMAForCausalLM"], [131, 10, 1, "", "MPTForCausalLM"], [131, 10, 1, "", "MPTModel"], [131, 10, 1, "", "MambaForCausalLM"], [131, 10, 1, "", "MedusaConfig"], [131, 10, 1, "", "MedusaForCausalLm"], [131, 10, 1, "", "OPTForCausalLM"], [131, 10, 1, "", "OPTModel"], [131, 10, 1, "", "Phi3ForCausalLM"], [131, 10, 1, "", "Phi3Model"], [131, 10, 1, "", "PhiForCausalLM"], [131, 10, 1, "", "PhiModel"], [131, 10, 1, "", "PretrainedConfig"], [131, 10, 1, "", "PretrainedModel"], [131, 10, 1, "", "ReDrafterForLLaMALM"], [131, 10, 1, "", "ReDrafterForQWenLM"], [131, 10, 1, "", "RecurrentGemmaForCausalLM"], [131, 11, 1, "", "RobertaForQuestionAnswering"], [131, 11, 1, "", "RobertaForSequenceClassification"], [131, 11, 1, "", "RobertaModel"], [131, 10, 1, "", "SD3Transformer2DModel"], [131, 10, 1, "", "SpeculativeDecodingMode"], [131, 10, 1, "", "WhisperEncoder"]], "tensorrt_llm.models.BaichuanForCausalLM": [[131, 11, 1, "", "config_class"], [131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "quantize"]], "tensorrt_llm.models.BertForQuestionAnswering": [[131, 12, 1, "", "forward"]], "tensorrt_llm.models.BertForSequenceClassification": [[131, 12, 1, "", "forward"]], "tensorrt_llm.models.BertModel": [[131, 12, 1, "", "forward"]], "tensorrt_llm.models.BloomModel": [[131, 12, 1, "", "forward"]], "tensorrt_llm.models.CLIPVisionTransformer": [[131, 12, 1, "", "forward"]], "tensorrt_llm.models.ChatGLMConfig": [[131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "to_dict"]], "tensorrt_llm.models.ChatGLMForCausalLM": [[131, 11, 1, "", "config_class"], [131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "prepare_inputs"], [131, 12, 1, "", "quantize"]], "tensorrt_llm.models.ChatGLMModel": [[131, 12, 1, "", "forward"]], "tensorrt_llm.models.CogVLMConfig": [[131, 12, 1, "", "to_dict"]], "tensorrt_llm.models.CogVLMForCausalLM": [[131, 11, 1, "", "config_class"], [131, 12, 1, "", "default_plugin_config"], [131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "quantize"]], "tensorrt_llm.models.CohereForCausalLM": [[131, 11, 1, "", "config_class"], [131, 12, 1, "", "from_hugging_face"]], "tensorrt_llm.models.DbrxConfig": [[131, 12, 1, "", "to_dict"]], "tensorrt_llm.models.DbrxForCausalLM": [[131, 11, 1, "", "config_class"]], "tensorrt_llm.models.DecoderModel": [[131, 12, 1, "", "check_config"], [131, 12, 1, "", "forward"], [131, 12, 1, "", "precompute_relative_attention_bias"], [131, 12, 1, "", "prepare_inputs"], [131, 12, 1, "", "use_lora"]], "tensorrt_llm.models.DeepseekForCausalLM": [[131, 11, 1, "", "config_class"], [131, 12, 1, "", "from_hugging_face"]], "tensorrt_llm.models.DeepseekV2ForCausalLM": [[131, 11, 1, "", "config_class"], [131, 12, 1, "", "from_hugging_face"]], "tensorrt_llm.models.DiT": [[131, 12, 1, "", "check_config"], [131, 12, 1, "", "forward"], [131, 12, 1, "", "forward_with_cfg"], [131, 12, 1, "", "forward_without_cfg"], [131, 12, 1, "", "prepare_inputs"], [131, 12, 1, "", "unpatchify"]], "tensorrt_llm.models.EagleForCausalLM": [[131, 11, 1, "", "config_class"], [131, 12, 1, "", "forward"], [131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "prepare_inputs"]], "tensorrt_llm.models.EncoderModel": [[131, 12, 1, "", "check_config"], [131, 12, 1, "", "forward"], [131, 12, 1, "", "precompute_relative_attention_bias"], [131, 12, 1, "", "prepare_inputs"], [131, 12, 1, "", "use_lora"], [131, 12, 1, "", "use_prompt_tuning"]], "tensorrt_llm.models.FalconConfig": [[131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "to_dict"]], "tensorrt_llm.models.FalconForCausalLM": [[131, 12, 1, "", "check_config"], [131, 11, 1, "", "config_class"], [131, 12, 1, "", "from_hugging_face"]], "tensorrt_llm.models.FalconModel": [[131, 12, 1, "", "forward"]], "tensorrt_llm.models.GPTConfig": [[131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "from_nemo"], [131, 12, 1, "", "to_dict"]], "tensorrt_llm.models.GPTForCausalLM": [[131, 11, 1, "", "config_class"], [131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "from_nemo"], [131, 12, 1, "", "quantize"], [131, 12, 1, "", "use_lora"]], "tensorrt_llm.models.GPTJConfig": [[131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "to_dict"]], "tensorrt_llm.models.GPTJForCausalLM": [[131, 11, 1, "", "config_class"], [131, 12, 1, "", "from_hugging_face"]], "tensorrt_llm.models.GPTJModel": [[131, 12, 1, "", "forward"]], "tensorrt_llm.models.GPTModel": [[131, 12, 1, "", "forward"]], "tensorrt_llm.models.GPTNeoXModel": [[131, 12, 1, "", "forward"]], "tensorrt_llm.models.GemmaConfig": [[131, 11, 1, "", "GEMMA2_ADDED_FIELDS"], [131, 11, 1, "", "GEMMA3_ADDED_FIELDS"], [131, 11, 1, "", "GEMMA_ADDED_FIELDS"], [131, 11, 1, "", "VERBATIM"], [131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "gemma2_config"], [131, 12, 1, "", "gemma3_config"], [131, 12, 1, "", "get_hf_config"], [131, 13, 1, "", "is_gemma_2"], [131, 13, 1, "", "is_gemma_3"], [131, 12, 1, "", "to_dict"]], "tensorrt_llm.models.GemmaForCausalLM": [[131, 11, 1, "", "NATIVE_QUANT_FLOW"], [131, 12, 1, "", "assert_valid_quant_algo"], [131, 11, 1, "", "config_class"], [131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "quantize"], [131, 12, 1, "", "use_lora"]], "tensorrt_llm.models.LLaMAConfig": [[131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "from_meta_ckpt"], [131, 12, 1, "", "to_dict"]], "tensorrt_llm.models.LLaMAForCausalLM": [[131, 11, 1, "", "config_class"], [131, 12, 1, "", "default_plugin_config"], [131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "from_meta_ckpt"], [131, 12, 1, "", "quantize"], [131, 12, 1, "", "use_lora"]], "tensorrt_llm.models.LLaMAModel": [[131, 12, 1, "", "forward"]], "tensorrt_llm.models.LlavaNextVisionConfig": [[131, 12, 1, "", "from_hugging_face"]], "tensorrt_llm.models.LlavaNextVisionWrapper": [[131, 12, 1, "", "forward"], [131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "prepare_inputs"], [131, 12, 1, "", "save_checkpoint"]], "tensorrt_llm.models.MLLaMAForCausalLM": [[131, 11, 1, "", "config_class"], [131, 12, 1, "", "forward"], [131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "prepare_inputs"], [131, 12, 1, "", "use_lora"]], "tensorrt_llm.models.MPTForCausalLM": [[131, 12, 1, "", "check_config"]], "tensorrt_llm.models.MPTModel": [[131, 12, 1, "", "forward"]], "tensorrt_llm.models.MambaForCausalLM": [[131, 11, 1, "", "config_class"], [131, 12, 1, "", "forward"], [131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "prepare_inputs"]], "tensorrt_llm.models.MedusaConfig": [[131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "to_dict"]], "tensorrt_llm.models.MedusaForCausalLm": [[131, 11, 1, "", "config_class"], [131, 12, 1, "", "from_hugging_face"]], "tensorrt_llm.models.OPTForCausalLM": [[131, 12, 1, "", "check_config"]], "tensorrt_llm.models.OPTModel": [[131, 12, 1, "", "forward"]], "tensorrt_llm.models.Phi3ForCausalLM": [[131, 11, 1, "", "config_class"], [131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "use_lora"]], "tensorrt_llm.models.Phi3Model": [[131, 12, 1, "", "forward"]], "tensorrt_llm.models.PhiForCausalLM": [[131, 12, 1, "", "check_config"], [131, 11, 1, "", "config_class"], [131, 12, 1, "", "from_hugging_face"], [131, 12, 1, "", "use_lora"]], "tensorrt_llm.models.PhiModel": [[131, 12, 1, "", "forward"]], "tensorrt_llm.models.PretrainedConfig": [[131, 12, 1, "", "create_runtime_defaults"], [131, 12, 1, "", "for_each_rank"], [131, 12, 1, "", "from_checkpoint"], [131, 12, 1, "", "from_dict"], [131, 12, 1, "", "from_json_file"], [131, 12, 1, "", "get_config_group"], [131, 12, 1, "", "has_config_group"], [131, 13, 1, "", "kv_dtype"], [131, 13, 1, "", "quant_algo"], [131, 13, 1, "", "quant_mode"], [131, 12, 1, "", "set_if_not_exist"], [131, 12, 1, "", "set_rank"], [131, 12, 1, "", "to_dict"], [131, 12, 1, "", "to_json_file"], [131, 12, 1, "", "to_layer_quant_config"]], "tensorrt_llm.models.PretrainedModel": [[131, 12, 1, "", "check_config"], [131, 12, 1, "", "from_checkpoint"], [131, 12, 1, "", "from_config"], [131, 12, 1, "", "load"], [131, 12, 1, "", "prepare_inputs"], [131, 12, 1, "", "quantize"], [131, 12, 1, "", "release"], [131, 12, 1, "", "save_checkpoint"]], "tensorrt_llm.models.RecurrentGemmaForCausalLM": [[131, 12, 1, "", "forward"], [131, 12, 1, "", "prepare_inputs"], [131, 12, 1, "", "prepare_recurrent_inputs"]], "tensorrt_llm.models.SD3Transformer2DModel": [[131, 13, 1, "", "attn_processors"], [131, 11, 1, "", "config_class"], [131, 12, 1, "", "disable_forward_chunking"], [131, 12, 1, "", "enable_forward_chunking"], [131, 12, 1, "", "forward"], [131, 12, 1, "", "from_pretrained"], [131, 12, 1, "", "fuse_qkv_projections"], [131, 12, 1, "", "load"], [131, 12, 1, "", "prepare_inputs"], [131, 12, 1, "", "set_attn_processor"], [131, 12, 1, "", "unfuse_qkv_projections"]], "tensorrt_llm.models.SpeculativeDecodingMode": [[131, 11, 1, "", "AUTO"], [131, 11, 1, "", "DRAFT_TOKENS_EXTERNAL"], [131, 11, 1, "", "EAGLE"], [131, 11, 1, "", "EXPLICIT_DRAFT_TOKENS"], [131, 11, 1, "", "LOOKAHEAD_DECODING"], [131, 11, 1, "", "MEDUSA"], [131, 11, 1, "", "NGRAM"], [131, 11, 1, "", "NONE"], [131, 11, 1, "", "USER_PROVIDED"], [131, 12, 1, "", "from_arguments"]], "tensorrt_llm.models.WhisperEncoder": [[131, 12, 1, "", "forward"], [131, 12, 1, "", "precompute_relative_attention_bias"], [131, 12, 1, "", "prepare_inputs"]], "tensorrt_llm.plugin": [[132, 10, 1, "", "PluginConfig"]], "tensorrt_llm.plugin.PluginConfig": [[132, 12, 1, "", "to_legacy_setting"]], "tensorrt_llm.quantization": [[133, 10, 1, "", "QuantAlgo"], [133, 10, 1, "", "QuantMode"], [133, 14, 1, "", "quantize_and_export"]], "tensorrt_llm.runtime": [[134, 10, 1, "", "ChatGLMGenerationSession"], [134, 10, 1, "", "EncDecModelRunner"], [134, 10, 1, "", "GenerationSequence"], [134, 10, 1, "", "GenerationSession"], [134, 10, 1, "", "KVCacheManager"], [134, 10, 1, "", "LogitsProcessor"], [134, 10, 1, "", "LogitsProcessorList"], [134, 10, 1, "", "ModelConfig"], [134, 10, 1, "", "ModelRunner"], [134, 10, 1, "", "ModelRunnerCpp"], [134, 10, 1, "", "MultimodalModelRunner"], [134, 10, 1, "", "QWenForCausalLMGenerationSession"], [134, 10, 1, "", "SamplingConfig"], [134, 10, 1, "", "Session"], [134, 10, 1, "", "StoppingCriteria"], [134, 10, 1, "", "StoppingCriteriaList"], [134, 10, 1, "", "TensorInfo"], [134, 14, 1, "", "decode_words_list"]], "tensorrt_llm.runtime.EncDecModelRunner": [[134, 12, 1, "", "encoder_run"], [134, 12, 1, "", "from_engine"], [134, 12, 1, "", "generate"], [134, 12, 1, "", "process_input"]], "tensorrt_llm.runtime.GenerationSequence": [[134, 12, 1, "", "get_batch_idx"], [134, 12, 1, "", "get_seq_idx"]], "tensorrt_llm.runtime.GenerationSession": [[134, 11, 1, "", "batch_size"], [134, 11, 1, "", "buffer_allocated"], [134, 13, 1, "", "context_mem_size"], [134, 13, 1, "", "conv_kernel"], [134, 13, 1, "", "cross_attention"], [134, 11, 1, "", "cuda_graph_mode"], [134, 12, 1, "", "cuda_stream_guard"], [134, 11, 1, "", "debug_mode"], [134, 11, 1, "", "debug_tensors_to_save"], [134, 12, 1, "", "decode"], [134, 12, 1, "", "decode_batch"], [134, 12, 1, "", "decode_regular"], [134, 12, 1, "", "decode_stream"], [134, 11, 1, "", "device"], [134, 13, 1, "", "dtype"], [134, 12, 1, "", "dump_debug_buffers"], [134, 12, 1, "", "early_stop_criteria"], [134, 13, 1, "", "engine_inspector"], [134, 12, 1, "", "filter_medusa_logits"], [134, 12, 1, "", "finalize_decoder"], [134, 12, 1, "", "find_best_medusa_path"], [134, 13, 1, "", "first_layer"], [134, 13, 1, "", "gather_context_logits"], [134, 13, 1, "", "gather_generation_logits"], [134, 13, 1, "", "gemm_allreduce_plugin"], [134, 12, 1, "", "get_next_medusa_tokens"], [134, 12, 1, "", "get_num_heads_kv"], [134, 12, 1, "", "handle_per_step"], [134, 13, 1, "", "has_position_embedding"], [134, 13, 1, "", "has_token_type_embedding"], [134, 13, 1, "", "head_size"], [134, 13, 1, "", "hidden_size"], [134, 13, 1, "", "is_medusa_mode"], [134, 13, 1, "", "is_redrafter_mode"], [134, 13, 1, "", "kv_cache_type"], [134, 13, 1, "", "last_layer"], [134, 12, 1, "", "locate_accepted_draft_tokens"], [134, 11, 1, "", "mapping"], [134, 13, 1, "", "max_draft_tokens"], [134, 13, 1, "", "max_prompt_embedding_table_size"], [134, 12, 1, "", "medusa_decode_and_verify"], [134, 11, 1, "", "medusa_paths"], [134, 11, 1, "", "medusa_position_offsets"], [134, 11, 1, "", "medusa_temperature"], [134, 11, 1, "", "medusa_topks"], [134, 11, 1, "", "medusa_tree_ids"], [134, 12, 1, "", "next_medusa_input_ids"], [134, 11, 1, "", "num_draft_tokens"], [134, 13, 1, "", "num_heads"], [134, 13, 1, "", "num_layers"], [134, 13, 1, "", "num_medusa_heads"], [134, 13, 1, "", "paged_kv_cache"], [134, 13, 1, "", "paged_state"], [134, 12, 1, "", "pp_communicate_final_output_ids"], [134, 12, 1, "", "pp_communicate_new_tokens"], [134, 12, 1, "", "process_logits_including_draft"], [134, 13, 1, "", "profiler"], [134, 13, 1, "", "quant_mode"], [134, 13, 1, "", "remove_input_padding"], [134, 12, 1, "", "reorder_kv_cache_for_beam_search"], [134, 13, 1, "", "rnn_conv_dim_size"], [134, 13, 1, "", "rnn_head_size"], [134, 13, 1, "", "rnn_hidden_size"], [134, 11, 1, "", "runtime"], [134, 12, 1, "", "setup"], [134, 13, 1, "", "state_dtype"], [134, 13, 1, "", "state_size"], [134, 13, 1, "", "tokens_per_block"], [134, 12, 1, "", "update_output_ids_by_offset"], [134, 13, 1, "", "use_gemm_allreduce_plugin"], [134, 13, 1, "", "use_gpt_attention_plugin"], [134, 13, 1, "", "use_kv_cache"], [134, 13, 1, "", "use_lora_plugin"], [134, 13, 1, "", "use_mamba_conv1d_plugin"], [134, 13, 1, "", "vocab_size"]], "tensorrt_llm.runtime.KVCacheManager": [[134, 12, 1, "", "add_sequence"], [134, 12, 1, "", "get_block_offsets"], [134, 12, 1, "", "step"]], "tensorrt_llm.runtime.ModelConfig": [[134, 11, 1, "", "conv_kernel"], [134, 11, 1, "", "cross_attention"], [134, 11, 1, "", "dtype"], [134, 11, 1, "", "gather_context_logits"], [134, 11, 1, "", "gather_generation_logits"], [134, 11, 1, "", "gemm_allreduce_plugin"], [134, 11, 1, "", "gpt_attention_plugin"], [134, 11, 1, "", "gpu_weights_percent"], [134, 11, 1, "", "has_position_embedding"], [134, 11, 1, "", "has_token_type_embedding"], [134, 11, 1, "", "head_size"], [134, 11, 1, "", "hidden_size"], [134, 11, 1, "", "kv_cache_type"], [134, 11, 1, "", "language_adapter_config"], [134, 11, 1, "", "layer_types"], [134, 11, 1, "", "lora_plugin"], [134, 11, 1, "", "lora_target_modules"], [134, 11, 1, "", "mamba_conv1d_plugin"], [134, 11, 1, "", "max_batch_size"], [134, 11, 1, "", "max_beam_width"], [134, 11, 1, "", "max_medusa_tokens"], [134, 11, 1, "", "max_prompt_embedding_table_size"], [134, 11, 1, "", "model_name"], [134, 11, 1, "", "num_heads"], [134, 11, 1, "", "num_kv_heads"], [134, 11, 1, "", "num_kv_heads_per_cross_attn_layer"], [134, 11, 1, "", "num_kv_heads_per_layer"], [134, 11, 1, "", "num_layers"], [134, 11, 1, "", "num_medusa_heads"], [134, 11, 1, "", "paged_state"], [134, 11, 1, "", "quant_mode"], [134, 11, 1, "", "redrafter_draft_len_per_beam"], [134, 11, 1, "", "redrafter_num_beams"], [134, 11, 1, "", "remove_input_padding"], [134, 11, 1, "", "rnn_conv_dim_size"], [134, 11, 1, "", "rnn_head_size"], [134, 11, 1, "", "rnn_hidden_size"], [134, 11, 1, "", "skip_cross_attn_blocks"], [134, 11, 1, "", "skip_cross_kv"], [134, 11, 1, "", "state_dtype"], [134, 11, 1, "", "state_size"], [134, 11, 1, "", "tokens_per_block"], [134, 11, 1, "", "trtllm_modules_to_hf_modules"], [134, 11, 1, "", "vocab_size"]], "tensorrt_llm.runtime.ModelRunner": [[134, 13, 1, "", "dtype"], [134, 12, 1, "", "from_dir"], [134, 12, 1, "", "from_engine"], [134, 13, 1, "", "gather_context_logits"], [134, 13, 1, "", "gather_generation_logits"], [134, 12, 1, "", "generate"], [134, 13, 1, "", "hidden_size"], [134, 13, 1, "", "mapping"], [134, 13, 1, "", "max_prompt_embedding_table_size"], [134, 13, 1, "", "max_sequence_length"], [134, 13, 1, "", "num_heads"], [134, 13, 1, "", "num_layers"], [134, 13, 1, "", "remove_input_padding"], [134, 12, 1, "", "serialize_engine"], [134, 13, 1, "", "use_lora_plugin"], [134, 13, 1, "", "vocab_size"], [134, 13, 1, "", "vocab_size_padded"]], "tensorrt_llm.runtime.ModelRunnerCpp": [[134, 13, 1, "", "dtype"], [134, 12, 1, "", "from_dir"], [134, 13, 1, "", "gather_context_logits"], [134, 13, 1, "", "gather_generation_logits"], [134, 12, 1, "", "generate"], [134, 13, 1, "", "hidden_size"], [134, 13, 1, "", "max_prompt_embedding_table_size"], [134, 13, 1, "", "max_sequence_length"], [134, 13, 1, "", "num_heads"], [134, 13, 1, "", "num_layers"], [134, 13, 1, "", "remove_input_padding"], [134, 13, 1, "", "vocab_size"], [134, 13, 1, "", "vocab_size_padded"]], "tensorrt_llm.runtime.MultimodalModelRunner": [[134, 13, 1, "", "audio_engine_dir"], [134, 13, 1, "", "cpp_e2e"], [134, 13, 1, "", "cpp_llm_only"], [134, 12, 1, "", "generate"], [134, 12, 1, "", "get_audio_features"], [134, 12, 1, "", "get_rope_index"], [134, 12, 1, "", "get_visual_features"], [134, 12, 1, "", "init_audio_encoder"], [134, 12, 1, "", "init_image_encoder"], [134, 12, 1, "", "init_llm"], [134, 12, 1, "", "init_processor"], [134, 12, 1, "", "init_tokenizer"], [134, 13, 1, "", "llm_engine_dir"], [134, 12, 1, "", "load_test_audio"], [134, 12, 1, "", "load_test_data"], [134, 12, 1, "", "prepare_position_ids_for_cogvlm"], [134, 12, 1, "", "preprocess"], [134, 12, 1, "", "ptuning_setup"], [134, 12, 1, "", "ptuning_setup_fuyu"], [134, 12, 1, "", "ptuning_setup_llava_next"], [134, 12, 1, "", "ptuning_setup_phi3"], [134, 12, 1, "", "ptuning_setup_pixtral"], [134, 13, 1, "", "python_e2e"], [134, 12, 1, "", "run"], [134, 12, 1, "", "setup_fake_prompts"], [134, 12, 1, "", "setup_fake_prompts_qwen2vl"], [134, 12, 1, "", "setup_fake_prompts_vila"], [134, 12, 1, "", "setup_inputs"], [134, 12, 1, "", "split_prompt_by_images"], [134, 12, 1, "", "tokenizer_image_token"], [134, 12, 1, "", "video_preprocess"], [134, 13, 1, "", "visual_engine_dir"]], "tensorrt_llm.runtime.QWenForCausalLMGenerationSession": [[134, 12, 1, "", "generate"]], "tensorrt_llm.runtime.SamplingConfig": [[134, 11, 1, "", "bad_words_list"], [134, 11, 1, "", "beam_search_diversity_rate"], [134, 11, 1, "", "early_stopping"], [134, 11, 1, "", "end_id"], [134, 11, 1, "", "frequency_penalty"], [134, 11, 1, "", "length_penalty"], [134, 11, 1, "", "max_attention_window_size"], [134, 11, 1, "", "max_new_tokens"], [134, 11, 1, "", "min_length"], [134, 11, 1, "", "min_p"], [134, 11, 1, "", "no_repeat_ngram_size"], [134, 11, 1, "", "num_beams"], [134, 11, 1, "", "num_return_sequences"], [134, 11, 1, "", "output_cum_log_probs"], [134, 11, 1, "", "output_log_probs"], [134, 11, 1, "", "output_sequence_lengths"], [134, 11, 1, "", "pad_id"], [134, 11, 1, "", "presence_penalty"], [134, 11, 1, "", "random_seed"], [134, 11, 1, "", "repetition_penalty"], [134, 11, 1, "", "return_dict"], [134, 11, 1, "", "sink_token_length"], [134, 11, 1, "", "stop_words_list"], [134, 11, 1, "", "temperature"], [134, 11, 1, "", "top_k"], [134, 11, 1, "", "top_p"], [134, 11, 1, "", "top_p_decay"], [134, 11, 1, "", "top_p_min"], [134, 11, 1, "", "top_p_reset_ids"], [134, 12, 1, "", "update"], [134, 11, 1, "", "use_beam_hyps"]], "tensorrt_llm.runtime.Session": [[134, 13, 1, "", "context"], [134, 13, 1, "", "context_mem_size"], [134, 13, 1, "", "engine"], [134, 12, 1, "", "from_engine"], [134, 12, 1, "", "from_serialized_engine"], [134, 12, 1, "", "infer_shapes"], [134, 12, 1, "", "run"], [134, 13, 1, "", "runtime"], [134, 12, 1, "", "set_shapes"]], "tensorrt_llm.runtime.TensorInfo": [[134, 11, 1, "", "dtype"], [134, 11, 1, "", "name"], [134, 12, 1, "", "numel"], [134, 11, 1, "", "shape"], [134, 12, 1, "", "squeeze"], [134, 12, 1, "", "view"]], "trtllm-bench": [[21, 17, 1, "cmdoption-trtllm-bench-log_level", "--log_level"], [21, 17, 1, "cmdoption-trtllm-bench-m", "--model"], [21, 17, 1, "cmdoption-trtllm-bench-model_path", "--model_path"], [21, 17, 1, "cmdoption-trtllm-bench-w", "--workspace"], [21, 17, 1, "cmdoption-trtllm-bench-m", "-m"], [21, 17, 1, "cmdoption-trtllm-bench-w", "-w"]], "trtllm-bench-build": [[21, 17, 1, "cmdoption-trtllm-bench-build-dataset", "--dataset"], [21, 17, 1, "cmdoption-trtllm-bench-build-max_batch_size", "--max_batch_size"], [21, 17, 1, "cmdoption-trtllm-bench-build-max_num_tokens", "--max_num_tokens"], [21, 17, 1, "cmdoption-trtllm-bench-build-max_seq_len", "--max_seq_len"], [21, 17, 1, "cmdoption-trtllm-bench-build-no_weights_loading", "--no_weights_loading"], [21, 17, 1, "cmdoption-trtllm-bench-build-pp", "--pp_size"], [21, 17, 1, "cmdoption-trtllm-bench-build-q", "--quantization"], [21, 17, 1, "cmdoption-trtllm-bench-build-target_input_len", "--target_input_len"], [21, 17, 1, "cmdoption-trtllm-bench-build-target_output_len", "--target_output_len"], [21, 17, 1, "cmdoption-trtllm-bench-build-tp", "--tp_size"], [21, 17, 1, "cmdoption-trtllm-bench-build-trust_remote_code", "--trust_remote_code"], [21, 17, 1, "cmdoption-trtllm-bench-build-pp", "-pp"], [21, 17, 1, "cmdoption-trtllm-bench-build-q", "-q"], [21, 17, 1, "cmdoption-trtllm-bench-build-tp", "-tp"]], "trtllm-bench-latency": [[21, 17, 1, "cmdoption-trtllm-bench-latency-backend", "--backend"], [21, 17, 1, "cmdoption-trtllm-bench-latency-beam_width", "--beam_width"], [21, 17, 1, "cmdoption-trtllm-bench-latency-concurrency", "--concurrency"], [21, 17, 1, "cmdoption-trtllm-bench-latency-dataset", "--dataset"], [21, 17, 1, "cmdoption-trtllm-bench-latency-engine_dir", "--engine_dir"], [21, 17, 1, "cmdoption-trtllm-bench-latency-ep", "--ep"], [21, 17, 1, "cmdoption-trtllm-bench-latency-extra_llm_api_options", "--extra_llm_api_options"], [21, 17, 1, "cmdoption-trtllm-bench-latency-iteration_log", "--iteration_log"], [21, 17, 1, "cmdoption-trtllm-bench-latency-kv_cache_free_gpu_mem_fraction", "--kv_cache_free_gpu_mem_fraction"], [21, 17, 1, "cmdoption-trtllm-bench-latency-max_input_len", "--max_input_len"], [21, 17, 1, "cmdoption-trtllm-bench-latency-max_seq_len", "--max_seq_len"], [21, 17, 1, "cmdoption-trtllm-bench-latency-medusa_choices", "--medusa_choices"], [21, 17, 1, "cmdoption-trtllm-bench-latency-modality", "--modality"], [21, 17, 1, "cmdoption-trtllm-bench-latency-num_requests", "--num_requests"], [21, 17, 1, "cmdoption-trtllm-bench-latency-pp", "--pp"], [21, 17, 1, "cmdoption-trtllm-bench-latency-report_json", "--report_json"], [21, 17, 1, "cmdoption-trtllm-bench-latency-sampler_options", "--sampler_options"], [21, 17, 1, "cmdoption-trtllm-bench-latency-tp", "--tp"], [21, 17, 1, "cmdoption-trtllm-bench-latency-warmup", "--warmup"]], "trtllm-bench-throughput": [[21, 17, 1, "cmdoption-trtllm-bench-throughput-backend", "--backend"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-beam_width", "--beam_width"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-cluster_size", "--cluster_size"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-concurrency", "--concurrency"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-custom_module_dirs", "--custom_module_dirs"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-data_device", "--data_device"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-dataset", "--dataset"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-enable_chunked_context", "--disable_chunked_context"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-enable_chunked_context", "--enable_chunked_context"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-engine_dir", "--engine_dir"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-eos_id", "--eos_id"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-ep", "--ep"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-extra_llm_api_options", "--extra_llm_api_options"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-image_data_format", "--image_data_format"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-iteration_log", "--iteration_log"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-kv_cache_free_gpu_mem_fraction", "--kv_cache_free_gpu_mem_fraction"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-max_batch_size", "--max_batch_size"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-max_input_len", "--max_input_len"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-max_num_tokens", "--max_num_tokens"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-max_seq_len", "--max_seq_len"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-modality", "--modality"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-no_skip_tokenizer_init", "--no_skip_tokenizer_init"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-num_requests", "--num_requests"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-output_json", "--output_json"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-pp", "--pp"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-report_json", "--report_json"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-request_json", "--request_json"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-sampler_options", "--sampler_options"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-scheduler_policy", "--scheduler_policy"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-streaming", "--streaming"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-target_input_len", "--target_input_len"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-target_output_len", "--target_output_len"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-tp", "--tp"], [21, 17, 1, "cmdoption-trtllm-bench-throughput-warmup", "--warmup"]], "trtllm-eval": [[23, 17, 1, "cmdoption-trtllm-eval-backend", "--backend"], [23, 17, 1, "cmdoption-trtllm-eval-disable_kv_cache_reuse", "--disable_kv_cache_reuse"], [23, 17, 1, "cmdoption-trtllm-eval-ep_size", "--ep_size"], [23, 17, 1, "cmdoption-trtllm-eval-extra_llm_api_options", "--extra_llm_api_options"], [23, 17, 1, "cmdoption-trtllm-eval-gpus_per_node", "--gpus_per_node"], [23, 17, 1, "cmdoption-trtllm-eval-kv_cache_free_gpu_memory_fraction", "--kv_cache_free_gpu_memory_fraction"], [23, 17, 1, "cmdoption-trtllm-eval-log_level", "--log_level"], [23, 17, 1, "cmdoption-trtllm-eval-max_batch_size", "--max_batch_size"], [23, 17, 1, "cmdoption-trtllm-eval-max_beam_width", "--max_beam_width"], [23, 17, 1, "cmdoption-trtllm-eval-max_num_tokens", "--max_num_tokens"], [23, 17, 1, "cmdoption-trtllm-eval-max_seq_len", "--max_seq_len"], [23, 17, 1, "cmdoption-trtllm-eval-model", "--model"], [23, 17, 1, "cmdoption-trtllm-eval-pp_size", "--pp_size"], [23, 17, 1, "cmdoption-trtllm-eval-tokenizer", "--tokenizer"], [23, 17, 1, "cmdoption-trtllm-eval-tp_size", "--tp_size"], [23, 17, 1, "cmdoption-trtllm-eval-trust_remote_code", "--trust_remote_code"]], "trtllm-eval-cnn_dailymail": [[23, 17, 1, "cmdoption-trtllm-eval-cnn_dailymail-apply_chat_template", "--apply_chat_template"], [23, 17, 1, "cmdoption-trtllm-eval-cnn_dailymail-dataset_path", "--dataset_path"], [23, 17, 1, "cmdoption-trtllm-eval-cnn_dailymail-max_input_length", "--max_input_length"], [23, 17, 1, "cmdoption-trtllm-eval-cnn_dailymail-max_output_length", "--max_output_length"], [23, 17, 1, "cmdoption-trtllm-eval-cnn_dailymail-num_samples", "--num_samples"], [23, 17, 1, "cmdoption-trtllm-eval-cnn_dailymail-random_seed", "--random_seed"], [23, 17, 1, "cmdoption-trtllm-eval-cnn_dailymail-rouge_path", "--rouge_path"], [23, 17, 1, "cmdoption-trtllm-eval-cnn_dailymail-system_prompt", "--system_prompt"]], "trtllm-eval-gpqa_diamond": [[23, 17, 1, "cmdoption-trtllm-eval-gpqa_diamond-apply_chat_template", "--apply_chat_template"], [23, 17, 1, "cmdoption-trtllm-eval-gpqa_diamond-dataset_path", "--dataset_path"], [23, 17, 1, "cmdoption-trtllm-eval-gpqa_diamond-max_input_length", "--max_input_length"], [23, 17, 1, "cmdoption-trtllm-eval-gpqa_diamond-max_output_length", "--max_output_length"], [23, 17, 1, "cmdoption-trtllm-eval-gpqa_diamond-num_samples", "--num_samples"], [23, 17, 1, "cmdoption-trtllm-eval-gpqa_diamond-random_seed", "--random_seed"], [23, 17, 1, "cmdoption-trtllm-eval-gpqa_diamond-system_prompt", "--system_prompt"]], "trtllm-eval-gpqa_extended": [[23, 17, 1, "cmdoption-trtllm-eval-gpqa_extended-apply_chat_template", "--apply_chat_template"], [23, 17, 1, "cmdoption-trtllm-eval-gpqa_extended-dataset_path", "--dataset_path"], [23, 17, 1, "cmdoption-trtllm-eval-gpqa_extended-max_input_length", "--max_input_length"], [23, 17, 1, "cmdoption-trtllm-eval-gpqa_extended-max_output_length", "--max_output_length"], [23, 17, 1, "cmdoption-trtllm-eval-gpqa_extended-num_samples", "--num_samples"], [23, 17, 1, "cmdoption-trtllm-eval-gpqa_extended-random_seed", "--random_seed"], [23, 17, 1, "cmdoption-trtllm-eval-gpqa_extended-system_prompt", "--system_prompt"]], "trtllm-eval-gpqa_main": [[23, 17, 1, "cmdoption-trtllm-eval-gpqa_main-apply_chat_template", "--apply_chat_template"], [23, 17, 1, "cmdoption-trtllm-eval-gpqa_main-dataset_path", "--dataset_path"], [23, 17, 1, "cmdoption-trtllm-eval-gpqa_main-max_input_length", "--max_input_length"], [23, 17, 1, "cmdoption-trtllm-eval-gpqa_main-max_output_length", "--max_output_length"], [23, 17, 1, "cmdoption-trtllm-eval-gpqa_main-num_samples", "--num_samples"], [23, 17, 1, "cmdoption-trtllm-eval-gpqa_main-random_seed", "--random_seed"], [23, 17, 1, "cmdoption-trtllm-eval-gpqa_main-system_prompt", "--system_prompt"]], "trtllm-eval-gsm8k": [[23, 17, 1, "cmdoption-trtllm-eval-gsm8k-apply_chat_template", "--apply_chat_template"], [23, 17, 1, "cmdoption-trtllm-eval-gsm8k-dataset_path", "--dataset_path"], [23, 17, 1, "cmdoption-trtllm-eval-gsm8k-fewshot_as_multiturn", "--fewshot_as_multiturn"], [23, 17, 1, "cmdoption-trtllm-eval-gsm8k-max_input_length", "--max_input_length"], [23, 17, 1, "cmdoption-trtllm-eval-gsm8k-max_output_length", "--max_output_length"], [23, 17, 1, "cmdoption-trtllm-eval-gsm8k-num_samples", "--num_samples"], [23, 17, 1, "cmdoption-trtllm-eval-gsm8k-random_seed", "--random_seed"], [23, 17, 1, "cmdoption-trtllm-eval-gsm8k-system_prompt", "--system_prompt"]], "trtllm-eval-json_mode_eval": [[23, 17, 1, "cmdoption-trtllm-eval-json_mode_eval-dataset_path", "--dataset_path"], [23, 17, 1, "cmdoption-trtllm-eval-json_mode_eval-max_input_length", "--max_input_length"], [23, 17, 1, "cmdoption-trtllm-eval-json_mode_eval-max_output_length", "--max_output_length"], [23, 17, 1, "cmdoption-trtllm-eval-json_mode_eval-num_samples", "--num_samples"], [23, 17, 1, "cmdoption-trtllm-eval-json_mode_eval-random_seed", "--random_seed"], [23, 17, 1, "cmdoption-trtllm-eval-json_mode_eval-system_prompt", "--system_prompt"]], "trtllm-eval-mmlu": [[23, 17, 1, "cmdoption-trtllm-eval-mmlu-accuracy_threshold", "--accuracy_threshold"], [23, 17, 1, "cmdoption-trtllm-eval-mmlu-apply_chat_template", "--apply_chat_template"], [23, 17, 1, "cmdoption-trtllm-eval-mmlu-check_accuracy", "--check_accuracy"], [23, 17, 1, "cmdoption-trtllm-eval-mmlu-dataset_path", "--dataset_path"], [23, 17, 1, "cmdoption-trtllm-eval-mmlu-max_input_length", "--max_input_length"], [23, 17, 1, "cmdoption-trtllm-eval-mmlu-max_output_length", "--max_output_length"], [23, 17, 1, "cmdoption-trtllm-eval-mmlu-num_fewshot", "--num_fewshot"], [23, 17, 1, "cmdoption-trtllm-eval-mmlu-num_samples", "--num_samples"], [23, 17, 1, "cmdoption-trtllm-eval-mmlu-random_seed", "--random_seed"], [23, 17, 1, "cmdoption-trtllm-eval-mmlu-system_prompt", "--system_prompt"]], "trtllm-eval-mmmu": [[23, 17, 1, "cmdoption-trtllm-eval-mmmu-dataset_path", "--dataset_path"], [23, 17, 1, "cmdoption-trtllm-eval-mmmu-max_input_length", "--max_input_length"], [23, 17, 1, "cmdoption-trtllm-eval-mmmu-max_output_length", "--max_output_length"], [23, 17, 1, "cmdoption-trtllm-eval-mmmu-num_samples", "--num_samples"], [23, 17, 1, "cmdoption-trtllm-eval-mmmu-random_seed", "--random_seed"], [23, 17, 1, "cmdoption-trtllm-eval-mmmu-system_prompt", "--system_prompt"]], "trtllm-serve-disaggregated": [[26, 17, 1, "cmdoption-trtllm-serve-disaggregated-c", "--config_file"], [26, 17, 1, "cmdoption-trtllm-serve-disaggregated-l", "--log_level"], [26, 17, 1, "cmdoption-trtllm-serve-disaggregated-m", "--metadata_server_config_file"], [26, 17, 1, "cmdoption-trtllm-serve-disaggregated-metrics-log-interval", "--metrics-log-interval"], [26, 17, 1, "cmdoption-trtllm-serve-disaggregated-r", "--request_timeout"], [26, 17, 1, "cmdoption-trtllm-serve-disaggregated-t", "--server_start_timeout"], [26, 17, 1, "cmdoption-trtllm-serve-disaggregated-c", "-c"], [26, 17, 1, "cmdoption-trtllm-serve-disaggregated-l", "-l"], [26, 17, 1, "cmdoption-trtllm-serve-disaggregated-m", "-m"], [26, 17, 1, "cmdoption-trtllm-serve-disaggregated-r", "-r"], [26, 17, 1, "cmdoption-trtllm-serve-disaggregated-t", "-t"]], "trtllm-serve-disaggregated_mpi_worker": [[26, 17, 1, "cmdoption-trtllm-serve-disaggregated_mpi_worker-c", "--config_file"], [26, 17, 1, "cmdoption-trtllm-serve-disaggregated_mpi_worker-log_level", "--log_level"], [26, 17, 1, "cmdoption-trtllm-serve-disaggregated_mpi_worker-c", "-c"]], "trtllm-serve-mm_embedding_serve": [[26, 17, 1, "cmdoption-trtllm-serve-mm_embedding_serve-extra_encoder_options", "--extra_encoder_options"], [26, 17, 1, "cmdoption-trtllm-serve-mm_embedding_serve-gpus_per_node", "--gpus_per_node"], [26, 17, 1, "cmdoption-trtllm-serve-mm_embedding_serve-host", "--host"], [26, 17, 1, "cmdoption-trtllm-serve-mm_embedding_serve-log_level", "--log_level"], [26, 17, 1, "cmdoption-trtllm-serve-mm_embedding_serve-max_batch_size", "--max_batch_size"], [26, 17, 1, "cmdoption-trtllm-serve-mm_embedding_serve-max_num_tokens", "--max_num_tokens"], [26, 17, 1, "cmdoption-trtllm-serve-mm_embedding_serve-metadata_server_config_file", "--metadata_server_config_file"], [26, 17, 1, "cmdoption-trtllm-serve-mm_embedding_serve-port", "--port"], [26, 17, 1, "cmdoption-trtllm-serve-mm_embedding_serve-trust_remote_code", "--trust_remote_code"], [26, 17, 1, "cmdoption-trtllm-serve-mm_embedding_serve-arg-MODEL", "MODEL"]], "trtllm-serve-serve": [[26, 17, 1, "cmdoption-trtllm-serve-serve-backend", "--backend"], [26, 17, 1, "cmdoption-trtllm-serve-serve-cluster_size", "--cluster_size"], [26, 17, 1, "cmdoption-trtllm-serve-serve-ep_size", "--ep_size"], [26, 17, 1, "cmdoption-trtllm-serve-serve-extra_llm_api_options", "--extra_llm_api_options"], [26, 17, 1, "cmdoption-trtllm-serve-serve-fail_fast_on_attention_window_too_large", "--fail_fast_on_attention_window_too_large"], [26, 17, 1, "cmdoption-trtllm-serve-serve-gpus_per_node", "--gpus_per_node"], [26, 17, 1, "cmdoption-trtllm-serve-serve-host", "--host"], [26, 17, 1, "cmdoption-trtllm-serve-serve-kv_cache_free_gpu_memory_fraction", "--kv_cache_free_gpu_memory_fraction"], [26, 17, 1, "cmdoption-trtllm-serve-serve-log_level", "--log_level"], [26, 17, 1, "cmdoption-trtllm-serve-serve-max_batch_size", "--max_batch_size"], [26, 17, 1, "cmdoption-trtllm-serve-serve-max_beam_width", "--max_beam_width"], [26, 17, 1, "cmdoption-trtllm-serve-serve-max_num_tokens", "--max_num_tokens"], [26, 17, 1, "cmdoption-trtllm-serve-serve-max_seq_len", "--max_seq_len"], [26, 17, 1, "cmdoption-trtllm-serve-serve-metadata_server_config_file", "--metadata_server_config_file"], [26, 17, 1, "cmdoption-trtllm-serve-serve-num_postprocess_workers", "--num_postprocess_workers"], [26, 17, 1, "cmdoption-trtllm-serve-serve-port", "--port"], [26, 17, 1, "cmdoption-trtllm-serve-serve-pp_size", "--pp_size"], [26, 17, 1, "cmdoption-trtllm-serve-serve-reasoning_parser", "--reasoning_parser"], [26, 17, 1, "cmdoption-trtllm-serve-serve-server_role", "--server_role"], [26, 17, 1, "cmdoption-trtllm-serve-serve-tokenizer", "--tokenizer"], [26, 17, 1, "cmdoption-trtllm-serve-serve-tp_size", "--tp_size"], [26, 17, 1, "cmdoption-trtllm-serve-serve-trust_remote_code", "--trust_remote_code"], [26, 17, 1, "cmdoption-trtllm-serve-serve-arg-MODEL", "MODEL"]]}, "objnames": {"0": ["c", "macro", "C macro"], "1": ["cpp", "type", "C++ type"], "2": ["cpp", "class", "C++ class"], "3": ["cpp", "function", "C++ function"], "4": ["cpp", "functionParam", "C++ function parameter"], "5": ["cpp", "member", "C++ member"], "6": ["cpp", "enum", "C++ enum"], "7": ["cpp", "enumerator", "C++ enumerator"], "8": ["cpp", "templateParam", "C++ template parameter"], "9": ["py", "module", "Python module"], "10": ["py", "class", "Python class"], "11": ["py", "attribute", "Python attribute"], "12": ["py", "method", "Python method"], "13": ["py", "property", "Python property"], "14": ["py", "function", "Python function"], "15": ["py", "pydantic_field", "Python field"], "16": ["py", "pydantic_validator", "Python validator"], "17": ["std", "cmdoption", "program option"]}, "objtypes": {"0": "c:macro", "1": "cpp:type", "2": "cpp:class", "3": "cpp:function", "4": "cpp:functionParam", "5": "cpp:member", "6": "cpp:enum", "7": "cpp:enumerator", "8": "cpp:templateParam", "9": "py:module", "10": "py:class", "11": "py:attribute", "12": "py:method", "13": "py:property", "14": "py:function", "15": "py:pydantic_field", "16": "py:pydantic_validator", "17": "std:cmdoption"}, "terms": {"": [0, 1, 2, 3, 5, 6, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 25, 26, 28, 29, 30, 31, 32, 33, 35, 37, 47, 51, 52, 55, 57, 58, 62, 73, 75, 77, 78, 80, 81, 84, 86, 87, 88, 89, 90, 92, 94, 97, 98, 99, 100, 102, 103, 104, 109, 111, 112, 113, 114, 115, 116, 120, 121, 123, 124, 125, 126, 127, 129, 130, 131, 134, 135, 137, 139, 140, 142, 143, 144, 146, 147, 148, 149, 150, 151, 152, 154, 156, 157, 158, 160, 166], "0": [0, 1, 2, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 25, 26, 28, 29, 30, 31, 32, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 49, 51, 52, 53, 54, 55, 56, 57, 58, 60, 61, 62, 63, 64, 65, 66, 68, 69, 70, 72, 73, 74, 75, 77, 78, 79, 81, 83, 85, 89, 91, 92, 94, 97, 98, 99, 101, 102, 103, 105, 106, 109, 110, 112, 113, 114, 115, 119, 120, 121, 126, 127, 128, 129, 130, 131, 134, 135, 138, 139, 140, 141, 143, 144, 145, 146, 147, 149, 152, 153, 154, 156, 157, 158, 159, 162, 165, 167], "00": [8, 12, 37, 59, 60, 61, 113, 120, 121, 139], "000": [2, 8, 11, 37, 93, 120, 146], "0000": [37, 120, 121, 143], "0012": 120, "00128": 26, "0017": 38, "003": 38, "0047": 139, "0058": 28, "0060": [28, 29], "0062": 28, "0063": 28, "0068": 30, "00688362121582": 26, "007": 38, "0070": 139, "0071": 139, "0075": 31, "007f": 143, "0080": 31, "0083": 31, "0086": 31, "0087": 30, "0096": 139, "00978": 137, "01": [7, 12, 59, 60, 61, 65, 120, 135, 148], "0101": 30, "0105": 2, "014": 5, "0158": 121, "0162": 123, "0165": 126, "02": 148, "020": 38, "0235": 139, "024": 23, "026": 38, "0260": 139, "027": 38, "0273": 139, "028": 38, "0294": 139, "03": [25, 126, 139, 148], "032": 12, "0339": 38, "035": 38, "03762": 129, "03961": 100, "03x": 13, "04": [87, 97, 141, 148, 163], "0449": 139, "045471": 15, "046": 38, "0461": 2, "0463": 38, "048": 38, "049": 38, "05": [129, 130, 131, 139, 148], "051": 38, "05100": 129, "0523": 139, "0528": [28, 38], "0554": 121, "0560": 139, "0563": 38, "057": 38, "06": [12, 18, 120, 129, 130, 148], "061": 38, "0630": 139, "0669": 2, "0675": 2, "0682": 139, "0689e": 120, "07": [7, 12, 148], "0704": 121, "0713": 139, "0723": 139, "0732": 139, "074": 38, "0772": 2, "0776": 139, "078": 38, "079": 18, "08": [12, 25, 126, 138], "0804": 139, "081947": 15, "082": 38, "0838": [2, 38], "088": 38, "0881": 127, "09": [12, 139], "0903": 139, "0910": 139, "092": 18, "092314": 15, "092623": 15, "093256": 15, "09353": 106, "094": 23, "096": 23, "0964": 38, "09685": 106, "09f": [0, 1], "0cf2f6f154b4a5765d89945b20aa3449b2be7933": 19, "0e": 102, "0f": [0, 102, 143], "0rc0": [25, 93, 95, 147], "0rc1": [20, 37, 120], "0rc6": [28, 29, 30, 31], "0u": 1, "0x": 4, "0x0000000000000000": 148, "1": [0, 1, 3, 4, 5, 6, 7, 10, 11, 12, 13, 14, 16, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 33, 34, 35, 36, 37, 39, 41, 42, 43, 45, 46, 47, 48, 49, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 62, 63, 64, 65, 66, 68, 69, 70, 72, 73, 74, 75, 78, 79, 81, 83, 84, 85, 87, 88, 90, 91, 92, 93, 95, 97, 98, 99, 101, 102, 103, 105, 106, 109, 110, 112, 114, 115, 119, 120, 121, 122, 123, 126, 128, 129, 130, 131, 133, 134, 136, 138, 139, 140, 143, 145, 146, 147, 152, 153, 154, 156, 158, 159, 162, 163, 164, 165, 166], "10": [0, 2, 7, 8, 9, 10, 12, 13, 15, 16, 18, 19, 20, 28, 37, 38, 42, 45, 58, 65, 75, 83, 85, 93, 105, 106, 109, 120, 121, 123, 127, 129, 135, 138, 139, 143, 154, 162], "100": [0, 2, 8, 10, 15, 19, 23, 25, 36, 37, 45, 49, 60, 63, 83, 105, 108, 119, 120, 121], "1000": [0, 17, 36, 37, 38, 119, 120, 121], "10000": [129, 130, 131], "1003": 148, "100gb": 14, "100m": 93, "101": [19, 105], "101029": 15, "101253": 26, "101256": 26, "101978": 38, "102": [4, 19, 105], "1024": [1, 2, 5, 7, 11, 15, 17, 20, 22, 25, 28, 29, 30, 31, 38, 42, 57, 63, 75, 85, 102, 112, 120, 121, 126, 129, 130, 139, 140, 143, 154, 162], "102415": [37, 120], "103": [15, 19, 105], "104": [19, 148], "1041": 20, "10438": 137, "1045": 139, "1047": [37, 120], "105": [19, 38], "1050": 139, "1051": 121, "1059": [37, 120], "106": [19, 38], "106563": 38, "107": [19, 38], "1072": 139, "107501": 38, "10774": 0, "1079": 115, "108": [19, 38], "1082": 139, "10858": 42, "109": [15, 19, 38], "10b": [129, 148], "10m": 4, "11": [0, 2, 5, 7, 8, 15, 16, 18, 19, 38, 88, 106, 109, 120, 123, 129, 139], "110": 19, "11023": [37, 120], "110804": 38, "110b": 148, "111": [4, 12, 19], "111302": 38, "111618": 38, "111668": 38, "1118": 148, "112": [19, 38], "1123": 148, "113": 19, "1134": 135, "113420": 15, "1135": 139, "114": [15, 19, 38], "1141": 139, "114688": 2, "1148": 148, "11489": 2, "11490": 120, "115": [15, 19], "1151": 2, "115378": 15, "115716": 38, "116": [19, 38], "1160": [26, 43], "117": [19, 38], "1178": [37, 120], "118": 19, "1181": 148, "1183": 148, "119": [15, 19, 37, 120], "11943": [37, 120], "11947": 42, "1196": 2, "119648": 15, "11b": [138, 145, 148], "11x": 16, "12": [0, 4, 8, 12, 15, 16, 17, 18, 19, 25, 38, 42, 75, 87, 88, 93, 97, 106, 112, 120, 123, 126, 129, 139, 154, 163], "120": [15, 19], "120b": [29, 93], "121": 19, "1212": 139, "121847": 120, "1219": 2, "122": [19, 120], "1225": 129, "12288": [37, 120], "123": [19, 45, 46], "1234": [131, 143], "1239": 148, "124": 19, "1242": 148, "1245": 25, "1248": 148, "125": [15, 19, 120], "1252": [37, 115, 120], "1256": 148, "1257": 2, "125m": [109, 112], "126": [19, 37, 120], "1267": 148, "127": [19, 129], "1272": 139, "128": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 15, 19, 25, 28, 29, 30, 31, 37, 38, 42, 45, 46, 60, 72, 73, 101, 105, 106, 110, 113, 120, 131, 143, 148, 152], "1284": 148, "1287": 123, "128798": 143, "128799": 143, "128e": [17, 38, 79, 159], "128k": [10, 79, 159], "129": [15, 19, 38], "1290": 139, "1291504": 121, "1293": 115, "12945": 2, "129498": 2, "13": [6, 14, 15, 16, 18, 19, 38, 101, 106, 120, 121, 129, 138, 139], "130": [19, 38], "1300": 51, "131": [19, 38], "131072": [9, 120, 121], "13195": [37, 120], "132": [19, 120], "1323": 148, "1328": 148, "1329": 148, "133": [19, 148], "133120": 29, "13368": 120, "1337": 148, "134": [19, 38], "1341": 2, "1343": 148, "1344": 148, "135": [19, 38], "13525": 120, "13598": [37, 120], "136": [19, 38], "137": [19, 37, 38, 120], "1378": 139, "138": [15, 19], "139": [19, 38], "1392": 148, "13b": [4, 79, 80, 93, 159, 160], "14": [7, 15, 16, 18, 19, 25, 37, 38, 106, 112, 120, 123, 126, 127, 139], "140": [8, 19], "140g": 115, "141": [5, 19], "1418": [37, 120], "141gb": [3, 38], "142": [14, 15, 19, 38], "1424": 148, "143": 19, "1436": [2, 148], "1437": 139, "144": 123, "1446": 148, "1447": 148, "14480": [37, 120], "1449": 148, "145": [126, 127], "1459": 139, "146": [38, 126, 127], "1467": 148, "147": [121, 123, 126, 127], "14702": 10, "148": 38, "1480": 148, "1486": 148, "149": [139, 148], "15": [12, 15, 16, 17, 18, 19, 25, 38, 106, 120, 127, 129, 139], "150": [19, 36, 75, 119, 154], "1500": 38, "15043": 42, "151": 38, "1514": 148, "152": [15, 37, 38, 120], "1529": 148, "153": 25, "1534": 148, "1535": 148, "1536": 2, "1537": 148, "1539": 148, "154": [12, 25, 26], "1552": 148, "1556": 139, "15585": [37, 120], "1562": 148, "1564": [121, 126, 127], "157": 38, "158": 2, "1583": 148, "1584": 2, "1585": 121, "1589": 148, "1590": 148, "1597": 123, "15b": [79, 159], "15u": 14, "16": [0, 2, 4, 7, 8, 11, 12, 15, 16, 19, 25, 26, 28, 29, 30, 31, 37, 38, 39, 41, 57, 59, 60, 61, 72, 73, 75, 81, 85, 94, 101, 106, 107, 109, 113, 120, 121, 122, 129, 130, 131, 135, 137, 139, 152, 154, 156, 162], "160": [15, 148], "1607": [37, 120], "161": [26, 37, 43, 120], "162": 15, "1625": 123, "1626": 148, "163": 3, "163061": 10, "163062": 10, "1637": 148, "16384": [28, 29, 30, 31, 123, 126], "164": [12, 15], "1642": 148, "1643": 25, "165": 38, "1650": 148, "1651165696": 19, "166": 38, "1660": 148, "1664": 29, "1669": 148, "167": [37, 120], "1671517696": 19, "1672": 139, "1674": 148, "1675": 148, "167507": 15, "1676": 148, "168": 12, "16e": [18, 31, 79, 138, 145, 159], "16x": [13, 135], "17": [0, 2, 15, 17, 18, 19, 20, 28, 29, 30, 31, 37, 38, 106, 120, 126, 139, 141], "170": 38, "1706": 129, "171": 15, "1721": 139, "1723": 148, "172321": 2, "17233": 2, "173": [12, 38], "1732": 148, "17323": 137, "1738": 148, "1741966075": 147, "1742": 148, "17453": 22, "17453v3": 1, "1748018634": 17, "175": 38, "1753843963": [30, 31], "1754294810": 28, "1754358426": 20, "1755815898": 29, "175b": 5, "176": 120, "1762": 148, "1774995776": 19, "1776": [79, 159], "1799": 148, "17b": [17, 18, 27, 38, 79, 93, 138, 145, 159], "17x": 19, "18": [14, 15, 18, 19, 38, 106, 120, 139], "180": [11, 12, 135], "180000000": 0, "1806": 2, "180b": [7, 37, 120], "180gb": 38, "181": 38, "1815": 148, "181540": 2, "182": 38, "1822": 42, "1834": 148, "185": [4, 37, 120], "1851": 148, "18527": 42, "18563": [37, 120], "1861": 127, "1866": 127, "187": 15, "188415": 19, "188416": 19, "1885": 121, "1886": 148, "1897": 148, "19": [2, 15, 18, 19, 20, 38, 127, 139], "190": 38, "1909": 148, "192": [3, 15], "1926": 148, "192gb": 38, "1937": 148, "1939": 148, "194": 38, "1944": 126, "195": 38, "1950": 15, "1953": 148, "1959": [37, 120], "1963": 15, "198": [12, 18], "1985": 148, "1987": 148, "199": 38, "1993": [56, 139], "1999": 148, "1_1b": [79, 159], "1_405b": 113, "1_70b": [25, 113], "1_output": 25, "1b": [16, 26, 35, 39, 41, 45, 47, 48, 49, 51, 52, 53, 54, 55, 58, 62, 63, 64, 66, 68, 70, 74, 75, 78, 79, 81, 97, 138, 140, 142, 145, 147, 148, 153, 154, 158, 159], "1d": [72, 101, 129, 134], "1e": [112, 129, 130, 131], "1e20f": 1, "1g": 139, "1gb": [81, 85, 98, 162], "1k": [2, 12, 13, 14, 19, 20, 28], "1k2k": 20, "1m": 127, "1st": [4, 25, 28, 29, 30, 31, 129, 135], "1u": [0, 1], "1x": [12, 20], "1xgpu": 20, "1xh200": 3, "1xtep": 16, "1ytic": 148, "2": [0, 1, 3, 4, 5, 7, 10, 11, 12, 13, 14, 16, 20, 23, 25, 26, 29, 30, 31, 33, 37, 38, 54, 55, 57, 58, 59, 60, 61, 62, 63, 72, 73, 74, 75, 77, 79, 81, 84, 85, 88, 90, 93, 95, 97, 98, 99, 101, 102, 103, 104, 105, 106, 107, 109, 110, 112, 113, 115, 120, 122, 123, 126, 127, 129, 131, 134, 137, 138, 139, 143, 145, 147, 152, 153, 154, 156, 157, 159, 162, 166], "20": [1, 10, 11, 14, 15, 16, 18, 19, 25, 28, 29, 30, 31, 38, 64, 66, 68, 69, 75, 85, 102, 109, 110, 120, 121, 126, 129, 134, 139, 154, 162], "200": [5, 9, 15, 17, 20, 28, 29, 30, 31, 58, 75, 134, 143, 154], "2000": [14, 38], "20000": [32, 38, 143], "200mb": 14, "2017": 126, "2018": 148, "202": 15, "2023": [3, 139], "2024": [12, 87, 163], "2025": [2, 11, 12, 120], "2028": 148, "2033": 127, "2039": 148, "204": [12, 38], "2040": 148, "2042": 2, "2044": [126, 127], "2045": 126, "2048": [2, 3, 5, 6, 14, 21, 22, 25, 28, 29, 30, 31, 37, 38, 75, 81, 112, 120, 121, 123, 124, 125, 126, 127, 131, 134, 139, 143, 148, 154], "205": [15, 38], "2056": 148, "206": 38, "20627": 42, "20685": [37, 120], "2079": 139, "208": 38, "2081": [123, 126, 148], "2087": 148, "2089": 38, "209": [15, 38], "20b": 148, "20k": 20, "21": [2, 7, 12, 15, 18, 19, 38, 109, 126, 139], "2101": 100, "2102": 38, "2106": 106, "2107": 139, "210g": 115, "211": 12, "2113": 148, "212": [15, 38], "2135": 148, "214": 18, "215": 29, "2152": 148, "2158": 38, "2168": 2, "2169": 148, "21747": [37, 120], "2176": 38, "21764": [37, 120], "2182": 148, "2191": 148, "22": [14, 15, 19, 28, 30, 31, 35, 38, 117, 129, 139], "220": 38, "22000": 38, "22056": [37, 120], "221": [38, 120], "2210": 137, "2211": [129, 137], "2219": 148, "222": 38, "22213": [37, 120], "2225": 139, "2232": 148, "224": 130, "2243": 148, "225": [29, 38], "2263": 148, "227": 6, "2288": 148, "2294": 148, "22b": [79, 159], "22x": 13, "23": [8, 15, 18, 19, 37, 38, 120, 139, 148], "2305": 139, "2306": 137, "2309": [1, 22], "232": [6, 15], "234": 15, "2352": 148, "2357": 148, "235b": [79, 159], "236": 12, "2366": 148, "2370": 148, "2373": 148, "2379": 148, "238": 38, "2388": 148, "239": 12, "2397": [37, 120], "24": [0, 15, 19, 37, 38, 97, 120, 139, 141, 148], "2401": 0, "2402": 106, "24189": 38, "2419": 148, "2425": 148, "243": 15, "2439": 148, "244": 38, "245": 12, "2458": 148, "246": 15, "2461": 126, "2466": 126, "2473": 148, "2474": [123, 126], "2484": 148, "2485": 148, "2487": 38, "249": 12, "24b": [138, 148], "24mib": 15, "25": [6, 8, 12, 15, 16, 19, 37, 38, 120, 138, 148], "250": [2, 12, 15], "2500": 38, "2503": 138, "25032": [37, 120], "251": [15, 38], "252u": 14, "253": [12, 38], "253b": [79, 159], "2552": 148, "256": [1, 2, 3, 6, 14, 15, 19, 23, 25, 28, 29, 30, 31, 38, 57, 73, 75, 83, 120, 129, 131, 139, 140, 143, 148, 152, 154], "25603": [37, 120], "257": 38, "2573": 148, "2581": [123, 126], "2590780": 120, "259840": 135, "26": [15, 19, 38, 120, 123, 147], "2602": 42, "2628": [126, 127], "263": [3, 42], "2640": 127, "2649": 139, "266": 38, "2671": 2, "2677": 148, "26778": [37, 120], "2679": 123, "2685": 148, "2691": 148, "27": [19, 38, 148], "270m": [79, 159], "271": 38, "2712": 148, "274": [2, 38, 148], "2742": 121, "275": 148, "2755": 2, "2766684": 2, "278": 42, "2782": 148, "2787": 148, "2796": 148, "27b": [79, 138, 159], "28": [12, 19, 38, 120, 139], "2820": 139, "283": 38, "28390": 120, "287113": 120, "288": [15, 28, 148], "29": [19, 135, 148], "290": 15, "2939": 139, "295": 38, "296": 38, "297": 42, "29892": 42, "299": [12, 37, 120], "29962": 42, "2998": 139, "2b": [37, 79, 115, 120, 159], "2cta": 14, "2d": [109, 129, 130, 137], "2gib": 148, "2k": [2, 12, 13, 14, 20], "2m": 127, "2nd": 129, "2u": 1, "2x": [4, 5, 10, 16], "2xdep": 16, "3": [0, 1, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 18, 20, 23, 25, 26, 27, 28, 31, 34, 37, 47, 52, 54, 57, 62, 64, 65, 72, 73, 75, 79, 81, 82, 84, 88, 90, 91, 93, 97, 99, 101, 103, 105, 106, 114, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 134, 136, 139, 140, 142, 143, 144, 145, 147, 148, 149, 152, 154, 156, 159, 161, 164, 165], "30": [0, 2, 10, 12, 16, 17, 18, 19, 109, 121, 123, 127, 129, 135, 143, 146], "300": [6, 15, 37, 120], "3000": [18, 37, 38, 120], "30000": 38, "30065": [37, 120], "300k": 18, "3019": [37, 120], "3021": 2, "3022": [37, 120], "303": 5, "3031": 126, "304": 42, "3040": [121, 126, 127], "305": 38, "306": 42, "307": [25, 38], "3072": 2, "309": 38, "3095": 148, "30990": [37, 120], "30b": [7, 79, 138, 145, 159], "30x": 7, "31": [8, 19, 38, 107, 121, 126, 127, 148], "3106": 148, "311": 38, "312": 8, "3121": 26, "3126": 26, "3132": [37, 120], "315": 12, "3159": 11, "316": 38, "319": 38, "32": [1, 2, 4, 5, 11, 15, 16, 18, 19, 22, 23, 25, 26, 28, 29, 30, 31, 38, 42, 48, 49, 56, 72, 73, 75, 101, 105, 107, 120, 121, 129, 130, 131, 134, 135, 137, 139, 143, 147, 148, 152, 154, 156], "3200": 28, "3201": 121, "321": [38, 120], "322": 42, "323": 38, "325": 38, "326": 38, "3276": [121, 126, 127], "32768": [29, 129], "329": 38, "3291": 139, "32b": [79, 138, 145, 148, 159], "32k": [10, 148], "32x": 7, "33": [8, 10, 18, 19, 38, 139, 148], "330": 38, "331": 38, "332": 38, "3328": 139, "332826": 2, "3338": 121, "336": 38, "338": [12, 42], "3389": 123, "339": 38, "339447": 15, "339981": 15, "33x": 13, "34": [2, 8, 15, 19, 38], "340": [11, 12, 38], "341": [5, 15], "342": 38, "3442": 139, "3445": 139, "345": 38, "3452": [37, 120, 139], "3476": 2, "348": 38, "348gib": 15, "349": [5, 38], "34b": [79, 148, 159], "35": [0, 8, 19, 38, 49, 83], "350": 29, "3504": 15, "351": 38, "352": 38, "3555": 139, "356": 38, "36": [12, 15, 18, 19, 122, 123], "360": 29, "36384": 2, "3671": [37, 120], "367714": 15, "368": 12, "369": 38, "37": [8, 15, 18, 19, 38, 120], "370318": 19, "3763": 12, "3764": 148, "378": 38, "38": [8, 19, 37, 120], "381": 15, "384": [2, 38], "3863": 38, "387b12598a9e": 120, "3887": 139, "389": 38, "39": [8, 12, 19, 38], "391": 38, "3914": 38, "393": 38, "3936": [37, 120], "3977": 139, "399": 38, "3_1": [79, 138, 145, 159], "3_3": [79, 138, 145, 159], "3_output": 30, "3b": [40, 46, 67, 79, 138, 159], "3d": [72, 101, 129, 134], "3rd": 129, "3u": 1, "3x": [7, 12, 14], "4": [0, 1, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 23, 25, 26, 29, 30, 31, 37, 42, 49, 57, 59, 60, 61, 63, 73, 75, 79, 81, 85, 87, 88, 89, 90, 91, 92, 93, 103, 105, 106, 107, 109, 113, 115, 120, 121, 123, 124, 125, 126, 127, 128, 129, 131, 134, 135, 136, 137, 138, 139, 143, 145, 146, 148, 152, 154, 156, 159, 162, 163], "40": [18, 19, 29, 38, 64, 102, 123, 129, 146, 148], "400": [14, 15], "4000": [14, 18], "401": 38, "403": 148, "405b": [37, 79, 120, 122, 148, 159], "4060": 135, "4066": 42, "407": 38, "408348": 15, "4089": 127, "4096": [3, 11, 14, 25, 28, 29, 37, 38, 42, 83, 120, 123, 129, 130, 134], "40b": 7, "40gb": 22, "40x": 7, "41": 19, "410": 38, "4101": 148, "41020": 120, "411": [37, 120], "4117e": [37, 120], "4133": 127, "4135": 148, "41375": 120, "414": 2, "4141": 148, "41607": 120, "4168": 2, "4192": 139, "42": [15, 19, 29, 38, 120], "420": [20, 21], "421": 38, "422": 38, "4224": 38, "4227": 148, "4248": 123, "4265": 120, "427": [37, 120], "4280": 12, "43": [18, 19, 38, 135, 147], "431": 38, "43146": 2, "434": 38, "435": 38, "437": 38, "4384": 15, "44": [10, 15, 18, 19, 25, 38, 135], "4408": 42, "442": 38, "4439": [37, 120], "4456": 38, "447": 38, "449": 148, "4493": [126, 127], "4495": 15, "4497": 38, "44x": 7, "45": [15, 19, 105, 138, 148], "450": 38, "45000000000": 105, "450m": [79, 159], "453": 38, "4532": 148, "4548": 2, "456": 38, "4566": 38, "457": 38, "458676": 15, "459": 38, "46": [7, 19], "4600": 14, "461014": 15, "463": 38, "464": [15, 38], "465004": 19, "4653": 42, "4656": 38, "4667": 38, "4678": 148, "47": [7, 15, 19, 123], "4701": [37, 120], "472": 42, "476": 38, "4767": 148, "478": 148, "4798": 148, "47x": 7, "48": [15, 19, 123, 135, 148], "4809": 148, "480gb": [15, 19], "481": 4, "482": 148, "4853": 148, "489795": 19, "489935": 19, "49": [15, 19, 123], "491": [15, 38], "49152": 2, "494": 38, "495": 38, "496": [38, 107], "4963": [37, 120], "4963654": 33, "498043": 15, "499": [8, 38], "4993": 148, "49b": [79, 138, 145, 159], "4b": [79, 148, 159], "4bit": 3, "4gb": 14, "4gpu": 20, "4k": [19, 79, 159], "4u": 1, "4x": [3, 4, 5, 16, 20, 93], "4xgpu": 20, "5": [0, 1, 3, 4, 5, 7, 8, 10, 11, 12, 13, 14, 15, 16, 18, 19, 25, 28, 29, 30, 31, 37, 38, 40, 46, 51, 63, 64, 67, 79, 81, 88, 90, 91, 105, 106, 109, 110, 112, 120, 126, 129, 131, 134, 136, 138, 139, 143, 145, 148, 159, 165], "50": [0, 7, 8, 10, 11, 15, 16, 19, 29, 37, 51, 63, 64, 75, 85, 120, 143, 148, 154, 162], "500": [12, 14, 29, 38], "5000": 38, "500000": 131, "5007": 42, "500m": 7, "50272": 112, "5029": 148, "505": 38, "5064": 38, "5073": 139, "50m": 15, "51": 19, "511": 38, "512": [1, 2, 5, 6, 19, 21, 23, 25, 28, 29, 38, 63, 106, 110, 120, 123, 126, 131, 143], "5120": 2, "512mb": [81, 98], "513": 38, "5141": 19, "518": 42, "51b": [79, 138, 145, 148, 159], "51x": 7, "52": [15, 19, 38], "5213": 19, "5215": 19, "5224": 19, "52269": 38, "524288": 19, "526": 148, "52667": 38, "5284": 19, "529514": 15, "5299": 123, "53": [15, 19, 120, 126, 127, 148], "530": 38, "5305": 123, "535": 17, "5371": 148, "5373": 148, "537602": 15, "538": 38, "5393": 2, "54": [7, 8, 19], "540": [37, 38, 120], "5417": 148, "5436": 148, "5443839": 2, "54576": 2, "5496": 123, "5497": 38, "55": [7, 10, 18, 19, 120], "5500": 38, "551": 38, "5510": [37, 120], "5514": [37, 120, 148], "5519": 148, "552": [8, 17], "5520": 148, "5530": 38, "5531": 148, "5534": 148, "554": 38, "5558": 148, "556": 38, "5563": 148, "5564": 148, "5568": 148, "5570": 19, "558": 38, "56": [7, 15, 19, 25], "560": 3, "562": [106, 110], "5636": 148, "564": 15, "5642": 148, "564272": 15, "566": 38, "5669": 148, "568": 120, "5698": 148, "57": [15, 19, 38, 120], "570": 15, "572": 38, "5739": 2, "5742": [123, 126], "575": [28, 29, 30, 31], "5761": 148, "5772": 148, "5779": 148, "5782": 148, "579": 38, "58": [12, 15, 19, 126], "5800": 148, "5801": 148, "5809": 148, "581": 38, "5815": 148, "5816": 148, "5821": 38, "5823": 148, "5825": 148, "5830": 139, "5835": 148, "5855": 148, "5874": 139, "5877": 123, "5879": 139, "58x": 12, "59": [10, 15, 18, 19, 25, 120], "590": 42, "5900": 148, "5902": 148, "5904": 148, "591": 38, "5918": 139, "5921": 148, "5925": 148, "5931": 148, "594": 38, "5941": 148, "5947": 148, "5949": 148, "5954": 148, "5957": 139, "5963": 19, "5975": 148, "5976": 123, "5980": 123, "599": 38, "5992": 148, "5b": [79, 148, 159], "5k": 20, "5m": 20, "5th": [14, 129], "5u": 1, "5vl": [25, 148], "5x": [4, 7, 12, 16], "6": [0, 1, 5, 7, 8, 10, 11, 12, 13, 14, 15, 16, 18, 19, 25, 28, 30, 31, 38, 81, 102, 105, 106, 109, 129, 134, 138, 139, 143, 148], "60": [0, 8, 10, 15, 18, 19, 143], "600": 116, "6000": [10, 37, 38, 120, 146, 148], "6014": 148, "602": 38, "603": 38, "6049": 123, "605": 38, "6059": [37, 120], "6064": 139, "6065": 148, "6075": 148, "608": 38, "6080": 148, "61": [19, 25, 83], "610": 38, "6100": 2, "6103": 148, "612328": 15, "613": 38, "6136": 148, "6140": 148, "615": 38, "6157": 139, "616": 38, "617": 38, "61954812": 34, "62": [12, 18, 19, 38, 126], "622": 25, "623": [25, 38], "623219": 15, "6255": 139, "626": 42, "6299": 139, "63": [18, 19, 38, 94, 120, 126, 131, 135], "630": 38, "6300": 10, "63266": 121, "633": 29, "63307": 121, "63308": 121, "63331": 121, "63374": 121, "634": 38, "6344": 148, "63456": 121, "6345624": 121, "636": 38, "6372": 123, "6376": 2, "639": 148, "64": [0, 1, 2, 4, 5, 11, 15, 19, 22, 25, 28, 29, 30, 31, 37, 38, 40, 46, 62, 67, 72, 73, 77, 84, 85, 101, 102, 112, 120, 126, 129, 130, 131, 135, 148, 152, 156, 157, 162], "640": [3, 20], "640gb": 14, "6429": 148, "643": 38, "6437": 148, "644": 38, "6447": 148, "6452": 127, "646": 38, "6475": 126, "649": 148, "6499": 10, "64x": 12, "65": [10, 19], "650": 38, "65024": 139, "651": 38, "65100": 2, "651199": 15, "6523": 127, "653": 8, "654": 5, "6550": 123, "65536": 19, "6554": 126, "658": 38, "6591": [37, 120], "66": [12, 18, 19, 28, 29, 30, 31, 38], "660": 38, "662": 38, "6628": [126, 127], "664": 8, "665": 38, "666": 38, "6678": 135, "6684": 127, "6695": 135, "67": [7, 12, 15, 19], "6704": 10, "671": 2, "67108864": [9, 17, 20, 25, 94, 147], "671b": 13, "672": 38, "673": 148, "675": 120, "6753e": [37, 120], "676": 38, "6769": 126, "677": 38, "6774": 10, "678": 38, "679": [4, 38], "68": [7, 12, 19, 38, 127], "6825": [37, 120], "684": 12, "6852": [123, 126], "6862": 120, "6890": 139, "69": [7, 12, 15, 19, 127, 147], "6925": [37, 120], "6938": 42, "6948": 10, "695": [38, 148], "697": [14, 38], "6975": 123, "6976": [121, 126, 127], "699": 38, "6a": 3, "6b": [4, 37, 79, 120, 129, 148, 159], "6x": [5, 16, 93], "7": [0, 1, 3, 4, 7, 8, 10, 12, 13, 14, 15, 16, 18, 19, 20, 26, 28, 29, 31, 37, 38, 48, 64, 75, 81, 92, 94, 97, 105, 106, 120, 121, 129, 134, 139, 154], "70": [0, 7, 8, 15, 18, 19, 127, 135], "700": 116, "7000": [37, 120], "701": 148, "703": 10, "7031": 123, "705": [14, 29, 148], "7063": [37, 120], "707": 38, "7072": 38, "708": 18, "709": 120, "7090": 139, "70b": [5, 7, 10, 23, 25, 27, 37, 72, 79, 92, 93, 115, 121, 123, 124, 125, 126, 127, 128, 138, 145, 148, 159], "70g": 115, "71": [12, 19, 120], "712": 38, "7128": 29, "7134": 139, "7136": 121, "714": 38, "7144": 139, "715": 38, "7168": [2, 12, 14], "717498": 15, "7187": 38, "7188": 2, "72": [10, 15, 19, 29, 122, 148], "720": 29, "7206": 2, "722": 38, "724": 38, "727": 38, "728516": 15, "72b": [138, 145, 148], "73": [12, 18, 19], "734": 38, "738": 38, "7382": 38, "739": [38, 148], "73x": 16, "74": [12, 19, 29], "741": [38, 148], "744": 38, "7456": 2, "74561": 2, "7480": 121, "7481": 10, "749": 38, "74x": 11, "75": [7, 15, 17, 19, 29, 120, 148], "750": 5, "7502": 121, "7520": 2, "755": [38, 116], "7584": 2, "75903": 38, "76": 19, "7607": 126, "762": 38, "7621": 38, "7638": [121, 126, 127], "7657": 2, "766": 38, "76665782272": 26, "767": 38, "768": [23, 112, 130], "77": [15, 18, 19, 25], "770": 38, "7743": 121, "7770": 121, "78": [12, 19, 123], "780": [37, 120], "781": 38, "782": 38, "783": 38, "7842": 123, "785": 38, "78509": 38, "787": 38, "7871": 10, "7876": 126, "789": 38, "79": [19, 38, 120, 135], "790": 10, "7900": 139, "791": 15, "792": 15, "7933": 126, "794": 148, "7949": 139, "795": 38, "797": 38, "7977": 123, "798": 38, "799": 38, "7a": 3, "7b": [7, 11, 25, 26, 37, 38, 69, 79, 86, 100, 106, 109, 110, 120, 138, 145, 148, 159], "7b_model": 25, "7k": 11, "7x": [4, 12, 16], "8": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 25, 26, 28, 29, 30, 31, 38, 42, 43, 47, 52, 53, 54, 55, 58, 59, 60, 61, 63, 64, 65, 72, 73, 75, 77, 85, 89, 91, 97, 100, 101, 105, 106, 107, 112, 113, 115, 120, 121, 122, 123, 128, 129, 130, 131, 135, 137, 139, 140, 142, 143, 147, 152, 154, 156, 157, 162, 165], "80": [0, 5, 12, 14, 19, 94, 102, 148], "800": [3, 29, 148], "8000": [9, 16, 17, 20, 25, 26, 28, 29, 30, 31, 39, 40, 41, 43, 45, 46, 66, 67, 68, 69, 70, 81, 85, 147, 156, 162], "8001": [16, 81], "8002": [16, 81, 120], "8003": [16, 81], "8004": 16, "8005": 38, "803": [3, 8], "804": 38, "8048": 120, "80833": 25, "80gb": [4, 7, 22, 38, 121, 123, 124, 125], "81": [12, 15, 19, 123], "812": 38, "813": 38, "8140": 2, "8149": 139, "817": 38, "8179": 139, "819": 5, "8192": [11, 22, 25, 28, 38, 88, 120, 121, 126, 129, 130, 139, 143, 148], "82": [12, 19, 123], "820": 120, "821": 38, "8212": [0, 1], "8218": 139, "8225": 123, "825": 148, "8259": [37, 120], "828": 38, "829": 38, "83": [19, 25], "830": 11, "8307": 127, "831": 38, "833": 38, "835": 38, "8351": [37, 120], "838": 38, "8393": 30, "84": [8, 12, 19, 25], "844": 38, "8441": [37, 120], "849": 38, "85": [2, 7, 19, 25, 120, 148], "852": 38, "854": 38, "856": 38, "86": [10, 19, 94], "863": [37, 120], "8672": 139, "87": [7, 8, 10, 15, 19, 29], "870": 38, "871": 38, "872": 38, "876": 38, "8779": 139, "878": 18, "88": [19, 25, 123, 127], "8804": 121, "880676": 15, "881": 38, "88226": [37, 120], "8828": 139, "8841": 123, "8870": 30, "89": [7, 10, 12, 19, 94, 138], "8908": 31, "893": 38, "8932": [37, 120], "895": 38, "8958": 127, "896": 2, "898": 38, "8984": 31, "899": 38, "8a": 6, "8b": [10, 23, 37, 47, 52, 57, 65, 73, 75, 79, 86, 90, 91, 92, 97, 120, 138, 140, 142, 145, 147, 152, 154, 156, 159, 164, 165], "8bit": 4, "8k": [19, 28, 148], "8tb": 5, "8x": [14, 16, 17, 20], "8x22b": [79, 159], "8x7b": [37, 100, 120, 138, 145, 148], "8xb200": [12, 17], "8xgpu": [14, 20], "8xh100": 6, "8xh200": 3, "9": [0, 1, 4, 11, 12, 13, 15, 16, 18, 19, 20, 25, 29, 30, 31, 34, 38, 48, 64, 73, 83, 106, 109, 115, 123, 129, 139, 152, 156], "90": [0, 2, 15, 18, 19, 29, 37, 38, 48, 64, 83, 94, 108, 120, 121, 123, 128, 135, 143], "9007": 2, "902": 38, "9028": 139, "907": 4, "9075": 31, "908": 38, "9087": 127, "909": 38, "91": 19, "910": 38, "9101": 38, "9115": 127, "912": 8, "912656": 2, "916": 38, "9184": 123, "9189": 31, "919": 38, "9197": 2, "92": [12, 19, 25], "9203": 123, "9214": 38, "9216": 28, "922": 38, "9223372036854775807": 143, "924": [23, 38, 112], "925": 18, "9263": 2, "9274": [10, 121], "93": [2, 15, 19], "931": 38, "932": 38, "933": 38, "9348": 30, "935": 148, "9353e": 121, "9356": 30, "937": 38, "9379": 2, "939": 38, "94": 19, "94022": 38, "941": [3, 6, 38], "943": 38, "944": 38, "9447": 28, "946": 3, "9462": 28, "948": [15, 38], "949": 38, "9494": 126, "95": [11, 19, 26, 38, 43, 47, 52, 53, 54, 55, 58, 63, 64, 97, 121, 128, 140, 142, 147], "9500": 28, "9521": 139, "953": 38, "9537": 123, "9538": 28, "954": [14, 38], "955200": 15, "957": 38, "96": [3, 12, 14, 18, 19, 123, 148], "960": 3, "9606": 14, "960gb": 15, "961": 38, "9613": 14, "9623": 126, "9629": 14, "9639": 38, "965": 38, "96583": 38, "967": 148, "9692": 139, "97": [14, 19, 120, 123], "972": 38, "976442": 15, "977": 38, "978": 38, "98": 19, "981": 38, "983": 148, "9845": 25, "987": 148, "9898": 2, "99": [12, 15, 18, 19, 105, 116], "992": 148, "9928": 127, "993": 38, "9938": 2, "994": 38, "9980": 25, "9982": [126, 127], "9b": [79, 159], "9f": 0, "9x": [5, 6], "A": [0, 1, 2, 7, 8, 10, 12, 15, 17, 18, 25, 28, 29, 30, 31, 33, 36, 37, 38, 57, 58, 59, 60, 61, 62, 72, 79, 81, 83, 88, 89, 92, 93, 98, 99, 101, 102, 104, 106, 109, 112, 113, 115, 119, 120, 129, 134, 136, 138, 140, 142, 143, 144, 145, 148, 149, 151, 159], "AND": 129, "And": [11, 13, 14, 15, 92, 109, 115, 129, 130, 135], "As": [10, 11, 13, 15, 16, 18, 19, 20, 28, 42, 73, 81, 84, 94, 100, 101, 103, 106, 109, 113, 123, 127, 128, 129, 135, 137, 139, 148, 151, 152, 166], "At": [10, 14, 19, 20, 23, 62, 87, 111, 123, 130, 135, 163], "Being": 33, "But": [11, 15, 19, 72, 101, 104], "By": [0, 1, 8, 10, 11, 12, 14, 15, 16, 19, 23, 29, 34, 35, 42, 72, 80, 81, 94, 98, 102, 108, 109, 120, 123, 127, 129, 139, 143, 151, 160], "For": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 25, 26, 28, 29, 30, 31, 32, 33, 35, 37, 38, 42, 47, 48, 56, 59, 60, 61, 71, 72, 73, 74, 75, 77, 80, 81, 83, 84, 85, 86, 88, 89, 92, 93, 94, 96, 98, 99, 101, 102, 103, 104, 105, 106, 108, 109, 112, 113, 114, 115, 120, 121, 122, 123, 126, 127, 128, 129, 134, 135, 139, 140, 142, 143, 144, 147, 148, 149, 150, 151, 152, 153, 154, 156, 157, 160, 162, 166, 167], "If": [0, 1, 2, 7, 9, 10, 11, 13, 15, 17, 18, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 34, 35, 37, 56, 58, 72, 80, 81, 83, 84, 88, 90, 91, 92, 94, 95, 97, 98, 99, 100, 101, 102, 103, 105, 106, 107, 108, 109, 112, 113, 114, 115, 116, 117, 120, 121, 122, 123, 126, 127, 128, 129, 131, 134, 135, 138, 139, 142, 143, 144, 147, 148, 149, 151, 160, 166, 167], "In": [0, 1, 2, 4, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 23, 34, 37, 38, 42, 62, 79, 80, 81, 84, 86, 92, 93, 94, 95, 103, 104, 107, 109, 113, 114, 115, 117, 120, 121, 122, 123, 126, 127, 129, 135, 137, 138, 139, 143, 146, 147, 148, 151, 159, 160, 166, 167], "It": [0, 1, 2, 3, 6, 7, 9, 10, 11, 12, 13, 14, 15, 18, 19, 20, 21, 22, 23, 28, 29, 30, 31, 34, 37, 38, 42, 56, 58, 62, 64, 72, 80, 81, 83, 84, 88, 89, 92, 93, 94, 99, 101, 102, 103, 106, 109, 111, 113, 114, 120, 123, 124, 125, 126, 127, 128, 129, 137, 139, 142, 143, 144, 146, 147, 149, 150, 151, 160, 167], "Its": [11, 57, 72, 88, 101, 129, 146, 151], "NOT": [56, 129], "Near": 8, "No": [0, 15, 26, 32, 37, 62, 81, 82, 98, 105, 120, 121, 136, 145, 161], "Not": [1, 2, 7, 20, 83], "ON": [120, 123, 126, 127], "OR": 129, "Of": [12, 28, 148], "On": [15, 18, 19, 29, 36, 72, 94, 97, 101, 105, 119, 122, 127, 129, 148], "One": [10, 11, 15, 19, 82, 83, 112, 113, 126, 129, 139, 145, 150, 161], "Or": [94, 129, 134, 164], "TO": 8, "That": [11, 33, 38, 72, 88, 99, 101, 102, 105, 113, 123, 129, 143], "The": [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 12, 13, 14, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 42, 47, 48, 49, 51, 52, 53, 54, 55, 57, 58, 59, 60, 61, 62, 63, 73, 74, 75, 76, 77, 78, 79, 80, 81, 85, 86, 87, 89, 90, 91, 92, 93, 94, 96, 97, 98, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 111, 112, 113, 114, 115, 119, 120, 121, 122, 123, 126, 127, 128, 129, 130, 131, 132, 134, 135, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 157, 158, 159, 160, 162, 163, 164, 165, 166, 167], "Their": [14, 17], "Then": [11, 13, 15, 19, 25, 26, 37, 75, 84, 106, 115, 116, 120, 121, 129, 142, 144, 149, 154, 166], "There": [5, 10, 12, 13, 14, 15, 16, 18, 28, 29, 42, 72, 80, 81, 88, 92, 94, 96, 97, 98, 101, 102, 103, 104, 105, 106, 112, 115, 129, 132, 135, 137, 139, 142, 148, 150, 151, 160, 166, 167], "These": [3, 5, 6, 8, 10, 11, 12, 14, 15, 16, 19, 28, 29, 30, 31, 32, 34, 37, 42, 81, 90, 92, 109, 115, 120, 121, 122, 130, 132, 148], "To": [2, 5, 8, 10, 11, 12, 13, 15, 16, 17, 19, 20, 23, 26, 28, 29, 30, 31, 33, 35, 36, 37, 38, 72, 74, 78, 80, 81, 83, 84, 87, 88, 89, 91, 92, 94, 99, 101, 105, 106, 108, 109, 110, 113, 114, 115, 119, 120, 123, 126, 127, 128, 129, 135, 137, 141, 142, 143, 144, 146, 147, 148, 149, 151, 153, 158, 160, 163, 166, 167], "WITH": 64, "Will": [0, 1], "With": [10, 11, 15, 16, 18, 20, 29, 42, 57, 72, 81, 84, 88, 93, 98, 101, 102, 109, 113, 116, 120, 143], "_": [0, 10, 11, 32, 35, 87, 88, 99, 114, 132], "__all__": [144, 149], "__call__": [58, 91], "__dict__": 143, "__file__": 56, "__getitem__": 143, "__init__": [11, 32, 56, 58, 72, 80, 103, 111, 113, 114, 120, 139, 143, 144, 148, 149, 151, 160, 167], "__main__": [47, 48, 49, 51, 52, 53, 54, 55, 56, 57, 58, 62, 63, 64, 65, 97, 121, 123, 127, 128, 140, 142, 144, 147, 148, 149], "__name__": [47, 48, 49, 51, 52, 53, 54, 55, 56, 57, 58, 62, 63, 64, 65, 97, 121, 123, 127, 128, 140, 142, 144, 147, 148, 149], "__post_init__": 148, "__pydantic_extra__": 143, "__pydantic_fields_set__": 143, "__repr__": 148, "__traceback__": 143, "__version__": 95, "_autodeploi": [21, 26, 73, 152, 156], "_autodeployllmarg": 148, "_callback": [80, 160], "_capac": 1, "_checkpoint_format": [80, 160], "_chunk_token": 56, "_config_load": [80, 160], "_context_logits_auto_en": 143, "_cpp_gen": 99, "_create_tensor": 113, "_custom_transform_funct": [80, 160], "_executor_loop": 92, "_explicitly_disable_gemm_plugin": 132, "_fields_set": 143, "_file_path": 56, "_forward_step": [35, 87], "_generation_logits_auto_en": 143, "_get_config_dict": [75, 154], "_get_pretrained_config": [80, 160], "_handl": 1, "_hash_token": 56, "_hf_model_dir": 143, "_llm": [28, 29, 31], "_llm_arg": 56, "_load_pretrained_config": [80, 160], "_map": [80, 160], "_mark_output": 139, "_metadata": 56, "_modelformatkind": 143, "_mpi_sess": 143, "_note": [72, 101], "_parallelconfig": 143, "_parent_namespace_depth": 143, "_path": 2, "_postproc_param": 143, "_postprocess_result": 143, "_process_previous_batch": [35, 87], "_releas": 1, "_return_log_prob": 143, "_run": 139, "_runtim": 134, "_sample_async": [35, 87], "_schedul": [35, 87], "_serv": [28, 29, 31], "_sliding_window_pattern": 131, "_static": 113, "_str_to_trt_dtype_dict": 129, "_tensorrt_engin": 140, "_torch": [10, 37, 56, 72, 75, 77, 80, 92, 120, 141, 143, 144, 148, 149, 151, 154, 157, 160, 164], "_torchllm": 143, "_types_namespac": 143, "_unsign": 1, "_util": 129, "_was_": 143, "_weight_load": [80, 160], "_weight_mapp": [80, 160], "a10": 22, "a100": [20, 22, 33, 93, 102, 115, 146], "a100x": 33, "a10g": 22, "a2": 148, "a22b": [79, 159], "a2a": 148, "a30": 22, "a3b": [79, 138, 145, 159], "a40": 22, "a8": 137, "a_": 129, "a_1": 129, "a_2": 129, "a_n": 129, "a_sf": 129, "aarch64": 138, "ab": [22, 56, 106, 129, 137, 143], "abbrevi": 26, "abc": [11, 13], "abcd": 13, "abi": [94, 148], "abil": [1, 37, 120, 148], "abl": [4, 10, 12, 15, 19, 37, 72, 88, 92, 97, 101, 120, 126, 129, 148], "ablat": [10, 13, 14], "abnorm": [15, 148], "abort": [143, 148], "about": [0, 1, 2, 3, 4, 6, 7, 11, 14, 15, 16, 19, 20, 29, 37, 56, 62, 64, 77, 81, 93, 99, 115, 120, 121, 123, 124, 125, 127, 129, 135, 139, 143, 147, 148, 157], "abov": [2, 7, 11, 14, 15, 16, 17, 19, 20, 26, 28, 29, 30, 31, 34, 37, 38, 42, 72, 81, 89, 94, 97, 106, 107, 113, 115, 120, 121, 123, 127, 135], "absenc": [16, 102], "absolut": 9, "absorb": 12, "abstract": [35, 92, 127, 130, 143], "abstractsetintstr": 143, "ac": 148, "acc": 129, "acceler": [4, 5, 6, 7, 11, 15, 20, 22, 28, 29, 30, 31, 72, 78, 88, 92, 101, 107, 109, 146, 158], "accept": [0, 1, 2, 10, 11, 15, 17, 18, 19, 20, 21, 23, 32, 33, 42, 47, 52, 53, 54, 55, 72, 81, 92, 93, 94, 97, 109, 121, 123, 129, 134, 140, 142, 143, 147, 148, 151], "accept_length": 134, "acceptancelength": 0, "acceptancer": 0, "acceptancethreshold": 0, "acceptedlen": 1, "acceptedlengthscumsum": 1, "acceptedpath": 1, "acceptedpathid": 1, "acceptedtoken": 1, "acceptedtokenslen": 1, "access": [10, 15, 17, 19, 20, 28, 29, 34, 51, 99, 117, 120, 121, 129, 143, 147, 148], "accessor": 1, "accommod": [16, 81, 100, 150, 166], "accomplish": 122, "accord": [10, 17, 18, 20, 72, 101, 114, 129, 130, 151], "accordingli": [10, 17, 18, 114], "account": [2, 8, 16, 59, 60, 61, 92, 94, 113, 116], "accumul": [0, 8, 11, 15, 19, 22, 58, 101, 102, 129, 134, 143], "accur": [3, 13, 37, 51, 81, 98, 120, 121, 146, 148], "accuraci": [2, 3, 11, 12, 14, 20, 22, 23, 123, 128, 129, 137, 147, 148], "accuracy_threshold": 23, "achiev": [2, 3, 7, 8, 10, 11, 12, 14, 15, 16, 18, 19, 20, 29, 38, 49, 84, 92, 93, 94, 109, 121, 123, 126, 128, 143, 144, 149], "acknowledg": 93, "acquir": [10, 19], "across": [0, 5, 8, 10, 11, 12, 15, 16, 17, 19, 20, 26, 28, 29, 30, 31, 32, 33, 38, 72, 81, 84, 89, 93, 98, 100, 101, 102, 103, 113, 114, 122, 123, 126, 127, 129, 134, 143, 146], "act": [12, 15, 16, 81, 90], "act_fn": 130, "act_typ": [113, 129], "action": 57, "activ": [0, 1, 3, 4, 7, 12, 14, 15, 16, 20, 22, 29, 32, 35, 72, 78, 81, 84, 89, 90, 101, 103, 113, 122, 129, 137, 138, 148, 158, 167], "activation_scaling_factor": 112, "activationtyp": [113, 129], "active_request": [92, 167], "actual": [2, 7, 8, 12, 13, 15, 17, 19, 21, 22, 25, 29, 32, 80, 88, 92, 103, 104, 109, 123, 126, 127, 128, 143, 148, 160, 166], "ad": [1, 8, 11, 13, 14, 16, 18, 19, 23, 29, 36, 48, 72, 75, 81, 83, 85, 88, 92, 93, 101, 102, 103, 105, 109, 110, 115, 117, 119, 122, 126, 127, 129, 131, 134, 141, 143, 148, 150, 154, 162], "ada": [7, 72, 90, 94, 101, 123, 138, 146, 148], "adalayernorm": 130, "adalayernormcontinu": 130, "adalayernormzero": 130, "adalayernormzerosingl": 130, "adapt": [0, 13, 15, 19, 21, 50, 78, 89, 93, 106, 129, 130, 143, 144, 146, 148, 149, 158], "adapter1": [85, 162], "adapter2": [85, 162], "adapter_id": 143, "adapter_s": 106, "adapters": 1, "add": [1, 9, 11, 13, 18, 20, 23, 28, 29, 30, 31, 33, 34, 37, 72, 80, 94, 99, 101, 103, 106, 111, 112, 113, 115, 116, 117, 120, 121, 123, 127, 129, 134, 139, 142, 143, 144, 147, 148, 149, 160, 166], "add_": 10, "add_activ": 113, "add_argu": [57, 62, 63], "add_bias_linear": 131, "add_generation_prompt": 12, "add_input": 129, "add_not": 143, "add_output": 129, "add_padding_request": 166, "add_prefix_spac": 58, "add_qkv_bia": 131, "add_rmsnorm": 12, "add_sequ": 134, "add_special_token": [11, 12, 28, 30, 58, 134, 143, 148], "addbadhandl": 1, "addcumlogprob": 148, "added_kv_proj_dim": 130, "added_proj_bia": 130, "addit": [0, 5, 8, 10, 11, 13, 14, 15, 16, 18, 19, 26, 28, 29, 30, 31, 32, 37, 38, 42, 51, 72, 74, 75, 79, 80, 81, 86, 88, 92, 94, 96, 97, 101, 102, 106, 109, 113, 115, 120, 122, 123, 126, 129, 130, 137, 138, 139, 143, 148, 151, 153, 154, 159, 160, 166], "addition": [19, 72, 92, 120, 121, 123, 127, 144, 149, 151], "additional_model_output": 143, "additional_opt": 61, "additionalmodeloutput": [0, 99, 143], "additionaloutput": [0, 99], "addr": 0, "address": [1, 2, 7, 8, 10, 12, 14, 15, 89, 114, 127, 135, 142, 143, 148], "addresswiths": 1, "adequ": [130, 143], "adher": 51, "adirondack": 29, "adjust": [9, 15, 16, 17, 20, 37, 58, 59, 81, 98, 120, 121, 135, 143, 167], "admin": 97, "adopt": [10, 15, 18, 102, 115, 148], "adp": [10, 93, 148], "advanc": [6, 10, 12, 13, 14, 15, 16, 22, 47, 52, 53, 55, 72, 75, 81, 89, 93, 97, 109, 113, 129, 140, 142, 147, 148, 151, 154], "advantag": [15, 16, 18, 19, 20, 81, 83, 102], "advers": [3, 22], "advertis": [37, 120], "affect": [2, 8, 10, 22, 75, 88, 107, 115, 121, 123, 126, 127, 135, 148, 154], "affin": 130, "aforement": [10, 15, 34], "after": [0, 1, 8, 9, 10, 11, 12, 13, 14, 15, 17, 19, 21, 22, 23, 25, 26, 28, 29, 30, 31, 33, 34, 48, 72, 78, 79, 81, 83, 88, 89, 92, 94, 98, 99, 101, 103, 104, 105, 106, 109, 113, 114, 116, 120, 123, 126, 127, 128, 129, 130, 132, 135, 143, 147, 148, 151, 158, 159, 167], "again": [10, 15, 57, 113, 121, 123, 127, 139], "against": [18, 37, 94, 120, 143, 148], "agent": [5, 10, 11], "agentdesc": 0, "agentnam": 0, "agentst": 0, "aggreg": [14, 15, 16, 44, 81], "aggress": [13, 48, 112, 123, 128], "agnost": [20, 148], "agre": [30, 31, 142], "agreement": 142, "ahead": [0, 17, 101, 109], "ai": [2, 4, 11, 12, 15, 17, 19, 20, 26, 28, 30, 36, 43, 47, 48, 49, 52, 53, 54, 55, 58, 63, 65, 79, 97, 119, 121, 128, 129, 138, 140, 142, 145, 146, 147, 148, 159], "aidc": 148, "aim": [2, 12, 15, 20, 35, 37, 100, 112, 120, 121, 123, 148], "aime2025": 29, "aime24": 11, "aime25": 29, "ainsli": 3, "air": [29, 148], "airport": 29, "aka": 129, "akhoroshev": 148, "al": [3, 18], "albani": 29, "albeit": 109, "alessionetti": 148, "algo": 65, "algorithm": [0, 7, 8, 10, 11, 12, 13, 14, 15, 21, 37, 65, 72, 92, 93, 101, 102, 109, 112, 113, 115, 120, 123, 129, 143, 146, 148], "alia": [32, 130, 131, 143], "alias": 143, "alibi": 129, "alibi_bias_max": [129, 130], "alibi_scal": 129, "alibi_slop": 129, "alibi_with_scal": 129, "align": [8, 18, 19, 86, 148, 167], "align_corn": 129, "all": [0, 1, 2, 5, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 25, 28, 29, 30, 31, 32, 33, 37, 38, 49, 56, 59, 60, 61, 62, 63, 64, 72, 75, 79, 80, 81, 83, 85, 88, 89, 92, 94, 95, 97, 98, 99, 100, 101, 102, 103, 104, 106, 109, 113, 114, 115, 120, 121, 122, 123, 126, 127, 128, 129, 130, 132, 134, 135, 137, 138, 139, 143, 147, 148, 151, 154, 159, 160, 162, 166, 167], "all2al": [15, 19, 148], "all_clos": 148, "all_reduc": 148, "all_reduce_param": [129, 130], "allbitset": [0, 1], "allgath": [14, 19, 22, 113, 127, 129, 148], "allgeneratedtoken": 0, "alllayersdrafttokenid": 1, "alllayersdrafttokenidspredecessor": 1, "alllayersscor": 1, "alloc": [0, 1, 17, 19, 23, 26, 28, 35, 42, 48, 72, 74, 81, 84, 88, 92, 94, 98, 101, 104, 105, 128, 129, 134, 135, 139, 143, 148, 150, 151, 153, 166, 167], "allocateipcmemori": 1, "allocnewblock": [0, 26], "allocnewblocksperrequest": 0, "alloctotalblock": [0, 26], "alloctotalblocksperrequest": 0, "allot": 0, "allottedtimem": [0, 148], "allow": [0, 1, 3, 6, 10, 11, 14, 15, 16, 19, 20, 22, 28, 29, 30, 31, 32, 34, 35, 36, 37, 38, 80, 81, 83, 84, 85, 87, 88, 89, 90, 91, 98, 99, 101, 102, 105, 109, 112, 119, 120, 121, 122, 123, 126, 127, 129, 132, 139, 142, 143, 146, 147, 148, 150, 160, 162, 163, 167], "allow_pickl": 143, "allreduc": [12, 14, 22, 113, 127, 129, 143, 148], "allreduce_gemm": 108, "allreduce_strategi": [107, 143, 148], "allreducebuff": 1, "allreducefusionkernel": 12, "allreducefusionop": 129, "allreduceparam": [129, 130], "allreducestrategi": [107, 129], "alltoal": [19, 148], "alltoall_prepare_maybe_dispatch": 148, "alltoallprepar": 19, "almost": [10, 14, 15, 113, 123, 126, 135], "alon": [8, 100], "along": [10, 13, 19, 72, 79, 94, 101, 109, 129, 148, 159], "alongsid": [15, 78, 79, 158, 159], "alpaca": 106, "alpha": [129, 130, 143, 148], "alphabet": [129, 143], "alreadi": [0, 2, 11, 12, 13, 14, 15, 16, 18, 20, 35, 56, 72, 80, 88, 101, 103, 105, 123, 126, 128, 129, 143, 144, 148, 149, 166], "also": [0, 2, 5, 6, 7, 10, 11, 12, 13, 14, 15, 16, 19, 20, 22, 23, 26, 28, 29, 30, 31, 33, 34, 37, 38, 42, 48, 49, 58, 72, 80, 81, 83, 84, 88, 92, 94, 95, 97, 98, 99, 101, 103, 109, 112, 113, 114, 115, 120, 121, 122, 123, 124, 125, 126, 129, 130, 135, 137, 142, 143, 144, 146, 147, 148, 149, 150, 151, 166], "altair": 148, "alter": [83, 99, 103], "altern": [10, 12, 19, 20, 23, 33, 37, 57, 78, 80, 94, 99, 108, 120, 144, 149, 150, 158, 160], "although": [10, 16, 19, 37, 75, 81, 98, 103, 113, 120, 123, 127, 154], "alwai": [0, 1, 10, 14, 15, 18, 19, 32, 56, 72, 75, 88, 92, 99, 101, 102, 105, 112, 113, 115, 126, 127, 129, 139, 143, 154], "always_share_across_beam": 134, "am": [47, 52, 53, 55, 97, 121, 128, 134, 140, 142, 147], "ambigu": 1, "amc23": 11, "amd": 148, "amen": [0, 10, 99, 143], "american": 56, "among": [10, 16, 18, 83, 117, 129], "amongst": 129, "amort": 18, "amount": [0, 14, 15, 19, 22, 37, 84, 88, 105, 113, 120, 126, 128, 134, 135, 139, 143], "amp": 20, "amper": [4, 20, 90, 94, 138, 146, 148], "amplifi": 10, "amsterdam": 29, "amtrak": 29, "an": [0, 1, 3, 5, 7, 8, 9, 10, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 25, 26, 28, 29, 30, 31, 33, 34, 35, 37, 38, 42, 47, 51, 52, 53, 54, 55, 56, 57, 58, 72, 75, 77, 78, 79, 80, 81, 83, 84, 85, 86, 88, 89, 90, 91, 92, 93, 94, 97, 98, 99, 101, 102, 103, 105, 106, 108, 109, 110, 111, 112, 113, 114, 115, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 134, 135, 137, 139, 140, 142, 143, 144, 147, 148, 149, 150, 151, 154, 156, 157, 158, 159, 160, 162, 166, 167], "analog": 34, "analys": 15, "analysi": [11, 12, 13, 14, 15, 16, 17, 19, 20, 93, 103, 135], "analysispatternmanag": 103, "analysisth": 29, "analyt": 4, "analyz": [19, 58, 103, 121], "andlength": 21, "ani": [0, 1, 2, 8, 10, 13, 14, 15, 19, 26, 29, 32, 33, 34, 35, 37, 38, 56, 72, 75, 80, 83, 88, 94, 99, 103, 104, 109, 114, 115, 120, 126, 127, 128, 129, 131, 134, 139, 143, 144, 149, 150, 151, 154, 160], "annot": [32, 143], "announc": [2, 3, 4, 6], "anoth": [0, 1, 4, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 26, 28, 29, 30, 31, 72, 81, 84, 88, 101, 103, 106, 115, 126, 129, 139, 142, 151, 167], "answer": [11, 13, 18, 20, 29, 51, 58, 63], "answer_suffix": 11, "answer_suffix_with_mark": 11, "antialia": 129, "antonin": [47, 97, 140, 142, 147], "anybitset": [0, 1], "anymor": [15, 92], "anyth": [38, 56, 62, 143], "aotman": 148, "apart": 42, "api": [2, 9, 10, 11, 13, 14, 15, 16, 17, 19, 20, 21, 23, 25, 35, 36, 37, 38, 42, 43, 47, 50, 56, 60, 61, 77, 78, 79, 80, 81, 84, 86, 94, 98, 102, 105, 109, 111, 112, 113, 119, 120, 123, 124, 125, 127, 128, 129, 135, 139, 141, 146, 149, 157, 158, 159, 160, 162], "api_kei": [66, 67, 68, 69, 70, 85, 162], "api_st": 32, "app": [94, 148], "appar": [8, 81, 98], "appeal": 18, "appear": [0, 19, 72, 88, 92, 97, 101, 102, 129, 139, 143, 148], "append": [11, 13, 18, 36, 56, 80, 119, 129, 160, 167], "append_paged_kv_cach": [72, 151], "appl": [79, 148, 159], "appli": [0, 12, 13, 14, 15, 19, 20, 22, 23, 34, 35, 37, 72, 78, 79, 80, 83, 84, 85, 94, 99, 101, 103, 106, 109, 112, 113, 114, 120, 129, 130, 134, 137, 143, 148, 151, 158, 159, 160, 162], "applic": [0, 4, 7, 8, 9, 12, 14, 15, 16, 17, 20, 25, 28, 29, 30, 31, 34, 36, 39, 40, 41, 56, 77, 81, 92, 97, 105, 109, 119, 134, 139, 142, 145, 146, 147, 148, 156, 157, 167], "apply_batched_logits_processor": 143, "apply_callback": [80, 160], "apply_chat_templ": [12, 23, 51], "apply_llama3_sc": 129, "apply_query_key_layer_sc": [130, 131], "apply_residual_connection_post_layernorm": 131, "apply_rotary_pos_emb": 129, "apply_rotary_pos_emb_chatglm": 129, "apply_rotary_pos_emb_cogvlm": 129, "apply_router_weight_on_input": 148, "apply_silu": 129, "applybiasropeupdatekvcach": 148, "applyrop": 12, "appreci": [10, 11, 14], "approach": [0, 8, 11, 12, 13, 14, 15, 16, 35, 37, 80, 81, 88, 92, 96, 100, 103, 105, 107, 109, 120, 128, 143, 160], "appropri": [2, 7, 11, 16, 32, 33, 35, 42, 75, 77, 81, 88, 139, 154, 157], "approx": 29, "approxim": [14, 15, 18, 94, 130], "apt": [2, 78, 94, 97, 116, 158], "ar": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 47, 49, 51, 52, 53, 55, 56, 57, 58, 59, 60, 61, 62, 66, 67, 70, 72, 74, 75, 78, 80, 81, 83, 84, 85, 86, 88, 89, 90, 91, 92, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 134, 135, 137, 139, 140, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 153, 154, 156, 158, 160, 162, 164, 165, 166, 167], "arang": 129, "arbitrag": [37, 120], "arbitrari": [10, 92, 114, 148], "arch": 92, "architectur": [4, 8, 13, 14, 15, 16, 19, 20, 28, 29, 30, 31, 73, 74, 78, 79, 80, 81, 93, 94, 98, 100, 102, 105, 112, 131, 134, 138, 141, 145, 148, 152, 153, 156, 158, 159, 160], "archiv": [80, 160], "arctic": [138, 148], "area": [14, 15, 29], "aresult": [42, 143], "arg": [0, 11, 21, 23, 26, 32, 57, 62, 63, 74, 75, 80, 103, 115, 130, 131, 134, 143, 148, 153, 154, 160], "arglist": 103, "argmax": [92, 129], "argpars": [57, 62, 63], "argument": [2, 10, 23, 26, 28, 29, 30, 31, 33, 37, 42, 48, 49, 56, 65, 72, 81, 92, 93, 94, 98, 99, 120, 122, 129, 135, 143, 148, 151], "argumentpars": [57, 62, 63], "aris": [8, 10, 94], "arithmet": [10, 20, 113], "arm": 33, "arm64": 20, "aros": 19, "around": [1, 19, 80, 84, 112, 115, 121, 127, 160], "arrai": [0, 1, 33, 129, 134, 143], "arrayview": [0, 1], "arriv": [0, 8, 11, 57, 100], "arrivaltim": 0, "arrow": 129, "art": [2, 12, 15, 16], "articl": [12, 13, 101, 109], "artifact": [34, 93, 94], "artifici": [17, 64], "arxiv": [0, 1, 22, 100, 106, 129, 137], "as_dtyp": 129, "as_lay": 103, "as_shap": 129, "ascii": [129, 143], "asciichar": 1, "ask": [11, 18, 20, 29, 62, 139], "aspect": [88, 101], "asscoiat": 80, "assembl": [11, 113], "assert": [56, 69, 70, 103, 129, 139, 148, 167], "assert_valid_quant_algo": 131, "asset": 9, "assign": [0, 10, 15, 19, 37, 83, 115, 120, 130, 132, 144, 149], "assist": [17, 20, 29, 39, 40, 51, 66, 67, 70, 102, 142, 147, 156], "assistant_model": 102, "assistantfin": 29, "associ": [1, 8, 14, 16, 34, 35, 81, 92, 94, 99, 100, 106, 121, 129, 160], "asssembl": 109, "assum": [1, 2, 9, 13, 14, 15, 37, 49, 99, 105, 106, 109, 110, 120, 129, 134, 143], "assumpt": [16, 22, 109], "assur": 15, "async": [1, 11, 42, 53, 54, 120, 134, 143], "asynchron": [1, 10, 11, 15, 35, 42, 50, 56, 86, 99, 143], "asynchroni": 15, "asyncio": [11, 53, 54], "asyncllmengin": 148, "aten": [78, 79, 158, 159], "atlant": 29, "atom": 1, "attach": [2, 92, 143, 147], "attack": 83, "attempt": [0, 1, 34, 38, 81, 98, 121, 123, 143], "attend": [84, 128], "attent": [0, 1, 2, 3, 8, 10, 15, 16, 19, 20, 22, 25, 26, 28, 29, 30, 31, 35, 73, 74, 77, 78, 81, 82, 92, 93, 98, 102, 104, 105, 106, 109, 111, 113, 114, 129, 134, 135, 139, 141, 143, 144, 145, 146, 148, 149, 152, 153, 156, 157, 158, 161, 166], "attention_backend": [72, 144, 149, 151], "attention_bia": 148, "attention_chunk_s": 84, "attention_dp": 148, "attention_dp_config": [8, 29, 143], "attention_dp_events_gather_period_m": 143, "attention_head_s": [129, 130], "attention_mask": [72, 129, 130, 131, 134, 151], "attention_mask_param": 131, "attention_mask_typ": 130, "attention_multipli": 131, "attention_output": 139, "attention_output_orig_quant_scal": 129, "attention_output_sf_scal": 129, "attention_packed_mask": [129, 130], "attention_param": [130, 131], "attention_qk_half_accumul": 148, "attention_window_s": [72, 104], "attentionconfig": 0, "attentiondpconfig": [93, 143], "attentiondpeventsgatherperiodm": 0, "attentiondprank": 0, "attentionheads": 1, "attentionlayernumperpp": 0, "attentionmask": [72, 151], "attentionmaskparam": 130, "attentionmasktyp": [129, 130], "attentionmetadata": [92, 144, 149], "attentionparam": [130, 131], "attentiontyp": 0, "attn": [74, 75, 153, 154], "attn_backend": [72, 73, 75, 77, 79, 143, 151, 152, 154, 156, 157, 159], "attn_bia": 131, "attn_dens": [22, 106], "attn_forward_funcnam": 130, "attn_k": [22, 37, 85, 106, 120, 162], "attn_logit_softcap": 131, "attn_logit_softcapping_scal": 129, "attn_metadata": [144, 149], "attn_page_s": [77, 157], "attn_processor": 131, "attn_q": [22, 37, 85, 106, 120, 162], "attn_qkv": [22, 106], "attn_v": [22, 37, 85, 106, 120, 162], "attribut": [0, 1, 92, 99, 103, 114, 115, 134, 143], "audienc": 58, "audio": [26, 134, 138, 145, 148], "audio_engine_dir": 134, "audio_featur": 134, "audio_path": 134, "audio_url": 26, "augment": 156, "authent": [9, 121, 142], "author": [10, 92], "authorized_kei": [116, 117], "auto": [0, 1, 15, 20, 21, 25, 28, 30, 31, 34, 37, 74, 75, 81, 83, 93, 98, 99, 101, 102, 107, 110, 113, 120, 127, 129, 131, 132, 133, 143, 148, 153, 154], "auto_deploi": [74, 75, 78, 148, 153, 154, 158], "auto_deploy_log_level": [76, 155], "auto_parallel": [22, 143, 148], "auto_parallel_config": 143, "auto_parallel_world_s": 143, "auto_quantize_bit": 133, "autoawq": 148, "autodecodingconfig": [93, 143], "autodeploi": [73, 74, 75, 77, 79, 93, 141, 148, 152, 153, 154, 156, 157, 159], "autodeploy_config": [75, 154, 156], "autodeploy_overrid": [75, 154], "autodeployconfi": [75, 154], "autodeployconfig": [75, 77, 154, 157], "autogptq": 148, "autom": [16, 33, 51, 58, 78, 79, 148, 158, 159], "automat": [0, 1, 10, 12, 15, 20, 23, 26, 32, 33, 35, 37, 42, 73, 75, 78, 92, 99, 103, 107, 113, 114, 120, 121, 129, 135, 137, 142, 143, 146, 148, 152, 154, 156, 158], "automodelforcausallm": [74, 77, 79, 153, 157, 159], "automodelforimagetexttotext": [79, 159], "automot": 56, "autonom": 17, "autoparallelconfig": 143, "autopp": 148, "autoq": 148, "autoregress": [0, 35, 72, 109, 151, 166], "autotoken": [11, 42], "autotun": [143, 148], "autotuner_en": 148, "aux": 135, "auxiliari": [10, 80, 109, 160], "avaiable_block": 167, "avail": [0, 1, 3, 5, 8, 9, 11, 15, 16, 19, 25, 26, 28, 29, 30, 31, 34, 35, 37, 42, 47, 52, 53, 55, 64, 72, 73, 74, 77, 81, 83, 86, 88, 90, 91, 92, 94, 95, 97, 99, 103, 105, 108, 109, 113, 120, 126, 127, 128, 134, 135, 137, 140, 141, 142, 143, 147, 148, 151, 152, 153, 157, 165, 166], "averag": [0, 2, 8, 10, 13, 15, 16, 18, 19, 21, 37, 38, 109, 120, 121, 123, 126, 127, 143], "avg": [8, 37, 120, 121, 129], "avg_decoded_tokens_per_it": 29, "avg_pool2d": 129, "avgdecodedtokensperit": 0, "avgnumdecodedtokensperit": 0, "avgpool2d": 130, "avoid": [1, 8, 10, 11, 12, 13, 14, 15, 20, 28, 30, 83, 92, 94, 115, 134, 135, 142, 148], "awai": [88, 126, 127], "await": [0, 11, 42, 53, 54, 99], "await_respons": 143, "awaitcontextrespons": 0, "awaitgenerationrespons": 0, "awaitrespons": [0, 99], "awar": [3, 16, 18, 20, 72, 92, 101, 139, 148], "awq": [7, 42, 90, 138, 148], "awq_block_s": 133, "ax": 129, "axi": [6, 15, 129], "b": [1, 3, 4, 5, 6, 33, 36, 57, 75, 103, 106, 113, 119, 129, 131, 134, 143, 148, 154], "b200": [13, 14, 15, 17, 18, 20, 28, 29, 38, 93, 146, 148], "b6261862419c33d6ce2313aff1e7116067d6037d": 2, "b_sf": 129, "back": [0, 1, 10, 15, 35, 38, 48, 56, 57, 83, 105, 107, 109, 148], "backend": [0, 2, 9, 10, 11, 13, 14, 15, 21, 23, 25, 26, 33, 36, 37, 38, 50, 51, 56, 64, 73, 74, 75, 77, 78, 80, 84, 85, 86, 90, 91, 92, 93, 99, 109, 113, 120, 142, 143, 144, 148, 150, 152, 153, 154, 156, 157, 158, 160, 162, 164, 165, 166, 167], "backend_token": [0, 99], "backendagentdesc": 0, "backendtyp": [0, 81], "background": [1, 15, 25, 35, 93], "backgroundconfigur": 1, "backlog": 33, "backstream": 1, "backtyp": 1, "backu": [0, 91, 99, 143], "backup": 1, "backward": [78, 80, 115, 158, 160], "bad": [0, 91, 99, 143, 146, 148, 165], "bad_token_id": 143, "bad_words_data": 134, "bad_words_list": 134, "badword": 0, "badwordslen": 1, "badwordslist": 1, "badwordsptr": 1, "baichuan": [137, 138, 148], "baichuan2": 138, "baichuanconfig": 131, "baichuanforcausallm": 131, "balanc": [11, 14, 16, 28, 64, 83, 88, 93, 100, 102, 109, 113, 126, 128, 143, 148], "band": 51, "bandwidth": [3, 4, 5, 7, 14, 15, 19, 20, 38, 51, 81, 98, 102, 113], "bangbang": 4, "bantoken": 0, "banword": 0, "bar": 143, "bare": [141, 148], "barissglc": 62, "barrier": [18, 146], "bart": [138, 148], "base": [0, 1, 2, 3, 4, 7, 8, 10, 12, 14, 15, 16, 19, 20, 22, 25, 30, 31, 34, 35, 37, 53, 54, 58, 75, 79, 80, 81, 85, 88, 89, 92, 93, 94, 99, 105, 106, 107, 109, 111, 114, 115, 120, 126, 128, 129, 130, 131, 132, 133, 134, 135, 138, 141, 143, 144, 145, 148, 149, 150, 154, 159, 160, 162, 166, 167], "base64": [26, 67], "base_checkpoint_load": [80, 160], "base_config_load": [80, 160], "base_model": 106, "base_s": 130, "base_url": [28, 30, 31, 66, 67, 68, 69, 70, 85, 162], "base_weight_load": [80, 160], "base_weight_mapp": [80, 160], "baseagentconfig": 0, "basecheckpointload": 143, "basekvcachemanag": 0, "baselin": [7, 11, 12, 13, 14, 19, 72, 75, 121, 126, 127, 151, 154], "baseline_fp8_engin": 123, "basellmarg": 143, "baseloopbackag": 0, "basemodel": 143, "baseresourcemanag": [150, 166], "basetransferag": 0, "bash": [9, 20, 25, 26, 28, 29, 30, 31, 36, 39, 40, 41, 43, 45, 46, 57, 59, 60, 61, 113, 117, 119, 147], "basi": 16, "basic": [36, 93, 111, 119, 129, 148], "basic_string_view": 0, "batch": [0, 1, 2, 4, 5, 7, 8, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 35, 37, 38, 57, 73, 74, 75, 78, 81, 84, 86, 87, 89, 92, 93, 102, 105, 106, 107, 109, 110, 113, 118, 120, 121, 123, 124, 125, 127, 128, 129, 130, 134, 135, 139, 143, 144, 146, 148, 149, 150, 151, 152, 153, 154, 158, 163, 166, 167], "batch_beam_s": [101, 129], "batch_dim": 129, "batch_idx": 134, "batch_input_id": 134, "batch_manag": [0, 1, 56, 148, 166], "batch_output": [35, 57, 87], "batch_schedul": 148, "batch_siz": [2, 3, 6, 10, 18, 25, 28, 30, 31, 38, 63, 72, 75, 92, 101, 103, 110, 112, 129, 130, 133, 134, 135, 143, 151, 154], "batch_wait_max_tokens_ratio": 143, "batch_wait_timeout_it": 143, "batch_wait_timeout_m": 143, "batchdon": 1, "batched_logits_processor": 143, "batchedlogitsprocessor": 143, "batchidx": 1, "batchindex": 1, "batching_typ": 143, "batching_wait_it": [8, 29, 143], "batchingtyp": [0, 93, 143], "batchsiz": [0, 1, 4, 102], "batchsizelimit": 0, "batchsizet": 0, "batchslot": 1, "batchslotshostcopi": 1, "bc": 129, "bc1393d529ce485c961d9ffee5b25d72": [30, 31], "beam": [0, 1, 6, 21, 22, 23, 26, 35, 42, 64, 81, 84, 93, 102, 109, 129, 134, 135, 139, 143, 148], "beam_search_diversity_r": [134, 143], "beam_width": [21, 42, 72, 88, 101, 102, 129, 134, 148], "beam_width_arrai": 143, "beamhypothes": 1, "beamsearch": [0, 143], "beamsearchbuff": 1, "beamsearchdiversityr": [0, 1, 102], "beamsiz": 0, "beamtoken": [0, 99], "beamwidth": [0, 1, 98, 99, 102, 143, 148], "beamwidtharrai": [0, 1, 102], "beat": [75, 154], "beauti": 64, "becam": [0, 19], "becaus": [0, 7, 8, 10, 11, 12, 13, 14, 15, 19, 22, 37, 38, 42, 62, 72, 81, 88, 89, 92, 95, 98, 99, 105, 120, 121, 122, 123, 126, 128, 129, 135, 142, 143], "becom": [7, 8, 10, 11, 12, 14, 15, 18, 19, 72, 75, 84, 86, 88, 89, 100, 101, 102, 103, 105, 106, 113, 114, 154], "been": [0, 4, 5, 10, 11, 12, 14, 15, 19, 37, 62, 72, 79, 83, 88, 94, 95, 97, 99, 101, 115, 117, 120, 123, 126, 129, 139, 143, 148, 159], "befor": [0, 1, 8, 10, 11, 12, 13, 15, 17, 19, 20, 21, 28, 30, 32, 36, 56, 58, 59, 60, 61, 72, 73, 81, 83, 88, 89, 91, 92, 93, 94, 96, 97, 98, 99, 101, 103, 105, 106, 107, 112, 113, 114, 119, 122, 123, 126, 128, 129, 131, 134, 135, 139, 142, 143, 144, 148, 149, 150, 151, 152, 166, 167], "beforehand": 121, "began": 19, "begin": [28, 29, 30, 31, 56, 78, 79, 109, 122, 143, 144, 148, 149, 158, 159], "begin_thinking_phase_token": 143, "behav": [0, 135, 143], "behavior": [8, 11, 19, 34, 37, 38, 57, 72, 73, 75, 81, 88, 91, 93, 98, 101, 126, 129, 134, 135, 143, 148, 152, 154], "behaviour": [0, 15, 129], "behind": [4, 11, 14, 35], "being": [0, 10, 14, 17, 18, 19, 20, 28, 29, 30, 31, 62, 72, 74, 80, 83, 88, 101, 105, 113, 115, 126, 139, 143, 148, 151, 153, 160], "believ": [15, 37, 120], "belong": [19, 88, 126], "below": [0, 2, 5, 6, 7, 9, 10, 11, 13, 14, 15, 16, 17, 18, 20, 25, 28, 29, 30, 31, 33, 37, 38, 72, 81, 88, 92, 94, 96, 101, 102, 103, 104, 106, 116, 117, 120, 123, 126, 127, 139, 147, 148], "bench": [2, 10, 13, 15, 18, 20, 25, 28, 29, 30, 31, 37, 38, 50, 62, 89, 93, 120, 124, 125, 147, 148, 156], "benchmark": [9, 10, 12, 13, 15, 16, 17, 19, 20, 21, 24, 26, 36, 60, 74, 75, 78, 81, 86, 89, 93, 94, 98, 119, 123, 124, 125, 127, 147, 148, 153, 154, 156, 158], "benchmark_2nod": 26, "benchmark_serv": [25, 28, 29, 30, 31, 148], "benefici": [10, 14, 16, 19, 37, 81, 88, 120, 126, 127], "benefit": [5, 7, 8, 10, 13, 14, 15, 16, 18, 19, 22, 38, 58, 63, 84, 88, 103, 105, 107, 126, 148], "berkelei": 23, "bert": [22, 90, 129, 137, 138, 145, 148], "bert_attent": 129, "bert_attention_plugin": 22, "bert_context_fmha_fp32_acc": 22, "bertattent": 130, "bertattentionplugin": 129, "bertbas": 131, "bertforquestionansw": 131, "bertforsequenceclassif": [131, 138, 145], "bertmodel": 131, "besid": [19, 26, 150], "best": [8, 10, 11, 12, 13, 14, 15, 16, 18, 20, 36, 37, 73, 93, 101, 113, 119, 120, 122, 124, 125, 126, 143, 148, 152, 156], "best_of": [91, 143, 148], "best_path": 134, "best_path_len": 134, "best_path_length": 134, "best_perf_practice_on_deepseek": [12, 148], "bestofn": 11, "bestpathindic": 1, "bestpathlength": 1, "beta": [26, 93, 129, 141, 143, 146], "beta_fast": 129, "beta_slow": 129, "better": [0, 6, 8, 10, 12, 13, 14, 15, 16, 18, 19, 20, 22, 35, 38, 58, 59, 60, 61, 63, 72, 81, 83, 84, 87, 88, 92, 98, 101, 102, 105, 107, 114, 115, 122, 123, 127, 128, 143, 148, 163], "between": [0, 8, 11, 12, 13, 14, 15, 16, 18, 19, 21, 25, 28, 29, 30, 31, 32, 36, 38, 40, 67, 72, 77, 80, 81, 83, 85, 89, 98, 101, 102, 104, 105, 108, 109, 113, 114, 115, 119, 122, 127, 128, 129, 130, 135, 139, 143, 144, 148, 149, 157, 160, 162], "beyond": [1, 4, 11, 18, 86, 88, 123], "bf16": [1, 2, 12, 14, 17, 72, 79, 90, 101, 107, 114, 115, 123, 127, 138, 145, 148, 159], "bfloat16": [22, 37, 72, 101, 113, 120, 121, 132, 137, 138, 143, 148], "bhuvanesh09": 148, "bi": [72, 101], "bia": [0, 14, 72, 99, 112, 113, 129, 130, 131, 143, 148], "bias": [112, 129], "bidirect": [129, 130], "bidirectionalglm": 129, "bielik": 138, "big": [18, 58], "bigcod": [79, 159], "bigger": 105, "biggest": [70, 105], "billion": 2, "bin": [2, 9, 20, 25, 28, 29, 30, 31, 36, 39, 40, 41, 43, 45, 46, 59, 60, 61, 80, 112, 113, 114, 119, 139, 148, 160], "binari": [36, 80, 109, 113, 119, 129, 160], "bind": [1, 9, 10, 11, 15, 56, 128, 134, 135, 143, 148, 150, 166, 167], "bindcapacityschedul": 167, "bindf": 34, "bit": [0, 1, 4, 15, 62, 72, 101, 129, 137], "bitmask": 148, "bl": [16, 109], "black": 103, "blackwel": [2, 8, 13, 15, 18, 19, 27, 38, 90, 92, 93, 97, 122, 123, 138, 146, 148], "blank": 83, "blip": [137, 148], "blip2": [90, 137, 138, 148], "blob": [12, 28, 29, 31], "block": [0, 1, 10, 11, 14, 15, 16, 19, 22, 42, 48, 56, 57, 72, 81, 83, 84, 88, 90, 92, 98, 101, 102, 105, 113, 126, 129, 134, 135, 142, 143, 146, 148, 156, 166], "block_controlnet_hidden_st": 131, "block_id": 56, "block_num": 129, "block_po": 56, "block_siz": [56, 129, 130, 134], "block_sparse_block_s": 129, "block_sparse_homo_head_pattern": 129, "block_sparse_num_local_block": 129, "block_sparse_param": 130, "block_sparse_vertical_strid": 129, "blockhash": 0, "blockidx": 1, "blockkei": 0, "blockptr": 1, "blocksiz": 0, "blockspars": 129, "blocksparseattnparam": 130, "blockwis": [56, 148], "blog": [2, 3, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 18, 19, 38, 81, 87, 89, 148, 163], "bloodeagle40234": 148, "bloom": [102, 114, 137, 138, 148], "bloom_dict": 114, "bloomforcausallm": 131, "bloommodel": 131, "bm": 1, "bmm": [113, 148], "bo": [28, 30], "board": [78, 127, 158], "bodi": 113, "book": 62, "bool": [0, 1, 56, 58, 72, 80, 103, 110, 112, 129, 130, 131, 132, 134, 143, 151, 160], "boolean": [1, 99, 106, 129, 131, 132], "boost": [2, 12, 14, 15, 17, 19, 58, 123, 126, 127], "boost_factor": 58, "boost_val": 58, "border": [28, 29, 30, 31], "born": [111, 113, 139], "borough": 29, "borrow": [37, 42, 120], "bos_token": 58, "bos_token_ad": 58, "bos_token_id": [58, 134], "boston": 29, "bot": 33, "both": [0, 2, 4, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 25, 28, 29, 30, 31, 33, 37, 38, 47, 64, 72, 75, 81, 84, 85, 88, 89, 92, 98, 100, 101, 103, 104, 106, 109, 113, 114, 120, 122, 126, 128, 129, 130, 135, 137, 143, 148, 150, 151, 154, 162], "bother": 56, "bottleneck": [2, 7, 8, 15, 19, 35, 84, 88, 100, 107, 122, 126], "bottom": [8, 10, 19, 117], "bound": [0, 5, 8, 12, 13, 14, 19, 20, 35, 87, 102, 111, 113, 120, 129, 134, 135, 143, 148], "boundari": [14, 15, 19, 93, 102, 113, 129, 131, 133, 135, 143], "box": [2, 11, 33, 103], "bpru": 148, "br": 20, "brace": 143, "bracket": 19, "brahma": [37, 120], "branch": [3, 6, 15, 17, 28, 29, 30, 31, 109, 143], "breadth": 109, "break": [12, 15, 19, 32, 37, 56, 109, 120, 127, 143, 146, 148, 167], "breakdown": [19, 36, 37, 38, 119, 120, 121], "breakthrough": 146, "breviti": [2, 19], "bridg": [10, 15], "brief": [20, 33, 72, 131, 134, 151], "briefli": [11, 40, 67], "brife": 0, "bright": [58, 92], "bring": [7, 12, 13, 14, 15, 79, 144, 148, 149, 159], "broad": [20, 142], "broadcast": [12, 99, 129], "broadcast_help": 129, "broader": [78, 101, 142, 146, 148, 158], "broadli": [14, 72, 90], "broken": [10, 88, 126, 148], "bronx": 29, "brooklyn": 29, "brought": 15, "bsz": 130, "bu": 94, "bubbl": [8, 10], "budget": [11, 88, 93, 110, 126], "buffer": [0, 1, 10, 15, 22, 23, 26, 28, 29, 30, 31, 81, 84, 92, 98, 99, 104, 105, 129, 143, 148, 166], "buffer_0": 1, "buffer_1": 1, "buffer_2": 1, "buffer_alloc": 134, "buffercast": 1, "buffercastornul": 1, "bufferdatatyp": 1, "buffermanag": 135, "buffermanagertest": 1, "bufferptr": 1, "bufferrang": 1, "buffers": 1, "bufferview": 0, "bug": [14, 33, 148], "build": [11, 19, 20, 28, 29, 30, 31, 34, 36, 37, 62, 73, 74, 77, 81, 88, 96, 98, 99, 101, 102, 103, 105, 106, 108, 109, 110, 111, 113, 119, 122, 123, 124, 126, 128, 131, 132, 135, 139, 143, 146, 148, 152, 153, 157], "build_and_run_ad": [74, 78, 153, 158], "build_config": [22, 42, 62, 115, 123, 126, 127, 131, 140, 143], "build_connector_meta": 56, "build_dir": 94, "build_engin": 113, "build_flags_multiple_profil": 127, "build_serialized_network": 113, "build_sqsh": 94, "build_wheel": [2, 36, 94, 108, 119], "buildcacheconfig": [93, 143], "buildconfig": [42, 93, 110, 115, 123, 126, 127, 140, 143, 148], "builder": [110, 113, 115, 148], "builder_force_num_profil": 148, "builder_opt": 148, "built": [10, 11, 14, 16, 18, 22, 23, 25, 28, 29, 30, 31, 34, 35, 38, 74, 76, 78, 81, 93, 94, 96, 97, 99, 102, 105, 113, 115, 120, 121, 127, 128, 129, 135, 139, 142, 146, 148, 153, 155, 158], "bulid": 88, "bulk": 15, "bump": 1, "bumptaskinprogress": 1, "burden": 122, "busi": [0, 10, 16, 35, 58, 87], "button": 148, "buvnswrn": 148, "bw": [8, 148], "by_alia": 143, "by_nam": 143, "bypass": [16, 34, 81], "byt5": [138, 148], "byte": [0, 1, 83, 107, 134, 143], "bytearrai": 143, "bytestostr": 1, "c": [0, 1, 2, 10, 13, 14, 16, 17, 19, 25, 26, 29, 36, 42, 59, 60, 61, 81, 82, 88, 91, 101, 103, 109, 113, 116, 117, 119, 126, 129, 131, 134, 143, 145, 148, 150, 161, 165, 166, 167], "c0": 8, "c1": 8, "c2c": [15, 19], "c440e2a3e7e14cd699295afc3739bf42": 20, "c4dep4_g1dep4": 16, "c5bf51b5cab94e10ba5da5266d12ee59": 29, "cach": [0, 1, 7, 8, 12, 13, 14, 17, 18, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 34, 35, 37, 38, 42, 49, 50, 63, 73, 74, 77, 78, 79, 82, 84, 86, 89, 93, 98, 99, 102, 106, 113, 115, 118, 120, 121, 126, 129, 134, 136, 137, 141, 143, 144, 145, 146, 148, 149, 150, 151, 152, 153, 156, 157, 158, 159, 161, 167], "cache_block_id": 56, "cache_fold": 56, "cache_indir": 134, "cache_indir_t": 129, "cache_indirect": [72, 101, 129, 130, 134, 139], "cache_root": 143, "cache_salt": [83, 143], "cache_transceiver_config": [81, 143], "cached_properti": 143, "cachehitr": [0, 26], "cacheindirect": 1, "cachelevel": 0, "cachelevelupd": 0, "caches": 0, "cachesaltid": 0, "cachesaltidtyp": [0, 1], "cachest": 0, "cachetransceiv": 0, "cachetransceiverconfig": [0, 81, 93, 143], "cachetyp": 166, "cachevalu": 1, "calcul": [0, 3, 4, 6, 8, 13, 14, 15, 16, 18, 19, 25, 28, 30, 31, 37, 83, 120, 128, 129, 134, 135, 143, 148], "calculate_speculative_resourc": 143, "calculatespeculativeresourc": 0, "calculatespeculativeresourcetupl": 0, "calib_batch": [123, 131, 143], "calib_batch_s": [123, 131, 143], "calib_config": [123, 143], "calib_dataset": [131, 133, 143], "calib_max_seq_length": [123, 131, 133, 143], "calib_s": [120, 133], "calibconfig": [93, 123, 143], "calibr": [7, 14, 15, 20, 22, 114, 123, 143, 148], "california": 56, "call": [0, 1, 10, 11, 13, 14, 15, 16, 19, 32, 35, 36, 42, 57, 58, 72, 80, 81, 84, 92, 99, 100, 101, 102, 103, 108, 113, 114, 115, 119, 121, 123, 129, 131, 133, 134, 135, 143, 144, 148, 149, 150, 151, 166], "callabl": [114, 131, 143], "callback": [80, 99, 143, 160], "callstack": 10, "campaign": 58, "can": [0, 1, 2, 3, 4, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 33, 34, 35, 37, 38, 42, 44, 47, 48, 49, 56, 58, 59, 60, 61, 62, 65, 72, 74, 75, 77, 78, 80, 81, 83, 84, 86, 88, 89, 90, 91, 92, 93, 94, 95, 97, 98, 99, 100, 101, 102, 103, 104, 107, 108, 109, 110, 113, 114, 115, 116, 117, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 131, 132, 134, 135, 137, 138, 139, 141, 143, 144, 145, 147, 148, 149, 150, 151, 153, 154, 156, 157, 158, 160, 164, 166, 167], "canaccessp": 1, "canada": 29, "canadian": 29, "cancel": [0, 99, 120, 143, 148], "cancelrequest": [0, 99], "candid": [0, 12, 13, 18, 91, 92, 102, 107, 109, 113, 143], "canenqueu": 0, "canenqueuerequest": 0, "cannot": [1, 8, 10, 11, 12, 14, 15, 16, 19, 20, 80, 88, 89, 92, 102, 113, 114, 126, 127, 128, 129, 135, 139, 143, 148, 160, 167], "cap": [21, 121], "capabl": [3, 11, 12, 15, 16, 36, 51, 75, 93, 94, 119, 123, 143, 148, 154], "capac": [0, 1, 3, 5, 7, 15, 18, 19, 143, 148, 167], "capacitor_schedul": 167, "capacity_scheduler_polici": [128, 143], "capacityschedul": [150, 166, 167], "capacityschedulerpolici": [0, 93, 128, 143, 148], "capit": [26, 29, 47, 48, 49, 52, 53, 54, 55, 63, 65, 85, 97, 121, 128, 140, 142, 143, 147, 162], "caption": 130, "captur": [13, 14, 15, 35, 72, 79, 92, 93, 143, 151, 159], "capturableguideddecod": 10, "capture_num_token": 143, "car": 29, "card": [62, 74, 153], "care": [11, 15, 16, 75, 81, 154], "carefulli": [2, 15, 18], "carlo": 11, "case": [0, 1, 2, 4, 7, 10, 11, 12, 13, 14, 15, 16, 19, 22, 23, 25, 29, 34, 37, 38, 42, 72, 81, 88, 92, 98, 101, 102, 104, 105, 106, 109, 120, 121, 123, 124, 125, 127, 129, 137, 142, 143, 148], "casefold": 143, "caseless": 143, "cast": [10, 14, 129], "cast_to_dtyp": 129, "castsiz": 1, "cat": [2, 9, 13, 15, 19, 20, 25, 26, 28, 29, 30, 31, 60, 75, 89, 154], "catalog": [25, 28, 29, 30, 31, 94, 95], "catch": 19, "categor": [14, 90, 109, 129], "categori": [32, 132], "categorical_sampl": 129, "caus": [8, 10, 15, 19, 22, 28, 81, 98, 99, 114, 115, 127, 139, 142, 143, 148], "causal": [13, 72, 129, 130, 151], "cautiou": 115, "caveat": [83, 123], "cd": [2, 13, 17, 74, 78, 90, 94, 111, 112, 120, 139, 153, 158, 164], "ceil": [1, 131], "ceil_mod": [129, 130], "ceildiv": 1, "center": [4, 5, 16, 20, 81, 143], "centr": 29, "central": [80, 132, 160], "certain": [8, 15, 16, 35, 83, 89, 103, 112, 129], "certainti": 11, "certainty_threshold": 11, "cg": 131, "cga": 148, "chain": [10, 11, 13, 58], "challeng": [8, 12, 15, 16, 19, 81, 84, 89, 93], "chanc": [15, 22, 105, 128], "chang": [2, 3, 5, 6, 10, 13, 14, 15, 33, 37, 65, 72, 78, 81, 83, 84, 89, 93, 94, 98, 101, 102, 104, 105, 106, 114, 115, 120, 127, 129, 131, 134, 135, 139, 140, 141, 142, 143, 158, 166], "channel": [15, 20, 22, 129, 137, 148], "chapter": 11, "char": [0, 1, 143], "charact": [33, 143], "characterist": [8, 16, 81], "charg": [72, 102, 113, 151], "chart": [4, 18], "chat": [5, 16, 17, 18, 20, 23, 25, 29, 35, 41, 45, 47, 48, 49, 51, 52, 53, 54, 55, 58, 62, 63, 64, 68, 70, 71, 74, 75, 78, 79, 81, 86, 93, 97, 109, 140, 142, 147, 148, 153, 154, 156, 158, 159], "chat_complet": 29, "chat_templ": 26, "chat_template_kwarg": 70, "chatbot": 62, "chatbot_lora_dir": 62, "chatcmpl": [17, 20, 29, 147], "chatglm": [129, 137, 138, 148], "chatglm2": [138, 148], "chatglm3": [131, 138, 148], "chatglm_vers": 131, "chatglmconfig": 131, "chatglmforcausallm": 131, "chatglmgenerationsess": 134, "chatglmmodel": 131, "chatgpt": 20, "cheap": 92, "check": [10, 11, 15, 17, 20, 28, 29, 30, 31, 32, 34, 35, 38, 47, 56, 57, 81, 87, 93, 94, 97, 98, 99, 122, 123, 126, 127, 129, 134, 135, 139, 143, 144, 147, 148, 149, 163], "check_accuraci": [23, 112], "check_config": 131, "check_eagle_choic": 143, "check_gpt_mem_usag": 135, "checkbeamsearchdiversityr": 0, "checkbeamwidth": 0, "checkbeamwidtharrai": 0, "checkearlystop": 0, "checklengthpenalti": 0, "checkminp": 0, "checkmintoken": 0, "checknorepeatngrams": 0, "checknumreturnsequ": 0, "checkpoint": [2, 12, 13, 14, 17, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 37, 47, 52, 74, 78, 79, 85, 90, 92, 93, 97, 111, 114, 115, 120, 121, 123, 133, 134, 137, 139, 140, 142, 143, 144, 146, 147, 148, 149, 153, 158, 159, 162], "checkpoint_dir": [22, 80, 106, 110, 111, 112, 113, 115, 120, 139, 160], "checkpoint_format": 143, "checkpoint_load": [80, 143, 160], "checkposteriorvalu": 0, "checkremotedesc": 0, "checkrepetitionpenalti": 0, "checktemperatur": 0, "checktopk": 0, "checktopp": 0, "checktoppdecai": 0, "checktoppmin": 0, "checktoppresetid": 0, "chef": 139, "chen": 11, "china": 70, "chip": 56, "chmod": [28, 29, 30, 31, 116], "choic": [0, 7, 13, 15, 17, 18, 20, 22, 23, 28, 29, 30, 31, 63, 64, 65, 70, 72, 109, 120, 122, 129, 134, 147, 151], "choos": [12, 14, 15, 16, 28, 29, 30, 31, 74, 77, 113, 115, 123, 129, 148, 153, 157], "chose": [15, 19], "chosen": [14, 91, 135, 165, 167], "chri": 56, "chrome": [36, 119], "chrono": 0, "chunk": [0, 11, 14, 21, 22, 56, 82, 86, 93, 104, 118, 127, 129, 134, 135, 136, 143, 145, 146, 148, 161], "chunk_dim": 130, "chunk_length": 148, "chunk_scan": 129, "chunk_siz": [11, 129, 131], "chunkedcontextnexttoken": 1, "chunkedcontextnexttokenshost": 1, "ci": [1, 8, 34, 93, 148], "circular": [72, 84, 101], "citi": [29, 70, 85, 147, 162], "ckpt": [37, 120], "ckpt_dir": [113, 115, 131], "ckpt_llama_3": 113, "ckpt_sourc": 143, "cl": [111, 115, 143], "claim": [1, 114], "claimpag": 1, "claimpageswithevict": 1, "clamp": [143, 148], "clamp_val": 143, "clara": [20, 56], "class": [0, 1, 7, 10, 11, 22, 25, 28, 29, 30, 31, 35, 42, 56, 58, 72, 74, 75, 77, 80, 83, 91, 92, 93, 94, 101, 102, 103, 104, 110, 111, 113, 114, 115, 122, 123, 127, 129, 130, 131, 132, 133, 134, 139, 142, 143, 144, 148, 149, 150, 151, 153, 154, 157, 160, 167], "class_dropout_prob": 130, "class_label": 130, "classic": [15, 20, 113], "classifi": [32, 130, 131], "classmethod": [111, 115, 130, 131, 134, 143], "classvar": 143, "clean": [2, 10, 15, 36, 80, 94, 119, 139, 160], "cleanup": [56, 80, 160], "clear": [8, 15, 20, 32, 126, 134], "clear_logprob_param": 143, "clearer": 23, "clearli": [8, 10, 11, 15, 128], "clearvirtualmemoryalloc": 1, "cli": [2, 9, 20, 37, 42, 112, 120, 122, 123, 126, 127, 147, 148, 156], "click": [56, 64, 65, 79, 116, 117, 159], "client": [0, 16, 19, 20, 26, 28, 29, 30, 31, 38, 71, 81, 83, 86, 93, 99, 109], "client_id": [58, 91, 143], "clientid": 0, "clip": 129, "clip_before_cast": 129, "clip_qkv": [130, 131], "clip_vision_model": 131, "clipvisiontransform": 131, "clock": 12, "clone": [2, 11, 90, 94, 106, 121, 139, 142, 164], "clone_input": 103, "close": [2, 10, 15, 16, 19, 22, 72, 101, 115, 127, 135], "closer": 8, "closur": 129, "cloud": [4, 20, 44, 116, 117], "cls_token": 130, "cluster": [12, 15, 17, 19, 20, 21, 22, 26, 44, 89, 97, 102, 113, 143, 148], "cluster_info": 148, "cluster_kei": [22, 148], "cluster_s": [21, 26], "clusteruuid": [81, 98], "cmake": [94, 148], "cmpl": [28, 30, 31], "cn": 8, "cnn": 23, "cnn_dailymail": [131, 143], "co": [0, 2, 8, 9, 10, 13, 14, 15, 40, 67, 106, 129, 130, 139, 142], "coast": [29, 147], "code": [7, 10, 11, 12, 14, 15, 17, 19, 20, 21, 25, 26, 28, 29, 30, 31, 32, 34, 35, 36, 42, 48, 59, 60, 61, 72, 78, 79, 83, 92, 93, 96, 101, 103, 104, 107, 108, 109, 113, 115, 119, 120, 129, 137, 138, 139, 142, 143, 144, 146, 148, 149, 156, 158, 159, 166, 167], "codebas": [32, 104, 144, 149], "codec": 143, "codegemma": [79, 159], "codellama": [79, 148, 159], "codepath": 148, "codeqwen": 148, "coderham": 148, "codestr": [79, 159], "cognit": 84, "cogvlm": [138, 148], "cogvlmattent": 130, "cogvlmconfig": 131, "cogvlmforcausallm": 131, "coher": [102, 148], "cohereconfig": 131, "cohereforcausallm": 131, "cold": 15, "collabor": [8, 10, 11, 12, 14, 15, 16, 19, 102, 129], "collect": [1, 8, 12, 14, 16, 19, 32, 38, 81, 103, 107, 109, 113, 129, 143, 144, 149], "collect_and_bia": 130, "collector": 15, "collis": 86, "color": [62, 88, 126], "column": [106, 129, 137], "columnlinear": [106, 111, 130], "com": [2, 9, 12, 17, 26, 28, 29, 30, 31, 33, 34, 58, 90, 94, 113, 115, 129, 139, 148, 164], "combin": [0, 5, 8, 11, 12, 13, 14, 15, 16, 17, 22, 25, 28, 29, 30, 31, 34, 38, 59, 60, 61, 64, 72, 75, 83, 86, 89, 92, 93, 103, 109, 120, 123, 126, 130, 131, 141, 143, 148, 151, 154, 167], "combinedtimesteplabelembed": 130, "combinedtimesteptextprojembed": 130, "come": [4, 10, 15, 16, 18, 20, 81, 88, 102, 106, 121, 122, 126, 128, 135, 139, 143], "comm": 143, "comma": [129, 134], "command": [2, 9, 16, 17, 20, 21, 23, 25, 26, 28, 29, 30, 31, 33, 34, 36, 37, 56, 59, 60, 61, 62, 64, 65, 73, 75, 85, 86, 94, 95, 105, 106, 108, 111, 112, 113, 115, 116, 117, 119, 120, 121, 127, 132, 135, 139, 142, 147, 148, 152, 154, 162, 164], "commandr": 148, "comment": [33, 148], "commerci": [30, 31], "commit": [2, 14, 17, 19, 33, 34, 148], "commmod": 0, "common": [0, 2, 14, 15, 18, 19, 21, 25, 34, 47, 72, 74, 88, 93, 101, 104, 105, 109, 129, 135, 142, 143, 153, 166], "commonli": [8, 11, 12, 26, 103, 148], "commstat": 0, "commtyp": 0, "commun": [0, 8, 10, 11, 14, 16, 20, 22, 30, 31, 78, 81, 93, 98, 102, 107, 113, 123, 129, 138, 143, 148, 158], "communicationmod": 0, "communicationtyp": 0, "compact": 143, "compani": [17, 56, 58, 81], "compar": [1, 2, 4, 5, 7, 10, 11, 13, 14, 15, 16, 18, 19, 20, 23, 72, 86, 89, 114, 123, 126, 127, 128, 129, 143, 151], "comparison": [4, 12, 13, 37, 102, 120, 143], "compat": [9, 13, 15, 16, 17, 19, 20, 21, 25, 26, 28, 29, 30, 31, 79, 80, 81, 92, 94, 109, 115, 127, 130, 138, 142, 144, 147, 148, 149, 156, 159, 160], "compati": 25, "compatibal": 25, "compbin": 106, "compel": 8, "compelet": 29, "compet": [10, 15, 19], "competit": 20, "compil": [10, 17, 20, 28, 29, 30, 31, 36, 37, 73, 74, 75, 77, 78, 102, 107, 108, 119, 120, 129, 139, 143, 148, 152, 153, 154, 156, 157, 158], "compile_backend": [73, 75, 77, 79, 152, 154, 156, 157, 159], "complementari": 8, "complet": [0, 1, 8, 10, 11, 15, 17, 19, 20, 25, 28, 29, 30, 31, 32, 35, 37, 38, 39, 40, 42, 56, 64, 66, 67, 71, 74, 79, 80, 81, 85, 87, 88, 89, 92, 93, 94, 99, 102, 104, 105, 109, 120, 121, 126, 127, 143, 147, 148, 153, 156, 159, 160, 162, 163, 166, 167], "complete_sent": 58, "completion_token": [17, 20, 28, 29, 30, 31, 147], "completionoutput": [42, 93, 143], "complex": [8, 11, 12, 15, 58, 75, 93, 96, 103, 104, 109, 113, 154], "compli": 26, "complic": [10, 13, 14, 15, 92, 144, 149], "compon": [7, 8, 10, 11, 12, 13, 14, 15, 21, 32, 35, 72, 90, 92, 93, 99, 101, 113, 137, 150], "compos": [0, 11, 15, 19, 93, 102, 120], "comprehens": [2, 8, 21, 26, 32, 73, 147, 152], "compress": [3, 14, 80, 160], "compris": [7, 8, 16], "comput": [0, 1, 3, 4, 5, 7, 8, 12, 13, 14, 15, 16, 17, 19, 20, 22, 35, 36, 37, 47, 52, 53, 55, 56, 57, 58, 72, 78, 81, 83, 84, 87, 88, 89, 90, 93, 94, 97, 100, 101, 102, 105, 109, 113, 119, 120, 122, 123, 128, 129, 135, 139, 140, 142, 143, 144, 147, 148, 149, 150, 151, 158, 163, 166], "computation": 8, "compute_relative_bia": 130, "computecontextlogit": 1, "computed_block": 56, "computed_posit": 56, "computegenerationlogit": 1, "computenumpackedmask": 1, "concat": [12, 111, 129], "concat_kvcach": 12, "concat_qkv": 19, "concaten": [12, 72, 101, 106, 114, 129, 143, 144, 149], "concept": [11, 16, 37, 89, 113, 120, 124, 125, 143, 148, 166], "conceptu": 1, "concern": [15, 113, 135], "concert": 8, "concis": 20, "conclud": [15, 19], "conclus": [18, 93, 124, 125], "concret": [15, 92, 143, 144, 149], "concurr": [1, 2, 4, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 25, 28, 29, 30, 31, 35, 37, 81, 84, 85, 86, 87, 98, 109, 120, 148, 162, 163], "concurrency_": [25, 28, 29, 30, 31], "concurrency_list": [25, 28, 29, 30, 31], "cond_proj_dim": 130, "conda": 148, "condit": [0, 1, 8, 11, 15, 16, 33, 37, 81, 99, 102, 103, 109, 120, 129, 130, 148], "condition": [34, 129], "conditioning_embed": 130, "conditioning_embedding_dim": 130, "conduct": [16, 19, 37, 81, 98, 101, 120], "confid": 11, "config": [0, 1, 2, 3, 9, 13, 14, 17, 19, 20, 21, 23, 25, 26, 37, 43, 49, 72, 75, 81, 83, 84, 92, 101, 105, 106, 110, 111, 114, 115, 120, 126, 130, 131, 132, 134, 139, 143, 144, 148, 149, 154, 166], "config_cl": 143, "config_class": 131, "config_dir": 131, "config_fil": [26, 131, 143], "config_load": [80, 160], "configdict": 143, "configur": [0, 1, 2, 5, 16, 17, 18, 19, 21, 22, 25, 26, 32, 33, 37, 38, 50, 51, 57, 62, 72, 74, 77, 78, 79, 80, 81, 87, 88, 89, 91, 92, 93, 94, 97, 100, 101, 104, 108, 109, 114, 120, 121, 124, 125, 126, 128, 131, 134, 135, 139, 143, 148, 151, 153, 157, 158, 159, 160, 163], "configuration_llama": [144, 149], "configuration_mymodel": [144, 149], "configuration_util": [144, 149], "configuratorptr": 1, "confirm": [47, 57, 97, 140, 142, 147], "conform": [10, 91, 143], "congest": 15, "conjunct": 126, "connect": [0, 10, 15, 17, 19, 20, 28, 29, 30, 31, 81, 98, 107, 113, 121, 122, 124, 125], "connecticut": 29, "connectioninfo": 0, "connectioninfotyp": 0, "connectionmanag": 0, "connector": [50, 143], "connector_cach": 56, "connector_cache_dir": 56, "connector_cache_fold": 56, "connector_cache_folder_kei": 56, "connector_modul": 56, "connector_scheduler_class": 56, "connector_worker_class": 56, "connectremoteag": 0, "consecut": 102, "consequ": [7, 122, 127], "conserv": [0, 33, 128], "consid": [0, 1, 2, 7, 8, 11, 15, 16, 17, 32, 38, 48, 49, 62, 64, 88, 91, 106, 109, 126, 129, 143, 144, 149, 167], "consider": [7, 15, 16, 19, 42, 81, 115], "consist": [4, 8, 11, 12, 19, 32, 37, 72, 103, 115, 120, 121, 129, 137, 139, 143, 148, 151], "consol": 116, "consolid": [15, 109], "const": [0, 1, 99], "const_iter": 1, "constant": [1, 15, 18, 19, 72, 101, 129, 135], "constant_to_tensor_": 129, "constantli": [47, 52, 53, 55, 97, 140, 142, 147], "constants_to_tensors_": 129, "constantthreshold": 1, "constexpr": [0, 1], "constitut": [16, 32], "constpointercast": 1, "constrain": [7, 10, 57, 89, 102, 148], "constraint": [0, 7, 10, 15, 16, 72, 88, 89, 91, 101, 102, 129, 146], "construct": [0, 1, 11, 16, 18, 72, 77, 91, 92, 99, 109, 113, 120, 129, 143, 148, 151, 157], "constructor": [0, 62, 72, 93, 110, 142, 143, 147, 151], "consult": [36, 94, 109, 119], "consum": [0, 14, 15, 19, 33, 103, 129, 143], "consumpt": [4, 13, 22, 72, 84, 101], "contact": 129, "contain": [0, 1, 10, 11, 12, 15, 17, 19, 21, 22, 26, 33, 37, 38, 57, 59, 60, 61, 72, 78, 79, 80, 81, 83, 88, 93, 96, 97, 99, 101, 102, 103, 104, 106, 107, 112, 113, 114, 115, 117, 120, 129, 131, 134, 137, 138, 141, 143, 147, 148, 150, 151, 158, 159, 160], "container_id": [17, 147], "container_imag": [59, 60, 61], "container_img": 26, "container_path": [28, 29, 31], "content": [1, 9, 17, 20, 23, 26, 28, 29, 30, 31, 34, 39, 40, 41, 51, 56, 66, 67, 70, 81, 93, 98, 106, 115, 116, 129, 135, 143, 147, 148, 156], "content_typ": 143, "context": [0, 7, 10, 11, 13, 14, 15, 16, 18, 19, 22, 29, 81, 91, 93, 98, 100, 105, 118, 120, 124, 125, 129, 134, 135, 139, 143, 146, 148, 151, 165, 166, 167], "context_and_gener": 143, "context_chunking_polici": [128, 143], "context_extra": 81, "context_fmha": [22, 88, 106], "context_fmha_fp32_acc": 148, "context_fmha_typ": [101, 135], "context_init": 167, "context_len": [72, 134, 151], "context_length": [129, 130, 134, 139], "context_logit": [28, 30, 31, 134, 143], "context_mem_s": 134, "context_onli": 143, "context_parallel_s": 143, "context_phas": [72, 101], "context_pre_onli": 130, "context_request": 167, "context_serv": [16, 81], "contextchunkingpolici": [0, 93, 128, 143, 148], "contextfmha": 1, "contextidx": 0, "contextlogit": 0, "contextmanag": 142, "contextparallel": [0, 1], "contextphaseparam": [0, 143], "contextpositionid": 1, "contextprefillposit": 0, "contextrequest": 1, "contigu": [81, 98, 104, 122, 129, 148], "continu": [1, 5, 7, 11, 16, 19, 22, 35, 56, 72, 78, 88, 91, 93, 94, 99, 101, 109, 123, 126, 134, 143, 147, 158, 167], "contract": [37, 120], "contrast": [72, 102, 109, 151], "contrib": [3, 148], "contribut": [8, 10, 11, 13, 14, 15, 37, 78, 115, 120, 129, 148, 158], "contributor": [8, 12, 15, 16, 135], "control": [0, 8, 17, 18, 19, 20, 25, 28, 29, 30, 31, 42, 50, 64, 72, 73, 81, 88, 91, 93, 98, 101, 102, 103, 108, 119, 120, 121, 128, 129, 130, 134, 137, 143, 148, 152], "contronl": 11, "conv": 129, "conv1d": [22, 129, 130], "conv2d": [129, 130], "conv3d": [129, 130], "conv_bia": 129, "conv_kernel": 134, "conv_stat": 131, "conv_state_or_ptr": 129, "conv_transpose2d": 129, "conv_weight": 129, "conveni": [1, 11, 94, 111, 115], "convent": [8, 80, 115, 129, 160], "converg": 19, "convers": [1, 6, 7, 16, 19, 62, 81, 84, 114, 146, 148], "convert": [0, 1, 15, 19, 26, 37, 58, 72, 78, 80, 90, 106, 110, 111, 112, 113, 114, 115, 120, 121, 123, 139, 143, 148, 151, 158, 160], "convert_and_load_weights_into_trtllm_llama": 115, "convert_checkpoint": [100, 106, 110, 111, 112, 113, 115, 121, 122, 139, 148], "convert_hf_mpt_legaci": 148, "convert_load_format": 143, "convert_util": 148, "convert_weights_from_custom_training_checkpoint": 115, "convkernel": 1, "convolut": [0, 134], "convtranspose2d": 130, "cooper": 93, "coordin": [15, 19, 29, 35, 93, 109, 129], "copi": [0, 1, 10, 11, 15, 22, 56, 72, 81, 83, 98, 105, 109, 117, 123, 129, 135, 143, 148, 151], "copy_": 56, "copy_on_partial_reus": [83, 143], "copyfrom": 1, "copyonpartialreus": 0, "copytask": 1, "copytaskmappag": 1, "copyto": 0, "copytocpu": 0, "copytogpu": 0, "copytomanag": 0, "copytopag": 1, "copytopin": 0, "copytopooledpin": 0, "core": [3, 4, 6, 8, 10, 14, 19, 20, 23, 35, 38, 75, 78, 79, 83, 93, 94, 102, 103, 106, 110, 113, 115, 120, 122, 139, 143, 148, 150, 154, 158, 159], "corner": [14, 29], "coroutin": [53, 54, 143], "corpor": 56, "correct": [10, 11, 13, 18, 19, 72, 78, 99, 101, 106, 109, 148, 158], "correctli": [10, 81, 98, 105, 129, 144, 148, 149], "correl": 18, "correspond": [0, 1, 8, 10, 13, 15, 16, 18, 26, 36, 72, 89, 92, 95, 100, 101, 103, 104, 106, 109, 114, 115, 119, 127, 129, 130, 134, 137, 139, 143, 144, 148, 149], "correspondingli": 15, "corridor": 29, "corrupt": 15, "cost": [8, 12, 13, 14, 15, 18, 20, 37, 58, 90, 105, 113, 120, 122, 135, 148], "costli": 12, "cot": [93, 148], "could": [0, 10, 15, 19, 20, 47, 52, 53, 54, 55, 81, 92, 94, 97, 103, 104, 105, 112, 121, 135, 139, 140, 142, 143, 147, 148], "couldn": [88, 126], "count": [0, 1, 8, 15, 19, 38, 45, 46, 58, 89, 92, 102, 120, 142, 143], "count_include_pad": [129, 130], "countlocallay": 1, "countlowerranklay": 1, "coupl": 10, "cours": 109, "court": [47, 97, 140, 142, 147], "cover": [2, 8, 15, 20, 25, 28, 29, 30, 31, 33, 123, 124, 125, 127], "coverag": [10, 15], "cp312": 94, "cp_config": 143, "cp_group": [129, 130], "cp_rank": [129, 130], "cp_size": [129, 130, 133, 148], "cp_split_plugin": 129, "cpp": [2, 14, 36, 37, 38, 60, 72, 94, 99, 101, 102, 108, 113, 119, 120, 121, 139, 148], "cpp_e2e": 134, "cpp_llm_onli": 134, "cpp_onli": 94, "cpu": [0, 1, 12, 13, 15, 18, 20, 21, 22, 26, 35, 56, 57, 72, 83, 85, 86, 87, 93, 104, 105, 106, 110, 113, 129, 135, 139, 143, 148, 151, 162, 163], "cpu_tensor": 56, "cpumemusag": [0, 143], "crash": 148, "crd": 44, "creat": [1, 8, 10, 11, 12, 17, 18, 19, 25, 26, 34, 37, 38, 42, 44, 47, 48, 52, 53, 54, 55, 58, 66, 67, 68, 69, 70, 72, 75, 81, 83, 85, 88, 89, 91, 92, 93, 94, 97, 98, 99, 103, 104, 105, 109, 110, 111, 113, 115, 116, 120, 121, 126, 127, 129, 130, 131, 134, 135, 140, 142, 143, 144, 147, 148, 149, 150, 151, 154, 162, 167], "create_allreduce_plugin": 129, "create_attention_const_param": 130, "create_builder_config": 110, "create_cuda_graph_metadata": [72, 92, 151], "create_execution_context": 134, "create_fake_weight": 129, "create_from_prompt": 11, "create_network": 113, "create_pytorch_model_based_executor": [166, 167], "create_runtime_default": 131, "create_scaffolding_output": 11, "create_sinusoidal_posit": 129, "create_sinusoidal_positions_for_attention_plugin": 129, "create_sinusoidal_positions_for_cogvlm_attention_plugin": 129, "create_sinusoidal_positions_long_rop": 129, "create_sinusoidal_positions_long_rope_for_attention_plugin": 129, "create_sinusoidal_positions_yarn": 129, "createloramodul": 1, "creation": [1, 35, 73, 129, 135, 143, 152], "creativ": [64, 102], "creator": [1, 143], "creatorptr": 1, "credenti": 9, "criteria": [35, 87, 134, 163], "critic": [8, 12, 15, 16, 35, 37, 120, 139], "crop": 130, "cropped_pos_emb": 130, "cross": [0, 10, 11, 12, 13, 15, 29, 106, 107, 129, 134, 143, 148], "cross_attent": [130, 134], "cross_attention_dim": 130, "cross_attention_mask": [130, 134], "cross_attention_mask_for_context": 134, "cross_attention_mask_for_gen": 134, "cross_attention_norm": 130, "cross_attention_norm_num_group": 130, "cross_attention_packed_mask": 130, "cross_attn_dens": [22, 106], "cross_attn_k": [22, 106], "cross_attn_q": [22, 106], "cross_attn_qkv": [22, 106], "cross_attn_v": [22, 106], "cross_kv": 129, "cross_kv_cache_block_offset": [130, 134], "cross_kv_cache_fract": [134, 143], "cross_kv_cache_gen": [130, 131], "cross_kv_length": 129, "cross_kv_reus": [130, 131], "crossattentionmask": 0, "crosskvcachefract": [0, 148], "crosskvcachestat": 0, "crucial": [7, 8, 10, 57, 109, 113, 150], "csv": 21, "cta": 19, "ctor": 129, "ctrl": 25, "ctx": [0, 2, 8, 16], "ctx1dep4": 16, "ctx_len": 8, "ctx_param": [16, 81], "ctx_request_id": 143, "ctxenginepath": 0, "ctxexecutorconfig": 0, "ctxreqrat": 16, "cu": [12, 113], "cu12": 148, "cu128": 97, "cuassert": 139, "cubin": 148, "cubla": [14, 20], "cublaslt": [22, 127], "cublasltmatmul": 14, "cublasscaledmm": 14, "cuda": [0, 1, 2, 13, 14, 15, 17, 19, 20, 21, 25, 28, 29, 30, 31, 36, 37, 56, 58, 63, 72, 79, 81, 82, 91, 92, 93, 94, 97, 98, 101, 107, 113, 119, 120, 131, 134, 135, 136, 139, 143, 145, 148, 151, 156, 159, 161, 166], "cuda_arch": 94, "cuda_architectur": [2, 94, 108], "cuda_graph": 63, "cuda_graph_batch_s": [25, 28, 29, 30, 31, 73, 143, 148, 152, 156], "cuda_graph_cache_s": 143, "cuda_graph_config": [2, 9, 13, 14, 20, 38, 56, 60, 63, 91, 143, 148], "cuda_graph_inst": 139, "cuda_graph_mod": [134, 139, 143], "cuda_hom": 97, "cuda_launch_block": 139, "cuda_stream": 139, "cuda_stream_guard": 134, "cuda_stream_sync": 129, "cuda_visible_devic": 81, "cudadevicegetstreampriorityrang": 1, "cudaevent_t": 1, "cudaeventdisabletim": 1, "cudagraph": [10, 73, 78, 79, 148, 152, 156, 158, 159], "cudagraphcaches": 0, "cudagraphconfig": [63, 93, 143], "cudagraphlaunch": [19, 139], "cudagraphmod": 0, "cudagraphrunn": 92, "cudahostregist": 19, "cudalaunchhostfunc": 10, "cudamalloc": [1, 19, 81, 98], "cudamallocasync": [1, 81, 98], "cudamallocmanag": 19, "cudamemadvis": 19, "cudamempool": 1, "cudamempoolptr": 1, "cudaprofilerapi": [36, 119], "cudart": 139, "cudastream": 0, "cudastream_t": 1, "cudastreamcreatewithflag": 1, "cudastreamnonblock": 1, "cudastreamptr": [0, 1], "cudavirtualmemori": 1, "cudavirtualmemoryalloc": 1, "cudavirtualmemorychunk": 1, "cudavirtualmemorymanag": 1, "cudevic": 1, "cudeviceptr": 1, "cudnn": [20, 148], "cufil": 0, "cuh": 12, "cumemaccessdesc": 1, "cumemallocationprop": 1, "cumemcr": 1, "cumemgenericallocationhandl": 1, "cumemimportfromshareablehandl": [81, 98], "cumlogprob": [0, 1], "cumlogprobscba": 1, "cumsum": [129, 148], "cumsumgenerationlength": 1, "cumsumlastdim": 129, "cumsumlength": 1, "cumul": [0, 1, 8, 64, 129, 143], "cumulative_logprob": [42, 143], "cupi": 10, "curand": 148, "curl": [9, 17, 20, 26, 28, 29, 30, 31, 71, 81, 86, 93, 147, 156], "currenc": [37, 120], "current": [0, 1, 2, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 22, 23, 26, 34, 35, 37, 42, 51, 62, 72, 75, 78, 80, 81, 83, 84, 87, 88, 91, 92, 94, 95, 98, 99, 101, 106, 109, 120, 123, 126, 127, 128, 129, 134, 135, 141, 143, 148, 150, 151, 154, 158, 160, 166, 167], "current_image_tag": 34, "current_prompt": 11, "current_stream": 139, "currentexpandindic": 1, "curti": 56, "curv": [6, 10, 16, 19], "custom": [3, 10, 11, 12, 13, 15, 16, 21, 22, 34, 44, 47, 58, 64, 72, 75, 77, 78, 79, 81, 91, 92, 93, 94, 102, 113, 115, 123, 127, 129, 134, 142, 143, 146, 148, 150, 151, 154, 157, 158, 159], "custom_all_reduc": 148, "custom_format": [80, 160], "custom_mask": 129, "custom_module_dir": 21, "customallreduc": 148, "customcheckpointload": [80, 160], "customconfigload": [80, 160], "customdataset": 148, "customized_key_dict": 114, "customized_preprocess": 114, "customizedmodulea": 114, "customizedmoduleb": 114, "customweightload": [80, 160], "customweightmapp": [80, 160], "custream": 1, "cut": 20, "cutedsl": 143, "cutlass": [14, 20, 25, 28, 29, 30, 31, 108, 143, 148], "cutlass_kernel": 108, "cxx11": [94, 148], "cycl": [19, 57], "cyclic": [8, 84, 134, 148], "d": [1, 9, 17, 20, 28, 29, 30, 31, 37, 39, 40, 41, 59, 60, 61, 62, 81, 106, 108, 116, 120, 129, 130, 139, 147, 148, 156], "d0": 12, "d04e592bb4f6aa9cfee91e2e20afa771667e1d4b": 120, "d1": 92, "d2": 92, "d2h": 10, "d2t": 10, "d3": 92, "d_": 13, "d_6": 13, "dai": [78, 146, 158], "dailymail": 23, "dangl": 103, "data": [0, 1, 3, 4, 5, 6, 7, 8, 11, 12, 15, 16, 18, 19, 20, 21, 22, 23, 25, 26, 28, 30, 31, 34, 37, 38, 56, 58, 67, 72, 81, 82, 83, 86, 90, 98, 101, 102, 104, 107, 113, 114, 120, 121, 129, 131, 138, 139, 143, 144, 145, 148, 149, 161], "data_devic": 21, "data_path": 60, "data_ptr": 148, "data_typ": [110, 112], "databas": [80, 160], "dataclass": [32, 56], "datacontext": 0, "dataset": [10, 11, 12, 13, 14, 17, 19, 20, 23, 25, 28, 29, 30, 31, 36, 40, 60, 67, 73, 85, 89, 93, 119, 123, 143, 148, 152, 162], "dataset_fil": 38, "dataset_path": [23, 85, 120, 162], "datatyp": [0, 1, 102, 113, 129, 134, 137, 139], "datatypetrait": 1, "date": 115, "datetim": 143, "db": 33, "dbrx": [137, 138, 148], "dbrxconfig": 131, "dbrxforcausallm": 131, "dconv": 129, "de": 1, "deactiv": 42, "dead": 148, "deal": [72, 101, 103, 139], "dealloc": [1, 35, 104, 167], "death": [47, 97, 140, 142, 147], "debug": [0, 15, 19, 21, 22, 23, 26, 57, 76, 79, 83, 94, 104, 134, 135, 143, 148, 155, 159], "debug_buff": 139, "debug_mod": [134, 139], "debug_tensors_to_sav": 134, "debugconfig": 0, "debuginputtensor": 0, "debugoutputtensor": 0, "debugtensor": 0, "debugtensornam": 0, "debugtensorsmaxiter": 0, "debugtensorsperiter": 0, "dec": [22, 134, 148], "decai": [0, 102, 143], "decid": [15, 37, 48, 72, 90, 101, 112, 120, 124, 126, 137, 150, 167], "decilmforcausallm": [138, 145], "decim": 143, "decis": [8, 15, 19, 62, 129], "declar": [1, 102, 103, 115, 150, 166], "decltyp": [0, 1], "decod": [0, 1, 8, 12, 14, 15, 16, 17, 23, 26, 35, 50, 57, 64, 70, 72, 78, 81, 82, 84, 86, 87, 93, 98, 101, 102, 111, 115, 120, 129, 131, 134, 138, 140, 142, 143, 144, 145, 146, 148, 149, 158, 161, 163, 165, 166], "decode_batch": 134, "decode_duration_m": [49, 83, 143], "decode_regular": 134, "decode_retention_polici": 83, "decode_retention_prior": [49, 143], "decode_stream": 134, "decode_words_list": 134, "decode_wrapp": [72, 151], "decodedurationm": 0, "decoder_batch": 1, "decoder_input_id": [131, 134], "decoder_language_adapter_rout": 134, "decoder_lay": [144, 149], "decoder_start_token_id": 22, "decoderbuff": 1, "decoderenginebuff": 0, "decoderetentionprior": 0, "decoderjsonconfigstr": 0, "decoderlay": [144, 149], "decoderlayerlist": 111, "decoderlookaheadbuff": 1, "decodermaskedmultiheadattent": [72, 101], "decodermodel": [0, 131, 144, 149], "decodermodelforcausallm": [111, 115, 131, 144, 149], "decodermodelpath": 0, "decoderst": 148, "decoderxqarunn": [72, 101], "decoding_config": 143, "decoding_typ": [2, 9, 13, 17, 28, 92, 143], "decodingbaseconfig": 143, "decodingconfig": [0, 1], "decodinginputptr": 1, "decodingit": 0, "decodinglayerworkspac": 1, "decodingmod": [0, 1, 148], "decodingoutputptr": 1, "decompos": [15, 72, 88, 101], "decomposit": [85, 162], "decor": [10, 32, 143, 144, 149], "decoupl": [10, 11, 12, 15, 16, 81, 84, 89, 108, 135], "decreas": [2, 3, 4, 8, 10, 19, 76, 123, 155], "dedic": [8, 10, 12, 14, 15, 16, 19, 20, 35, 139], "deduc": [15, 22, 23, 26, 148], "deem": 11, "deep": [4, 5, 17, 20, 36, 93, 113, 119, 129, 143, 148], "deepcopi": 11, "deepep": [15, 148], "deeper": 13, "deepgemm": [2, 28, 143, 148], "deeplearn": [113, 129, 139], "deepli": 15, "deepseek": [8, 10, 11, 15, 19, 26, 27, 36, 65, 71, 79, 89, 90, 92, 93, 119, 138, 145, 146, 148, 159], "deepseek_model": 92, "deepseek_r1_output": 28, "deepseek_v1": 148, "deepseek_v2": 148, "deepseek_v3": [12, 148], "deepseekforcausallm": 131, "deepseekv1config": 131, "deepseekv2": 129, "deepseekv2attent": 130, "deepseekv2config": 131, "deepseekv2forcausallm": 131, "deepseekv3forcausallm": [138, 145], "deepseekv3routingimpl": 14, "deepspe": 112, "def": [10, 11, 32, 47, 48, 49, 51, 52, 53, 54, 55, 56, 57, 58, 62, 63, 64, 65, 80, 91, 97, 103, 111, 113, 114, 115, 121, 123, 127, 128, 139, 140, 142, 143, 144, 147, 149, 160, 167], "default": [0, 1, 10, 11, 13, 14, 18, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 42, 48, 49, 57, 58, 62, 63, 64, 65, 72, 73, 74, 76, 79, 80, 81, 83, 84, 87, 88, 89, 90, 91, 92, 94, 95, 98, 99, 100, 101, 102, 105, 108, 112, 114, 115, 116, 119, 123, 124, 125, 126, 127, 128, 129, 131, 134, 135, 137, 139, 143, 144, 148, 149, 151, 152, 153, 155, 159, 160, 163], "default_factori": 56, "default_net": 129, "default_plugin_config": 131, "default_record_cr": 143, "default_trtnet": 113, "defaultvalu": 1, "defer": [19, 129], "defin": [0, 1, 2, 5, 8, 11, 15, 21, 22, 32, 33, 34, 37, 57, 72, 75, 80, 81, 83, 88, 91, 92, 99, 101, 103, 109, 112, 113, 114, 115, 120, 127, 129, 130, 137, 143, 144, 146, 148, 149, 151, 154, 160], "definit": [11, 12, 44, 72, 78, 80, 93, 99, 101, 104, 115, 129, 139, 146, 148, 158, 160], "defrag": 19, "deftruth": 148, "degrad": [0, 17, 22, 123], "degre": [15, 34, 38, 47, 52, 53, 55, 97, 123, 127, 140, 142, 147], "del": 56, "delai": [8, 15, 16, 19, 25, 28, 29, 30, 31, 38, 81, 143, 148], "deleg": [72, 129, 151], "delet": [0, 1, 15, 132, 139, 143], "delimit": 143, "deliv": [2, 3, 6, 8, 12, 13, 18, 20, 25, 38, 89, 146], "delta": [0, 12, 13, 129, 130], "delta_bia": 129, "delta_softplu": 129, "delv": 14, "demand": [12, 14, 15, 16, 58, 81, 89], "demo": [11, 12, 40, 57, 64, 67, 78, 158], "demo_prompt": 64, "demollm": [73, 74, 79, 152, 153, 156, 159], "demonstr": [4, 8, 10, 11, 12, 15, 16, 19, 20, 35, 57, 63, 64, 77, 86, 91, 99, 114, 121, 123, 126, 127, 142, 157], "demonstrate_beam_search": 64, "demonstrate_combined_sampl": 64, "demonstrate_greedy_decod": 64, "demonstrate_multiple_sequ": 64, "demonstrate_temperature_sampl": 64, "demonstrate_top_k_sampl": 64, "demonstrate_top_p_sampl": 64, "demonstrate_with_logprob": 64, "denois": 130, "denot": 109, "dens": [29, 38, 72, 100, 101, 106, 112, 114, 129, 148], "dense_4h_to_h": 114, "dense_bia": 130, "dense_h_to_4h": 114, "densiti": [7, 20], "dep": 94, "dep4": [16, 29], "dep8": [16, 29], "depend": [0, 5, 10, 15, 16, 17, 19, 23, 26, 29, 38, 48, 72, 81, 84, 91, 94, 97, 99, 101, 102, 103, 108, 109, 112, 121, 123, 127, 129, 135, 139, 143, 148, 166], "deploi": [11, 15, 19, 26, 44, 78, 79, 89, 93, 109, 112, 158, 159], "deplot": [138, 148], "deploy": [7, 8, 12, 15, 16, 17, 18, 19, 20, 44, 75, 79, 81, 89, 120, 123, 141, 142, 146, 147, 148, 154, 159], "deprec": [22, 108, 120, 143, 148], "deprecatedparseprotocol": 143, "deprecationwarn": [32, 120], "depriorit": 108, "depriv": 103, "depth": [79, 109, 143, 159], "dequ": [0, 1], "dequant": [72, 101, 107, 129], "deregistermemori": 0, "deriv": [18, 92, 113, 114, 129, 135, 150], "desc": [0, 1], "descend": 83, "descendli": 102, "describ": [0, 2, 6, 15, 16, 20, 33, 37, 38, 40, 64, 67, 72, 83, 92, 94, 97, 101, 102, 104, 105, 106, 109, 111, 113, 114, 117, 120, 127, 129, 137, 139, 151], "descript": [0, 1, 21, 25, 26, 28, 29, 30, 31, 32, 37, 38, 57, 62, 63, 72, 73, 74, 79, 93, 102, 106, 120, 127, 129, 143, 151, 152, 153, 159], "descriptor": 143, "deseri": [0, 19, 115], "deserializeadditionalmodeloutput": 0, "deserializeadditionaloutput": 0, "deserializeagentst": 0, "deserializeblockkei": 0, "deserializebool": 0, "deserializecachest": 0, "deserializecachetransceiverconfig": 0, "deserializecommst": 0, "deserializecontextphaseparam": 0, "deserializedatatransceiverst": 0, "deserializedebugconfig": 0, "deserializedecodingconfig": 0, "deserializedecodingmod": 0, "deserializedisservingrequeststat": 0, "deserializedynamicbatchconfig": 0, "deserializeeagleconfig": 0, "deserializeexecutorconfig": 0, "deserializeextendedruntimeperfknobconfig": 0, "deserializeexternaldrafttokensconfig": 0, "deserializeguideddecodingconfig": 0, "deserializeguideddecodingparam": 0, "deserializeinflightbatchingstat": 0, "deserializeiterationstat": 0, "deserializeiterationstatsvec": 0, "deserializekvcacheconfig": 0, "deserializekvcachecreateddata": 0, "deserializekvcacheev": 0, "deserializekvcacheeventdiff": 0, "deserializekvcacheremoveddata": 0, "deserializekvcacheretentionconfig": 0, "deserializekvcachestat": 0, "deserializekvcachestoredblockdata": 0, "deserializekvcachestoreddata": 0, "deserializekvcacheupdateddata": 0, "deserializelookaheaddecodingconfig": 0, "deserializeloraconfig": 0, "deserializemodeltyp": 0, "deserializemropeconfig": 0, "deserializemultimodalinput": 0, "deserializeorchestratorconfig": 0, "deserializeoutputconfig": 0, "deserializeparallelconfig": 0, "deserializepeftcacheconfig": 0, "deserializeprompttuningconfig": 0, "deserializerequest": 0, "deserializerequestperfmetr": 0, "deserializerequeststag": 0, "deserializerequeststat": 0, "deserializerequeststatsperiter": 0, "deserializerequeststatsperiterationvec": 0, "deserializerespons": 0, "deserializeresult": 0, "deserializesamplingconfig": 0, "deserializeschedulerconfig": 0, "deserializesocketst": 0, "deserializespecdecfastlogitsinfo": 0, "deserializespecdecodingstat": 0, "deserializespeculativedecodingconfig": 0, "deserializestaticbatchingstat": 0, "deserializestr": 0, "deserializetensor": 0, "deserializetimepoint": 0, "deserializetokenrangeretentionconfig": 0, "deserializeuniquetoken": 0, "design": [1, 2, 7, 8, 10, 11, 12, 13, 14, 16, 17, 18, 19, 21, 35, 57, 72, 78, 79, 80, 81, 83, 89, 93, 107, 109, 113, 114, 115, 121, 142, 146, 147, 150, 151, 158, 159, 160, 166], "desir": [21, 34, 38, 72, 89, 91, 99, 129, 143, 151, 160], "desired_world_s": [77, 157], "despit": [8, 11], "destin": [59, 60, 61], "destroi": [1, 135], "destroyipcmemori": 1, "destruct": 1, "destructor": 1, "detach": 17, "detail": [0, 2, 8, 10, 11, 12, 14, 15, 16, 19, 20, 22, 23, 26, 28, 33, 34, 37, 38, 42, 44, 47, 51, 56, 72, 81, 83, 86, 89, 91, 99, 101, 107, 109, 111, 113, 120, 121, 123, 128, 129, 131, 135, 139, 143, 147, 148, 150, 151, 166], "detect": [0, 15, 19, 21, 23, 26, 34, 99, 129, 143, 148], "detect_format": 114, "determin": [0, 1, 11, 13, 15, 16, 19, 35, 72, 78, 79, 81, 83, 84, 88, 92, 101, 102, 106, 115, 122, 123, 128, 129, 131, 137, 143, 150, 158, 159, 166, 167], "determinenumpag": 1, "determinist": [64, 127, 143, 148], "detoken": [19, 20, 35, 143, 148, 150], "detokenizedgenerationresultbas": 143, "dev": [9, 15, 17, 20, 23, 28, 29, 30, 31, 78, 93, 97, 148, 158], "dev_container_imag": 34, "dev_trtllm_imag": 94, "devcontain": 34, "devel": [94, 116, 117], "develop": [8, 11, 12, 13, 15, 16, 17, 19, 20, 23, 28, 29, 30, 31, 32, 33, 34, 47, 52, 53, 55, 56, 78, 79, 81, 86, 94, 97, 111, 112, 113, 115, 116, 121, 129, 138, 140, 142, 144, 146, 147, 148, 149, 158, 159], "deviat": [15, 21, 38], "devic": [0, 1, 10, 15, 16, 19, 20, 21, 28, 29, 56, 58, 81, 98, 123, 129, 131, 133, 134, 139, 143], "device_cache_perc": [85, 162], "device_id": 134, "device_map": 133, "device_memory_size_v2": 135, "device_num_expert": 129, "device_request_typ": 131, "deviceallocationnvl": 1, "devicecach": 1, "devicecacheperc": 0, "deviceid": [0, 1, 98], "dgx": [2, 14, 20, 33, 38, 102, 113], "di": [13, 15, 16, 81], "diagnost": 1, "diagon": 129, "diagram": [11, 14, 16, 81, 109], "dial": 48, "diamond": [12, 14, 23], "dict": [32, 75, 80, 111, 114, 115, 129, 131, 134, 143, 144, 148, 149, 154, 160, 166], "dict_kei": 139, "dictat": 126, "dictionari": [18, 75, 80, 112, 114, 130, 143, 154, 160], "didn": [88, 126], "differ": [0, 1, 2, 7, 8, 11, 13, 14, 15, 16, 18, 19, 20, 21, 22, 37, 38, 40, 57, 64, 67, 72, 75, 79, 80, 81, 84, 85, 88, 89, 90, 91, 92, 94, 98, 100, 101, 102, 104, 105, 107, 111, 112, 113, 114, 115, 120, 121, 123, 126, 127, 129, 131, 134, 135, 137, 143, 146, 147, 148, 151, 154, 159, 160, 162, 165], "differenti": 129, "difficult": [8, 10], "difficulti": [11, 92], "difftyp": 1, "diffus": [40, 67, 130, 148], "diffusersattent": 130, "difi": 11, "digit": [18, 143], "dilat": [129, 130], "dim": [0, 1, 129, 130, 131, 134, 139], "dim0": 129, "dim1": 129, "dim_head": 130, "dim_in": 130, "dim_out": 130, "dim_rang": 129, "dimems": 1, "dimens": [0, 1, 14, 15, 25, 72, 74, 101, 102, 106, 129, 130, 131, 135, 139, 144, 148, 149, 153], "dimension": 129, "diminish": [11, 15], "dimrang": 129, "dimtype64": [0, 1], "dir": [9, 21, 25, 28, 29, 30, 31, 37, 42, 94, 120], "direct": [0, 16, 29, 75, 81, 97, 98, 107, 115, 139, 154], "directli": [0, 10, 11, 13, 14, 15, 16, 19, 28, 29, 30, 31, 32, 42, 72, 78, 80, 81, 83, 90, 92, 94, 98, 102, 103, 109, 113, 115, 117, 120, 127, 128, 129, 142, 143, 147, 148, 151, 158, 160, 167], "directori": [0, 9, 15, 19, 20, 21, 22, 28, 29, 30, 31, 37, 38, 59, 60, 61, 62, 78, 80, 94, 99, 111, 112, 113, 114, 115, 120, 121, 131, 134, 142, 143, 144, 148, 149, 158, 160], "dirnam": 56, "disabl": [0, 1, 10, 15, 20, 21, 22, 23, 26, 34, 48, 57, 81, 87, 91, 92, 101, 102, 105, 110, 114, 120, 123, 127, 128, 129, 132, 134, 135, 143, 148, 156, 163], "disable_chunked_context": 21, "disable_finalize_fus": 143, "disable_forward_chunk": 131, "disable_kv_cach": 134, "disable_kv_cache_reus": 23, "disable_overlap_schedul": [14, 65, 81, 87, 91, 92, 143, 163], "disable_weight_only_quant_plugin": 131, "disable_xqa": 101, "disablelookahead": 1, "disablelookaheaddecod": 1, "disableseamlesslookaheaddecod": 1, "disadvantag": [115, 122], "disagg": 148, "disagg_config": [16, 81], "disagg_executor": 0, "disaggexecutororchestr": 0, "disaggreg": [0, 8, 10, 19, 44, 82, 93, 143, 145, 146, 148, 156, 161], "disaggregated_param": [20, 28, 29, 30, 31, 143], "disaggregatedparam": [93, 143], "disaggserverbenchmark": 148, "disallow": 10, "disappear": 10, "discard": [11, 57, 123, 143], "disclaim": [13, 121, 123, 126, 127], "disclosur": 148, "disconnect": 148, "discourag": [0, 58, 102, 143], "discov": [10, 19, 64, 97, 113], "discoveri": 34, "discrep": [16, 94, 144, 149], "discret": 83, "discuss": [11, 13, 15, 19, 101, 121, 123, 127, 128, 148], "disk": [80, 94, 99, 115, 160], "dispar": 8, "dispatch": [0, 11, 12, 15, 16, 42, 81, 89, 92, 100, 115, 148], "displai": [8, 15, 143], "disrupt": 89, "disservingrequeststat": 0, "disservingstat": 0, "dist": [2, 36, 37, 38, 60, 119, 120, 121], "distanc": [19, 72, 101, 129], "distil": [11, 79, 148, 159], "distinct": [8, 12, 16, 57, 80, 104, 106, 109, 129, 160], "distinguish": 105, "distribut": [1, 12, 15, 18, 21, 25, 50, 59, 72, 88, 89, 100, 101, 102, 113, 120, 129, 134, 135, 140, 142, 146], "distserv": 98, "dit": [131, 148], "div": 129, "dive": [13, 36, 93, 119], "diverg": [75, 154], "divers": [0, 8, 36, 102, 119], "diversity_penalti": 102, "divid": [11, 13, 15, 83, 84, 114, 129, 148], "divup": 129, "dl": 7, "dlsym": 0, "do": [1, 2, 7, 10, 11, 12, 13, 14, 15, 16, 19, 20, 21, 25, 28, 29, 30, 31, 33, 42, 56, 72, 81, 90, 92, 93, 94, 98, 103, 114, 115, 121, 123, 127, 129, 139, 143, 144, 149, 151], "do_cross_attent": [129, 130], "do_layer_norm_befor": 112, "do_sampl": 102, "doactivationkernel": 19, "doc": [1, 2, 6, 12, 15, 20, 113, 117, 123, 127, 129, 139, 148], "docker": [2, 9, 25, 59, 60, 61, 93, 139, 148], "docker_run_arg": 2, "dockerfil": [94, 116], "docstr": 32, "document": [0, 4, 5, 7, 10, 13, 15, 16, 21, 25, 28, 32, 36, 37, 38, 39, 40, 41, 42, 43, 45, 46, 51, 66, 67, 68, 69, 70, 72, 81, 84, 85, 94, 95, 98, 101, 102, 104, 105, 106, 109, 111, 112, 113, 115, 118, 119, 121, 128, 129, 135, 137, 139, 143, 150, 151, 162], "doe": [0, 2, 3, 10, 11, 14, 15, 18, 22, 33, 34, 37, 38, 56, 69, 72, 73, 81, 88, 92, 95, 98, 101, 102, 106, 109, 115, 120, 127, 129, 134, 135, 138, 143, 144, 148, 149, 152, 167], "doesn": [1, 8, 12, 19, 28, 29, 30, 31, 35, 37, 42, 65, 72, 84, 101, 116, 120, 126, 127, 148, 156], "dollar": [37, 120], "domain": [15, 19, 81, 98, 107], "domin": [8, 12, 15, 148], "don": [8, 9, 11, 14, 15, 56, 81, 89, 98, 109, 115, 116, 122, 127, 129], "done": [1, 2, 14, 15, 16, 17, 25, 28, 29, 30, 31, 48, 81, 88, 92, 105, 113, 120, 123, 126, 129, 132, 143, 144, 149], "dongjiyingdji": 148, "dora": [22, 129, 130], "dora_plugin": [22, 106, 129], "dot": [12, 18, 114, 129], "doubl": [0, 4, 11, 18, 124, 125, 127, 139], "doubt": 11, "down": [0, 3, 10, 11, 13, 14, 15, 19, 62, 99, 106, 122, 129, 134], "down_proj": 114, "downgrad": 148, "download": [20, 23, 28, 29, 30, 31, 37, 59, 60, 61, 62, 65, 93, 94, 96, 97, 120, 121, 139, 142, 148], "downscale_freq_shift": 130, "downsid": 127, "downstream": [86, 137], "dp": [0, 2, 3, 6, 8, 12, 14, 16, 20, 29, 143, 148], "dp4ep4": 20, "dp8": [12, 14], "dprank": 0, "dpsize": 0, "dpu": 20, "draft": [0, 1, 12, 13, 17, 18, 19, 22, 93, 131, 134, 143, 148], "draft_len": 131, "draft_model": 92, "draft_path": 134, "draft_target": [65, 143], "draft_target_model": 109, "draft_token": [131, 143], "draft_tokens_extern": [22, 131], "draftacceptancethreshold": 1, "draftbuff": 1, "drafter": [10, 92, 109, 143, 148], "draftindic": 1, "draftlen": 1, "draftlogit": 1, "draftlogitshost": 1, "draftoverhead": 0, "draftparticipantid": 0, "draftpath": 1, "draftpathshost": 1, "draftprob": 1, "draftrequestid": 0, "drafttarget": 92, "drafttargetdecodingconfig": [92, 93, 143], "drafttoken": [0, 1], "drafttokenid": 1, "drafttokenidshost": 1, "drafttokensextern": 1, "dram": [0, 113], "dramat": [8, 84], "drastic": [14, 35], "drat": 92, "draw": 18, "dreamgenx": 148, "drive": [58, 113, 120], "driven": [8, 15], "driver": [15, 19, 28, 29, 30, 31, 35, 81, 98, 135, 148], "drop": [2, 8, 13, 14, 81, 98, 123, 126, 128], "dropout": [130, 148], "dropout_prob": 130, "dry": 143, "dry_run": [22, 143, 148], "dst": 1, "dstate": 129, "dstdesc": 0, "dsttype": 1, "dt_proj": 129, "dt_rank": 129, "dtype": [1, 10, 25, 28, 30, 31, 37, 83, 90, 103, 106, 110, 111, 112, 113, 115, 120, 121, 122, 129, 130, 131, 132, 133, 134, 139, 143, 145, 148, 166], "dual": 94, "duck": 143, "due": [0, 1, 2, 5, 8, 11, 12, 14, 15, 17, 19, 20, 26, 28, 29, 30, 31, 33, 34, 37, 57, 72, 88, 89, 91, 94, 108, 109, 115, 120, 121, 126, 128, 134, 148, 151, 165], "duke": 29, "dummi": [21, 85, 121, 143, 148, 162], "dump": [0, 15, 94, 99, 143], "dump_debug_buff": 134, "dumps_kwarg": 143, "duplic": [14, 19, 75, 148, 154], "duplicate_data": 129, "durat": [0, 15, 19, 25, 28, 29, 30, 31, 83, 121], "duration_m": [83, 143], "durationm": 0, "dure": [0, 1, 6, 8, 10, 11, 12, 13, 14, 15, 16, 18, 19, 22, 33, 35, 36, 37, 72, 73, 74, 80, 81, 83, 85, 88, 91, 92, 94, 98, 101, 102, 103, 107, 108, 109, 110, 113, 119, 120, 127, 128, 134, 135, 139, 143, 148, 151, 152, 153, 160, 162, 166], "dutch": 29, "dynam": [0, 11, 12, 13, 15, 16, 19, 22, 37, 75, 79, 81, 89, 92, 98, 120, 129, 131, 134, 135, 143, 146, 148, 154, 159, 167], "dynamic_batch_config": 143, "dynamic_batch_moving_average_window": 143, "dynamic_quant_bf16tonvfp4": 12, "dynamic_tree_max_topk": 143, "dynamicbatchconfig": [0, 93, 143], "dynamicbatchmovingaveragewindow": 0, "dynamicbatchsizeconfig": 0, "dynamicdecodelay": 1, "dynamicqu": 12, "dynamictreemaxtopk": 0, "dynamictreemaxtopkhost": 1, "dynamicyamlmixinforset": [75, 154], "dynamo": [93, 109, 142, 146], "dynamodeploymentgraph": 44, "dynasor": [93, 148], "dynasor_generation_control": 11, "dynasorgenerationcontrol": 11, "dynlibload": 0, "e": [0, 10, 11, 13, 14, 17, 19, 20, 23, 25, 26, 29, 32, 34, 35, 36, 37, 59, 60, 61, 72, 75, 80, 81, 87, 89, 94, 95, 98, 99, 101, 104, 105, 106, 107, 114, 116, 119, 120, 129, 132, 134, 137, 139, 142, 143, 144, 147, 148, 149, 154, 163], "e2": [14, 16, 18, 93, 148], "e2el": [25, 28, 29, 30, 31], "e4m3": [4, 90, 107], "e5m2": 4, "e728f08114c042309efeae4df86a50ca": 28, "e752184d1181494c940579c007ab2c5f": 17, "each": [0, 1, 2, 8, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 33, 35, 37, 38, 42, 49, 56, 58, 59, 60, 61, 64, 72, 80, 81, 83, 84, 88, 89, 92, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 109, 112, 113, 120, 121, 122, 126, 127, 128, 129, 130, 132, 134, 135, 137, 139, 143, 148, 150, 151, 160, 166, 167], "eager": [14, 35, 148], "eagl": [0, 1, 9, 10, 17, 22, 82, 91, 131, 134, 143, 145, 146, 148, 161, 165], "eagle3": [10, 65, 93, 109, 143, 148], "eagle3_layers_to_captur": 143, "eagle3_one_model": [17, 65, 92, 143], "eagle_choic": [134, 143], "eagle_dynamic_tree_max_top_k": 134, "eagle_posterior_threshold": 134, "eagle_temperatur": 131, "eagle_use_dynamic_tre": 134, "eaglechoic": [0, 1], "eagleconfig": [0, 131], "eagledecodingconfig": [65, 92, 93, 143], "eagleforcausallm": 131, "eagleinput": 1, "eaglenetctxcontextlengthshost": 1, "eaglenetctxpastkeyvaluelengthshost": 1, "eaglenetctxrequesttypeshost": 1, "eaglenetgencontextlengthshost": 1, "eaglenetgenpastkeyvaluelengthshost": 1, "eaglenetgenrequesttypeshost": 1, "ealge2": 13, "earli": [1, 8, 11, 32, 134, 139, 148], "earlier": [0, 11, 112, 123, 139], "early_stop": [102, 134, 143, 148], "early_stop_criteria": 134, "earlystop": [0, 1, 102], "eas": [11, 15, 16, 38], "easi": [7, 11, 15, 57, 78, 80, 121, 142, 146, 158, 160], "easier": [2, 13, 15, 19, 37, 113, 115, 120], "easili": [2, 10, 12, 15, 114, 129], "east": [29, 111, 113, 139], "eastern": 147, "ebnf": [0, 91, 99, 143], "echo": [17, 19, 26, 60, 61, 81, 116, 117], "econom": 20, "ecosystem": [20, 142, 146], "eddi": 148, "edg": [4, 20], "edit": [34, 38, 94, 109], "edu": 23, "eec": 23, "ef648e7489c040679d87ed12db5d3214": 147, "effect": [0, 8, 10, 11, 12, 13, 14, 19, 20, 22, 28, 29, 30, 31, 35, 57, 58, 64, 81, 83, 84, 88, 89, 98, 102, 107, 109, 123, 126, 127, 143, 148], "effici": [8, 11, 12, 13, 14, 15, 16, 19, 20, 22, 33, 35, 40, 47, 52, 53, 55, 67, 72, 78, 79, 81, 84, 85, 86, 88, 89, 97, 100, 101, 102, 105, 109, 113, 135, 138, 140, 141, 142, 143, 146, 147, 150, 151, 158, 159, 162, 166], "effort": [8, 10, 13, 14, 15, 16, 19, 20, 29, 109, 112, 123, 148], "eg": 38, "egx": 20, "eight": [2, 3], "einop": 129, "einstein": 129, "einsum": 129, "einsum_eq": 129, "either": [0, 1, 11, 12, 14, 19, 23, 33, 80, 89, 99, 129, 135, 139, 142, 143, 148, 160], "elaps": [8, 25, 28, 29, 30, 31, 83], "element": [0, 1, 15, 33, 72, 84, 88, 101, 102, 106, 107, 129, 130, 137, 143], "element_typ": 1, "elementwis": [103, 129], "elementwise_affin": 130, "elementwise_binari": 129, "elementwise_sub": 103, "elementwise_sum": 103, "elementwiseoper": [103, 129], "eleutherai": [28, 30, 31, 37, 120], "elicit": 11, "elif": [64, 65, 167], "elimin": [8, 10, 12, 14, 22, 37, 81, 88, 98, 109, 120, 123, 126, 146, 148], "ellipsi": 129, "els": [0, 11, 42, 56, 57, 58, 65, 92, 113, 114, 115, 129, 139, 167], "emb": [67, 113, 130], "embed": [0, 13, 22, 86, 105, 111, 120, 129, 134, 143, 144, 148, 149, 151], "embed_dim": 130, "embed_posit": 130, "embed_positions_for_gpt_attent": 130, "embed_positions_for_gpt_attention_loc": 130, "embed_positions_loc": 130, "embed_token": [114, 144, 149], "embedding_bia": 143, "embedding_dim": 130, "embedding_multipli": 131, "embedding_parallel_mod": 143, "embedding_scal": 131, "embedding_sharding_dim": [112, 131], "embeddingbia": [0, 1], "embeddingt": [0, 1], "emerg": [7, 12, 15], "emit": 143, "emot": 64, "emphas": 11, "emphasi": 112, "empir": [8, 15], "emploi": [8, 15, 16, 35, 80, 109, 150, 167], "employe": 58, "empow": [8, 12], "empti": [0, 1, 42, 83, 109, 129, 143, 148, 167], "emptybuff": 1, "emptygenslot": 0, "emptytensor": 1, "emul": [129, 148], "en": 148, "enabl": [0, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 21, 22, 26, 28, 30, 31, 34, 35, 37, 38, 42, 54, 55, 57, 58, 63, 64, 72, 73, 74, 75, 79, 81, 84, 85, 86, 87, 88, 90, 91, 92, 93, 94, 98, 99, 101, 102, 103, 106, 107, 108, 109, 110, 113, 114, 117, 120, 121, 126, 128, 129, 130, 131, 132, 134, 137, 139, 143, 144, 145, 146, 147, 148, 149, 151, 152, 153, 154, 159, 162, 163, 166], "enable_allreduc": 129, "enable_attention_dp": [2, 15, 20, 25, 26, 28, 29, 30, 31, 38, 60, 89, 143], "enable_autotun": [17, 143, 148], "enable_bal": [8, 29, 143], "enable_batch_size_tun": 143, "enable_block_reus": [9, 17, 26, 48, 57, 63, 83, 143], "enable_build_cach": [143, 148], "enable_chunked_context": [2, 21, 134, 148], "enable_chunked_prefil": [84, 143, 148], "enable_context_fmha_fp32_acc": [134, 143], "enable_debug_output": [22, 139, 143], "enable_forward_chunk": 131, "enable_fp8": 107, "enable_fullgraph": 143, "enable_if_t": 1, "enable_inductor": 143, "enable_iter_perf_stat": [26, 143], "enable_iter_req_stat": 143, "enable_kv_cache_reus": 105, "enable_layerwise_nvtx_mark": 143, "enable_lm_head_tp_in_adp": 143, "enable_lora": 143, "enable_max_num_tokens_tun": [143, 148], "enable_min_lat": [17, 143], "enable_mixed_sampl": 148, "enable_multi_devic": 148, "enable_offload": 57, "enable_overlap_schedul": 26, "enable_pad": [2, 14, 20, 25, 28, 29, 30, 31, 38, 63, 143], "enable_partial_reus": [83, 143], "enable_piecewise_cuda_graph": 143, "enable_prompt_adapt": [143, 148], "enable_qkv": 130, "enable_think": 70, "enable_tqdm": 143, "enable_trt_overlap": 148, "enable_trtllm_sampl": 91, "enable_ucx": 148, "enable_userbuff": 143, "enable_xqa": 148, "enableattentiondp": [0, 1], "enablebatchsizetun": 0, "enableblockreus": [0, 105], "enablechunkedcontext": 0, "enablecontextfmhafp32acc": 0, "enabled_with_fp32_acc": 101, "enablelookaheaddecod": 1, "enablemaxnumtokenstun": 0, "enablepartialreus": 0, "enableseamlesslookaheaddecod": [0, 1], "enabletrtoverlap": 0, "enc": [22, 134, 148], "enc_dec": 102, "encapsul": [72, 101, 102, 113, 129], "encdecmodelrunn": 134, "encod": [0, 4, 11, 12, 22, 26, 35, 58, 72, 86, 101, 102, 129, 134, 136, 137, 138, 143, 148], "encode_base64_content_from_url": 67, "encoded_vocab": [0, 99], "encodedvocab": [0, 99], "encoder_hidden_st": [130, 131], "encoder_input_featur": 134, "encoder_input_id": 134, "encoder_input_len_rang": 148, "encoder_input_length": [129, 130, 134], "encoder_language_adapter_rout": 134, "encoder_max_input_length": [130, 134], "encoder_output": [130, 131, 134], "encoder_output_length": 134, "encoder_run": 134, "encoderenginebuff": 0, "encoderhiddens": 1, "encoderinputfeatur": 0, "encoderinputtokenid": 0, "encoderjsonconfigstr": 0, "encoderlen": 0, "encodermodel": [0, 131], "encodermodelpath": 0, "encoderoutput": 0, "encoderoutputlength": 0, "encount": [2, 10, 17, 19, 20, 28, 29, 30, 31, 38, 81, 97, 98, 114, 139, 143], "encourag": [0, 15, 25, 58, 102, 115, 143], "end": [0, 1, 8, 11, 13, 18, 20, 21, 22, 25, 28, 29, 30, 31, 35, 37, 58, 72, 74, 88, 93, 101, 102, 113, 120, 123, 127, 128, 129, 143, 148, 153, 166], "end_dim": 129, "end_id": [134, 143, 148], "end_thinking_phase_token": 143, "end_token": [0, 143], "endeavor": [12, 15, 16], "endid": [0, 1], "endpoint": [9, 17, 20, 45, 46, 81, 86, 143, 147, 148], "endswith": [114, 143], "enforc": [121, 129, 143], "engag": 11, "engin": [0, 1, 6, 8, 11, 12, 13, 14, 15, 16, 19, 20, 21, 22, 23, 26, 37, 38, 42, 48, 49, 62, 73, 74, 81, 82, 83, 88, 89, 98, 99, 101, 102, 103, 106, 109, 110, 115, 122, 123, 126, 127, 128, 129, 131, 134, 135, 139, 143, 145, 148, 152, 153, 156, 161], "engine_buff": 134, "engine_dir": [21, 110, 111, 112, 113, 115, 120, 121, 134, 139], "engine_inspector": 134, "engine_llama_3": 113, "engine_nam": 134, "engine_output": 22, "engineaddr": 1, "enginebuff": [0, 1], "enginefilenam": 1, "engineinput": 1, "engineoutput": 1, "enginepath": 1, "engines": 1, "english": [18, 29], "enhanc": [2, 8, 11, 12, 13, 14, 15, 16, 21, 35, 58, 72, 78, 79, 81, 86, 100, 102, 109, 128, 135, 141, 143, 151, 158, 159], "enjoi": [47, 52, 53, 55, 97, 117, 140, 142, 147], "enough": [2, 13, 20, 57, 72, 83, 101, 105, 126, 135, 148, 150, 167], "enqueu": [0, 99, 113, 134, 135, 148], "enqueuecontext": 0, "enqueuegener": 0, "enqueuerequest": [0, 99], "enroot": 94, "ensembl": 16, "ensur": [8, 9, 10, 13, 15, 17, 19, 21, 28, 29, 30, 31, 32, 35, 36, 37, 75, 81, 91, 94, 98, 99, 100, 103, 108, 115, 120, 126, 132, 143, 144, 149, 154, 166], "enter": [11, 38, 88, 94, 103, 116, 126, 166], "enterpris": [20, 51], "entir": [0, 3, 8, 11, 12, 15, 35, 37, 38, 75, 89, 92, 99, 106, 113, 120, 129, 135, 143, 154, 166], "entri": [0, 1, 8, 11, 19, 21, 23, 32, 33, 35, 37, 55, 78, 94, 106, 120, 129, 148, 158], "entrypoint": [26, 116, 121, 142], "enum": [0, 1, 11, 143], "enumer": [0, 1, 48, 49, 54, 58, 63, 64, 92], "env": [29, 34, 39, 40, 41, 43, 45, 46, 76, 81, 120, 155], "envelop": 15, "environ": [2, 8, 9, 12, 15, 16, 19, 28, 29, 30, 31, 36, 37, 40, 56, 59, 60, 61, 67, 69, 72, 78, 93, 94, 97, 102, 107, 109, 119, 120, 121, 123, 126, 127, 139, 141, 142, 148, 151, 158], "environment": 114, "eo": [21, 25, 28, 29, 30, 31, 58, 102, 143], "eof": [2, 9, 13, 15, 20, 25, 26, 28, 29, 30, 31, 60, 89], "eos_id": [15, 21], "eos_token": 58, "eos_token_id": [58, 99, 134], "ep": [2, 8, 12, 13, 16, 19, 20, 21, 26, 28, 37, 38, 93, 100, 120, 129, 130, 148], "ep16": 19, "ep2": 12, "ep2tp4": 12, "ep32": [15, 19], "ep4": [15, 19], "ep4tp2": 12, "ep8": [14, 15, 19], "ep8tp8": 12, "ep_load_balanc": [15, 89], "ep_siz": [9, 15, 17, 20, 23, 25, 26, 38, 43], "epd": 145, "eplb": [8, 28, 148], "epsilon": [0, 129], "eq": 129, "equal": [0, 1, 8, 14, 15, 22, 42, 81, 89, 91, 99, 100, 122, 129, 130, 135, 143], "equal_progress": [128, 143], "equat": [6, 129], "equilibr": 8, "equival": [11, 12, 14, 96, 123, 129, 144, 149], "equvili": 22, "era": 146, "erenup": 148, "err": [59, 60, 61], "error": [0, 1, 14, 17, 20, 21, 22, 23, 26, 28, 29, 30, 31, 34, 38, 76, 81, 94, 97, 98, 99, 106, 115, 121, 126, 134, 135, 143, 148, 155], "errormsg": 0, "especi": [10, 11, 13, 15, 16, 18, 19, 22, 47, 52, 53, 55, 81, 97, 98, 103, 122, 126, 140, 142, 147, 166], "essenti": [11, 15, 35, 37, 94, 109, 120], "establish": [14, 15, 16, 81, 98], "estim": [15, 120, 148, 167], "et": 3, "etc": [0, 1, 10, 15, 20, 29, 32, 35, 36, 37, 80, 89, 109, 119, 120, 123, 127, 134, 135, 139, 142, 143, 144, 149], "ethnzhng": 148, "etp": 89, "euo": 29, "eval": [10, 28, 29, 30, 31, 51, 93, 147], "evalu": [2, 4, 5, 8, 11, 14, 16, 23, 86, 93, 107, 147, 148], "even": [0, 7, 10, 11, 12, 15, 16, 18, 19, 20, 22, 26, 34, 72, 81, 88, 90, 101, 102, 113, 115, 121, 126, 129, 134, 135, 143], "evenli": [12, 89, 100], "event": [0, 1, 10, 143, 148], "event_buffer_max_s": 143, "eventbuffermaxs": 0, "eventid": 0, "eventptr": 1, "eventu": 108, "ever": [0, 92, 127], "everi": [0, 8, 10, 11, 12, 14, 15, 16, 19, 20, 28, 33, 37, 58, 89, 92, 99, 114, 120, 121, 122, 129, 134], "everyon": [10, 11, 13], "everyth": [92, 113], "evict": [0, 1, 11, 13, 21, 37, 57, 83, 88, 104, 105, 106, 120, 121, 126, 148], "evidenc": 8, "evolv": [12, 101, 115, 166], "ewr": 29, "ex": [60, 61], "exact": [2, 72, 101, 135], "exact_match": [28, 30, 31], "exactli": [10, 33, 92], "exam": [11, 12], "examin": [15, 109], "exampl": [0, 3, 5, 7, 9, 10, 13, 15, 16, 19, 20, 25, 26, 28, 29, 30, 31, 32, 33, 34, 37, 38, 42, 48, 49, 51, 56, 59, 64, 70, 72, 75, 77, 78, 80, 81, 83, 84, 85, 88, 89, 90, 91, 92, 93, 94, 95, 98, 101, 102, 103, 105, 108, 109, 110, 111, 115, 121, 122, 123, 124, 125, 126, 127, 128, 129, 134, 135, 137, 138, 139, 140, 141, 143, 144, 145, 147, 148, 149, 151, 154, 156, 157, 158, 160, 162, 164, 165, 167], "example_cuda_graph_config": 63, "example_kv_cache_config": 63, "exaon": [90, 114, 138, 145, 148], "exaone4forcausallm": [138, 145], "exc": 54, "exce": [0, 8, 18, 128, 129, 143], "exceed": [0, 135], "excel": [11, 18, 19], "except": [0, 1, 10, 11, 12, 13, 15, 19, 22, 58, 70, 72, 88, 89, 92, 99, 101, 102, 115, 122, 129, 139, 143, 148], "excess": [15, 72, 101], "exchang": [93, 143], "excit": [47, 52, 53, 54, 55, 97, 140, 142, 147], "excl": [25, 28, 29, 30, 31], "exclud": [1, 10, 123, 129, 143, 148], "exclude_default": 143, "exclude_input_from_output": 143, "exclude_modul": [112, 143, 148], "exclude_non": 143, "exclude_unset": 143, "excludeinputfromoutput": 0, "exclus": [1, 10, 75, 92, 102, 137, 148, 154], "exec": [28, 30, 31, 36, 119, 147], "execut": [0, 8, 10, 11, 12, 14, 15, 16, 19, 20, 25, 28, 29, 30, 31, 33, 34, 35, 36, 37, 73, 78, 81, 87, 88, 89, 99, 102, 106, 109, 113, 115, 119, 120, 126, 128, 129, 134, 135, 143, 146, 150, 152, 158, 167], "executeloopbackrequest": 0, "executor": [1, 10, 16, 42, 62, 81, 85, 86, 92, 98, 105, 109, 110, 120, 128, 134, 135, 143, 148, 150, 162], "executor_config": 166, "executorconfig": [0, 81, 99, 110, 143], "executorexamplefastlogit": 148, "exhaust": [0, 16, 74, 153], "exhibit": [8, 18], "exist": [1, 10, 11, 12, 14, 15, 18, 19, 22, 28, 29, 30, 31, 34, 35, 37, 56, 69, 72, 77, 80, 85, 94, 102, 105, 106, 109, 114, 115, 120, 134, 143, 146, 148, 151, 157, 160, 162], "exist_ok": 56, "exit": [11, 15, 19, 26, 38, 134], "exp": 129, "expand": [0, 5, 7, 11, 13, 19, 78, 79, 93, 129, 134, 143, 148, 158, 159], "expand_dim": 129, "expand_dims_lik": 129, "expand_mask": 129, "expand_shap": 129, "expanded_idx_to_permuted_idx": 129, "expandinputrowskernel": 19, "expandtab": 143, "expans": 129, "expect": [0, 5, 9, 10, 13, 15, 16, 17, 19, 22, 28, 29, 30, 31, 37, 42, 57, 59, 60, 61, 72, 73, 83, 92, 101, 102, 107, 111, 113, 114, 115, 120, 121, 124, 125, 129, 139, 143, 148, 152, 156], "expend": 11, "expens": [16, 81, 99, 109, 122, 123, 128], "experi": [6, 7, 10, 12, 14, 15, 16, 18, 19, 20, 28, 29, 30, 31, 35, 36, 57, 58, 74, 93, 108, 109, 119, 139, 142, 146, 148, 153], "experiment": [8, 13, 26, 32, 59, 60, 61, 102, 148, 158], "experiment_config": [75, 154], "experimentconfig": [74, 75, 153, 154], "expert": [2, 8, 20, 21, 23, 25, 26, 29, 30, 31, 38, 55, 78, 81, 93, 98, 106, 127, 143, 146, 148, 158], "expert_scale_factor": 129, "expert_statist": 15, "expert_statistic_eplb": 15, "expert_statistic_iter_rang": 15, "expert_statistic_path": 15, "expertid": 15, "expertis": [8, 12, 14, 15, 16, 19], "expir": [0, 83], "explain": [14, 18, 32, 33, 72, 81, 88, 98, 102, 113, 126, 129, 135, 137, 150, 151], "explan": [2, 14, 20, 28, 29, 30, 31, 32, 33, 127, 134, 135], "explicit": [0, 1, 15, 26, 109, 129, 148], "explicit_draft_token": [22, 109, 131], "explicitdrafttoken": [0, 1], "explicitdrafttokensdtyp": 1, "explicitdrafttokensinput": 1, "explicitdrafttokensmodul": 1, "expliciteosstop": 0, "explicitli": [1, 14, 15, 22, 28, 29, 42, 75, 80, 98, 103, 109, 113, 114, 143, 148, 154, 160], "explor": [11, 12, 14, 15, 19, 91, 93, 109, 146, 147], "expon": 4, "exponenti": [16, 109], "export": [2, 9, 12, 13, 15, 19, 22, 37, 45, 46, 59, 60, 61, 75, 78, 79, 94, 112, 115, 120, 133, 134, 139, 142, 148, 154, 158, 159], "export_fmt": [90, 164], "expos": [0, 10, 11, 17, 19, 20, 32, 75, 80, 94, 102, 113, 117, 123, 147, 148, 154, 156, 160], "express": [0, 91, 99, 129, 143], "extend": [0, 11, 12, 13, 14, 15, 18, 19, 91, 99, 105, 113, 127, 129, 142, 143, 146, 148], "extend_ctx": 92, "extended_runtime_perf_knob_config": [143, 148], "extendedruntimeperfknobconfig": [0, 93, 143], "extens": [16, 19, 33, 37, 80, 112, 120, 148, 160], "extent": 19, "extern": [0, 10, 11, 103, 104, 114, 134, 135, 147, 148], "external_checkpoint_dir": 114, "external_kei": 114, "external_weight": 114, "externaldrafttoken": 0, "externaldrafttokensconfig": [0, 1], "externaldrafttokensinput": 1, "externalstream": 58, "extra": [0, 2, 10, 11, 12, 13, 18, 19, 22, 25, 26, 33, 35, 37, 38, 43, 72, 74, 81, 87, 92, 97, 98, 101, 105, 109, 112, 120, 122, 123, 134, 143, 148, 153, 162, 163], "extra_arg": 60, "extra_bodi": [69, 85, 162], "extra_encoder_opt": 26, "extra_id": 105, "extra_llm_api_fil": [25, 28, 29, 30, 31], "extra_llm_api_opt": [2, 9, 13, 15, 17, 20, 21, 23, 26, 28, 29, 30, 31, 37, 38, 43, 60, 70, 73, 81, 85, 92, 120, 152, 156, 162], "extra_llm_api_options_eplb": 15, "extra_llm_config": 26, "extra_resource_manag": 143, "extra_token": 130, "extract": [0, 10, 11, 15, 21, 28, 30, 31, 36, 78, 92, 94, 99, 119, 124, 125, 129, 134, 143, 158], "extrapol": 129, "extrem": [8, 12, 15, 19, 88, 92, 113, 123, 126, 127], "f": [0, 17, 29, 36, 47, 48, 49, 51, 52, 53, 54, 55, 56, 57, 58, 62, 63, 64, 65, 69, 70, 72, 97, 101, 102, 116, 119, 121, 128, 129, 139, 140, 142, 143, 147], "fabric": [81, 98, 148], "face": [15, 23, 28, 29, 30, 31, 37, 42, 78, 79, 90, 99, 106, 110, 115, 120, 131, 143, 147, 148, 158, 159], "facilit": [15, 16, 92, 103, 109, 147], "fact": [29, 37, 92, 120, 127], "factor": [7, 14, 15, 19, 58, 122, 123, 129, 130, 135, 137, 148], "factori": [74, 77, 115, 134, 143, 148, 153, 157], "factual": 102, "fail": [0, 8, 15, 17, 19, 20, 28, 29, 30, 31, 34, 70, 81, 98, 134, 135, 139, 143, 148, 167], "fail_fast_on_attention_window_too_larg": [26, 134, 143], "failfastonattentionwindowtoolarg": 0, "failur": [15, 81, 98, 114, 143, 148], "fairli": 113, "fairseq": [138, 148], "fake": [105, 148], "fakebuff": 1, "falcon": [7, 37, 112, 120, 137, 138, 148], "falconconfig": 131, "falconforcausallm": 131, "falconmodel": 131, "fall": [35, 38, 107, 148], "fallback": [92, 114, 143, 148], "fals": [0, 1, 9, 11, 12, 14, 17, 18, 20, 22, 25, 26, 28, 29, 30, 31, 48, 51, 56, 58, 60, 70, 73, 74, 75, 77, 83, 89, 92, 99, 101, 102, 103, 105, 112, 129, 130, 131, 132, 133, 134, 143, 148, 152, 153, 154, 156, 157], "false_output_valu": 129, "false_valu": 129, "famili": [15, 33, 101, 114, 138, 148], "familiar": [73, 102, 113, 121, 122, 124, 125, 142, 152], "famou": [29, 102], "faq": 93, "far": [0, 11, 13, 99], "fast": [0, 9, 10, 15, 17, 18, 19, 20, 57, 72, 78, 101, 104, 109, 120, 122, 134, 143, 148, 158], "fast_build": [22, 143, 148], "fastapi": 148, "fastapi_serv": 148, "faster": [4, 5, 13, 14, 19, 20, 22, 38, 57, 72, 92, 101, 115, 121, 129], "fastest": 18, "fastlogit": 0, "fault": [15, 148], "favor": [8, 148], "favorit": 62, "fc": [112, 113, 114, 139], "fc2": 143, "fc_gate": 130, "fc_gate_dora": 130, "fc_gate_lora": 130, "fc_gate_plugin": 130, "fd": 0, "featur": [0, 7, 10, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 28, 29, 30, 31, 33, 34, 37, 38, 59, 60, 61, 79, 81, 83, 84, 88, 91, 94, 96, 98, 99, 101, 103, 104, 106, 107, 109, 112, 113, 115, 120, 123, 126, 127, 128, 129, 132, 134, 138, 142, 143, 144, 149, 151, 159, 165], "feature_dim": 134, "feb": 11, "februari": 14, "fed": [38, 86, 131], "feed": [100, 129], "feedback": [15, 148], "feel": 62, "fetch": [0, 10, 13, 26, 35, 150], "few": [7, 13, 14, 15, 81, 88, 92, 97, 98, 105, 113, 115, 126], "fewer": [3, 8, 18, 72, 88, 101, 109, 143, 151], "fewshot": 23, "fewshot_as_multiturn": 23, "ffn": [12, 100], "ffn_hidden_s": 130, "fhma": 148, "field": [0, 11, 26, 28, 37, 42, 56, 72, 75, 81, 92, 98, 102, 107, 112, 115, 117, 120, 123, 131, 132, 137, 143, 148, 151, 154], "field_nam": 143, "fieldinfo": 143, "fifo": [15, 19], "figur": [8, 10, 11, 12, 13, 15, 16, 18, 19, 81, 84], "file": [0, 2, 9, 13, 15, 17, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 32, 33, 34, 36, 37, 38, 44, 45, 46, 56, 70, 73, 74, 80, 85, 89, 92, 99, 100, 101, 103, 105, 112, 113, 114, 115, 119, 120, 134, 143, 144, 148, 149, 152, 153, 156, 160, 162], "file_path": 56, "filedesc": 0, "filenam": [0, 21, 25, 28, 29, 30, 31, 94], "filepath": 1, "filesystem": [0, 1], "fill": [1, 10, 11, 47, 72, 83, 97, 114, 117, 129, 140, 142, 143, 147, 151], "fill_attention_const_params_for_long_rop": 130, "fill_attention_const_params_for_rop": 130, "fill_attention_param": 130, "fill_none_tensor_list": 130, "fill_valu": 129, "fillchar": 143, "fillemptyfieldsfromruntimedefault": 0, "filloper": 129, "filltaskstensor": 1, "filter": [18, 28, 30, 31, 80, 92, 160], "filter_medusa_logit": 134, "filter_weight": [80, 160], "final": [0, 1, 8, 10, 11, 12, 13, 15, 16, 19, 20, 22, 25, 28, 29, 30, 31, 35, 42, 106, 116, 129, 143, 148, 167], "final_logit_softcap": 131, "final_output_id": 134, "finalize_decod": 134, "finalizemoeroutingkernel": 19, "find": [2, 8, 10, 11, 14, 15, 16, 91, 93, 123, 129, 139, 143, 148], "find_best_medusa_path": 134, "fine": [2, 14, 15, 85, 89, 109, 120, 127, 130, 143, 146, 162], "finer": [28, 29, 30, 31, 103], "finetun": 12, "finish": [0, 1, 10, 13, 15, 19, 35, 42, 99, 102, 104, 115, 120, 134, 143, 150, 166], "finish_reason": [17, 20, 28, 29, 30, 31, 143, 147, 148], "finished_gen_req_id": 56, "finishedst": 1, "finishedsum": 1, "finishreason": [0, 1, 148], "first": [0, 1, 5, 7, 8, 10, 11, 13, 14, 15, 16, 17, 19, 20, 22, 25, 26, 28, 29, 30, 31, 33, 37, 38, 49, 56, 57, 64, 72, 75, 80, 81, 83, 86, 88, 93, 94, 98, 99, 101, 102, 103, 105, 106, 109, 116, 120, 121, 123, 126, 127, 128, 129, 135, 139, 142, 143, 144, 148, 149, 151, 154, 160, 166, 167], "first_come_first_serv": [128, 143], "first_gen_token": 143, "first_lay": 134, "firstgentoken": 0, "firstit": 0, "firstli": [14, 15, 88, 116, 126, 135], "firstscheduledtim": 0, "firsttokentim": 0, "fit": [0, 1, 3, 4, 26, 89, 101, 122, 123, 134, 143, 167], "fitting_request": 167, "five": [18, 29], "fix": [10, 11, 13, 14, 16, 18, 19, 37, 81, 83, 84, 98, 104, 106, 109, 120, 135], "fjosw": 148, "flag": [0, 1, 6, 9, 15, 19, 20, 21, 23, 26, 28, 29, 30, 31, 37, 42, 73, 74, 88, 99, 101, 106, 115, 120, 123, 124, 126, 128, 129, 135, 148, 152, 153], "flags_siz": 1, "flan": [137, 138], "flash": [72, 101, 113], "flashattent": [72, 101, 113], "flashinf": [28, 30, 31, 72, 73, 74, 75, 77, 79, 148, 151, 152, 153, 154, 156, 157, 159], "flashinferattent": [72, 151], "flashmla": [13, 148], "flatten": [1, 6, 15, 106, 129, 130], "flattenedinouts": 1, "flattenn": 1, "flavor": 92, "flayer": 103, "flayerinfomemo": 103, "flexibl": [10, 12, 15, 20, 28, 30, 31, 42, 75, 80, 92, 94, 109, 115, 142, 146, 154, 160], "flexibli": [11, 19], "flight": [1, 8, 37, 78, 81, 86, 93, 120, 126, 128, 135, 146, 148, 158], "flip": 129, "flip_sin_to_co": 130, "float": [0, 1, 4, 58, 90, 102, 110, 112, 113, 128, 129, 130, 131, 134, 137, 143], "float16": [22, 103, 106, 110, 111, 112, 115, 122, 129, 131, 132, 139, 143], "float2": 129, "float32": [0, 22, 112, 129, 130, 131, 132, 143], "floattensor": [144, 149], "floattyp": [0, 1], "floor_div": 129, "floordiv": 129, "flop": 14, "flow": [9, 11, 12, 14, 16, 92, 103, 114, 115, 121, 122, 123, 126, 127, 148, 150, 167], "fluctuat": [8, 28, 29, 30, 31, 81, 98], "fly": [72, 101, 129, 137], "fmha": [0, 22, 88, 129, 134, 135, 143, 148], "fmt_dim": 1, "focu": [7, 10, 11, 12, 15, 36, 58, 103, 119, 148], "focus": [11, 19, 20, 32, 64, 109, 120, 123, 124, 125, 148], "fold": 135, "folder": [0, 34, 56, 74, 99, 102, 115, 121, 137, 138, 148, 153], "folder_trt_llm": 113, "follow": [1, 2, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 19, 20, 22, 23, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 37, 38, 42, 44, 48, 53, 54, 59, 60, 61, 72, 74, 75, 76, 77, 78, 79, 80, 81, 86, 89, 90, 91, 92, 94, 97, 98, 99, 102, 103, 106, 108, 109, 111, 112, 113, 114, 115, 117, 120, 121, 122, 123, 124, 125, 126, 127, 129, 137, 138, 142, 143, 144, 145, 147, 148, 149, 151, 153, 154, 155, 157, 158, 159, 160, 164, 165, 166], "footprint": [3, 14, 72, 90, 101, 135], "for_each_rank": 131, "forbid": 143, "forc": [0, 11, 12, 15, 16, 34, 37, 57, 72, 81, 101, 120, 143, 148], "force_drop_id": 130, "force_dynamic_quant": 143, "force_multi_block_mod": 120, "force_nccl_all_reduce_strategi": 148, "force_num_profil": 143, "force_words_id": 102, "forcefulli": 10, "forecast": 109, "forev": 92, "fork": [36, 119], "form": [0, 11, 16, 19, 72, 89, 91, 92, 99, 101, 109, 129, 143], "formal": 148, "format": [0, 4, 7, 13, 14, 17, 21, 26, 28, 29, 30, 31, 46, 70, 72, 79, 86, 91, 93, 94, 99, 107, 112, 114, 115, 121, 123, 134, 135, 139, 143, 146, 148, 151, 159], "format_map": 143, "former": [7, 113], "formula": [14, 16, 129], "forth": 15, "forthcom": [17, 20], "fortun": 10, "forum": 148, "forward": [0, 1, 10, 11, 13, 15, 16, 19, 35, 56, 72, 81, 92, 100, 103, 109, 111, 113, 128, 129, 130, 131, 139, 144, 148, 149, 150, 151, 166, 167], "forward_loop": 120, "forward_with_cfg": 131, "forward_without_cfg": 131, "forwardasync": 1, "forwarddispatch": 1, "forwardref": 143, "forwardsync": 1, "found": [2, 4, 10, 11, 15, 19, 34, 48, 49, 56, 58, 72, 81, 94, 98, 99, 100, 101, 102, 103, 109, 113, 120, 121, 123, 127, 137, 143, 167], "foundat": [10, 11, 13, 19], "four": [12, 13, 49, 57, 80, 99, 103, 109, 112, 130, 160], "fourth": [83, 99], "fp": [137, 148], "fp16": [3, 4, 7, 20, 22, 37, 72, 79, 90, 101, 106, 107, 110, 112, 114, 120, 123, 127, 129, 138, 139, 148, 159], "fp32": [0, 12, 14, 22, 72, 79, 90, 101, 129, 134, 138, 139, 143, 148, 159], "fp4": [2, 13, 14, 15, 19, 22, 28, 30, 31, 65, 90, 138, 142, 148], "fp4_gemm": 108, "fp8": [3, 5, 6, 7, 12, 13, 14, 15, 17, 18, 20, 21, 22, 25, 28, 30, 31, 37, 47, 52, 79, 85, 91, 93, 97, 107, 115, 120, 124, 127, 129, 132, 135, 138, 140, 142, 143, 145, 147, 148, 151, 159, 162, 164, 165], "fp8_block_scal": 143, "fp8_blockscale_gemm": 148, "fp8_inputs_overrid": 129, "fp8_kv_cach": [72, 101, 137], "fp8_pb_wo": 148, "fp8_per_channel_per_token": 143, "fp8_qdq": 137, "fp8_rowwise_gemm_plugin": 22, "fp_valu": [72, 101], "fpa_intb": 148, "frac": [8, 16], "fraction": [0, 16, 23, 26, 28, 29, 30, 31, 73, 77, 83, 129, 130, 134, 143, 152, 156, 157], "fragment": 28, "framework": [8, 20, 78, 80, 93, 109, 111, 112, 115, 129, 148, 158, 160], "franc": [47, 48, 49, 52, 53, 54, 55, 63, 65, 85, 97, 111, 113, 121, 128, 139, 140, 142, 147, 162], "free": [0, 1, 10, 11, 14, 15, 19, 23, 26, 28, 29, 30, 31, 48, 58, 73, 83, 85, 88, 92, 104, 106, 113, 114, 126, 130, 131, 134, 135, 143, 148, 152, 156, 162, 166], "free_gpu_memory_fract": [9, 42, 48, 63, 65, 83, 128, 143, 148], "free_hostfunc_user_data": 10, "free_mem_ratio": [73, 77, 152, 156, 157], "free_resourc": [92, 150, 166], "freed": [8, 83, 92, 120], "freedom": 115, "freegpumemoryfract": [0, 135, 148], "freenumblock": [0, 26], "freez": 14, "french": [85, 162], "freq": 129, "frequenc": [37, 120, 130], "frequency_penalti": [134, 143, 148], "frequencypenalti": [0, 1, 102], "frequent": [8, 10, 11, 19, 32, 89, 105, 139, 143], "friend": [0, 1, 37, 120], "friendli": [15, 129], "from": [0, 1, 2, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 33, 34, 35, 37, 38, 42, 47, 48, 49, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 72, 75, 77, 79, 80, 81, 83, 84, 85, 87, 88, 89, 90, 91, 92, 93, 95, 96, 97, 98, 99, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 120, 121, 122, 123, 126, 127, 128, 129, 130, 131, 133, 134, 135, 139, 140, 141, 143, 144, 146, 147, 148, 149, 150, 151, 154, 157, 159, 160, 162, 163, 164, 165, 166, 167], "from_argu": 131, "from_attribut": 143, "from_checkpoint": [115, 131], "from_config": 131, "from_dict": [131, 143], "from_dir": 134, "from_engin": 134, "from_hugging_fac": [111, 114, 115, 131], "from_jax": 115, "from_json_fil": [131, 143], "from_kera": 115, "from_kwarg": 143, "from_meta_ckpt": [115, 131], "from_nemo": [115, 131], "from_orm": 143, "from_pretrain": [11, 131], "from_prun": 131, "from_pybind": 143, "from_serialized_engin": 134, "from_str": 129, "fromfil": 113, "front": 143, "frontier": [8, 146], "fruit": 14, "fu": 11, "full": [0, 4, 5, 13, 14, 15, 16, 23, 25, 26, 28, 29, 30, 31, 33, 36, 37, 56, 58, 72, 75, 83, 89, 100, 101, 102, 105, 106, 109, 119, 120, 122, 129, 134, 135, 139, 143, 146, 148, 154], "full_stop_token": 58, "fulli": [14, 33, 47, 148], "fun": 29, "funcnam": 0, "function": [0, 1, 10, 11, 12, 13, 15, 19, 26, 35, 36, 63, 72, 75, 78, 80, 91, 92, 99, 101, 110, 111, 113, 115, 119, 127, 131, 132, 134, 135, 137, 138, 139, 142, 143, 146, 148, 154, 158, 160, 166, 167], "functiont": 0, "functool": 143, "fundament": [8, 19], "further": [3, 7, 10, 11, 13, 14, 15, 16, 17, 22, 37, 72, 81, 86, 89, 92, 99, 100, 101, 109, 113, 120, 123, 127, 151], "furthermor": [12, 15, 16, 81, 109, 123], "fuse": [10, 12, 14, 19, 20, 22, 72, 79, 86, 101, 109, 113, 127, 129, 144, 148, 149, 151, 159], "fuse_a": [12, 14], "fuse_fp4_qu": 22, "fuse_qkv_project": 131, "fuseattentionwithbiaspass": 103, "fused_a": 89, "fused_gate_up_dora": 130, "fused_gate_up_lora": 130, "fused_mo": 143, "fusedgatedmlp": [129, 130], "fusedmo": 148, "fusevalu": 1, "fusion": [14, 22, 72, 78, 79, 103, 126, 135, 137, 143, 148, 151, 158, 159], "fusion_op": 129, "futur": [7, 10, 15, 22, 37, 47, 48, 49, 51, 52, 53, 54, 55, 58, 63, 64, 65, 72, 81, 83, 88, 92, 93, 94, 97, 98, 101, 102, 104, 108, 109, 114, 115, 120, 121, 128, 129, 135, 140, 141, 142, 143, 147, 148], "fuyu": [138, 148], "fw": [80, 160], "g": [10, 11, 13, 14, 19, 20, 23, 25, 29, 32, 34, 35, 37, 59, 60, 61, 75, 80, 87, 88, 89, 95, 99, 104, 107, 114, 120, 126, 134, 143, 144, 147, 149, 154, 163], "g00": 8, "g01": 8, "g0m": 8, "g1": [88, 126], "g10": 8, "g11": 8, "g1m": 8, "g2": [88, 126], "gain": [8, 15, 18, 35, 122, 126], "game": 17, "gamma": 129, "gap": [8, 10, 11, 16, 18, 19], "garbag": [19, 32, 143], "garbage_collection_gen0_threshold": [32, 143], "gate": [22, 106, 114, 121, 129, 148], "gate_a": 129, "gate_a_bia": 129, "gate_bia": 129, "gate_proj": 114, "gate_up_proj": 148, "gate_x": 129, "gate_x_bia": 129, "gatedmlp": [129, 130], "gather": [0, 1, 22, 53, 54, 129, 134, 143], "gather_all_token_logit": [22, 148], "gather_context_logit": [22, 131, 134, 143], "gather_dim": [113, 129], "gather_generation_logit": [22, 131, 134, 143], "gather_last_token_logit": 129, "gather_nd": 129, "gather_output": 130, "gathercontext": [0, 148], "gatheredid": 1, "gatherel": 129, "gathergenerationlogit": 0, "gathermod": 129, "gathertre": 1, "gatherv2": 129, "gb": [5, 14, 57, 81, 94, 98, 120, 143], "gb200": [8, 14, 16, 19, 20, 28, 29, 38, 81, 89, 93, 98, 138, 146, 148], "gc": 19, "gcc": [94, 148], "gd": 0, "geforc": 148, "gegelu": 129, "gegelu_limit": 130, "geglu": 129, "gelu": [129, 131], "gelu_pytorch_tanh": 148, "gelu_tanh": 130, "gemm": [10, 14, 15, 19, 22, 78, 79, 89, 90, 103, 126, 129, 135, 148, 158, 159], "gemm_allreduc": 129, "gemm_allreduce_plugin": [22, 134], "gemm_fc1": 12, "gemm_plugin": [22, 106, 110, 112, 113, 120, 123, 127, 130], "gemm_swiglu": 129, "gemm_swiglu_plugin": [22, 123, 132], "gemma": [79, 90, 115, 136, 137, 138, 145, 148, 159], "gemma2": 138, "gemma2_added_field": 131, "gemma2_config": 131, "gemma3": [80, 148, 160], "gemma3_added_field": 131, "gemma3_config": 131, "gemma3_weight_mapp": [80, 160], "gemma3forcausallm": [80, 138, 145, 160], "gemma3forconditionalgener": [138, 145], "gemma3hfweightmapp": [80, 160], "gemma_added_field": 131, "gemma_config_kwarg": 131, "gemmaconfig": 131, "gemmaforcausallm": 131, "gen": [8, 10, 16, 143, 148], "gen2dep4": 16, "gen4": 16, "gen8": 16, "gen_extra": 81, "gen_kwarg": [28, 30, 31], "genai": [7, 71, 93], "genattent": 12, "genenginepath": 0, "gener": [0, 1, 2, 3, 4, 6, 8, 10, 11, 12, 13, 14, 16, 18, 19, 20, 21, 22, 23, 25, 28, 29, 30, 31, 32, 34, 35, 36, 37, 38, 47, 48, 49, 50, 56, 57, 63, 64, 65, 73, 74, 78, 79, 83, 85, 88, 90, 92, 93, 95, 97, 99, 102, 105, 109, 112, 113, 114, 115, 119, 120, 121, 122, 124, 125, 126, 127, 128, 129, 131, 134, 135, 138, 139, 140, 143, 144, 146, 147, 148, 149, 150, 151, 152, 153, 158, 159, 162, 164, 165, 166, 167], "generate_alibi_bias": 129, "generate_alibi_slop": 129, "generate_async": [11, 42, 53, 54, 143, 148], "generate_eplb_config": 15, "generate_logn_sc": 129, "generate_tllm_weight": 114, "generate_with_stream": 32, "generated_text": [48, 49, 62, 121, 128], "generatejsonschema": 143, "generation_complet": 167, "generation_control": 11, "generation_dir": 11, "generation_in_progress": 167, "generation_kwarg": 11, "generation_kwargs_list": 11, "generation_logit": [134, 143], "generation_onli": 143, "generation_phas": [72, 101], "generation_request": 167, "generation_serv": [16, 81], "generation_task": 11, "generation_to_complet": 167, "generation_with_dynasor_cot": 11, "generationexecutor": [81, 98, 148], "generationlength": 1, "generationlengthsdevic": 1, "generationlengthshost": 1, "generationlengthshostcopi": 1, "generationlogit": 0, "generationmixin": 131, "generationoutput": 32, "generationresult": [11, 143], "generationresultbas": 143, "generationsequ": 134, "generationsess": [101, 134, 135], "generationstep": 1, "generationtask": 11, "genericprompttuningparam": 1, "genert": 98, "genexecutorconfig": 0, "genidx": 0, "genlengthlogitsprocessor": 58, "genlenthlogitsprocesor": 58, "genreqr": 16, "genrequest": 1, "geograph": 29, "get": [0, 1, 6, 11, 13, 14, 15, 17, 19, 20, 26, 28, 29, 30, 31, 36, 42, 56, 64, 69, 72, 75, 84, 92, 94, 97, 99, 101, 103, 106, 110, 114, 116, 117, 119, 121, 123, 129, 134, 139, 143, 146, 147, 148, 154, 164, 167], "get_1d_sincos_pos_embed_from_grid": 130, "get_2d_sincos_pos_emb": 130, "get_2d_sincos_pos_embed_from_grid": 130, "get_audio_featur": 134, "get_batch_cache_indic": 166, "get_batch_idx": 134, "get_block_offset": 134, "get_buff": 166, "get_build_config_default": 143, "get_comm": 143, "get_config_group": 131, "get_context_phase_param": 143, "get_default_config_load": [80, 160], "get_default_weight_load": [80, 160], "get_executor_config": 143, "get_finish": 56, "get_first_past_key_valu": 130, "get_hf_config": 131, "get_initialized_weight_mapp": [80, 160], "get_input": 103, "get_kv_cache_ev": 143, "get_kv_cache_events_async": 143, "get_max_resource_count": [166, 167], "get_needed_resource_to_complet": [166, 167], "get_next_medusa_token": 134, "get_num_free_block": 166, "get_num_heads_kv": 134, "get_num_new_matched_token": 56, "get_output": [103, 113], "get_par": [103, 129], "get_pybind_enum_field": 143, "get_pybind_variable_field": 143, "get_pytorch_backend_config": 143, "get_request_typ": 143, "get_rope_index": 134, "get_runtime_s": 143, "get_seq_idx": 134, "get_shap": 114, "get_slic": 114, "get_stat": [143, 148], "get_stats_async": 143, "get_timestep_embed": 130, "get_token": 56, "get_us": [103, 129], "get_visual_featur": 134, "get_vocab": [0, 99], "get_weight": 130, "get_zcopi": [81, 98], "getacceptancethreshold": 0, "getacceptedlengthscumsum": 1, "getacceptedpackedpath": 1, "getadditionalmodeloutput": 0, "getadditionaloutputnam": 0, "getaddr": 0, "getaddress": 1, "getagentst": 0, "getallnewtoken": 1, "getallottedtimem": 0, "getattentionconfig": 0, "getattentiondpeventsgatherperiodm": 0, "getattr": [11, 58], "getbackend": 0, "getbackendagentdesc": 0, "getbackendtyp": 0, "getbadword": 0, "getbatchingtyp": 0, "getbatchsizet": 0, "getbeamsearchbuff": 1, "getbeamsearchdiversityr": 0, "getbeamwidth": 0, "getbeamwidtharrai": 0, "getbuffermanag": 1, "getcacheindirectioninput": 1, "getcacheindirectionoutput": 1, "getcachesaltid": 0, "getcachest": 0, "getcachetransceiverconfig": 0, "getcapac": 1, "getcapacityschedulerpolici": 0, "getclientid": 0, "getcommptr": 1, "getcommst": 0, "getcommunicationmod": 0, "getcommunicationtyp": 0, "getconf": 19, "getconfig": 0, "getconnect": 0, "getconnectioninfo": 0, "getcontextchunkingpolici": 0, "getcontextexecutor": 0, "getcontextfmha": 1, "getcontextparallel": 1, "getcontextparallelgroup": 1, "getcontextparallelrank": 1, "getcontextphaseparam": 0, "getcopyonpartialreus": 0, "getcpu": 1, "getcpudiff": 1, "getcrossattentionmask": 0, "getcrosskvcachefract": 0, "getcudagraphcaches": 0, "getcudagraphmod": 0, "getcumlogprob": 1, "getdata": 0, "getdatatyp": [0, 1], "getdatatypenam": 1, "getdebugconfig": 0, "getdebuginputtensor": 0, "getdebugoutputtensor": 0, "getdebugtensornam": 0, "getdebugtensorsmaxiter": 0, "getdecodedurationm": 0, "getdecoderetentionprior": 0, "getdecoderstream": 1, "getdecodingconfig": 0, "getdecodingmod": 0, "getdefaultbatchslot": 1, "getdefaulteaglechoic": 1, "getdesc": 0, "getdevic": 1, "getdevicecacheperc": 0, "getdeviceid": 0, "getdeviceof": 1, "getdimens": 1, "getdirectori": 0, "getdrafttoken": 0, "getdstdesc": 0, "getdynamicbatchconfig": 0, "getdynamicbatchmovingaveragewindow": 0, "getdynamictreemaxtopk": 0, "geteaglebuff": 1, "geteaglechoic": 0, "geteagleconfig": 0, "getearlystop": 0, "getembeddingbia": 0, "getembeddingt": 0, "getenablebatchsizetun": 0, "getenableblockreus": 0, "getenablechunkedcontext": 0, "getenablecontextfmhafp32acc": 0, "getenablemaxnumtokenstun": 0, "getenablepartialreus": 0, "getenabletrtoverlap": 0, "getencodedvocab": 0, "getencoderhiddens": 1, "getencoderinputfeatur": 0, "getencoderinputtokenid": 0, "getencoderoutputlength": 0, "getendid": 0, "geterrormsg": 0, "geteventbuffermaxs": 0, "getexecutionconfig": 1, "getexplicitdrafttokensbuff": 1, "getextendedruntimeperfknobconfig": 0, "getexternaldrafttokensconfig": 0, "getfailfastonattentionwindowtoolarg": 0, "getfastlogit": 0, "getfd": 0, "getfinishedsum": 1, "getfinishreason": 1, "getfirstgentoken": 0, "getfirstlocallay": 1, "getfreegpumemoryfract": 0, "getfrequencypenalti": 0, "getfunctionpoint": 0, "getgatheredid": 1, "getgathergenerationlogit": 0, "getgemmallreducedtyp": 1, "getgenerationstep": 1, "getgenexecutor": 0, "getgpu": 1, "getgpudiff": 1, "getgpuspergroup": 1, "getgpuspernod": 1, "getgpuweightsperc": [0, 110], "getguid": 0, "getguideddecodingconfig": 0, "getguideddecodingparam": 0, "getguidetyp": 0, "gethandl": 0, "gethiddens": 1, "gethostcaches": 0, "gethostmemori": 1, "getid": 1, "getinittozero": 1, "getinputtokenextraid": 0, "getinputtokenid": 0, "getinst": [0, 1], "getipcunicastpoint": 1, "getisorchestr": 0, "getiterstatsmaxiter": 0, "getjointdecodinginput": 1, "getjointdecodingoutput": 1, "getkvcacheconfig": 0, "getkvcacheconfigref": 0, "getkvcacheeventmanag": 0, "getkvcacheretentionconfig": 0, "getkvcachetyp": 1, "getkvdatatyp": 1, "getlanguageadapteruid": 0, "getlastrank": 1, "getlatestdebugtensor": 0, "getlatestev": 0, "getlatestiterationstat": [0, 99], "getlatestrequeststat": 0, "getlayertyp": 1, "getlen": 0, "getlengthpenalti": 0, "getlevel": 1, "getlocalagentdesc": 0, "getlocalrank": 1, "getlogit": 0, "getlogitsdtyp": 1, "getlogitspostprocessor": 0, "getlogitspostprocessorconfig": 0, "getlogitspostprocessornam": 0, "getlogprob": 1, "getlookaheadbuff": 1, "getlookaheadconfig": 0, "getlookaheaddecodingconfig": 0, "getlookaheaddecodingmaxnumrequest": 0, "getloraconfig": 0, "getloramodul": 1, "getloraprefetchdir": 0, "getmanagedweightsmapopt": 1, "getmanageweightstyp": 1, "getmaxadapters": 0, "getmaxattentionwindowvec": 0, "getmaxbatchs": [0, 1], "getmaxbeamwidth": [0, 1], "getmaxdecodingdecodertoken": 1, "getmaxdecodingdrafttoken": 1, "getmaxdecodingenginetoken": 1, "getmaxdecodingtoken": 1, "getmaxdraftpathlen": 1, "getmaxencoderlen": 1, "getmaxgputotalbyt": 0, "getmaxinputlen": 1, "getmaxlorarank": 1, "getmaxnonleafnodesperlay": 1, "getmaxnumpath": 1, "getmaxnumsequ": 1, "getmaxnumtoken": [0, 1], "getmaxpagesperblock": 1, "getmaxpagesperblockdevic": 0, "getmaxpagesperblockhost": 0, "getmaxpathlen": 1, "getmaxpositionembed": 1, "getmaxpromptembeddingtables": 1, "getmaxqueues": 0, "getmaxseqidlemicrosecond": 0, "getmaxsequencelen": 1, "getmaxsequencelength": 1, "getmaxtoken": 0, "getmaxtokensinbuff": 0, "getmedusachoic": [0, 1], "getmemorytyp": [0, 1], "getmemorytypenam": 1, "getminp": 0, "getmintoken": 0, "getmlphiddens": 1, "getmodelconfig": [0, 1], "getmodelconfigmut": 1, "getmodelnam": 1, "getmodelvari": 1, "getmpist": 0, "getmropeconfig": 0, "getmropepositiondelta": 0, "getmroperotarycossin": 0, "getmultiblockmod": 0, "getmulticastpoint": 1, "getmultimodalembed": 0, "getmultimodalhash": 0, "getmultimodalinput": 0, "getmultimodallength": 0, "getmultimodalposit": 0, "getnam": [0, 1], "getnbattentionlay": 1, "getnbhead": 1, "getnbkvhead": 1, "getnblay": 1, "getnbrnnlay": 1, "getnextdrafttoken": 1, "getnextdrafttokenslength": 1, "getngrams": 0, "getnoderank": 1, "getnoderankof": 1, "getnorepeatngrams": 0, "getnormalizelogprob": 0, "getnotifiedsyncmessag": 0, "getnumcopystream": [0, 1], "getnumdecodingenginetoken": 1, "getnumdevicemodulelay": 0, "getnumensurework": 0, "getnumhostmodulelay": 0, "getnumkvheadsforgivenlay": 1, "getnumkvheadsperlay": 1, "getnumkvheadsperlayerlocalrang": 1, "getnumlanguag": 1, "getnumnod": 0, "getnumpackedmask": 1, "getnumpag": 1, "getnumputwork": 0, "getnumresponsesreadi": 0, "getnumreturnbeam": [0, 1], "getnumreturnsequ": 0, "getnumtransformerlay": 1, "getonboardblock": 0, "getop": 0, "getoptimaladapters": 0, "getoptprofilessplitpoint": 1, "getorchestratorconfig": 0, "getorchleadercomm": 0, "getoutputconfig": 0, "getpadid": 0, "getpagedcontextfmha": 1, "getpageptr": 1, "getpagewidth": 1, "getparallelconfig": 0, "getparentid": 1, "getparticipantid": 0, "getpath": 1, "getpathopt": 1, "getpeftcacheconfig": 0, "getperblockretentionprioritydur": 0, "getpin": 1, "getpinneddiff": 1, "getpinnedpool": 1, "getpinnedpooldiff": 1, "getpipelineparallel": 1, "getpipelineparallelgroup": 1, "getpipelineparallelrank": 1, "getpositionid": 0, "getposteriorthreshold": 0, "getppreducescatt": 1, "getprecis": 1, "getpresencepenalti": 0, "getprevdrafttokenslength": 1, "getprior": 0, "getprocessorbatch": 0, "getprocessormap": 0, "getprompttableoffload": 0, "getprompttuningconfig": 0, "getquantmod": 1, "getrank": 1, "getrecvpollperiodm": 0, "getremotenam": 0, "getrepetitionpenalti": 0, "getrepl": 0, "getreqid": 0, "getrequestid": 0, "getrequeststatsmaxiter": 0, "getrequesttyp": 0, "getresult": [0, 99], "getreturnallgeneratedtoken": 0, "getrnnconfig": 1, "getrotaryembeddingdim": 1, "getruntimedefault": 1, "getruntimetyp": 0, "getsamplingconfig": [0, 1], "getschedulerconfig": 0, "getschedulerconfigref": 0, "getse": 0, "getsecondaryoffloadminprior": 0, "getselfidx": 0, "getsequencelength": 1, "getserializedst": 0, "getshap": [0, 1], "getsinktokenlength": 0, "getsiz": [0, 1], "getsizeinbit": 1, "getsizeinbyt": [0, 1], "getsizeperhead": 1, "getskipcrossattnblock": 0, "getslotsperpag": 1, "getsocketst": 0, "getspawnprocess": 0, "getspecdecconfig": 0, "getspeculativedecodingmod": 1, "getspeculativedecodingmodul": 1, "getspeculativedecodingmoduleptr": 1, "getsrcdesc": 0, "getstat": 0, "getstatu": 1, "getstoptokenid": 0, "getstopword": 0, "getstream": [0, 1], "getsyncmessag": 0, "gettag": 0, "gettaskid": 0, "gettemperatur": 0, "gettensorparallel": 1, "gettensorparallelgroup": 1, "gettensorparallelrank": 1, "getter": 102, "gettoken": 0, "gettokenizerstr": 0, "gettokenrangeretentionconfig": 0, "gettokensperblock": 1, "gettopk": 0, "gettopp": 0, "gettoppdecai": 0, "gettoppmin": 0, "gettoppresetid": 0, "gettotalnumpag": 1, "gettransfermod": 0, "gettyp": [0, 1], "getunderlyingdecod": 1, "getunicastpoint": 1, "getusegpudirectstorag": 0, "getuseuvm": 0, "getuvm": 1, "getuvmdiff": 1, "getverificationsets": 0, "getvers": 1, "getvirtualmemoryalloc": 1, "getvirtualmemorymanag": 1, "getvocabs": 1, "getvocabsizepad": 1, "getweight": 0, "getwindows": 0, "getworkerexecutablepath": 0, "getworlds": 1, "gh200": [33, 146, 148], "ghz": 51, "gib": [105, 135], "gid": [0, 34], "gigabyt": 5, "gij": 8, "gil": 19, "git": [2, 9, 17, 34, 90, 94, 106, 139, 142, 164], "github": [2, 11, 12, 15, 17, 21, 28, 29, 30, 31, 58, 78, 90, 93, 94, 95, 115, 148, 158, 164], "give": [9, 13, 14, 18, 19, 20, 70, 75, 91, 99, 123, 126, 131, 154, 165], "given": [0, 1, 2, 5, 8, 10, 15, 19, 29, 33, 36, 83, 88, 90, 92, 99, 102, 106, 114, 115, 119, 124, 125, 126, 129, 130, 131, 133, 134, 135, 137, 143, 148, 166], "givyboi": 62, "glm": [129, 138, 148], "glm4": 148, "glob": 33, "global": [0, 8, 12, 14, 18, 19, 101, 104, 113, 148], "global_max_input_length": 134, "global_max_output_length": 134, "globalrequestid": 0, "glossari": [3, 6], "gm": [75, 139, 154], "gn0": 8, "gn1": 8, "gnm": 8, "gnu": 94, "go": [10, 11, 72, 88, 101, 102, 122, 148], "goal": [15, 20, 128], "goe": [13, 37, 120, 142], "golden": 10, "good": [2, 11, 14, 15, 19, 37, 99, 113, 120, 122, 126, 127, 143], "googl": [79, 138, 145, 159], "got": [0, 1, 11, 47, 51, 52, 53, 54, 55, 58, 62, 97, 120, 139, 140, 143, 147], "gpqa": [12, 14, 23, 29], "gpt": [1, 4, 7, 22, 27, 33, 37, 72, 90, 93, 101, 109, 113, 120, 129, 135, 137, 138, 139, 145, 148], "gpt2": [131, 139], "gpt3": 5, "gpt_attent": [6, 101, 103, 129, 148], "gpt_attention_plugin": [22, 106, 113, 120, 130, 134, 139, 148], "gpt_attention_plugin_remove_pad": 103, "gpt_ib_ptun": 33, "gpt_oss": 29, "gpt_oss_output": 29, "gpt_variant": [131, 148], "gptattent": 103, "gptattentionpluginremovepaddingrewritepass": 103, "gptconfig": 131, "gptdecod": 102, "gptdecoderbatch": 148, "gptdecoderptr": 1, "gptforcausallm": 131, "gptj": 131, "gptjconfig": 131, "gptjforcausallm": 131, "gptjmodel": 131, "gptlmheadmodel": 139, "gptmanag": 148, "gptmanagerbenchmark": [94, 105, 148], "gptmodel": 131, "gptmodelconfig": 148, "gptneoxforcausallm": 131, "gptneoxmodel": 131, "gptq": [7, 90, 138, 148], "gptsession": 148, "gptsessionbenchmark": 148, "gpu": [0, 1, 4, 5, 6, 7, 8, 9, 13, 16, 17, 18, 19, 20, 22, 23, 25, 26, 28, 29, 30, 31, 33, 35, 38, 42, 48, 56, 57, 59, 60, 61, 63, 72, 73, 74, 78, 81, 83, 84, 85, 86, 87, 88, 89, 90, 93, 94, 97, 98, 99, 100, 101, 102, 104, 105, 106, 107, 109, 112, 115, 121, 122, 123, 127, 129, 131, 134, 138, 139, 142, 143, 148, 150, 151, 152, 153, 156, 158, 162, 163], "gpu_typ": 33, "gpu_weights_perc": [110, 134], "gpudirect": 0, "gpumemusag": [0, 26], "gpus_per_nod": [22, 23, 26, 143], "gpuspernod": [1, 102], "gpusync": 1, "gpuweightsperc": [0, 110], "gqa": [3, 6, 22, 72, 89, 93, 101, 104, 129, 148, 151], "grace": [15, 19, 105, 138, 143], "grade": [79, 159], "gradient": 4, "gradual": [108, 115], "grain": [15, 89, 103], "gram": [92, 93, 109, 148], "grammar": [0, 91, 93, 99, 143], "granit": [79, 138, 148, 159], "granular": [19, 73, 152], "graph": [0, 2, 13, 14, 15, 18, 19, 25, 28, 29, 30, 31, 36, 37, 44, 63, 72, 74, 75, 78, 79, 81, 82, 91, 92, 93, 98, 113, 119, 120, 129, 134, 135, 136, 139, 143, 145, 148, 151, 153, 154, 156, 158, 159, 161, 166], "graph_rewrit": 103, "graphic": [17, 56], "gratitud": 13, "gre": 26, "great": [3, 8, 10, 15, 17, 20, 29, 81], "greater": [0, 6, 7, 8, 12, 15, 22, 72, 81, 83, 98, 101, 129, 143], "greatli": [10, 13, 19, 105, 115, 123, 127], "greedi": [0, 9, 35, 64, 91, 92, 102, 150], "greedy_sampl": 143, "greedysampl": 0, "greedysamplinghost": 1, "greener": 11, "grid": [113, 123, 126, 129, 130], "grid_search_engin": 121, "grid_siz": 130, "grok": [138, 148], "groovi": 33, "gross": 28, "ground": [36, 119], "group": [0, 3, 14, 15, 19, 28, 29, 30, 31, 83, 93, 99, 100, 102, 104, 113, 129, 130, 137, 143, 148, 151], "group_cl": 131, "group_norm": 129, "group_rms_norm": 148, "group_siz": [112, 129, 143], "groupedrmsnorm": 12, "groupgemm": [14, 15], "groupnorm": [129, 130], "grow": [1, 16, 18, 72, 88, 109, 126], "gsm8k": [14, 28, 30, 31], "gt": 129, "gtc": [2, 12], "guarante": [0, 8, 10, 15, 19, 37, 38, 78, 93, 102, 105, 115, 120, 121, 123, 128, 158], "guaranteed_no_evict": [0, 21, 37, 120, 128, 143], "guaranteednoevictschedul": 167, "guard": 121, "guardian": [79, 159], "guid": [0, 2, 7, 9, 17, 20, 28, 29, 30, 31, 36, 38, 44, 50, 72, 73, 78, 82, 86, 90, 97, 113, 119, 121, 122, 123, 125, 127, 129, 139, 142, 143, 145, 146, 148, 151, 152, 158, 161], "guidanc": [8, 19, 72, 91, 109, 127, 130, 131], "guided_decod": [51, 91, 143], "guided_decoding_backend": [51, 70, 91, 143], "guided_decoding_param": 91, "guideddecod": 10, "guideddecodingbackend": 0, "guideddecodingconfig": [0, 99], "guideddecodingparam": [0, 51, 91, 93, 99, 143], "guidedrequest": 10, "guidelin": [78, 122, 158], "guidetyp": [0, 99], "gw": 103, "h": [9, 11, 13, 17, 19, 20, 22, 28, 29, 30, 31, 39, 40, 41, 72, 74, 81, 99, 101, 109, 114, 121, 129, 131, 147, 148, 153, 156], "h0": 13, "h1": 129, "h100": [7, 17, 22, 28, 33, 38, 93, 115, 121, 123, 124, 125, 126, 146, 148], "h20": 22, "h200": [4, 10, 22, 28, 38, 93, 146, 148], "ha": [0, 1, 2, 3, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 28, 29, 30, 31, 35, 36, 37, 38, 58, 72, 80, 81, 83, 88, 92, 93, 94, 95, 99, 101, 105, 106, 107, 112, 113, 114, 115, 117, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 134, 135, 137, 139, 143, 147, 148, 150, 160, 166, 167], "habitu": 33, "had": [2, 14, 88, 115, 123, 126], "half": [0, 1, 14, 19, 113, 121, 129], "halv": [4, 129], "hand": [33, 105, 109, 122], "handl": [0, 1, 3, 8, 10, 11, 12, 16, 19, 20, 35, 80, 81, 84, 86, 88, 92, 98, 100, 104, 114, 115, 121, 123, 126, 127, 128, 129, 130, 143, 144, 146, 149, 150, 160], "handle_per_step": 134, "handler": 11, "hang": [0, 10, 15, 81, 98, 139, 148], "hao": 11, "happen": [1, 10, 11, 15, 83, 97, 99, 102, 105, 113, 135, 139], "happi": 134, "har": [14, 28, 30, 31], "hard": [11, 72, 79, 101, 143, 159], "harder": 102, "hardwar": [7, 14, 17, 18, 20, 25, 27, 33, 35, 42, 93, 94, 104, 148], "has_affin": 129, "has_bia": 129, "has_config_group": 131, "has_position_embed": 134, "has_scal": 129, "has_token_type_embed": 134, "has_zero_point": [112, 143], "hascontextawaitthread": 0, "hasdraftlogit": 1, "haserror": [0, 99], "hasgenawaitthread": 0, "hash": [0, 56, 83, 86, 143], "hash_valu": 56, "hashed_token": 56, "hasresult": 0, "hasrnnconfig": 1, "hasspeculativedecodingmodul": 1, "hattizai": 148, "have": [0, 1, 2, 3, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 28, 29, 30, 31, 35, 36, 37, 38, 49, 56, 59, 60, 61, 62, 65, 72, 75, 79, 80, 81, 83, 84, 88, 90, 92, 97, 98, 99, 100, 101, 102, 105, 106, 108, 109, 112, 113, 114, 115, 116, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 134, 135, 138, 139, 142, 143, 144, 147, 148, 149, 154, 159, 160], "hbm": 93, "hbm3e": 5, "hcxvisionforcausallm": [138, 145], "head": [1, 3, 13, 14, 18, 22, 74, 79, 81, 83, 93, 102, 104, 109, 113, 120, 129, 130, 143, 148, 151, 153, 159], "head_dim": [72, 151, 166], "head_siz": [72, 101, 129, 131, 134, 148], "header": 98, "headquart": 56, "headsiz": 129, "headsperlay": 1, "health": [17, 20, 26, 28, 29, 30, 31, 62, 93], "healthi": 9, "heat": 102, "heavi": [19, 33, 127], "heavier": 122, "heavili": [8, 15, 19], "height": [8, 25, 46, 130, 134], "held": [10, 19, 83], "hello": [35, 47, 48, 49, 52, 53, 54, 55, 59, 62, 63, 85, 90, 91, 97, 121, 128, 140, 142, 147, 162, 164, 165], "help": [10, 12, 13, 15, 16, 19, 22, 23, 28, 29, 30, 31, 36, 37, 38, 39, 40, 51, 57, 58, 62, 63, 64, 65, 66, 67, 70, 72, 74, 81, 84, 88, 94, 98, 99, 101, 103, 113, 119, 120, 121, 124, 125, 126, 127, 128, 129, 142, 143, 147, 148, 150, 153, 156], "helper": [1, 33, 129], "henc": [10, 19, 144, 149], "hendryck": 23, "here": [2, 4, 5, 10, 11, 13, 14, 15, 23, 25, 28, 29, 30, 31, 32, 42, 47, 51, 58, 72, 80, 83, 90, 91, 94, 97, 99, 103, 106, 110, 111, 112, 113, 114, 115, 117, 119, 121, 122, 123, 126, 127, 129, 134, 135, 137, 139, 141, 143, 147, 151, 160, 166, 167], "hesit": 11, "heterogen": [8, 81, 98], "heurist": [14, 21, 34, 37, 72, 93, 101, 120, 129, 143, 148], "hf": [20, 21, 22, 23, 26, 37, 38, 47, 52, 53, 54, 55, 59, 60, 61, 69, 74, 78, 79, 80, 90, 97, 102, 106, 110, 114, 120, 121, 134, 138, 139, 140, 142, 143, 147, 153, 156, 158, 159, 160, 164], "hf_config_or_dir": 131, "hf_gemma3": [80, 160], "hf_home": 34, "hf_lora_convert": 106, "hf_model": [120, 131], "hf_model_card_or_dir": [77, 157], "hf_model_dir": [80, 110, 111, 112, 115, 131, 160], "hf_model_nam": 120, "hf_model_or_dir": 131, "hf_quant_config": [37, 120], "hf_token": [9, 37, 120], "hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx": 9, "hfcheckpointload": [80, 160], "hfconfigordir": 131, "hgx": 5, "hi": 106, "hidden": [0, 11, 12, 13, 19, 20, 72, 75, 89, 92, 99, 100, 101, 102, 106, 109, 129, 130, 143, 148, 154], "hidden_act": [112, 130, 131], "hidden_dim": [0, 72, 101, 129], "hidden_dim_per_head": [72, 88, 101, 129], "hidden_dtyp": 130, "hidden_s": [0, 72, 75, 103, 112, 114, 129, 130, 131, 134, 144, 149, 151, 154], "hidden_size_in": 106, "hidden_size_out": 106, "hidden_size_per_head": 129, "hidden_st": [92, 111, 129, 130, 131, 134, 139, 144, 149], "hidden_states_for_emb": 131, "hiddens": [0, 1, 102], "hide": [10, 12, 14, 35], "hierarch": 112, "hierarchi": [115, 129], "high": [3, 7, 8, 10, 11, 12, 13, 14, 16, 17, 18, 19, 28, 29, 30, 31, 32, 49, 56, 77, 79, 81, 88, 89, 90, 93, 99, 109, 111, 113, 115, 120, 128, 129, 135, 142, 146, 148, 157, 159], "higher": [0, 1, 3, 4, 6, 8, 10, 14, 15, 16, 18, 19, 20, 38, 64, 72, 75, 83, 84, 88, 91, 101, 102, 105, 106, 109, 114, 128, 135, 144, 148, 149, 154], "highest": [4, 5, 8, 20, 32, 49, 75, 102, 103, 143, 154], "highli": [11, 14, 15, 18, 19, 20, 35, 36, 109, 113, 119, 123], "highlight": [4, 7, 88, 93, 123, 126], "highwai": 29, "hin": 13, "hinderlit": 28, "hint": [19, 32, 129], "histor": [19, 89], "histori": 14, "hit": [0, 14, 18, 35, 38, 56, 57, 126, 127, 143, 148], "hk": 109, "hmac": 143, "hmm": 11, "ho": 106, "hoc": [115, 134], "hold": [0, 1, 10, 11, 15, 57, 80, 83, 89, 99, 100, 103, 104, 105, 106, 109, 122, 130, 135, 143, 150, 160], "home": [2, 34, 37, 120], "home_dir": 34, "homo_head_pattern": 130, "homogen": [81, 98], "honor": 29, "hood": 142, "hope": [8, 11, 12, 15, 16], "hopefulli": 10, "hopper": [2, 3, 4, 7, 9, 13, 14, 15, 20, 22, 27, 72, 90, 93, 94, 101, 105, 123, 138, 146, 148], "horizont": [14, 22], "host": [1, 9, 14, 16, 17, 18, 20, 25, 26, 28, 29, 30, 31, 35, 43, 57, 61, 81, 94, 97, 106, 117, 127, 129, 143, 147, 148], "host_cache_s": [57, 83, 85, 143, 162], "host_context_length": [129, 130, 131, 134, 139], "host_context_progress": [129, 130, 139], "host_cross_kv_cache_block_offset": [130, 134], "host_cross_kv_cache_pool_map": 130, "host_cross_kv_cache_pool_point": 130, "host_kv_cache_block_offset": [129, 130, 134, 139], "host_kv_cache_block_point": 139, "host_kv_cache_pool_map": [129, 130, 139], "host_kv_cache_pool_point": [129, 130, 139], "host_max_attention_window_s": [129, 130, 139], "host_past_key_value_length": [129, 130, 139], "host_path": [28, 29, 31], "host_request_typ": [129, 130, 131, 139], "host_runtime_perf_knob": [129, 130, 139], "host_sink_token_length": [129, 130, 139], "hostcaches": [0, 105], "hostfunc": 10, "hostmemori": 1, "hostnam": [16, 26, 81], "hot": [15, 19, 89], "hottest": 15, "hour": 121, "hous": [15, 122], "how": [0, 8, 15, 16, 18, 19, 20, 22, 25, 29, 32, 33, 36, 37, 47, 57, 59, 72, 74, 77, 81, 85, 86, 88, 90, 93, 98, 99, 109, 111, 113, 114, 115, 117, 119, 121, 123, 124, 125, 127, 129, 135, 137, 139, 141, 143, 147, 150, 151, 153, 156, 157, 162], "howev": [2, 3, 8, 10, 11, 12, 13, 14, 15, 19, 20, 37, 72, 75, 84, 88, 92, 99, 101, 109, 115, 120, 122, 123, 126, 127, 128, 135, 144, 148, 149, 150, 154], "hpc": 4, "html": [1, 21, 28, 29, 30, 31, 113, 129, 139], "http": [0, 1, 2, 9, 11, 12, 17, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 33, 34, 39, 40, 41, 58, 66, 67, 68, 69, 70, 81, 85, 87, 90, 94, 97, 100, 106, 113, 115, 129, 137, 139, 142, 147, 148, 156, 162, 163, 164], "http_code": [9, 17, 20, 28, 29, 30, 31], "huang": 56, "hub": [17, 20, 23, 28, 29, 30, 31, 37, 62, 90, 120, 143, 148, 164], "hudson": 29, "hug": [23, 28, 29, 30, 31, 37, 42, 78, 79, 90, 99, 106, 110, 115, 120, 131, 143, 147, 148, 158, 159], "huge": [19, 148], "hugepag": 19, "hugepages": 19, "hugepages_fre": 19, "hugepages_rsvd": 19, "hugepages_surp": 19, "hugepages_tot": 19, "hugetlb": 19, "huggingfac": [0, 2, 9, 20, 21, 28, 29, 30, 31, 34, 37, 38, 40, 62, 67, 78, 85, 92, 106, 111, 112, 114, 115, 120, 121, 138, 139, 142, 144, 145, 146, 147, 148, 149, 158, 162], "huggingface_exampl": [90, 164], "huggingface_hub": [9, 62], "huggingface_model_card": [90, 164], "human": [12, 35, 37, 120], "hundr": 15, "hurt": [10, 14, 15, 88, 127], "hw": [12, 14, 15], "hybrid": [89, 100, 148], "hyper": 112, "hyperclova": 136, "hyperclovax": [138, 148], "hypothes": 91, "hypothesi": 109, "i": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 37, 38, 39, 41, 42, 47, 48, 49, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 68, 72, 73, 74, 75, 76, 78, 79, 80, 81, 84, 85, 87, 88, 89, 90, 91, 92, 93, 94, 95, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 109, 110, 111, 112, 113, 114, 115, 117, 120, 121, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 134, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 158, 159, 160, 162, 164, 165, 166, 167], "ia3": [72, 101], "iactivationlay": 113, "ib": [33, 81, 98], "ibm": [79, 159], "ibrahimamin1": 148, "ibufferptr": 1, "iconstantlay": 129, "icudaengin": [134, 135], "id": [0, 1, 10, 13, 15, 17, 19, 20, 21, 23, 28, 29, 30, 31, 37, 38, 42, 54, 58, 72, 83, 89, 92, 99, 105, 120, 129, 130, 134, 142, 143, 147, 151, 166], "idea": [10, 14, 15, 106, 127], "ideal": [8, 15, 16, 92, 103, 123, 126, 148], "idempot": 143, "ident": [14, 19, 22, 91, 94, 99, 105, 129, 165], "identifi": [0, 11, 15, 16, 35, 37, 102, 106, 109, 113, 120, 126, 129, 143, 148], "idl": [0, 19, 35, 87, 163], "idtyp": [0, 99], "idx": 134, "ieee": 137, "ieinsumlay": 129, "ielementwiselay": 129, "iexecutioncontext": [134, 135], "ifb": [8, 16, 81, 93, 109, 136, 148], "ifilllay": 129, "igatherlay": 129, "ignor": [9, 22, 25, 28, 29, 30, 31, 34, 37, 75, 120, 129, 134, 143, 154], "ignore_eo": [143, 148], "igptdecod": 1, "ihostmemori": [1, 113, 134], "ii": [29, 101, 129], "ij": 129, "ijk": 129, "ijl": 129, "ik": 129, "ikl": 129, "ilay": [103, 113], "illeg": 148, "illustr": [8, 11, 12, 13, 15, 16, 25, 28, 35, 81, 84, 87, 88, 103, 109], "ilogg": 1, "ilooplay": 129, "imag": [9, 20, 21, 25, 26, 28, 29, 30, 31, 37, 40, 46, 59, 60, 61, 67, 86, 93, 96, 97, 120, 130, 134, 138, 145, 148], "image64": 67, "image_base64": 26, "image_data_format": 21, "image_grid_thw": 134, "image_patches_indic": 134, "image_path": 134, "image_s": 131, "image_tag": 95, "image_token_index": 134, "image_url": [26, 40, 67], "imagin": 11, "imatrixmultiplylay": 129, "imb": 15, "imbal": [8, 15, 19, 20, 89, 126], "imbalanc": 15, "immedi": [8, 11, 19, 35, 72, 78, 101, 109, 121, 139, 158], "immut": 1, "impact": [3, 7, 8, 12, 13, 14, 15, 19, 26, 35, 62, 88, 107, 109, 122, 123, 126, 127, 128], "imped": [7, 15], "impl": [0, 167], "implement": [3, 10, 14, 16, 17, 20, 21, 26, 56, 58, 74, 78, 79, 80, 81, 88, 89, 91, 92, 93, 98, 99, 101, 102, 104, 108, 109, 112, 113, 115, 129, 130, 131, 137, 138, 139, 143, 144, 148, 149, 150, 153, 158, 159, 160, 166, 167], "impli": 10, "implic": [8, 10], "implicit": [1, 10, 101, 109, 129], "implicitli": 1, "import": [1, 3, 7, 10, 11, 13, 14, 17, 19, 21, 32, 33, 35, 42, 47, 48, 49, 51, 52, 53, 54, 55, 56, 57, 58, 62, 63, 64, 65, 66, 67, 68, 69, 70, 77, 80, 83, 84, 85, 88, 90, 91, 92, 97, 107, 109, 114, 115, 121, 123, 126, 127, 128, 138, 140, 141, 142, 144, 147, 148, 149, 150, 157, 160, 162, 164, 165, 166], "importantli": [15, 34], "impos": 7, "imposs": 10, "improv": [3, 4, 5, 6, 7, 8, 10, 12, 13, 14, 15, 16, 18, 19, 20, 22, 25, 28, 29, 30, 31, 35, 37, 38, 47, 52, 53, 55, 58, 63, 72, 86, 87, 88, 97, 101, 105, 107, 113, 120, 121, 123, 124, 125, 126, 127, 140, 141, 142, 143, 147, 148, 151, 163], "in_channel": 130, "in_featur": [112, 113, 130], "in_hidden_s": 129, "in_len": 103, "in_point": 129, "in_progress": 134, "incex": 143, "includ": [0, 1, 3, 4, 6, 8, 10, 11, 13, 14, 15, 16, 19, 20, 21, 22, 23, 26, 28, 29, 30, 31, 32, 33, 35, 37, 38, 42, 51, 72, 75, 78, 79, 81, 86, 88, 92, 94, 99, 101, 102, 105, 106, 108, 109, 112, 113, 114, 120, 123, 127, 129, 137, 139, 140, 142, 143, 146, 148, 150, 151, 154, 158, 159, 166, 167], "include_stop_str_in_output": 143, "inclus": [81, 98, 129], "incom": [8, 35], "incompat": [22, 78, 141, 148, 158], "inconsist": 11, "incorpor": [0, 12, 15, 78, 86, 148, 158], "incorrect": [105, 109, 148], "increas": [0, 2, 4, 5, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 33, 35, 36, 37, 58, 72, 81, 83, 84, 88, 93, 101, 105, 109, 113, 119, 120, 121, 123, 127, 128, 129, 148, 167], "increasingli": [8, 11, 19], "increment": [15, 94, 148], "incur": [12, 16, 35, 81, 98, 113], "inde": 135, "indent": 143, "independ": [0, 1, 16, 81, 89, 98, 99, 109, 129], "index": [0, 1, 12, 17, 19, 20, 28, 29, 30, 31, 33, 42, 72, 93, 97, 99, 104, 109, 114, 129, 134, 143, 147, 148, 151], "index_select": 129, "indic": [0, 1, 8, 10, 11, 16, 19, 20, 72, 83, 99, 101, 102, 109, 112, 128, 129, 130, 134, 135, 143, 166], "indim": 1, "indimfirst": 1, "indirect": 1, "individu": [8, 12, 15, 16, 33, 148, 160], "indivis": 148, "inductor": 143, "industri": [20, 37, 120], "ineffect": 10, "ineffici": [11, 12, 72, 88, 101], "inetworkdefinit": [103, 113, 129], "inevit": [10, 113], "infeas": 99, "infer": [0, 2, 3, 4, 5, 8, 10, 12, 14, 19, 22, 28, 29, 30, 31, 35, 36, 38, 40, 44, 67, 74, 75, 78, 79, 81, 83, 84, 85, 86, 89, 90, 92, 93, 98, 102, 106, 109, 113, 115, 119, 121, 122, 123, 124, 125, 127, 128, 129, 134, 137, 139, 140, 142, 143, 146, 148, 150, 153, 154, 158, 159, 162], "infer_shap": 134, "inferenceoptim": [75, 154], "inferencerequest": 148, "infin": [10, 117], "infiniband": [81, 98], "infinit": [37, 38, 113, 120], "inflat": 12, "inflight": [0, 26, 72, 88, 101, 106, 109, 118, 120, 124, 125, 126, 129, 143, 148, 151, 167], "inflight_request_id": 167, "inflightbatch": 0, "inflightbatchingstat": 0, "influenc": [12, 127], "info": [0, 15, 21, 22, 23, 25, 26, 29, 56, 76, 120, 135, 139, 148, 155], "inform": [0, 1, 3, 6, 10, 12, 13, 15, 16, 19, 21, 26, 28, 29, 30, 31, 37, 38, 64, 70, 72, 77, 78, 80, 81, 83, 86, 94, 97, 98, 99, 101, 102, 104, 109, 112, 113, 120, 121, 138, 139, 148, 157, 158, 160], "infrastructur": [8, 11, 80, 160], "infti": [8, 102], "ingest": [75, 154], "inher": [8, 15, 89], "inherit": [11, 75, 91, 114, 115, 129, 143, 144, 149, 150, 154, 166, 167], "init": [1, 2, 14, 17, 94, 148], "init_audio_encod": 134, "init_backend": 143, "init_build_config": 143, "init_calib_config": 143, "init_image_encod": 134, "init_llm": 134, "init_model_and_config": [80, 160], "init_processor": 134, "init_token": 134, "init_with_new_llm": 11, "initi": [1, 8, 9, 10, 11, 12, 15, 16, 17, 18, 20, 21, 28, 29, 30, 31, 34, 35, 37, 72, 73, 74, 75, 80, 81, 83, 85, 88, 92, 98, 109, 114, 120, 123, 126, 127, 135, 139, 143, 144, 148, 149, 151, 152, 153, 154, 160, 162, 167], "initial_global_assign": 15, "initial_prompt": 11, "initial_prompt_token_num": 11, "initialis": 143, "initializecommand": 34, "initializer_list": [0, 1], "initmemorypool": 135, "inittozero": 1, "inject": 92, "inlin": [0, 1], "inner": [75, 129, 154], "inner_layernorm": [130, 131], "innov": [14, 15, 17], "inp": [129, 143], "inpaint": [40, 67], "inplac": [10, 91], "inprogress": 1, "input": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 13, 14, 15, 16, 18, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 32, 35, 36, 37, 38, 42, 45, 46, 60, 67, 81, 83, 84, 86, 88, 89, 92, 93, 99, 102, 103, 105, 106, 107, 109, 113, 114, 119, 120, 121, 122, 124, 125, 127, 128, 129, 130, 131, 134, 135, 139, 143, 144, 148, 149, 150, 151, 167], "input_1": 129, "input_1_": 129, "input_audio": 134, "input_featur": 131, "input_fil": 148, "input_id": [12, 37, 105, 111, 120, 129, 131, 134, 139, 144, 149], "input_imag": 134, "input_layernorm": [111, 112, 114, 144, 149], "input_length": [129, 130, 131, 134], "input_list": 129, "input_n": 129, "input_n_": 129, "input_sequence_len": 18, "input_str": 11, "input_text": [111, 113, 134], "input_timing_cach": [22, 143], "input_token_extra_id": 134, "inputbuff": 1, "inputdesc": 113, "inputdtyp": 1, "inputgentokenshost": 1, "inputpack": [1, 102], "inputs_emb": [144, 149], "inputtokenextraid": 0, "inputtokenid": 0, "ins": 143, "insert": [10, 15, 78, 79, 92, 103, 113, 120, 129, 143, 158, 159], "insertinputtensor": 1, "insid": [1, 2, 9, 13, 14, 19, 28, 29, 30, 31, 72, 94, 109, 114, 115, 129, 135, 147, 151], "insight": [8, 12, 15, 16], "insiz": 1, "inspect": [20, 22, 36, 75, 119, 135, 154], "inspir": [10, 13], "instabl": 8, "instal": [9, 17, 20, 23, 28, 29, 30, 31, 44, 59, 60, 61, 78, 93, 94, 115, 116, 121, 142, 144, 147, 148, 149, 158], "instanc": [0, 8, 10, 11, 12, 15, 16, 19, 20, 28, 29, 30, 31, 42, 56, 58, 72, 81, 83, 91, 92, 98, 99, 102, 103, 104, 109, 113, 134, 135, 142, 143, 147, 148, 151], "instance_idx": 139, "instanti": [121, 128, 143, 166], "instead": [2, 3, 8, 11, 15, 19, 20, 21, 32, 33, 42, 57, 72, 78, 83, 84, 85, 92, 94, 97, 103, 105, 109, 113, 115, 127, 128, 129, 135, 143, 148, 158, 162], "instruct": [2, 14, 16, 17, 19, 20, 23, 26, 28, 29, 30, 31, 37, 38, 40, 46, 47, 52, 65, 67, 75, 79, 86, 90, 91, 94, 95, 97, 109, 120, 121, 122, 127, 128, 138, 140, 142, 144, 145, 147, 148, 149, 154, 156, 159, 164, 165], "instrument": 14, "int": [0, 1, 32, 54, 56, 58, 72, 81, 84, 91, 102, 111, 112, 113, 115, 126, 129, 130, 131, 134, 143, 144, 149, 151, 166, 167], "int32": [1, 10, 22, 101, 129, 132, 139], "int32_t": [0, 1, 129], "int4": [7, 15, 20, 22, 42, 114, 115, 138, 148], "int4_weight": 137, "int64": [1, 10, 102, 129, 139], "int64_t": [0, 1], "int8": [1, 7, 15, 20, 21, 22, 90, 112, 114, 115, 123, 129, 135, 138, 143, 148], "int8_kv_cach": [72, 101, 137, 148], "int8_t": [0, 1], "int8_weight": 137, "int8awq": 123, "int_clip": 129, "integ": [21, 37, 70, 72, 83, 84, 88, 101, 120, 129, 137, 143, 148], "integr": [11, 15, 16, 19, 20, 21, 35, 72, 73, 77, 78, 81, 86, 93, 109, 142, 146, 148, 150, 151, 152, 156, 157, 158, 166, 167], "intellig": [17, 64, 89, 146], "intend": [23, 28, 29, 30, 31, 34, 94, 135], "intens": [8, 10, 14, 15], "intent": 121, "intention": [9, 115, 143], "intenum": [92, 129], "inter": [15, 20, 25, 28, 29, 30, 31, 37, 81, 98, 121, 122, 123, 126, 127, 139, 148], "inter_layernorm": 131, "inter_s": 114, "interact": [15, 16, 20, 35, 81, 94, 99, 109, 139, 147], "interchang": [10, 104], "interconect": 122, "interconnect": [102, 121, 122, 123, 126, 127], "interest": [15, 37, 120], "interestingli": 19, "interf": 19, "interfac": [1, 11, 19, 32, 56, 77, 80, 92, 113, 115, 121, 134, 142, 144, 148, 149, 150, 157, 160], "interfer": [15, 16, 81, 139], "interleav": [14, 72, 88, 101, 113], "intermedi": [10, 11, 14, 21, 72, 101, 113, 139], "intermediate_s": [112, 131], "intermitt": 10, "intern": [1, 2, 12, 14, 26, 29, 34, 35, 56, 72, 80, 99, 101, 104, 108, 115, 121, 124, 125, 135, 139, 148, 160, 166], "internal_cutlass_kernel": 108, "internal_error": [21, 22, 23, 26, 76, 155], "internlm": [137, 138, 148], "internlm2": [137, 138, 148], "internvl2": 148, "interpol": 129, "interpolation_scal": 130, "interpret": [10, 19, 33, 94, 99, 126, 143], "interrupt": 19, "intersect": 98, "interst": 29, "intertwin": 127, "interv": [8, 26, 143], "interven": 10, "intflag": [131, 133], "intpsplitdim": 1, "intra": 122, "introduc": [2, 4, 8, 10, 11, 12, 13, 15, 16, 18, 19, 35, 72, 80, 81, 84, 85, 87, 89, 115, 117, 137, 148, 160, 162, 163], "introduct": [19, 20, 93, 124, 125, 147, 148], "inttensor": [134, 144, 149], "intuit": [14, 124, 125], "inv": 129, "inv_freq": 129, "invalid": [1, 15, 19, 65, 81, 98, 139, 148], "invalid_st": 1, "invalidateremoteag": 0, "inventori": [37, 120], "invers": [72, 101], "invest": [37, 120], "investig": [2, 10, 19, 148], "invoc": 148, "invok": [0, 10, 11, 15, 19, 33, 35, 73, 77, 92, 99, 103, 139, 152, 157, 167], "invokequant": 113, "involv": [0, 1, 7, 13, 14, 16, 19, 25, 72, 81, 86, 109, 113, 130, 150, 151, 166], "io": [9, 11, 17, 20, 21, 25, 28, 29, 30, 31, 72, 101, 116, 117, 135, 147, 148], "ip": [0, 148], "ipc": [9, 17, 20, 25, 28, 29, 30, 31, 94, 142, 143, 147, 148], "ipc_uc_handl": 1, "ipc_uc_ptr": 1, "ipc_uc_va": 1, "ipcmemori": 1, "ipcnvl": 1, "ipcnvlsalloc": 1, "ipcnvlsfre": 1, "ipcnvlshandl": 1, "ipcnvlssupport": 1, "ipluginv3lay": 129, "ireducelay": 129, "irrelev": 10, "irrespect": [0, 102, 143], "is_alibi": 129, "is_caus": 130, "is_comm_sess": 143, "is_const_v": 1, "is_cuda_graph": [72, 151], "is_cutlass_min_lat": 129, "is_def": 129, "is_dora": 106, "is_dynam": 129, "is_enc_dec": 134, "is_expert": 130, "is_fin": 143, "is_gated_activ": 129, "is_gemma_2": 131, "is_gemma_3": 131, "is_keep_al": [18, 65, 92, 143], "is_loc": 130, "is_medusa_mod": 134, "is_mla_en": 129, "is_mla_enabled_flag": 129, "is_module_excluded_from_quant": 143, "is_mrop": 129, "is_network_input": 129, "is_orchestrator_mod": 134, "is_public_pool": [18, 65, 92, 143], "is_qkv": 130, "is_redrafter_mod": 134, "is_rop": 129, "is_trt_wrapp": 129, "is_use_oldest": [18, 65, 92, 143], "is_valid": [129, 130], "is_valid_cross_attn": 130, "isagentst": 0, "isalnum": 143, "isalpha": 143, "isascii": 143, "isauto": 0, "isbeamsearch": 0, "iscomplet": 0, "iscontextparallel": 1, "iscontinuouskvcach": 1, "iscrossattent": 1, "isdecim": 143, "isdigit": 143, "isdon": 1, "isdora": 1, "isdrafttokensextern": 1, "iseagl": [0, 1], "iselectlay": 129, "isexplicitdrafttoken": [0, 1], "isexternaldrafttoken": 0, "isfin": [0, 99], "isfirstcontextparallelrank": 1, "isfirstpipelineparallelrank": 1, "isfirsttensorparallelrank": 1, "isgreedysampl": 0, "ishufflelay": 129, "isidentifi": 143, "iskeyword": 143, "iskvcacheen": 1, "isl": [0, 3, 4, 5, 6, 8, 12, 13, 14, 15, 19, 20, 23, 25, 28, 29, 30, 31, 37, 38, 75, 81, 93, 120, 127, 154], "isl8192": 16, "island": 29, "islastpipelineparallelrank": 1, "isleg": 0, "islicelay": 129, "isload": 1, "islookahead": 0, "islookaheaddecod": 1, "islow": 143, "ismedusa": [0, 1], "ismpist": 0, "ismultimod": 1, "isn": [15, 56, 139], "isnon": 1, "isnumer": 143, "isoffload": 0, "isoftmaxlay": 129, "isol": 20, "isorchestr": 0, "ispagedkvcach": 1, "isparticip": [0, 148], "ispipelineparallel": 1, "ispoint": 1, "isprint": 143, "isrnnbas": 1, "issequencefin": [0, 99], "issocketst": 0, "isspac": 143, "issu": [8, 10, 11, 13, 15, 17, 20, 28, 29, 30, 31, 34, 37, 38, 62, 72, 81, 83, 94, 97, 98, 101, 113, 115, 120, 121, 129, 139], "istensorparallel": 1, "isthreadsaf": 0, "istitl": 143, "istopk": 0, "istopkandtopp": 0, "istopkortopp": 0, "istopp": 0, "istransformerbas": 1, "istream": [0, 1], "isunsign": 1, "isupp": 143, "isusebantoken": 0, "isusebanword": 0, "isuseexpliciteosstop": 0, "isusefrequencypenalti": 0, "isusemaxlengthstop": 0, "isuseminlength": 0, "isuseminp": 0, "isusenorepeatngrams": 0, "isuseoccurrencepenalti": 0, "isusepenalti": 0, "isusepresencepenalti": 0, "isuserepetitionpenalti": 0, "isusestopcriteria": 0, "isusestopword": 0, "isusetemperatur": 0, "isusevariablebeamwidthsearch": 0, "iswhisp": 1, "ite": 134, "item": [0, 11, 14, 99, 134, 143], "itensor": [0, 129], "itensorbind": 1, "itensorptr": 1, "iter": [0, 1, 8, 10, 11, 12, 13, 15, 18, 19, 20, 21, 26, 32, 35, 57, 72, 84, 88, 92, 99, 101, 109, 114, 120, 121, 126, 127, 128, 134, 143, 148], "iter_i": 8, "iter_stat": 148, "iter_stats_max_iter": [57, 143], "iteration_log": 21, "iterationresult": 143, "iterationstat": 0, "iterationtyp": 0, "iterlatencym": [0, 26], "iterlatencymillisec": 148, "iterstat": 0, "iterstatsmaxiter": 0, "iterstatsvec": 0, "ith": 129, "itl": [15, 25, 28, 29, 30, 31, 37, 123, 127, 148], "its": [0, 1, 3, 5, 11, 12, 13, 15, 16, 18, 20, 33, 35, 37, 51, 57, 72, 81, 83, 88, 89, 92, 99, 101, 102, 103, 104, 110, 112, 113, 114, 115, 120, 122, 124, 125, 126, 127, 129, 135, 142, 143, 150, 151, 167], "itself": [10, 14, 15, 75, 99, 134, 143, 148, 154], "itsuji": [37, 120], "iunarylay": 129, "j": [4, 7, 8, 11, 13, 37, 59, 60, 61, 72, 101, 102, 120, 129, 137, 138, 148], "jacobi": 109, "jai": 148, "jame": 29, "jamesthez": 148, "janpetrov": 148, "japanes": [37, 106, 120], "jax": [20, 112, 115], "jenkin": [34, 93], "jensen": 56, "jersei": 29, "jetson": 20, "jfk": 29, "ji": 129, "jit": [2, 148], "jj": 129, "jk": 129, "jl749": 148, "job": [33, 60, 61, 113], "john": [28, 29], "join": [11, 16, 56, 80, 81, 143, 160], "joint": 14, "joint_attention_kwarg": 131, "joint_attn_forward": 130, "joke": 64, "journei": [12, 147], "jpeg": 26, "jpg": [37, 120], "json": [0, 1, 9, 10, 15, 17, 20, 21, 23, 25, 28, 29, 30, 31, 34, 36, 37, 39, 40, 41, 45, 46, 51, 71, 81, 91, 93, 99, 112, 119, 120, 143, 147, 148, 156], "json_data": 143, "json_object": 143, "json_schema": 143, "json_schema_extra": 143, "jsonconfigstr": 0, "jsondecodeerror": 70, "jsonl": [21, 37, 120], "jsonseri": 0, "judgement": 15, "just": [0, 1, 11, 13, 14, 15, 18, 28, 29, 30, 31, 37, 58, 59, 60, 61, 62, 86, 92, 94, 109, 120, 121, 128, 134, 135, 148], "justic": [47, 62, 97, 140, 142, 147], "justifi": 143, "k": [1, 12, 13, 14, 18, 35, 64, 72, 89, 91, 92, 93, 100, 101, 102, 106, 109, 129, 137, 139, 143, 144, 148, 149, 151, 165], "k8": 93, "k_b_proj_tran": 129, "k_dim": 129, "k_proj": [37, 80, 85, 114, 120, 144, 149, 160, 162], "kattent": 1, "kattn_dens": 1, "kattn_k": 1, "kattn_q": 1, "kattn_qkv": 1, "kattn_v": 1, "kauto": 0, "kb": 19, "kbatchedpostprocessornam": [0, 99], "kbeamsearch": 0, "kbf16": 0, "kblk": 0, "kbool": [0, 1], "kbyte_typ": 1, "kc_cache_retention_config": 148, "kcancel": 0, "kchatglm": 1, "kcontext": 1, "kcontext_in_progress": 0, "kcontinu": 1, "kcpu": [0, 1], "kcpu_pin": 0, "kcpu_pinnedpool": 0, "kcross_attn_dens": 1, "kcross_attn_k": 1, "kcross_attn_q": 1, "kcross_attn_qkv": 1, "kcross_attn_v": 1, "kdatatyp": 1, "kdecoder_onli": [0, 110], "kdefault": 0, "kdefault_num_tokens_per_block": 1, "kdefaultbatchsizet": 0, "kdefaultdynamicbatchmovingaveragewindow": 0, "kdefaultgpumemfract": 0, "kdefaultgpuspernod": 1, "kdefaultiterstatsmaxiter": 0, "kdefaultlookaheaddecodingngram": 0, "kdefaultlookaheaddecodingverificationset": 0, "kdefaultlookaheaddecodingwindow": 0, "kdefaultmaxadapters": 0, "kdefaultmaxpagesperblockdevic": 0, "kdefaultmaxpagesperblockhost": 0, "kdefaultmaxseqidlemicrosecond": 0, "kdefaultoptimaladapters": 0, "kdefaultprior": 0, "kdefaultrequeststatsmaxiter": 0, "kdefaultretentionprior": 0, "kdisabl": 1, "kdrafttokensextern": 1, "kdram": 0, "kdynamicpostprocessornameprefix": 0, "keagl": [0, 1], "kebnf_grammar": [0, 99], "keep": [0, 12, 14, 15, 18, 20, 33, 38, 72, 88, 101, 102, 108, 115, 128, 129, 143, 148], "keepdim": 129, "keepend": 143, "kei": [0, 3, 7, 8, 9, 11, 13, 14, 15, 18, 19, 20, 21, 35, 37, 38, 57, 72, 74, 80, 83, 86, 88, 92, 93, 99, 105, 113, 120, 126, 131, 134, 139, 143, 150, 151, 153, 160, 166], "kenabl": 1, "kencdec": 1, "kencoder_decod": 0, "kencoder_in_progress": 0, "kencoder_onli": 0, "kend_id": 0, "kennedi": 29, "kept": [15, 18, 72, 101, 115, 129, 143], "kequal_progress": 0, "kera": 115, "kernel": [1, 3, 13, 14, 16, 22, 28, 30, 31, 35, 36, 63, 72, 74, 79, 88, 90, 92, 93, 101, 105, 108, 113, 119, 123, 127, 129, 134, 135, 139, 143, 146, 148, 153, 159], "kernel_s": [129, 130], "kexplicitdrafttoken": [0, 1], "kexternaldrafttoken": 0, "key_length": [129, 130], "keyvaluecacheparam": [130, 131], "keyword": [75, 114, 129, 135, 143, 148, 154], "kfile": 0, "kfirst_come_first_serv": 0, "kfloat": [1, 113], "kfp16": 0, "kfp32": [0, 143], "kfp8": 0, "kgener": 1, "kgeneration_complet": 0, "kgeneration_in_progress": 0, "kglm": 1, "kgpt": 1, "kgpu": [0, 1], "kguaranteed_no_evict": 0, "khalf": 1, "kick": 33, "kill": 17, "kind": [10, 11, 12, 15, 100, 101, 103, 167], "kinflight": 0, "king": 29, "kint32": [0, 1], "kint64": [0, 1], "kint8": [0, 1], "kinvalid": 1, "kispoint": 1, "kisunsign": 1, "kj": 129, "kjson": [0, 99], "kjson_schema": [0, 99], "kleader": 0, "klength": 0, "klinear": 1, "kllguidanc": 0, "klookahead": 0, "klookaheaddecod": 1, "km": 29, "kmamba": 1, "kmax_util": 0, "kmaxretentionprior": 0, "kmedusa": [0, 1], "kminretentionprior": 0, "kmla": 0, "kmlp_4h_to_h": 1, "kmlp_gate": 1, "kmlp_gate_up": 1, "kmlp_h_to_4h": 1, "kmlp_router": 1, "kmoe_4h_to_h": 1, "kmoe_gat": 1, "kmoe_h_to_4h": 1, "kmoe_rout": 1, "kmpi": 0, "knegativeinfin": 1, "knob": [0, 25, 128, 129, 143, 148], "knone": 1, "knoop": 1, "knot_finish": 0, "know": [11, 29, 36, 49, 102, 119, 128, 129], "knowledg": 18, "known": [11, 13, 15, 28, 33, 72, 88, 89, 97, 101, 108, 109, 113, 129, 138, 147], "knumflag": 0, "kobj": 0, "kopt_profiles_split_point": 1, "korchestr": 0, "korea": 26, "kosmo": [138, 148], "kpage": 1, "kpin": 1, "kpinnedpool": 1, "kqueu": 0, "kread": 0, "krecurr": 1, "krecurrentgemma": 1, "kregex": [0, 99], "kstatic": 0, "kstatic_batch": 0, "kstop_word": 0, "kstructural_tag": 0, "ktimed_out": 0, "ktopk": 0, "ktopktopp": 0, "ktopp": 0, "ktrtpointertyp": 1, "kubernet": [16, 44, 81], "kuint8": [0, 1], "kunderlyingtyp": 1, "kunish": 106, "kunknown": 0, "kunsign": 1, "kusebantoken": 0, "kusebanword": 0, "kuseexpliciteosstop": 0, "kusefrequencypenalti": 0, "kusemaxlengthstop": 0, "kuseminlength": 0, "kuseminp": 0, "kusenorepeatngrams": 0, "kuseoccurrencepenalti": 0, "kusepenalti": 0, "kusepresencepenalti": 0, "kuserepetitionpenalti": 0, "kusestandardstopcriteria": 0, "kusestopword": 0, "kusetemperatur": 0, "kusevariablebeamwidthsearch": 0, "kuvm": [0, 1], "kv": [0, 1, 3, 7, 8, 13, 14, 17, 21, 22, 23, 25, 26, 28, 29, 30, 31, 35, 37, 38, 42, 49, 50, 63, 73, 74, 78, 79, 82, 84, 86, 89, 93, 98, 99, 106, 113, 115, 118, 120, 121, 126, 129, 134, 136, 141, 143, 144, 145, 146, 148, 149, 150, 151, 152, 153, 156, 158, 159, 161, 167], "kv_b_proj": 129, "kv_cach": [0, 63], "kv_cache_block_offset": [129, 130, 134, 139], "kv_cache_block_point": 139, "kv_cache_config": [9, 26, 37, 42, 48, 56, 57, 63, 65, 84, 90, 128, 143, 166], "kv_cache_connector": 56, "kv_cache_dtyp": [2, 25, 120, 123, 133, 166], "kv_cache_enable_block_reus": [134, 148], "kv_cache_free_gpu_mem_fract": [2, 15, 20, 21, 38, 128], "kv_cache_free_gpu_memory_fract": [16, 17, 20, 23, 25, 26, 43, 134, 148], "kv_cache_host_memory_byt": 105, "kv_cache_host_s": 57, "kv_cache_manag": [0, 72, 148, 150, 151, 166, 167], "kv_cache_max_token": 57, "kv_cache_page_s": 57, "kv_cache_param": [72, 130, 131, 151], "kv_cache_quant_algo": [37, 85, 112, 120, 123, 143, 162], "kv_cache_quant_mod": [101, 129], "kv_cache_retention_config": [49, 143], "kv_cache_reus": 26, "kv_cache_scaling_factor": [72, 101, 112], "kv_cache_tensor": 56, "kv_cache_typ": [22, 113, 134, 143, 148], "kv_connector_config": [56, 143], "kv_dtype": 131, "kv_head": 130, "kv_host_cache_byt": 105, "kv_lora_rank": [129, 130], "kv_orig_quant_scal": 129, "kv_quant_orig_scal": 129, "kvalue_status_load": 1, "kvalue_status_miss": 1, "kvalue_status_process": 1, "kvcach": [0, 11, 12, 81, 98, 134, 148], "kvcacheblock": 104, "kvcacheblockpool": 104, "kvcacheconfig": [0, 42, 48, 57, 63, 65, 83, 84, 90, 93, 101, 105, 128, 135, 143, 148], "kvcacheconnectorconfig": [56, 143], "kvcacheconnectorschedul": 56, "kvcacheconnectorwork": 56, "kvcachecreateddata": [0, 143], "kvcacheev": 0, "kvcacheeventdata": 0, "kvcacheeventdiff": 0, "kvcacheeventmanag": 0, "kvcachehitr": 0, "kvcachehitrateperrequest": 0, "kvcacheindex": 1, "kvcachemanag": [0, 35, 72, 88, 101, 105, 134, 151, 166], "kvcachemetr": 0, "kvcacheparam": [72, 151], "kvcacheremoveddata": [0, 143], "kvcacheretentionconfig": [0, 49, 93, 143], "kvcaches": 0, "kvcachestat": [0, 26], "kvcachestoredblockdata": 0, "kvcachestoreddata": [0, 143], "kvcachetransferend": 0, "kvcachetransferm": 0, "kvcachetransfermod": [0, 143], "kvcachetransferstart": 0, "kvcachetyp": [1, 134, 143], "kvcachetypefromstr": 1, "kvcacheupdateddata": [0, 143], "kvfactor": 0, "kvheadnum": 129, "kvram": 0, "kwarg": [11, 32, 74, 75, 77, 80, 114, 115, 129, 130, 131, 134, 143, 144, 148, 149, 153, 154, 157, 160], "kwd": 143, "kwrite": 0, "kxgrammar": 0, "l": [26, 37, 59, 60, 61, 109, 120, 138, 145], "l0_a100": 33, "l0_mergerequest": 33, "l0_sanity_check": 33, "l0_test": 33, "l2": [22, 148], "l20": 22, "l304": 12, "l345": 12, "l4": [20, 22], "l40": [20, 22, 146], "l440": 12, "l506": 12, "l546": 12, "l823": 12, "lab": [11, 37, 120], "label": [103, 129, 130, 131, 143], "labelembed": 130, "lack": [0, 1], "laguardia": 29, "lai": 13, "lake": 29, "lambda": [0, 99], "lamportinitializeal": 1, "landmark": 29, "langchain": 11, "languag": [0, 3, 8, 12, 15, 16, 18, 35, 36, 78, 85, 87, 90, 92, 102, 109, 113, 119, 129, 137, 138, 145, 146, 148, 150, 158, 162, 163], "language_adapt": [134, 148], "language_adapter_config": 134, "language_adapter_rout": [131, 134], "language_adapter_uid": 134, "language_model": 114, "languageadapterconfig": 134, "languageadapteruid": 0, "laps": 49, "laptop": 20, "larg": [0, 2, 3, 7, 8, 12, 14, 16, 17, 19, 22, 26, 35, 36, 37, 40, 57, 67, 85, 86, 87, 88, 89, 92, 93, 101, 105, 107, 109, 113, 115, 119, 120, 122, 123, 126, 127, 129, 134, 135, 138, 139, 143, 146, 148, 150, 162, 163], "larger": [0, 4, 5, 7, 14, 16, 18, 19, 20, 35, 38, 57, 72, 101, 102, 105, 109, 110, 120, 129, 134, 135, 143, 148], "largest": [3, 4, 5, 29, 92, 102, 129], "last": [0, 1, 10, 12, 13, 15, 17, 18, 57, 58, 72, 84, 88, 99, 101, 106, 107, 109, 111, 126, 128, 129, 131, 143], "last_lay": 134, "last_process_for_ub": 129, "last_token_id": [129, 131, 139], "last_token_ids_for_logit": 131, "last_tokens_id": 129, "lastdraftindic": 1, "lastdraftlen": 1, "lastdraftpath": 1, "lastdrafttoken": 1, "lastgenerationlength": 1, "lastit": 0, "lastli": 20, "lastpositionidsbas": 1, "lasttokentim": 0, "late": [11, 62], "latenc": [0, 4, 5, 7, 9, 10, 11, 13, 14, 15, 16, 17, 18, 19, 22, 25, 28, 29, 30, 31, 35, 37, 38, 72, 73, 81, 88, 92, 93, 101, 105, 109, 126, 127, 128, 129, 143, 146, 148, 152], "latent": [14, 74, 130, 131, 153], "later": [0, 1, 5, 17, 28, 29, 30, 31, 75, 83, 102, 106, 109, 113, 115, 123, 127, 134, 135, 139, 141, 154], "latest": [0, 2, 14, 17, 28, 29, 30, 31, 86, 90, 93, 94, 113, 116, 148], "latter": [7, 29, 34, 88, 99, 148], "launch": [10, 14, 15, 16, 19, 26, 35, 59, 60, 61, 63, 86, 87, 92, 93, 94, 97, 105, 113, 121, 139, 141, 142, 148, 156, 163], "launch_hostfunc": 10, "launch_llama_3": 113, "layer": [0, 1, 13, 15, 16, 19, 20, 22, 72, 75, 81, 83, 84, 88, 89, 92, 93, 98, 100, 101, 102, 103, 104, 106, 109, 111, 112, 113, 114, 122, 129, 134, 135, 137, 139, 143, 144, 148, 149, 151, 154, 166], "layer1": 106, "layer_id": 92, "layer_idx": [56, 72, 106, 111, 129, 134, 144, 149, 151], "layer_names_onli": [22, 143], "layer_norm": [129, 130], "layer_quant_mod": 143, "layer_typ": 134, "layer_updates_per_it": [15, 28], "layerid": [1, 106], "layeridx": 1, "layernorm": [22, 111, 127, 129, 130, 148], "layernorm_shar": 130, "layernorm_typ": 130, "layernormpositiontyp": 129, "layernormtyp": [129, 130], "layertyp": [1, 103], "layerwis": 143, "layout": [9, 72, 88, 126, 148], "lead": [8, 11, 15, 16, 19, 20, 22, 28, 30, 37, 38, 62, 81, 88, 94, 98, 103, 105, 109, 113, 120, 121, 123, 126, 127, 143, 148], "leader": [0, 17, 134], "leaf": 83, "leak": [10, 148], "learn": [4, 5, 7, 15, 17, 18, 20, 47, 52, 53, 55, 97, 123, 129, 140, 142, 147], "learned_absolut": [112, 129, 130, 131], "least": [0, 15, 18, 26, 62, 72, 83, 88, 99, 101, 115, 126, 134, 143], "leav": [16, 81, 83, 88, 126, 127, 128], "left": [11, 16, 18, 19, 38, 75, 126, 128, 129, 143, 154], "legaci": [11, 114, 128, 132, 148], "legend": 8, "lego": 11, "len": [0, 1, 11, 25, 28, 29, 30, 31, 37, 56, 58, 74, 120, 129, 134, 143, 153, 167], "length": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 16, 18, 20, 21, 22, 23, 26, 28, 29, 30, 31, 37, 38, 49, 58, 72, 74, 81, 83, 84, 88, 92, 101, 105, 120, 121, 124, 125, 127, 128, 129, 134, 135, 139, 143, 148, 151, 153, 166], "length_penalti": [102, 134, 143], "lengthlengthpenalti": 102, "lengthpenalti": [0, 1, 102], "leq": 8, "less": [0, 4, 10, 13, 15, 18, 38, 72, 83, 84, 91, 99, 101, 102, 113, 129, 143], "lesser": 83, "let": [10, 11, 12, 15, 20, 29, 37, 42, 88, 103, 111, 112, 114, 116, 120, 126, 129], "letter": 129, "level": [0, 1, 8, 10, 11, 13, 14, 16, 19, 20, 21, 22, 23, 26, 32, 36, 37, 72, 75, 77, 78, 88, 91, 92, 93, 99, 101, 104, 106, 108, 111, 112, 114, 115, 119, 120, 135, 142, 143, 144, 146, 148, 149, 154, 157, 158], "leverag": [3, 8, 10, 12, 13, 15, 16, 17, 18, 20, 28, 29, 30, 31, 78, 79, 86, 109, 123, 146, 158, 159, 160], "lf": [2, 17, 94, 106, 142], "lfz941": 148, "lga": 29, "lgai": [138, 145], "lh": 1, "lib": [115, 120], "liberti": 29, "libnam": 0, "libnvinfer_plugin_tensorrt_llm": 94, "libopenmpi": [78, 97, 158], "librari": [15, 16, 19, 20, 21, 30, 72, 78, 79, 81, 94, 108, 113, 139, 142, 146, 148, 151, 158, 159], "libtensorrt_llm": 94, "licens": [96, 97, 142], "life": 62, "lifecycl": [44, 104], "lifetim": 19, "light": [79, 159], "lightweight": [10, 15, 18, 23, 78, 79, 101, 158, 159], "like": [0, 7, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 25, 28, 29, 30, 31, 33, 34, 35, 37, 42, 47, 51, 52, 53, 54, 55, 57, 58, 59, 60, 61, 62, 64, 72, 75, 78, 79, 80, 81, 83, 88, 89, 90, 92, 95, 97, 98, 99, 101, 102, 103, 105, 109, 112, 113, 115, 120, 121, 122, 123, 126, 127, 128, 129, 135, 139, 140, 141, 142, 143, 144, 147, 148, 149, 150, 154, 158, 159, 160, 166], "likelihood": [58, 100, 105, 109], "likewis": 80, "limit": [0, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 21, 26, 42, 57, 72, 81, 84, 88, 92, 93, 97, 98, 99, 101, 102, 103, 113, 115, 121, 126, 128, 129, 132, 134, 135, 138, 142, 143, 151], "lin": 3, "line": [2, 7, 16, 21, 25, 28, 29, 30, 31, 33, 34, 37, 48, 58, 62, 75, 83, 105, 120, 121, 123, 127, 135, 143, 147, 148, 154, 166, 167], "linear": [1, 14, 72, 78, 106, 109, 111, 112, 113, 129, 135, 137, 144, 148, 149, 151, 158], "linearactiv": 130, "linearapproximategelu": 130, "linearbas": 130, "lineargeglu": 130, "lineargelu": 130, "linearli": 135, "linearswiglu": 130, "link": [2, 12, 18, 19, 20, 33, 105, 116, 117, 148], "linspac": 129, "lint": 143, "linux": [19, 28, 29, 30, 31, 34, 96, 138, 148], "linux_x86_64": 94, "list": [0, 1, 10, 13, 15, 25, 28, 29, 30, 31, 32, 33, 34, 37, 42, 49, 56, 58, 72, 73, 74, 75, 79, 80, 83, 84, 91, 92, 93, 94, 99, 101, 102, 103, 112, 113, 114, 118, 120, 121, 129, 130, 131, 134, 138, 139, 143, 148, 151, 152, 153, 154, 159, 160, 166, 167], "list_siz": 130, "listen": 9, "liter": 143, "littl": [13, 15, 127], "live": [33, 135], "livecodebench": 12, "ljust": 143, "lkm2835": 148, "ll": [7, 13, 20, 92], "ll128": [15, 19], "llama": [4, 5, 7, 10, 17, 18, 22, 23, 25, 30, 31, 37, 47, 52, 65, 69, 73, 75, 78, 79, 80, 90, 91, 92, 93, 97, 102, 106, 109, 110, 114, 115, 121, 122, 124, 125, 126, 128, 131, 136, 137, 138, 140, 141, 142, 144, 145, 146, 147, 148, 149, 152, 154, 156, 158, 159, 160, 164, 165], "llama2": [3, 4, 72, 93, 106, 148], "llama3": [25, 27, 65, 93, 129, 148], "llama4": [15, 27, 84, 89, 93, 143, 148], "llama4_output": 31, "llama4forconditionalgener": [138, 145], "llama_13b": 5, "llama_70b": 5, "llama_7b": [106, 110], "llama_7b_with_lora_qkv": 106, "llama_model_path": 42, "llamaconfig": [131, 144, 149], "llamaforcausallm": [114, 115, 131, 138, 145], "llamamodel": 131, "llava": [90, 114, 136, 137, 138, 148], "llava_dict": 114, "llavallamamodel": [138, 145], "llavanextforconditionalgener": [138, 145], "llavanextvisionconfig": 131, "llavanextvisionwrapp": 131, "llguidanc": [0, 10, 51, 70, 91, 143, 148], "llm": [0, 3, 6, 8, 12, 14, 21, 22, 23, 25, 26, 27, 33, 34, 35, 36, 38, 39, 40, 41, 43, 44, 45, 46, 48, 49, 51, 52, 53, 54, 56, 57, 58, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 72, 73, 77, 79, 80, 81, 83, 84, 85, 88, 91, 92, 95, 96, 97, 98, 99, 101, 102, 103, 104, 105, 106, 107, 111, 113, 118, 119, 122, 123, 124, 125, 127, 128, 129, 131, 133, 134, 137, 139, 141, 143, 144, 149, 150, 151, 152, 157, 159, 160, 162, 164, 165, 166, 167], "llm_advanc": 63, "llm_arg": [32, 38, 56, 75, 77, 85, 143, 154, 157, 162], "llm_engine_dir": 134, "llm_id": 143, "llm_inference_distribut": 142, "llm_kv_cache_offload": 57, "llm_mgmn_": 148, "llm_models_root": 69, "llm_option": 38, "llm_ptq": [90, 164], "llm_sampl": 64, "llm_util": 143, "llm_worker": 11, "llmapi": [15, 25, 26, 32, 38, 42, 48, 49, 51, 56, 57, 59, 60, 61, 63, 65, 85, 90, 91, 92, 99, 123, 143, 148, 162], "llmarg": [32, 38, 75, 93, 107, 143, 148, 154], "llmrequest": [1, 56, 148, 166, 167], "llmrequestptr": 1, "llmrequestst": 167, "lm": [28, 30, 31, 109, 143], "lm_eval": [23, 28, 30, 31], "lm_head": [10, 111, 114, 120, 148], "lmm": [37, 102, 120], "lmsy": [87, 163], "ln_emb": 114, "ln_f": [111, 114], "lo": 142, "load": [0, 1, 6, 9, 12, 14, 16, 17, 20, 21, 22, 28, 29, 30, 31, 35, 38, 56, 57, 70, 73, 74, 75, 78, 81, 85, 93, 106, 111, 112, 113, 115, 120, 121, 127, 128, 131, 133, 134, 135, 142, 143, 146, 147, 148, 152, 153, 154, 156, 158, 162], "load_balanc": [15, 28, 143], "load_base64_imag": 26, "load_config": [80, 160], "load_format": 143, "load_model_on_cpu": 131, "load_tensor": 114, "load_test_audio": 134, "load_test_data": 134, "load_weight": [80, 144, 149, 160], "loaded_weight": 130, "loader": [93, 143, 148], "loadformat": 143, "loadinprogress": 1, "loadremoteag": 0, "loadweight": 1, "local": [1, 2, 9, 12, 15, 20, 22, 23, 28, 30, 31, 34, 37, 38, 47, 52, 53, 54, 55, 59, 60, 61, 65, 78, 89, 94, 95, 97, 112, 113, 120, 123, 140, 143, 147, 148, 156, 158, 166], "local_build": 34, "local_in_featur": 130, "local_layer_idx": 130, "local_model": [59, 60, 61], "local_model_path": 20, "local_out_featur": 130, "local_path": 30, "local_path_to_model": 142, "local_us": [2, 94], "localcr": 1, "localhost": [9, 16, 17, 20, 25, 26, 28, 29, 30, 31, 39, 40, 41, 43, 45, 46, 66, 67, 68, 69, 70, 81, 85, 147, 156, 162], "localinadapters": 1, "localindim": 1, "localinouts": 1, "localins": 1, "localn": 143, "localoutadapters": 1, "localoutdim": 1, "localouts": 1, "localreduct": 12, "localscaless": 1, "localtotals": 1, "locat": [14, 15, 19, 28, 29, 30, 31, 33, 38, 72, 94, 102, 103, 113, 120, 129, 139, 147, 151], "locate_accepted_draft_token": 134, "lock": [10, 15, 19, 37, 120], "lockstep": 0, "log": [0, 1, 9, 19, 20, 21, 22, 23, 25, 26, 37, 57, 59, 60, 61, 64, 72, 78, 91, 92, 101, 104, 116, 120, 129, 135, 143, 148, 158, 165], "log_ctx_0": 81, "log_ctx_1": 81, "log_gen_0": 81, "log_level": [21, 22, 23, 26], "log_sampl": [28, 30, 31], "log_softmax": 129, "logger": [56, 76, 155], "logic": [10, 11, 16, 19, 23, 34, 35, 58, 80, 87, 91, 92, 99, 104, 114, 115, 130, 131, 143, 144, 148, 149, 150, 160, 167], "login": [9, 116], "logit": [0, 1, 10, 12, 13, 35, 37, 50, 82, 92, 93, 102, 109, 120, 129, 134, 139, 143, 145, 148, 161, 165], "logits_dtyp": [22, 112, 131], "logits_processor": [58, 91, 134, 143], "logits_processor_map": 134, "logits_processor_nam": 134, "logitspostprocessor": 0, "logitspostprocessorbatch": [0, 99], "logitspostprocessorconfig": [0, 99, 148], "logitspostprocessormap": 0, "logitspostprocessornam": 0, "logitsprocessor": [58, 91, 134, 143, 148], "logitsprocessorlist": 134, "logitsvec": 1, "logn": [129, 148], "logn_scal": 129, "logprob": [0, 1, 17, 20, 28, 29, 30, 31, 42, 64, 143, 147, 148], "logprobs_diff": 143, "logprobscba": 1, "logprobstil": 1, "london": 139, "long": [7, 8, 10, 15, 16, 22, 28, 29, 30, 31, 36, 72, 81, 83, 85, 88, 89, 93, 101, 119, 120, 121, 122, 123, 126, 127, 135, 146, 148, 162], "long_factor": 129, "long_mscal": [129, 130], "long_rop": 129, "long_rope_embed_posit": 130, "long_rope_embed_positions_for_gpt_attent": 130, "long_rope_rotary_cos_sin": 129, "long_rope_rotary_inv_freq": [129, 130], "longer": [0, 8, 10, 12, 14, 15, 17, 19, 20, 28, 29, 30, 31, 38, 84, 88, 102, 105, 126, 129, 143, 167], "longest": [13, 88, 126, 129], "longrop": 129, "longtensor": [58, 134], "look": [0, 6, 10, 11, 15, 37, 56, 81, 92, 94, 99, 115, 120, 148], "lookahead": [0, 1, 143, 148], "lookahead_config": [134, 143], "lookahead_decod": [22, 131], "lookaheadalgoconfig": 1, "lookaheadconfig": 0, "lookaheaddecod": 1, "lookaheaddecodingbuff": 1, "lookaheaddecodingconfig": [0, 1, 93, 143], "lookaheadinput": 1, "lookaheadoutput": 1, "lookaheadprompt": 1, "lookaheadruntimebuff": 1, "lookup": [18, 33, 92, 129, 130, 143, 148], "lookup_plugin": 129, "lookuperror": 143, "loop": [0, 10, 11, 18, 19, 35, 92, 99, 102, 113, 114, 128, 143], "lopuhin": 148, "lora": [0, 1, 21, 50, 71, 78, 80, 82, 93, 99, 118, 129, 130, 131, 134, 143, 146, 148, 158, 160], "lora_0": [37, 120], "lora_adapt": [85, 162], "lora_ckpt_sourc": [22, 85, 134, 143, 162], "lora_config": [37, 62, 85, 120, 131, 143, 148, 162], "lora_dir": [22, 37, 62, 85, 106, 120, 134, 162], "lora_help": [62, 143, 162], "lora_hidden_st": 130, "lora_int_id": [37, 69, 85, 120, 143, 162], "lora_layer_param": 130, "lora_manag": [85, 134, 148], "lora_nam": [37, 69, 85, 120, 143, 162], "lora_param": 131, "lora_path": [37, 69, 85, 120, 143, 162], "lora_plugin": [22, 106, 129, 134], "lora_rank": [106, 129], "lora_req1": [85, 162], "lora_req2": [85, 162], "lora_request": [37, 62, 69, 85, 120, 143, 162], "lora_runtime_param": 130, "lora_target_modul": [22, 37, 85, 106, 120, 131, 134, 162], "lora_task_uid": 134, "lora_uid": 134, "lora_weights_point": 129, "loracachefullexcept": 1, "loracachepagemanag": 1, "loraconfig": [0, 62, 85, 106, 131, 143, 148, 162], "loraexpectedexcept": 1, "loraid": 0, "loramanag": 134, "loramodulenam": 1, "loraparam": 131, "loraprefetchdir": 0, "lorarequest": [62, 85, 93, 143, 162], "loraruntimeparam": 130, "lorataskidtyp": [0, 1], "loraweight": 106, "loss": [7, 8, 20, 123], "lost": 148, "lot": [13, 72, 88, 101, 105, 113], "loudspeak": 5, "love": 17, "lovelac": [90, 138, 146, 148], "low": [2, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 22, 28, 29, 30, 31, 37, 72, 78, 91, 92, 93, 101, 111, 115, 129, 143, 146, 148, 158], "low_lat": 20, "low_latency_benchmark": 20, "low_latency_gemm": [108, 129], "low_latency_gemm_plugin": [22, 120, 123, 130], "low_latency_gemm_swiglu": 129, "low_latency_gemm_swiglu_plugin": [22, 123, 132], "low_rank": 129, "lower": [0, 1, 2, 6, 7, 8, 14, 16, 20, 28, 29, 30, 31, 32, 38, 64, 75, 81, 83, 88, 90, 92, 98, 102, 103, 105, 106, 123, 129, 135, 143, 154], "lowercas": 143, "lowest": [20, 49, 75, 83, 143, 154], "lowprecis": [107, 129, 143], "loyalti": 58, "lpddr5x": 15, "lru": [1, 83, 105, 129], "lstrip": 143, "lt": 129, "lunch": 15, "luotuo": [69, 106], "m": [0, 2, 4, 8, 12, 15, 16, 19, 21, 25, 26, 28, 29, 30, 31, 32, 37, 38, 45, 46, 51, 62, 85, 120, 121, 123, 126, 127, 129, 135, 137, 162], "ma": 29, "macceptancethreshold": 0, "machin": [2, 7, 17, 105, 148], "macro": 108, "madditionalmodeloutput": 0, "maddr": 0, "maddress": 1, "made": [19, 83, 95, 148, 167], "madv_hugepag": 19, "madvis": 19, "magentnam": 0, "magic": 15, "magpi": 18, "mahmoudashraf97": 148, "mai": [0, 1, 2, 8, 10, 11, 12, 13, 15, 16, 17, 19, 20, 22, 25, 28, 29, 30, 31, 32, 34, 36, 37, 38, 48, 59, 60, 61, 72, 75, 78, 81, 88, 92, 94, 98, 99, 101, 102, 105, 106, 107, 108, 109, 112, 113, 114, 115, 116, 119, 120, 121, 127, 128, 129, 130, 132, 135, 139, 142, 143, 144, 145, 148, 149, 150, 151, 154, 158, 166], "main": [3, 6, 10, 11, 12, 13, 15, 17, 18, 19, 21, 28, 29, 30, 31, 36, 40, 42, 47, 48, 49, 51, 52, 53, 54, 55, 56, 57, 58, 62, 63, 64, 65, 67, 80, 92, 97, 99, 102, 104, 119, 121, 123, 127, 128, 129, 135, 139, 140, 142, 143, 144, 147, 149, 160], "mainland": 29, "mainli": [13, 15, 143], "mainstream": [16, 81], "maintain": [3, 4, 7, 8, 15, 37, 78, 80, 89, 91, 92, 106, 108, 120, 123, 137, 148, 158, 160], "mainten": [8, 35], "major": [12, 19, 29, 32, 115, 135, 148], "majority_vote_control": 11, "majorityvot": 11, "majorityvotecontrol": 11, "make": [1, 2, 7, 8, 11, 12, 13, 15, 17, 18, 19, 20, 28, 29, 30, 31, 37, 56, 57, 62, 72, 75, 81, 88, 92, 93, 94, 97, 98, 101, 103, 106, 108, 113, 115, 116, 117, 120, 121, 128, 129, 139, 142, 143, 146, 148, 154], "make_causal_mask": 130, "make_env": 34, "makedir": 56, "makeloopbackag": 0, "makeshap": 1, "maketran": 143, "maketransferag": 0, "malachowski": 56, "malici": 83, "mallotedtim": 0, "mallreducecommptr": 1, "mamba": [22, 129, 137, 138, 143, 148], "mamba1": 129, "mamba2": [129, 148], "mamba_conv1d": 129, "mamba_conv1d_plugin": [22, 134], "mamba_ssm_cache_dtyp": 143, "mamba_vers": 129, "mambaconfig": 131, "mambaforcausallm": 131, "manag": [0, 1, 10, 11, 14, 15, 16, 20, 22, 35, 42, 44, 57, 58, 72, 73, 79, 80, 81, 88, 92, 93, 97, 98, 101, 109, 113, 121, 128, 132, 134, 135, 141, 142, 146, 147, 148, 150, 151, 152, 159, 160], "managedweight": 0, "managedweightsmap": 1, "manageweightstyp": 1, "manageweighttyp": 1, "mandatori": [1, 32, 99, 112], "manhattan": 29, "mani": [0, 10, 11, 13, 14, 15, 17, 18, 20, 22, 38, 72, 81, 83, 84, 88, 92, 98, 101, 104, 105, 109, 113, 115, 117, 123, 126, 128, 129, 138, 139, 143], "manipul": 103, "manner": [15, 19, 103], "mantissa": 4, "manual": [10, 14, 15, 33, 42, 75, 78, 90, 97, 134, 139, 142, 143, 154, 158], "manufactur": [37, 120], "map": [0, 1, 8, 10, 11, 12, 15, 16, 17, 19, 20, 28, 29, 30, 31, 32, 33, 37, 38, 72, 80, 81, 89, 92, 99, 101, 103, 107, 111, 112, 113, 114, 115, 129, 130, 131, 134, 143, 148, 160, 166], "map_loc": 56, "map_weight": [80, 160], "mappingintstrani": 143, "mappingnamespac": 143, "mard1no": 148, "margin": [10, 37, 120, 126], "mark": [1, 18, 32, 33, 81, 88, 92, 103, 126, 129, 139, 145], "mark_as_remov": 103, "mark_output": [99, 129], "markalldon": 1, "markdon": 1, "marker": [11, 19, 33, 143], "market": 58, "marks101": 148, "marktaskdon": 1, "mask": [0, 1, 12, 13, 72, 84, 101, 109, 129, 130, 131, 134, 151], "mask_typ": 129, "masked_scatt": 129, "masked_scatter_": 129, "masked_select": [129, 148], "massachusett": 29, "massiv": [2, 20], "master": [122, 123, 124, 125], "mat2": 129, "match": [0, 13, 16, 18, 19, 23, 28, 29, 30, 31, 32, 33, 35, 37, 56, 73, 79, 83, 91, 92, 100, 103, 109, 120, 129, 130, 134, 139, 143, 148, 152, 156, 159], "match_and_rewrit": 103, "materi": [1, 99], "materializewithtag": 1, "math": [11, 12, 14, 138, 145], "math500": 11, "matichon": 148, "matmul": [22, 101, 113, 123, 129, 137], "matric": [85, 100, 162], "matrix": [6, 14, 20, 37, 72, 89, 93, 101, 113, 120, 122, 129, 141, 151], "mattentionconfig": 0, "mattentiondpeventsgatherperiodm": 0, "mattentionlayernumperpp": 0, "mattentiontyp": 0, "matter": [20, 105], "maverick": [79, 92, 93, 109, 146, 159], "max": [0, 1, 3, 4, 5, 8, 9, 10, 14, 15, 18, 21, 25, 28, 29, 30, 31, 37, 57, 72, 74, 75, 84, 88, 106, 121, 123, 124, 127, 129, 134, 135, 139, 143, 148, 151, 153, 154], "max_": 8, "max_all_reduce_block": 1, "max_attention_window": [83, 84, 128, 143, 148], "max_attention_window_s": [101, 128, 129, 134], "max_attn_valu": 130, "max_batch_s": [2, 9, 10, 11, 13, 17, 18, 20, 21, 22, 23, 25, 26, 37, 42, 43, 57, 63, 72, 73, 75, 77, 101, 106, 110, 112, 113, 115, 120, 123, 126, 127, 129, 131, 134, 135, 139, 140, 143, 148, 152, 154, 157, 166], "max_beam_width": [22, 23, 26, 42, 72, 88, 91, 99, 101, 129, 131, 134, 135, 143], "max_block": [129, 167], "max_blocks_per_seq": 134, "max_blocks_per_sequ": 129, "max_boost_slid": [37, 120], "max_cache_storage_gb": 143, "max_concurr": 143, "max_context_length": [129, 130, 134, 135], "max_cpu_lora": [62, 85, 162], "max_decoder_input_len": 131, "max_decoder_seq_len": 22, "max_dist": [101, 129, 130], "max_draft_len": [9, 17, 18, 22, 65, 92, 131, 133, 143], "max_draft_token": 134, "max_encoder_input_len": [22, 131, 143], "max_gen_token": 131, "max_gpu_total_byt": 143, "max_input_len": [21, 22, 37, 88, 106, 110, 112, 113, 120, 131, 134, 135, 143], "max_input_length": [23, 129, 130, 131, 134], "max_kv_seqlen": 129, "max_lora": [62, 85, 162], "max_lora_rank": [22, 37, 62, 85, 106, 120, 162], "max_low_rank": 129, "max_matching_ngram_s": [18, 65, 92, 143], "max_medusa_token": 134, "max_multimodal_len": 22, "max_new_token": [134, 135], "max_ngram_s": 143, "max_non_leaves_per_lay": 143, "max_num_draft_token": 10, "max_num_request": [72, 151, 166, 167], "max_num_sequ": 148, "max_num_stream": 143, "max_num_token": [2, 9, 11, 20, 21, 22, 23, 25, 26, 42, 43, 72, 84, 89, 92, 120, 123, 126, 127, 131, 135, 140, 143, 148, 151], "max_output_len": [88, 113, 134, 139, 148], "max_output_length": 23, "max_period": 130, "max_position_embed": [88, 112, 129, 130, 131], "max_position_embedding_len": 129, "max_power_limit": [37, 120], "max_prompt_adapter_token": 143, "max_prompt_embedding_table_s": [22, 134, 143, 148], "max_record": 143, "max_retri": [28, 30, 31], "max_seq_len": [9, 17, 21, 22, 23, 25, 26, 57, 63, 75, 77, 106, 110, 112, 113, 120, 128, 129, 130, 131, 134, 135, 143, 148, 154, 157, 166], "max_seqlen": [72, 88, 101, 129], "max_seqlen_for_logn_sc": 130, "max_sequence_length": [72, 101, 134], "max_split_size_mb": 28, "max_throughput": 20, "max_throughput_benchmark": 20, "max_token": [9, 11, 17, 20, 28, 29, 30, 31, 39, 40, 41, 48, 49, 51, 56, 57, 58, 63, 64, 65, 66, 67, 68, 69, 75, 81, 83, 85, 128, 143, 147, 154, 156, 162], "max_token_count": 58, "max_tokens_in_buff": [81, 143], "max_tokens_in_paged_kv_cach": [128, 134, 148], "max_total_draft_token": 143, "max_util": [0, 21, 128, 143], "max_verification_set_s": 143, "max_window_s": 143, "maxaccepteddrafttokensperstep": 1, "maxacceptedtoken": 1, "maxadapters": 0, "maxattentionwindow": 1, "maxattentionwindowvec": [0, 1], "maxbadwordslen": 1, "maxbatchs": [0, 1, 102], "maxbatchsizeruntim": 0, "maxbatchsizeruntimeupperbound": 0, "maxbatchsizestat": 0, "maxbatchsizetunerrecommend": 0, "maxbeamwidth": [0, 1, 99, 148], "maxdecoderstep": 1, "maxdecodingdrafttoken": 1, "maxdecodingtoken": [0, 1], "maxdraftpathlen": [0, 1], "maxdrafttoken": [0, 1], "maxencoderlen": 1, "maxgenerationlength": 1, "maxgenlengthdevic": 1, "maxgenlengthhost": 1, "maxgentoken": 1, "maxgputotalbyt": 0, "maxim": [0, 3, 5, 8, 12, 14, 17, 20, 35, 37, 87, 120, 128, 163], "maximum": [0, 1, 2, 5, 8, 11, 15, 18, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 37, 38, 72, 74, 81, 83, 84, 88, 92, 98, 99, 101, 102, 120, 123, 129, 130, 134, 135, 139, 143, 146, 148, 153, 166], "maxinputlen": [1, 102], "maxinputlength": 1, "maxlength": 1, "maxlengthstop": 0, "maxlorarank": 1, "maxmedusahead": 1, "maxnewtoken": [1, 148], "maxnonleafnodesperlay": 1, "maxnumactiverequest": 0, "maxnumblock": [0, 26], "maxnumpath": 1, "maxnumsequ": [1, 148], "maxnumtoken": [0, 1], "maxnumtokensruntim": 0, "maxnumtokensstat": 0, "maxnumtokenstunerrecommend": 0, "maxoutputlength": 99, "maxpagesperblock": 1, "maxpagesperblockdevic": 0, "maxpagesperblockhost": 0, "maxpathdraftlen": 1, "maxpathlen": [0, 1], "maxpositionembed": [0, 1], "maxpromptembeddingtables": 1, "maxqueues": 0, "maxseqidlemicrosecond": 0, "maxseqlen": 1, "maxsequencelen": [1, 102], "maxsequencelength": 1, "maxsplit": 143, "maxstopwordslen": 1, "maxtoken": [0, 135, 148], "maxtokensinbuff": 0, "maxtokensperenginestep": 1, "maxtokensperstep": 1, "maybe_capture_hidden_st": 92, "maybe_to_pybind": 143, "mb": [19, 135, 143], "mbackedstorag": 1, "mbackend": 0, "mbackendagentdesc": 0, "mbackendtyp": 0, "mbackground": 1, "mbackstream": 1, "mbacktyp": 1, "mbadhandl": 1, "mbart": [138, 148], "mbatchingtyp": 0, "mbatchsizet": 0, "mbeamsearchbuff": 1, "mbeamsearchdiversityr": 0, "mbeamwidth": 0, "mbeamwidtharrai": 0, "mbind": 19, "mbindoffset": 1, "mbp": 51, "mbuffer": 1, "mbuffermanag": 1, "mc_handl": 1, "mc_ptr": 1, "mc_va": 1, "mcachemap": 1, "mcachemutex": 1, "mcachepagemanag": 1, "mcachest": 0, "mcachetransceiverconfig": 0, "mcapacityschedulerpolici": 0, "mcommmod": 0, "mcommptr": 1, "mcommstat": 0, "mcommtyp": 0, "mcomputecontextlogit": 1, "mcomputegenerationlogit": 1, "mconfig": [0, 1], "mconfigur": 1, "mconnectioninfo": 0, "mcontextchunkingpolici": 0, "mcontextfmha": 1, "mcontextparallel": [0, 1], "mcopyonpartialreus": 0, "mcp": [11, 148], "mcpu": 1, "mcpudiff": 1, "mcreator": 1, "mcrosskvcachefract": 0, "mct": 11, "mcudagraphcaches": 0, "mcudagraphmod": 0, "mcumlogprobstmp": 1, "md": [12, 78, 92, 109, 111, 129, 143, 148, 150, 158], "mdatatyp": [0, 1], "mdebugconfig": 0, "mdebuginputtensor": 0, "mdebugoutputtensor": 0, "mdebugtensornam": 0, "mdebugtensorsmaxiter": 0, "mdecod": 1, "mdecodedurationm": 0, "mdecoderetentionprior": 0, "mdecoderstream": 1, "mdecodingconfig": 0, "mdecodinglayerworkspac": 1, "mdecodingmod": [0, 1], "mdefaulteaglechoic": 1, "mdefaultmedusachoic": 1, "mdefaultposteriorthreshold": 1, "mdesc": [0, 1], "mdevic": 1, "mdevicebuffermanag": 1, "mdevicecacheperc": 0, "mdeviceid": [0, 1], "mdirectori": 0, "mdllmutex": 0, "mdogreedysampl": 1, "mdonetask": 1, "mdprank": 0, "mdpsize": 0, "mdrafttoken": 0, "mdstdesc": 0, "mdynamicbatchconfig": 0, "mdynamicbatchmovingaveragewindow": 0, "mdynamicdecodelay": 1, "mdynamictreemaxtopk": 0, "me": [9, 29, 40, 56, 62, 64, 67, 70, 147, 156], "meaglechoic": 0, "meagleconfig": 0, "mean": [1, 2, 4, 5, 10, 11, 13, 14, 15, 16, 18, 21, 23, 25, 28, 29, 30, 31, 36, 37, 38, 45, 46, 48, 60, 62, 72, 83, 88, 92, 100, 101, 102, 105, 109, 112, 114, 115, 119, 120, 121, 122, 128, 129, 132, 134, 135, 143], "meaning": [1, 2, 14, 123, 127], "meant": [124, 125, 143], "meantim": 148, "meanwhil": [10, 19], "mearlystop": 0, "measur": [0, 3, 5, 6, 7, 8, 13, 14, 15, 18, 19, 20, 37, 73, 81, 98, 120, 121, 148, 152], "mechan": [15, 16, 19, 34, 81, 83, 99, 113, 143, 166, 167], "media": [37, 120, 148], "media_path": [37, 120], "median": [11, 25, 28, 29, 30, 31], "medic": 20, "medium": [7, 29, 79, 139, 148, 159], "medusa": [0, 1, 21, 22, 129, 131, 134, 143, 148], "medusa_choic": [21, 109, 120, 134, 143], "medusa_decode_and_verifi": 134, "medusa_hidden_act": 133, "medusa_logit": 134, "medusa_model_dir": 133, "medusa_output_token": 134, "medusa_path": 134, "medusa_position_offset": 134, "medusa_temperatur": [109, 134], "medusa_topk": 134, "medusa_tree_id": 134, "medusachoic": [0, 1], "medusaconfig": 131, "medusacurtokensperstep": 1, "medusadecodingconfig": [93, 143], "medusaforcausallm": 131, "medusainput": 1, "medusalogit": 1, "medusapath": 1, "medusatargettokensperstep": 1, "medusatreeid": 1, "meet": [7, 15, 16, 29, 129], "megan": 28, "mem": 156, "memavail": 19, "membeddingt": 0, "member": [0, 1, 11, 19, 102, 103, 110, 113, 129], "memcpi": 19, "memfre": 19, "meminfo": 19, "memlock": [9, 17, 20, 25, 94, 139, 147], "memori": [0, 1, 3, 4, 6, 7, 10, 12, 13, 14, 15, 16, 17, 18, 20, 21, 22, 23, 26, 28, 29, 30, 31, 37, 38, 42, 48, 72, 73, 77, 80, 81, 84, 85, 88, 89, 90, 92, 93, 98, 100, 101, 102, 104, 106, 113, 114, 115, 120, 121, 126, 127, 129, 134, 139, 143, 146, 148, 151, 152, 156, 157, 160, 162, 166], "memorydesc": 0, "memorypoolfre": [1, 135], "memorypoolreserv": [1, 135], "memorypooltrimto": 1, "memorypoolus": 1, "memorytyp": [0, 1], "memorytypestr": 1, "memset": 1, "memsetconfigur": 1, "memtot": 19, "memtyp": 1, "memusagechang": 135, "menableattentiondp": [0, 1], "menablebatchsizetun": 0, "menableblockreus": 0, "menablechunkedcontext": 0, "menablecontextfmhafp32acc": 0, "menablemaxnumtokenstun": 0, "menablepartialreus": 0, "menabletrtoverlap": 0, "mencodedvocab": 0, "mencoderhiddens": 1, "mengineaddr": 1, "menginebuff": 1, "menginepath": 1, "mengines": 1, "mental": 62, "mental_health_lora_dir": 62, "mention": [2, 11, 19, 20, 28, 29, 30, 31, 42, 102, 115, 123], "mentri": 1, "mentryit": 1, "menu": [116, 117], "merg": [12, 15, 29, 34, 129, 148], "merlin": 20, "meshgrid": 129, "meshgrid2d": 129, "messag": [9, 12, 17, 19, 20, 23, 29, 38, 39, 40, 66, 67, 70, 72, 81, 98, 107, 129, 135, 143, 147, 148, 156], "met": [0, 1, 99, 109], "meta": [23, 30, 31, 37, 38, 65, 73, 75, 79, 115, 120, 121, 128, 138, 142, 143, 145, 152, 154, 156, 159], "meta_ckpt_dir": 131, "metadata": [16, 19, 26, 37, 56, 72, 81, 92, 104, 120, 144, 149, 151], "metadata_server_config_fil": 26, "metal": [141, 148], "meth": 142, "method": [0, 1, 3, 8, 10, 11, 13, 14, 15, 16, 19, 34, 37, 42, 57, 58, 72, 78, 80, 81, 84, 90, 91, 92, 93, 97, 99, 101, 102, 108, 109, 110, 112, 113, 115, 120, 134, 137, 139, 143, 144, 148, 149, 150, 158, 160, 166, 167], "metric": [0, 8, 14, 15, 16, 19, 23, 36, 37, 38, 73, 81, 119, 120, 121, 123, 126, 127, 143, 148, 152], "metrics_log_interv": 26, "metro": 29, "metropoli": [20, 29], "mevent": 1, "meventbuffermaxs": 0, "mexecutionconfig": 1, "mextendedruntimeperfknobconfig": 0, "mfailfastonattentionwindowtoolarg": 0, "mfastlogit": 0, "mfirstgentoken": 0, "mfirsttim": 1, "mflagptr": 1, "mfreegpumemoryfract": 0, "mfreepageid": 1, "mfrequencypenalti": 0, "mfuntowicz": 148, "mgathergenerationlogit": 0, "mgemmallreducedtyp": 1, "mgmn": [15, 148], "mgpu": 1, "mgpudiff": 1, "mgpuspernod": 1, "mgpuweightsperc": 0, "mgreedysampl": 0, "mguid": 0, "mguideddecodingconfig": 0, "mguidetyp": 0, "mh": 109, "mh1": 109, "mha": [3, 14, 22, 72, 78, 79, 83, 88, 101, 104, 129, 134, 151, 158, 159], "mhandl": 1, "mhandler": 0, "mhiddens": 1, "mhostcaches": 0, "mi": [29, 137], "mib": 135, "micro": [0, 135], "microbatchid": 0, "microbatchschedul": [150, 167], "microsecond": 0, "microsoft": [75, 79, 112, 138, 145, 154, 159], "mid": [18, 32, 148], "middl": [11, 36, 119], "mig": [19, 20], "might": [0, 2, 7, 10, 15, 22, 37, 83, 84, 94, 99, 113, 115, 120, 121, 122, 127, 134, 135, 139, 143, 148, 166], "migrat": [32, 115, 132, 148], "million": [37, 120], "millisecond": [0, 83, 143], "millisecondstyp": 0, "mimpl": 0, "min": [0, 1, 4, 12, 13, 14, 21, 37, 102, 120, 121, 127, 129, 139, 143], "min_lat": 129, "min_length": [102, 134], "min_p": [0, 102, 134, 143], "min_token": 143, "mind": [7, 128, 142], "mindim": 1, "mindimfirst": 1, "mini": [79, 148, 159], "minim": [8, 12, 15, 16, 19, 20, 35, 80, 81, 86, 88, 89, 126, 148, 160], "minimum": [0, 8, 17, 20, 21, 37, 38, 72, 101, 102, 120, 123, 129, 134, 135, 143], "minitron": [79, 138, 145, 148, 159], "minittozero": 1, "minlat": [107, 143], "minlength": [1, 102, 148], "minnormedscorescba": 1, "minor": [35, 148], "minp": [0, 1, 102], "minprogresstask": 1, "minputpack": 1, "minputtokenextraid": 0, "mintoken": [0, 148], "mintpsplitdim": 1, "minut": [0, 7, 17, 20, 121], "mip": 0, "mipcmemoryhandl": 1, "mirco": 0, "mirror": 143, "mirror_pybind_enum": 143, "mirror_pybind_field": 143, "mish": 130, "mismatch": [115, 139], "misorchestr": 0, "mispagefre": 1, "miss": [0, 37, 103, 120, 148], "missedblock": [0, 26], "missedblocksperrequest": 0, "mission": [8, 12, 15, 16], "mistral": [37, 79, 90, 100, 120, 123, 127, 136, 137, 138, 145, 148, 159], "mistral3": [138, 148], "mistral3forconditionalgener": [138, 145], "mistralai": [37, 79, 120, 138, 145, 159], "mistralforcausallm": [138, 145], "misus": 148, "miterstatsmaxiter": 0, "mitig": [8, 12, 15, 115], "mix": [14, 20, 81, 98, 122, 127, 148], "mixed_precis": 143, "mixed_sampl": 148, "mixer": 148, "mixtral": [37, 79, 90, 100, 106, 120, 123, 127, 137, 138, 145, 148, 159], "mixtralforcausallm": [138, 145], "mixtur": [14, 15, 25, 28, 29, 30, 31, 127, 148], "mjointdecodinginput": 1, "mjointdecodingoutput": 1, "mkdir": [9, 28, 29, 30, 31, 116], "mkvcacheconfig": 0, "mkvcachetyp": 1, "mkvfactor": 0, "ml": [129, 148], "mla": [8, 12, 13, 74, 77, 89, 93, 129, 145, 148, 153, 157], "mla_backend": [77, 157], "mlayertyp": 1, "mlen": 0, "mlengthpenalti": 0, "mllama": [138, 148], "mllamaconfig": 131, "mllamaforcausallm": 131, "mllamaforconditionalgener": [138, 145], "mlogit": 0, "mlogitsdtyp": 1, "mlogitspostprocessorconfig": 0, "mlookaheaddecodingconfig": 0, "mlookaheaddecodingmaxnumrequest": 0, "mlop": 20, "mloramodul": 1, "mloraprefetchdir": 0, "mlp": [22, 106, 111, 113, 114, 129, 139, 144, 148, 149], "mlp_4h_to_h": [22, 106], "mlp_bia": 131, "mlp_gate": [22, 106], "mlp_gate_up": [22, 106], "mlp_h_to_4h": [22, 106], "mlp_output": 139, "mlp_router": [22, 106], "mlperf": [20, 93], "mlphiddens": 1, "mlptype": 129, "mm": [19, 148], "mm_data": [37, 120], "mm_embed": [57, 143], "mm_embedding_handl": [29, 143], "mm_embedding_offload": 134, "mm_encoder_onli": 143, "mma": [14, 129], "mmanag": 1, "mmanagedweightsmap": 1, "mmanageweightstyp": 1, "mmap": 19, "mmaxadapters": 0, "mmaxattentionwindow": 0, "mmaxattentionwindowvec": 0, "mmaxbatchs": [0, 1], "mmaxbeamwidth": [0, 1], "mmaxdecodingdecodertoken": 1, "mmaxdecodingdrafttoken": 1, "mmaxdecodingenginetoken": 1, "mmaxdraftpathlen": 1, "mmaxencoderlen": 1, "mmaxgputotalbyt": 0, "mmaxinputlen": 1, "mmaxlorarank": 1, "mmaxnonleafnodesperlay": 1, "mmaxnumpackedmask": 1, "mmaxnumpath": 1, "mmaxnumsequ": 1, "mmaxnumtoken": [0, 1], "mmaxpagesperblock": 1, "mmaxpagesperblockdevic": 0, "mmaxpagesperblockhost": 0, "mmaxpositionembed": 1, "mmaxpromptembeddingtables": 1, "mmaxqueues": 0, "mmaxseqidlemicrosecond": 0, "mmaxsequencelen": 1, "mmaxsequencelength": 1, "mmaxtoken": 0, "mmaxtokensinbuff": 0, "mmedusachoic": 0, "mmemori": 1, "mmemorytyp": 1, "mmha": [129, 148], "mminp": 0, "mmintoken": 0, "mmlphiddens": 1, "mmlu": [7, 12, 148], "mmlu_llmapi": 148, "mmmu": [37, 120], "mmode": 1, "mmodelconfig": [0, 1], "mmodelnam": 1, "mmodelvari": 1, "mmoduleidtomodul": 1, "mmprojector": 148, "mmropepositiondelta": 0, "mmroperotarycossin": 0, "mmultiblockmod": 0, "mmulticast": 1, "mmultimodalhash": 0, "mmultimodallength": 0, "mmultimodalposit": 0, "mmutex": 1, "mname": [0, 1], "mnbattentionlay": 1, "mnbhead": 1, "mnbkvheadsperlay": 0, "mnblayer": 1, "mnbrnnlayer": 1, "mngramsiz": 0, "mnnvl": [15, 89, 107, 129, 143, 148], "mnorepeatngrams": 0, "mnormalizelogprob": 0, "mnumcopystream": [0, 1], "mnumdecodingenginetoken": 1, "mnumdevicemodulelay": 0, "mnumensurework": 0, "mnumhostmodulelay": 0, "mnumkvheadsperattentionlay": 1, "mnumkvheadspercrossattentionlay": 1, "mnumlanguag": 1, "mnumnod": 0, "mnumputwork": 0, "mnumreturnbeam": 0, "mnumreturnsequ": 0, "mnumsm": 1, "mnumtransformerslay": 1, "mobil": 56, "modal": [21, 86, 90, 137, 145], "mode": [0, 1, 10, 14, 17, 19, 20, 21, 22, 23, 26, 34, 35, 51, 59, 60, 61, 72, 81, 85, 92, 100, 101, 103, 113, 114, 128, 129, 130, 134, 135, 137, 143, 144, 148, 149, 162], "mode_t": 0, "model": [0, 1, 3, 4, 5, 6, 7, 11, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 32, 35, 36, 39, 40, 41, 42, 47, 48, 49, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 72, 73, 74, 75, 77, 80, 81, 82, 83, 84, 85, 87, 88, 91, 97, 98, 99, 100, 101, 104, 105, 106, 107, 110, 112, 115, 119, 121, 124, 128, 129, 130, 132, 133, 134, 135, 136, 137, 140, 141, 143, 147, 151, 152, 153, 154, 156, 157, 160, 161, 162, 163, 164, 165, 166, 167], "model_architectur": 143, "model_arg": [28, 30, 31], "model_cl": 130, "model_cls_fil": 22, "model_cls_nam": 22, "model_computed_field": 143, "model_config": [22, 80, 134, 143, 144, 149, 160], "model_construct": 143, "model_copi": 143, "model_dir": [11, 13, 59, 86, 106, 110, 111, 112, 113, 114, 115, 120, 122, 131, 133, 139], "model_dump": 143, "model_dump_json": 143, "model_engin": [150, 166], "model_extra": 143, "model_factori": [77, 157], "model_field": 143, "model_fields_set": 143, "model_format": 143, "model_json_schema": 143, "model_kwarg": [75, 77, 154, 157], "model_nam": [15, 38, 60, 134], "model_parametrized_nam": 143, "model_path": [15, 20, 21, 25, 28, 30, 31, 36, 37, 60, 64, 85, 110, 119, 120, 162], "model_post_init": 143, "model_qu": 120, "model_rebuild": 143, "model_valid": 143, "model_validate_json": 143, "model_validate_str": 143, "model_weights_load": [114, 148], "modelconfig": [0, 80, 102, 134, 144, 148, 149, 160], "modelengin": [35, 92, 150, 166], "modelidtomodel": 1, "modeling_deepseekv3": [12, 14], "modeling_gemma3": [80, 160], "modeling_llama": [144, 149], "modeling_mymodel": [144, 149], "modeling_opt": [144, 149], "modeling_util": [80, 85, 143, 144, 149, 160, 162], "modelmodel_dump": 143, "modelmodel_dump_json": 143, "modelnam": 1, "modelopt": [28, 30, 31, 37, 38, 112, 115, 120, 133, 148], "modelpath": 0, "modelrunn": [112, 134, 148], "modelrunnercpp": [134, 148], "modelrunnermixin": 134, "models_as_dict": 143, "modeltyp": [0, 110], "modelvari": 1, "modelweightsformat": 114, "modelweightsload": [114, 148], "moder": [8, 16, 64, 81], "modern": 134, "modif": [29, 103, 113, 143], "modifi": [10, 11, 19, 37, 75, 85, 91, 92, 93, 94, 99, 103, 120, 123, 127, 128, 139, 146, 148, 154, 162], "modul": [0, 1, 8, 11, 12, 15, 16, 20, 21, 22, 72, 80, 81, 92, 93, 94, 101, 102, 111, 112, 113, 114, 127, 129, 130, 131, 133, 134, 139, 143, 144, 148, 149, 160], "modular": [11, 15, 80, 142, 146, 160], "modularli": [16, 81], "module1": 12, "module10": 12, "module11": 12, "module12": 12, "module13": 12, "module2": 12, "module3": 12, "module4": 12, "module5": 12, "module6": 12, "module7": 12, "module8": 12, "module9": 12, "module_id": 106, "module_nam": [80, 160], "module_names_breakdown": [80, 160], "module_weight": [80, 160], "moduleid": [1, 106], "moduleidtomodel": 1, "modulelist": [144, 149], "moduletyp": 1, "modulo": 129, "moe": [8, 12, 13, 15, 22, 25, 30, 31, 38, 55, 93, 106, 114, 127, 129, 131, 143, 148], "moe_4h_to_h": [22, 106], "moe_backend": [20, 28], "moe_cluster_parallel_s": 143, "moe_config": [2, 9, 13, 15, 20, 143], "moe_ep_s": 100, "moe_expert_parallel_s": [55, 89, 143], "moe_finalize_allreduce_residual_rms_norm": 129, "moe_gat": [22, 106], "moe_gemm": 108, "moe_h_to_4h": [22, 106], "moe_load_balanc": 15, "moe_plugin": 22, "moe_rout": [22, 106], "moe_shared_": 15, "moe_tensor_parallel_s": [55, 89, 143], "moe_tp_siz": 100, "moeallreduceparam": 129, "moeconfig": [93, 131, 143], "moeloadbalancerconfig": 143, "moetopk": 148, "moment": 99, "monboardblock": 0, "mondemand": 1, "monitor": [16, 22, 81, 89, 104, 143], "monitor_memori": [22, 143], "monolith": [72, 88, 101], "monost": 0, "mont": 11, "month": [10, 37, 120], "monthli": [28, 29, 30, 31], "mop": 0, "mopenipc": 1, "moptimaladapters": 0, "morchestratorconfig": 0, "morchleadercomm": 0, "more": [0, 1, 3, 4, 5, 7, 8, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23, 26, 28, 29, 30, 31, 32, 33, 34, 37, 38, 42, 44, 47, 58, 64, 72, 73, 74, 77, 78, 81, 83, 84, 88, 89, 91, 93, 94, 98, 99, 100, 101, 102, 103, 104, 109, 111, 112, 113, 120, 121, 123, 126, 127, 128, 129, 135, 139, 140, 142, 143, 144, 147, 148, 149, 151, 152, 153, 157, 158, 165, 167], "most": [0, 1, 3, 4, 5, 7, 8, 10, 11, 12, 14, 16, 19, 20, 32, 33, 34, 36, 47, 52, 53, 55, 57, 64, 73, 81, 83, 84, 88, 91, 96, 97, 102, 104, 109, 113, 115, 119, 124, 125, 127, 128, 129, 135, 139, 140, 142, 143, 146, 147, 148, 152, 165], "mostli": [15, 92], "motiv": 93, "mount": [9, 20, 26, 28, 29, 30, 31, 59, 60, 61, 93, 94], "mount_dest": [59, 60, 61], "mount_dir": [59, 60, 61], "mountain": 29, "moutdim": 1, "moutdimfirst": 1, "moutputbeamhypothes": 1, "mouttpsplitdim": 1, "move": [0, 1, 15, 32, 92, 104, 115, 129, 139, 143, 148], "movement": [104, 113], "mownsev": 1, "mownsstream": 1, "mp3": 26, "mp4": [26, 40, 67], "mpageblock": 1, "mpagedcontextfmha": 1, "mpagedst": 1, "mpagemanagerconfig": 1, "mpages": 1, "mpagesmutex": 1, "mpagewidth": 1, "mparallelconfig": 0, "mparticipantid": 0, "mpeftcacheconfig": 0, "mpi": [0, 1, 16, 22, 26, 36, 59, 60, 61, 81, 97, 102, 113, 115, 119, 120, 121, 129, 139, 143, 148], "mpi4pi": [121, 139, 142, 148], "mpi_barri": 115, "mpi_comm_world": 102, "mpi_group_barri": 1, "mpi_sess": 143, "mpicomm": 0, "mpicommsess": [93, 143], "mpin": 1, "mpinneddiff": 1, "mpinnedpool": 1, "mpinnedpooldiff": 1, "mpipelineparallel": [0, 1], "mpirun": [20, 112, 113, 121, 139, 142, 148], "mpisess": 143, "mpistat": 0, "mpointer": 1, "mpool": 1, "mport": 0, "mposteriorthreshold": 0, "mppreducescatt": 1, "mprecis": 1, "mpresencepenalti": 0, "mprocessorbatch": 0, "mprocessormap": 0, "mprompttableoffload": 0, "mprop": 1, "mpt": [7, 137, 138, 148], "mptforcausallm": 131, "mptmodel": 131, "mptrexpertcount": 148, "mqa": [3, 6, 12, 22, 72, 89, 93, 101, 104, 129, 148, 151], "mquantmod": 1, "mrank": [0, 1], "mrecvpollperiodm": 0, "mremotenam": 0, "mrepetitionpenalti": 0, "mreplic": 0, "mreqid": 0, "mrequeststatsmaxiter": 0, "mrnnconfig": 1, "mrope": [0, 129], "mrope_param": [130, 134], "mrope_position_delta": [129, 130, 134], "mrope_rotary_cos_sin": [129, 130], "mrope_rotary_cos_sin_s": 131, "mropeconfig": 0, "mropeparam": [130, 134], "mropepositiondelta": 0, "mroperoratysinco": 0, "mrotaryembeddingdim": 1, "mruntimedefault": 1, "mruntimestream": 1, "msamplingconfig": 1, "mscale": 129, "mscale_all_dim": 129, "mschedulerconfig": 0, "msecondaryofflineminprior": [0, 143], "msecondaryoffloadminprior": 0, "mseed": 0, "mselfidx": 0, "msg": [0, 1, 12, 143], "msinktokenlength": 0, "msize": 1, "msizeperhead": [0, 1], "mskipcrossattnblock": 1, "msl": 1, "mslotsperpag": 1, "mspawnprocess": 0, "mspeculativedecodingconfig": 0, "mspeculativedecodingmod": 1, "mspeculativedecodingmodul": 1, "msrcdesc": 0, "mstate": [0, 1], "mstoptokenid": 0, "mstream": 1, "msyncmessag": 0, "mt": 18, "mt5": 138, "mtag": [0, 1], "mtaskid": 0, "mtemperatur": 0, "mtensor": 0, "mtensorparallel": [0, 1], "mtoken": 0, "mtokenizerstr": 0, "mtokenrangeretentionconfig": 0, "mtokensperblock": [0, 1], "mtopk": 0, "mtopp": 0, "mtoppdecai": 0, "mtoppmin": 0, "mtoppresetid": 0, "mtotalnumpag": 1, "mtp": [2, 10, 15, 16, 28, 65, 82, 91, 93, 143, 145, 146, 148, 161, 165], "mtp3": 16, "mtp3_autoregress": 12, "mtp3_top1": 12, "mtp3_top10": 12, "mtp3_top15": 12, "mtp3_vanilla": 12, "mtp_eagle_one_model": 143, "mtpdecodingconfig": [65, 92, 93, 143], "mtprank": 1, "mtransfermod": 0, "mtrimpool": 1, "mtype": [0, 1], "much": [10, 11, 13, 15, 19, 36, 38, 88, 105, 113, 119, 126, 135], "mul": 129, "multi": [0, 3, 10, 11, 13, 14, 15, 20, 22, 25, 40, 59, 60, 61, 67, 73, 74, 78, 79, 83, 84, 89, 90, 93, 94, 98, 99, 100, 102, 105, 106, 109, 112, 115, 121, 129, 131, 135, 137, 143, 146, 148, 151, 152, 153, 158, 159], "multi_block_mod": [101, 134, 143, 148], "multi_round": [25, 28, 29, 30, 31], "multiblockmod": 0, "multicast": 1, "multicastconfigur": 1, "multidimension": 129, "multihead": [3, 113], "multiheadlatentattent": [77, 157], "multimap": 1, "multimod": [0, 21, 22, 37, 71, 93, 120, 134, 138, 140, 142, 143, 148], "multimodal_embedding_handl": 143, "multimodal_hash": 143, "multimodalembed": 0, "multimodalencod": [93, 143], "multimodalhash": 0, "multimodalinput": 0, "multimodallength": 0, "multimodalmodelrunn": 134, "multimodalposit": 0, "multinod": 122, "multinomi": 102, "multipl": [0, 1, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 22, 23, 28, 29, 32, 33, 50, 57, 64, 72, 75, 79, 81, 83, 84, 85, 88, 89, 91, 96, 98, 99, 100, 101, 102, 103, 104, 105, 109, 113, 114, 121, 122, 123, 126, 129, 130, 134, 139, 143, 146, 148, 151, 154, 159, 162], "multiple_profil": [22, 120, 123, 127, 148], "multipli": [11, 14, 20, 72, 101, 114, 129], "multiply_and_lora": 130, "multiply_collect": 130, "multiprocessor": 113, "multithread": 0, "multiturn": 23, "munsign": 1, "musecrossattent": 1, "musedynamictre": 0, "musegemmallreduceplugin": 1, "musegptattentionplugin": 1, "musegpudirectstorag": 0, "museloraplugin": 1, "musemambaconv1dplugin": 1, "musemrop": 1, "musepositionembed": 1, "museshapeinfer": 1, "musetokentypeembed": 1, "museuvm": 0, "must": [0, 1, 10, 15, 16, 19, 20, 22, 28, 29, 30, 31, 32, 49, 51, 57, 72, 80, 81, 83, 88, 89, 91, 92, 97, 98, 99, 100, 101, 102, 105, 106, 109, 113, 117, 123, 129, 130, 132, 134, 137, 139, 143, 160], "mutabl": [0, 1], "mutablepageptr": 1, "mutex": [0, 1, 19], "mutlictaskvmod": 148, "mutual": [19, 102, 137], "muvm": 1, "muvmdiff": 1, "mvalu": 1, "mverificationsets": 0, "mversion": 1, "mvocabs": 1, "mvocabsizepad": 1, "mweight": 0, "mwindows": 0, "mworkerexecutablepath": 0, "mworldconfig": 1, "mxfp4": [29, 90, 148], "mxfp8": [29, 90, 148], "mxfp8xmxfp4": 148, "my": [1, 35, 37, 47, 48, 49, 52, 53, 54, 55, 63, 81, 85, 90, 91, 97, 98, 120, 140, 142, 147, 162, 164, 165], "my_config": [75, 154], "my_faster_on": 42, "my_llm_task": 142, "my_model": 111, "my_profile_export": [45, 46], "my_test": 33, "myattent": [144, 149], "myconfig": [144, 149], "mycustomlogitsprocessor": 91, "mycustomweightload": [80, 160], "mydecoderlay": [111, 144, 149], "mydraft": 92, "mymodel": [111, 144, 149], "mymodelforcausallm": [111, 144, 149], "n": [1, 8, 9, 11, 13, 14, 17, 19, 20, 26, 28, 29, 30, 31, 35, 37, 47, 51, 52, 53, 54, 55, 57, 58, 59, 60, 61, 62, 63, 64, 72, 73, 81, 84, 87, 88, 91, 92, 93, 97, 98, 101, 106, 109, 112, 113, 120, 121, 126, 129, 130, 131, 135, 136, 137, 139, 140, 142, 143, 145, 147, 148, 152, 163], "n1": [17, 58, 63], "n2": [17, 58], "n3": 58, "n4": 58, "n_worker": 143, "na": [37, 120, 148], "naiv": [72, 79, 127, 159], "naivepatternrewriter_replaceaddwithsub": 103, "nalso": 20, "name": [0, 1, 9, 10, 21, 23, 25, 26, 28, 29, 30, 31, 32, 35, 37, 38, 47, 48, 49, 52, 53, 54, 55, 56, 60, 63, 64, 65, 70, 80, 89, 90, 91, 92, 93, 97, 99, 102, 103, 106, 108, 112, 113, 116, 120, 121, 129, 131, 132, 133, 134, 139, 140, 142, 143, 144, 147, 148, 149, 156, 160, 164, 165], "named_network_output": 139, "named_paramet": 114, "namedtupl": 143, "namespac": [0, 1, 131, 142, 143], "nano": [79, 159], "nanobind": 148, "nanoflow": [87, 163], "nation": [37, 120], "nationwid": [37, 120], "nativ": [4, 14, 15, 20, 21, 79, 115, 142, 144, 146, 148, 149, 159], "native_quant_flow": 131, "nativegenerationcontrol": 11, "natur": [8, 14, 15, 18, 40, 67, 115, 121, 143], "naur": [0, 91, 99, 143], "naver": 138, "nbattentionlay": [0, 1], "nbdim": 1, "nbhead": 1, "nbkvhead": [0, 1], "nbkvheadperlay": 0, "nblayer": 1, "nbook": 28, "nbrnnlayer": 1, "ncache_transceiver_config": 81, "nccl": [12, 15, 22, 81, 98, 107, 113, 129, 139, 143, 148], "nccl_graph_mixing_support": [81, 98], "nccl_p2p_level": 148, "nccl_plugin": 22, "nccl_symmetr": [129, 143], "ncclplugin": 113, "ncclrecv": [15, 129], "ncclsend": [15, 129], "ncuda_graph_config": 17, "nd": [37, 120, 129], "ndarrai": [129, 130, 134], "ndim": 129, "nearest": [14, 25, 28, 29, 30, 31, 35, 129, 143], "nearli": [4, 14, 19, 93, 103], "necess": 109, "necessari": [1, 10, 11, 12, 14, 15, 34, 35, 62, 83, 89, 100, 109, 123, 129, 143, 148, 166], "necessarili": [1, 81, 98, 113, 135], "necessit": [8, 15], "need": [1, 2, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 26, 28, 29, 30, 31, 32, 33, 37, 38, 42, 51, 55, 56, 57, 59, 60, 61, 62, 72, 80, 81, 83, 84, 88, 91, 92, 94, 97, 98, 99, 101, 102, 103, 105, 109, 110, 111, 112, 113, 114, 115, 116, 120, 121, 122, 123, 126, 127, 128, 129, 131, 132, 134, 135, 139, 142, 143, 144, 146, 148, 149, 150, 151, 160, 166, 167], "needed_block": 167, "needs_kv_cache_rewind": 92, "needsdecoderprologu": 1, "needskvcacherewind": 1, "neg": [1, 10, 58, 128, 129, 143], "neglig": [7, 105, 126], "neighborhood": 29, "neither": [99, 129, 135], "nemo": [22, 79, 112, 121, 134, 137, 138, 146, 148, 159], "nemo_ckpt_dir": 131, "nemo_prompt_convert": 134, "nemotron": [79, 138, 145, 148, 159], "nemotron_na": 148, "nemotronforcausallm": [138, 145], "nemotronh_nano_vl_v2": 145, "nemotronna": [138, 145, 148], "nemotronnasforcausallm": [138, 145], "nenable_attention_dp": 17, "nenable_min_lat": 17, "neox": [72, 101, 102, 137, 138, 148], "nest": [75, 103, 154], "net": [18, 105, 139, 143], "net_guard": 103, "network": [14, 15, 20, 22, 38, 51, 72, 91, 99, 100, 101, 103, 107, 113, 115, 129, 135, 137, 139, 143, 148], "neural": [100, 103, 113, 148], "neva": [138, 148], "never": [1, 10, 19, 37, 49, 83, 103, 120, 128, 143], "nevertheless": 15, "new": [0, 1, 4, 5, 8, 10, 11, 12, 13, 15, 18, 19, 28, 29, 30, 31, 35, 38, 39, 41, 47, 52, 53, 54, 55, 56, 58, 66, 68, 79, 80, 83, 88, 89, 92, 93, 94, 97, 99, 101, 102, 103, 105, 106, 109, 110, 115, 116, 126, 127, 129, 134, 140, 141, 142, 143, 146, 147, 148, 150, 156, 159, 160, 166], "new_block_id": 56, "new_decoder_architectur": [112, 131], "new_generated_id": 134, "new_input": 103, "new_line_token": 58, "new_method": 32, "new_nam": [80, 160], "new_out": 103, "new_request": 56, "new_shap": 129, "new_tensor": 129, "new_token": [56, 134], "new_workflow": 148, "newactiverequestsqueuelatencym": 0, "newark": 29, "newer": [138, 148], "newest": [5, 18, 92, 143], "newli": [0, 10, 11, 13, 15, 19, 88, 126, 143], "newlin": 33, "newsiz": 1, "newtoken": 1, "newtokensstep": 1, "newtokensvec": 1, "newvalu": 0, "next": [1, 4, 10, 11, 13, 15, 25, 28, 29, 30, 31, 35, 83, 87, 88, 92, 93, 94, 106, 109, 113, 115, 122, 123, 126, 127, 128, 134, 135, 136, 138, 146, 148, 163], "next_logit": 134, "next_medusa_input_id": 134, "next_medusa_logit": 134, "next_step_buff": 134, "next_step_tensor": 134, "nextdraftindic": 1, "nextdraftlen": 1, "nextdraftpath": 1, "nextdraftprob": 1, "nextdrafttoken": 1, "nextdrafttokenslen": 1, "nextflattoken": 1, "nextgenerationlength": 1, "nextn": 13, "nextpositionoffset": 1, "ngc": [9, 20, 28, 29, 30, 31, 34, 94, 96, 97, 141, 148], "ngoanpv": 148, "ngram": [0, 10, 65, 102, 131, 143, 146, 148], "ngramdecodingconfig": [18, 65, 92, 93, 143], "ngramsiz": 0, "ngroup": 129, "nhead": 129, "nhere": 51, "ni": [51, 137], "nic": [15, 81, 98], "nice": 15, "nif": 29, "nixl": [0, 16, 81, 143, 148], "nj": 54, "njane": [47, 52, 53, 54, 55, 97, 140, 142, 147], "njason": 62, "nkv_cache_config": 17, "nlet": 20, "nmh": 134, "nmt": [134, 138, 148], "nn": [80, 129, 144, 149, 160], "no_kv_cache_reus": 148, "no_quant": 143, "no_repeat_ngram_s": [102, 134, 143], "no_schedule_after_st": 167, "no_schedule_until_st": 167, "no_skip_tokenizer_init": 21, "no_weights_load": 21, "noauxtckernel": 12, "node": [0, 13, 14, 15, 16, 17, 19, 22, 23, 25, 36, 59, 60, 61, 81, 89, 93, 94, 98, 102, 107, 119, 121, 122, 129, 134, 137, 139, 143, 146, 148], "noexcept": [0, 1], "nomin": [47, 54, 97, 140, 142, 147], "non": [0, 7, 10, 11, 12, 13, 14, 15, 19, 22, 28, 29, 30, 31, 37, 38, 57, 72, 74, 81, 83, 86, 98, 101, 104, 110, 113, 115, 129, 139, 143, 148, 153], "non_block": [56, 58], "non_gated_vers": 129, "none": [1, 11, 18, 22, 23, 25, 26, 28, 30, 31, 35, 37, 42, 49, 56, 58, 62, 64, 65, 72, 80, 83, 84, 87, 88, 91, 102, 103, 111, 114, 115, 120, 121, 126, 129, 130, 131, 132, 133, 134, 139, 143, 144, 148, 149, 151, 160], "nonetyp": [134, 143], "nontrivi": 92, "nonzero": 129, "nor": [15, 135], "norepeatngrams": [0, 1, 102], "norm": [2, 14, 36, 37, 38, 60, 114, 119, 120, 121, 129, 144, 148, 149], "norm_before_bmm1": [130, 131], "norm_elementwise_affin": 130, "norm_ep": 130, "norm_epsilon": [112, 131], "norm_factor": [72, 101], "norm_num_group": 130, "norm_pre_residual_weight": 129, "norm_quant_fus": 22, "norm_typ": 130, "norm_weight": 129, "normal": [0, 7, 10, 11, 12, 13, 14, 15, 18, 19, 21, 37, 102, 105, 106, 110, 120, 129, 135, 143, 148], "normalize_log_prob": 143, "normalize_weight": 106, "normalized_shap": [129, 130], "normalizelogprob": [0, 1], "normedscorescba": 1, "north": [29, 111, 113, 139], "northeast": 29, "northeastern": [28, 29, 30, 31, 147], "not_op": 129, "notabl": 7, "notat": [13, 143], "note": [1, 2, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 17, 19, 20, 22, 25, 28, 29, 30, 31, 33, 37, 42, 56, 59, 60, 61, 72, 75, 80, 81, 84, 88, 91, 94, 96, 98, 103, 105, 106, 107, 108, 109, 113, 117, 120, 123, 126, 128, 129, 132, 134, 135, 137, 138, 139, 140, 141, 142, 143, 144, 145, 149, 154, 160, 166], "noth": [19, 92], "notic": 62, "notifysyncmessag": 0, "notimplementederror": [11, 115], "nougat": [90, 137, 138, 148], "novel": 8, "now": [3, 11, 12, 13, 15, 18, 19, 28, 29, 30, 31, 37, 38, 78, 88, 102, 108, 109, 112, 114, 120, 126, 132, 135, 143, 148, 158], "np": 129, "npy": 134, "npytorch_backend_config": 26, "nsight": 93, "nspeculative_config": 17, "nsy": [36, 119], "ntask": [26, 59, 60, 61, 113], "nucleu": 64, "null": [1, 9, 17, 20, 28, 29, 30, 31, 37, 60, 73, 112, 120, 147, 152], "nullopt": [0, 1], "nullptr": [0, 1], "num": [0, 1, 2, 21, 25, 28, 29, 30, 31, 36, 37, 38, 60, 75, 88, 119, 120, 121, 123, 124, 127, 143, 154], "num_accepted_token": 92, "num_attention_head": [112, 129, 130, 131], "num_aud_token": 134, "num_beam": [102, 134], "num_beam_group": 102, "num_block": [134, 166], "num_bucket": [129, 130], "num_capture_lay": 143, "num_channel": [130, 131], "num_class": 130, "num_computed_block": 56, "num_computed_token": 56, "num_concurr": [28, 30, 31], "num_context": [72, 151], "num_ctx_token": [72, 151], "num_draft_token": [0, 129, 134], "num_draft_tokens_alloc": 92, "num_eagle_lay": 143, "num_embed": 130, "num_experts_per_tok": 100, "num_fewshot": [23, 28, 30, 31], "num_gener": [72, 151], "num_gpu": [20, 29], "num_group": [129, 130], "num_head": [72, 88, 89, 101, 114, 129, 134, 151], "num_hidden_lay": [75, 77, 112, 131, 144, 149, 154, 157, 166], "num_imag": 134, "num_img_token": 134, "num_inst": [16, 81], "num_key_value_head": [112, 131, 166], "num_kv_head": [72, 104, 129, 130, 134, 151, 166], "num_kv_heads_origin": 129, "num_kv_heads_per_cross_attn_lay": 134, "num_kv_heads_per_lay": 134, "num_lay": [129, 130, 134, 166], "num_ln_in_parallel_attn": 131, "num_local_block": 130, "num_local_expert": 100, "num_lora_module_lay": 106, "num_lora_modules_lay": 106, "num_medusa_head": [131, 133, 134, 143], "num_medusa_lay": [131, 133], "num_multimodal_token": 0, "num_nextn_predict_lay": [2, 13, 28, 65, 92, 143], "num_nextn_predict_layers_from_model_config": 143, "num_orig_po": 129, "num_po": 129, "num_postprocess_work": [26, 29, 143], "num_profil": 131, "num_prompt": [25, 28, 29, 30, 31], "num_q_head": 12, "num_request": [2, 13, 20, 21, 37, 38, 85, 120, 162], "num_return_sequ": [134, 148], "num_sampl": [23, 36, 119], "num_slot": [15, 28], "num_task": 130, "num_token": [8, 12, 72, 101, 129, 151], "num_tokens_per_block": [129, 166], "num_tokens_per_task": 130, "num_video": 134, "numa": [15, 107], "numa_alloc_onnod": 19, "numacceptedtoken": 0, "numactiverequest": [0, 26], "numactl": [15, 19], "numattentionhead": 1, "numavailablepag": 1, "numbeamscba": 1, "number": [0, 1, 2, 6, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 33, 37, 38, 57, 59, 60, 61, 72, 73, 74, 81, 83, 84, 88, 90, 91, 92, 98, 99, 100, 101, 102, 104, 109, 113, 120, 121, 122, 123, 126, 127, 128, 129, 130, 134, 135, 137, 139, 143, 144, 148, 149, 151, 152, 153, 166], "numblockspercachelevel": 0, "numcompletedrequest": 0, "numcontextrequest": [0, 1], "numcopystream": [0, 1], "numctxgpu": 16, "numctxsequ": 1, "numctxtoken": 0, "numdevicemodulelay": 0, "numdrafttoken": [0, 1], "numdrafttokenshost": 1, "numeaglelay": 1, "numel": 134, "numensurework": 0, "numer": [12, 19, 32, 35, 102, 107, 120, 138, 143], "numexpert": 1, "numgeneratedtoken": 0, "numgengpu": 16, "numgenrequest": 0, "numgensequ": 1, "numgentoken": [0, 1], "numhead": 102, "numhostmodulelay": 0, "numkvattentionhead": 1, "numkvhead": 102, "numlanguag": 1, "numlay": 102, "nummissedblock": 0, "numnewactiverequest": 0, "numnewallocatedblock": 0, "numnewtokenscumsum": 148, "numnod": [0, 148], "numpag": 1, "numpausedrequest": 0, "numpi": [106, 129, 134], "numputwork": 0, "numqueuedrequest": [0, 148], "numrequestswithdrafttoken": 0, "numreturnbeam": 0, "numreturnsequ": [0, 1, 99], "numreusedblock": 0, "numscheduledrequest": 0, "numsequ": 1, "numslot": 1, "numtoken": 1, "numtotalallocatedblock": 0, "numtransformerslay": 1, "nvbugspro": 33, "nvcc": 2, "nvcr": [9, 20, 25, 28, 29, 30, 31, 147, 148], "nvfp4": [12, 15, 21, 22, 28, 30, 31, 37, 79, 90, 120, 143, 148, 159, 164], "nvidia": [2, 3, 4, 5, 7, 8, 9, 10, 11, 13, 15, 16, 17, 19, 20, 21, 22, 25, 26, 28, 29, 30, 31, 33, 34, 37, 38, 39, 40, 41, 43, 45, 46, 47, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 72, 79, 81, 88, 89, 90, 91, 92, 93, 94, 95, 97, 98, 112, 113, 115, 116, 120, 121, 127, 129, 135, 138, 139, 140, 142, 145, 146, 148, 159, 164, 165], "nvila": [86, 138, 148], "nvinfer1": [0, 1], "nvl": [1, 22, 148], "nvl36": 122, "nvl72": [14, 19, 20, 28, 38, 122, 138], "nvlink": [15, 16, 20, 81, 89, 98, 102, 107, 121, 122, 124, 125, 148], "nvswitch": [12, 20, 113], "nvtx": [19, 143], "nwe": 20, "nyc": 29, "nyou": 51, "n\u7b54\u6848": 69, "o": [0, 1, 6, 9, 10, 12, 14, 17, 19, 20, 28, 29, 30, 31, 36, 56, 59, 60, 61, 69, 103, 106, 115, 119, 139], "o_proj": 114, "oai": [40, 67], "oauthtoken": 9, "obei": 139, "obj": 143, "obj0": 143, "obj1": 143, "object": [0, 1, 8, 17, 19, 20, 28, 29, 30, 31, 32, 42, 49, 51, 58, 70, 75, 77, 80, 83, 91, 92, 99, 105, 111, 113, 114, 115, 129, 130, 131, 132, 134, 135, 143, 147, 148, 150, 154, 157, 160], "observ": [8, 14, 16, 18, 19, 38, 57, 81, 92, 98], "obtain": [1, 16, 38, 92, 95, 129], "obtain_answ": 11, "obviou": [2, 14], "occas": 139, "occasion": 148, "occup": [35, 72, 87, 101, 135, 163], "occupi": [7, 10, 14, 15, 29, 135], "occur": [8, 16, 81, 92, 102, 105, 143, 148, 166, 167], "occurr": 143, "ocean": [29, 64], "off": [2, 10, 14, 16, 33, 35, 36, 78, 92, 105, 108, 119, 123, 126, 127, 135, 148, 158], "offer": [7, 8, 10, 11, 12, 16, 18, 19, 34, 72, 90, 113, 151], "offic": 51, "officenetsecur": 51, "offici": [2, 13, 29, 37, 72, 79, 101, 120, 159], "offlin": [5, 8, 14, 23, 38, 47, 86, 89, 93, 111, 120, 148], "offload": [0, 1, 15, 22, 50, 104, 110, 143, 148], "offloadconfigur": 1, "offloading_dis": 57, "offloading_en": 57, "offset": [1, 10, 129, 134, 137, 148], "offsetdim": 1, "ofitensor": 0, "often": [0, 3, 7, 11, 12, 15, 16, 35, 81, 99, 104, 109, 122, 123, 129, 143], "oh": 11, "ok": 139, "okai": 11, "old": [13, 103, 106, 139, 143], "older": [94, 105, 115, 138], "oldest": [18, 92, 106, 143], "oldvalu": 0, "omegaconf": [75, 154], "omit": [1, 9, 91, 95, 99, 115, 129], "ompi": [97, 139], "ompi_mca_btl_tcp_if_includ": 142, "ompi_mca_oob_tcp_if_includ": 142, "onboard": [0, 79, 105, 135, 143, 159], "onboard_block": 143, "onboardblock": 0, "onc": [0, 10, 11, 13, 15, 16, 17, 18, 20, 25, 26, 33, 56, 64, 72, 81, 88, 92, 94, 97, 99, 101, 102, 103, 113, 123, 129, 135, 142, 143, 156], "ondemand": 1, "one": [0, 1, 3, 8, 9, 10, 11, 12, 13, 14, 16, 18, 19, 21, 22, 23, 25, 26, 28, 29, 30, 31, 34, 35, 37, 62, 72, 78, 79, 80, 81, 83, 87, 88, 92, 98, 99, 100, 101, 102, 103, 104, 105, 107, 109, 112, 113, 114, 115, 116, 120, 121, 122, 123, 127, 128, 129, 130, 132, 134, 135, 139, 143, 144, 147, 148, 149, 158, 159, 163, 167], "ones": [0, 10, 19, 75, 106, 143, 147, 154], "oneshot": [12, 107, 129, 143], "oneshotallreduc": 12, "oneshotar": 12, "onevis": [138, 148], "ongo": [8, 10, 15, 20, 28, 29, 30, 31, 35, 115], "onli": [0, 1, 2, 7, 8, 9, 10, 11, 13, 14, 15, 16, 18, 19, 21, 22, 23, 25, 26, 28, 29, 30, 31, 32, 33, 37, 38, 42, 56, 57, 58, 64, 72, 73, 74, 75, 80, 81, 83, 84, 88, 89, 92, 93, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 108, 109, 110, 111, 113, 114, 115, 120, 121, 122, 123, 126, 127, 128, 129, 130, 132, 134, 135, 138, 143, 145, 148, 150, 152, 153, 154, 156, 160, 167], "onlin": [5, 11, 23, 25, 28, 47, 89, 93, 148], "only_cross_attent": 130, "onnx": [20, 22, 129], "onnx__gathernd": 129, "ontario": 29, "onto": 102, "oom": [1, 2, 3, 6, 14, 28, 29, 30, 31, 135], "ootb": [14, 148], "op": [0, 1, 9, 14, 20, 72, 81, 98, 103, 129, 143, 148], "op_and": 129, "op_or": 129, "op_xor": 129, "opaqu": 103, "opaque_st": 143, "open": [3, 11, 12, 14, 15, 19, 36, 96, 97, 102, 108, 119, 139, 146, 147, 148], "openai": [9, 11, 16, 17, 19, 25, 26, 28, 29, 30, 31, 71, 81, 85, 93, 147, 148, 156, 162], "openaiapi": 11, "openelm": [79, 159], "openipc": 1, "openmpi": 148, "opensora": 148, "opensourc": 148, "openssh": 116, "oper": [0, 1, 8, 10, 12, 14, 15, 16, 19, 22, 25, 28, 29, 30, 31, 35, 37, 63, 72, 78, 79, 80, 99, 101, 102, 103, 107, 109, 112, 113, 114, 120, 122, 123, 127, 129, 135, 138, 143, 148, 150, 151, 158, 159, 160, 166], "opportun": [11, 37, 120], "opposit": [10, 58], "opt": [7, 14, 73, 75, 78, 79, 90, 99, 112, 116, 129, 137, 138, 139, 148, 152, 154, 156, 158, 159], "opt_batch_s": [131, 143], "opt_num_token": [22, 131, 143], "optforcausallm": [112, 131], "optim": [1, 3, 4, 5, 6, 7, 9, 11, 15, 17, 20, 21, 22, 25, 28, 29, 30, 31, 37, 38, 47, 52, 58, 63, 75, 78, 79, 80, 83, 84, 87, 88, 89, 90, 92, 93, 94, 97, 99, 102, 103, 104, 107, 108, 109, 113, 115, 120, 122, 123, 124, 125, 129, 135, 138, 139, 140, 141, 142, 143, 147, 148, 150, 151, 154, 156, 158, 159, 160, 163, 164, 166], "optimaladapters": [0, 1], "option": [0, 1, 4, 9, 13, 19, 21, 22, 23, 25, 26, 32, 33, 34, 36, 37, 38, 42, 48, 49, 58, 60, 62, 64, 65, 72, 74, 75, 80, 81, 83, 88, 90, 91, 92, 95, 96, 97, 99, 102, 103, 104, 107, 108, 109, 111, 115, 119, 120, 121, 122, 124, 125, 126, 129, 132, 134, 135, 139, 142, 143, 144, 147, 148, 149, 151, 153, 154, 156, 160, 162, 165, 166], "optionalbufferptr": 1, "optionaltensorptr": 1, "optmodel": 131, "optvec": 1, "orchestr": [0, 15, 16, 35, 80, 81, 109, 139, 148, 160], "orchestratorconfig": 0, "orchleadercomm": 0, "order": [0, 1, 3, 10, 34, 37, 38, 75, 76, 81, 83, 96, 98, 101, 104, 114, 120, 123, 128, 129, 130, 135, 143, 154, 155], "ordin": 143, "org": [0, 1, 22, 28, 29, 30, 31, 87, 97, 100, 106, 129, 137, 163], "organ": [33, 104, 166], "orient": [8, 14, 15, 16], "origin": [0, 10, 13, 14, 15, 18, 19, 23, 29, 58, 72, 78, 85, 88, 92, 101, 103, 106, 107, 129, 143, 144, 148, 149, 158, 162], "original_batch": 18, "original_max_po": 129, "original_max_position_embed": [129, 130], "originaltemperatur": 1, "orin": 20, "oserror": 148, "osl": [3, 4, 5, 6, 8, 12, 13, 14, 15, 19, 20, 23, 25, 28, 29, 30, 31, 37, 38, 75, 93, 120, 127, 154], "osl256": 16, "oss": [27, 90, 93, 108, 145], "ostream": [0, 1], "other": [0, 1, 3, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 32, 33, 37, 38, 42, 49, 59, 60, 61, 72, 80, 81, 83, 84, 88, 89, 92, 94, 98, 99, 100, 101, 102, 105, 107, 108, 109, 113, 114, 115, 121, 122, 123, 126, 127, 128, 129, 132, 135, 139, 142, 143, 145, 148, 151, 160, 167], "other_audio_input": 134, "other_decoder_input": 134, "other_vision_input": 134, "othercach": 1, "otherwis": [0, 1, 10, 11, 19, 37, 42, 56, 57, 72, 81, 92, 98, 99, 101, 102, 120, 129, 134, 139, 143, 151], "our": [2, 7, 8, 10, 11, 12, 13, 14, 15, 18, 19, 37, 38, 47, 51, 52, 53, 55, 56, 76, 78, 86, 88, 97, 120, 123, 126, 127, 129, 139, 140, 142, 144, 147, 148, 149, 155, 158], "out": [0, 1, 3, 4, 5, 6, 10, 12, 13, 14, 15, 17, 18, 20, 28, 29, 30, 31, 33, 34, 36, 38, 47, 59, 60, 61, 81, 86, 88, 98, 106, 115, 119, 123, 126, 127, 129, 135, 142, 147, 148], "out_bia": 130, "out_channel": 130, "out_context_dim": 130, "out_dim": 130, "out_featur": [112, 113, 130], "out_hidden_s": 129, "out_of_tree_exampl": [144, 149], "out_point": 129, "out_tp": [3, 6], "outcom": [19, 57], "outdim": 1, "outdimfirst": 1, "outer": [75, 129, 154], "outermost": 11, "outgrow": 18, "outlin": [36, 119], "outperform": 16, "output": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 32, 35, 36, 37, 38, 42, 45, 46, 47, 48, 49, 51, 52, 53, 54, 55, 56, 57, 58, 60, 62, 63, 64, 65, 72, 81, 85, 88, 91, 97, 101, 102, 103, 105, 106, 109, 113, 119, 121, 122, 123, 124, 125, 127, 128, 129, 130, 134, 139, 140, 142, 143, 147, 148, 150, 151, 162, 167], "output_a": 57, "output_b": 57, "output_bench": 25, "output_ctx0": 16, "output_ctx1": 16, "output_cum_log_prob": 134, "output_dim": 130, "output_dir": [22, 106, 110, 111, 112, 113, 115, 120, 122, 131, 133, 139], "output_dtyp": [129, 130], "output_gen0": 16, "output_gen1": 16, "output_generation_logit": 134, "output_id": 134, "output_json": 21, "output_log_prob": 134, "output_multiplier_scal": 131, "output_pad": [129, 130], "output_path": [15, 28, 30, 31], "output_s": 130, "output_seqlen": [3, 6], "output_sequence_length": 134, "output_str": 11, "output_timing_cach": [22, 143], "output_token": [37, 120], "outputbuff": 1, "outputconfig": [0, 42, 99, 148], "outputidscba": 1, "outputlen": 0, "outputlogprob": 1, "outputtokenid": [0, 99], "outsid": [72, 81, 98, 109, 115, 147, 151], "outsiz": 1, "outstand": [10, 11, 13, 19], "outtpsplitdim": 1, "outweigh": 122, "over": [0, 1, 2, 4, 5, 7, 8, 10, 11, 12, 14, 16, 17, 18, 20, 28, 29, 30, 31, 36, 64, 73, 88, 105, 109, 114, 117, 119, 120, 122, 126, 127, 129, 148, 152, 156], "overal": [2, 8, 10, 11, 13, 14, 15, 16, 25, 35, 72, 81, 88, 99, 101, 105, 107, 109, 122, 123, 126, 127, 128, 144, 149], "overcom": [12, 72, 101, 113], "overflow": 1, "overhead": [0, 8, 10, 12, 13, 14, 16, 18, 20, 35, 63, 72, 81, 87, 89, 98, 99, 113, 122, 143, 148, 151, 163], "overiew": 120, "overlap": [0, 2, 10, 12, 13, 14, 15, 75, 78, 82, 86, 91, 92, 93, 98, 109, 141, 143, 145, 148, 154, 158, 161, 167], "overlap_schedul": 65, "overload": [0, 1], "overrid": [1, 11, 42, 75, 93, 114, 115, 129, 134, 143, 148, 154, 156], "overridden": [34, 94, 143], "override_field": 131, "overshadow": 122, "oversubscrib": [20, 121, 142], "overus": 33, "overview": [2, 7, 8, 15, 36, 37, 78, 93, 94, 98, 99, 104, 119, 120, 121, 141, 150, 156, 158], "overwhelm": 62, "overwrit": [21, 23, 25, 26, 72, 75, 101, 154], "own": [0, 1, 2, 10, 11, 13, 15, 17, 18, 20, 37, 42, 75, 78, 79, 80, 92, 94, 105, 109, 112, 113, 114, 115, 144, 149, 154, 158, 159], "ownership": 0, "ownsev": 1, "ownsstream": 1, "p": [0, 9, 17, 20, 25, 28, 29, 30, 31, 35, 58, 59, 60, 61, 64, 91, 92, 102, 109, 116, 131, 134, 143, 147, 148, 165], "p2p": [15, 129], "p50": [37, 38, 120], "p90": [37, 38, 120, 121], "p95": [37, 38, 120, 121], "p99": [25, 28, 29, 30, 31, 37, 38, 120, 121], "p_max": 0, "p_x": 0, "pack": [0, 1, 10, 22, 83, 88, 102, 128, 129, 131, 135, 144, 149], "packag": [23, 37, 78, 94, 97, 99, 120, 121, 139, 148, 158], "packed_length": 131, "packedinput": 1, "packedmask": 1, "packedmaskhost": 1, "packedmaskhostcopi": 1, "packedmasksdevic": 1, "packedpositionid": 1, "pad": [0, 1, 10, 14, 22, 23, 25, 26, 28, 29, 30, 31, 35, 72, 75, 88, 92, 102, 103, 106, 129, 130, 134, 135, 143, 148, 154], "pad_id": [134, 143], "pad_lda": 130, "pad_ldc": 130, "pad_token_id": 134, "padding_2d": 129, "padding_back": 129, "padding_bottom": 129, "padding_en": [25, 148], "padding_front": 129, "padding_left": 129, "padding_mod": 130, "padding_right": 129, "padding_sid": 11, "padding_top": 129, "padid": 0, "page": [1, 5, 22, 33, 37, 74, 77, 78, 92, 93, 98, 102, 105, 113, 120, 121, 123, 129, 135, 143, 146, 148, 151, 153, 156, 157, 158], "page_s": 19, "pagealign": 1, "paged_context_fmha": [123, 148], "paged_kv_cach": [22, 106, 120, 134], "paged_st": [22, 134], "pagedcontextfmha": 1, "pagedkvcach": 102, "pagedst": 1, "pageid": 1, "pageidx": 1, "pagemanagerconfig": 1, "pageptr": 1, "pagewidth": 1, "paid": 15, "pair": [0, 1, 3, 13, 18, 83, 123, 127, 129, 143], "panel": 11, "paper": [4, 10, 13, 14, 16, 72, 81, 89, 92, 98, 106, 109, 137, 151], "par": [126, 127], "parallel": [0, 1, 2, 3, 5, 6, 8, 10, 11, 13, 16, 17, 18, 20, 21, 23, 26, 29, 30, 31, 38, 55, 72, 77, 78, 81, 82, 84, 93, 98, 99, 101, 102, 109, 112, 113, 123, 124, 125, 129, 130, 131, 135, 142, 143, 144, 145, 146, 148, 149, 157, 158, 161, 167], "parallel_attent": [112, 131], "parallel_config": [89, 143], "parallelconfig": [0, 148], "parallelprocess": 11, "param": [0, 1, 47, 52, 53, 54, 55, 97, 114, 129, 130, 134, 140, 142, 143, 147], "paramet": [0, 1, 2, 8, 9, 11, 14, 16, 17, 18, 20, 21, 22, 23, 25, 26, 32, 33, 34, 37, 58, 59, 64, 72, 73, 75, 80, 81, 83, 84, 85, 88, 91, 99, 100, 101, 104, 105, 106, 109, 110, 112, 113, 114, 115, 120, 122, 123, 124, 125, 128, 129, 130, 131, 134, 135, 143, 148, 151, 152, 154, 160, 162, 165], "parametr": [134, 143], "parent": [0, 1, 37, 114, 115, 120, 143], "parenthash": 0, "parentid": 1, "pareto": [10, 16, 19], "pari": [47, 52, 53, 54, 55, 97, 140, 142, 147], "pars": [1, 21, 75, 80, 143, 154, 160], "parse_arg": [57, 62, 63], "parse_fil": 143, "parse_obj": 143, "parse_raw": 143, "parser": [26, 57, 62, 63, 71, 75, 80, 93, 143, 154, 160], "part": [1, 10, 11, 14, 18, 29, 33, 38, 83, 92, 93, 94, 99, 100, 103, 113, 114, 115, 126, 127, 128, 129, 134, 135, 142, 143, 148], "part2": 148, "parti": [10, 96, 97, 148], "partial": [0, 8, 10, 12, 33, 35, 56, 100, 105, 113, 122, 143], "particip": [0, 19, 129, 148], "participantid": [0, 98], "particular": [0, 10, 29, 34, 83, 92, 99, 124, 125, 126, 127], "particularli": [8, 10, 12, 14, 15, 16, 19, 35, 75, 81, 94, 127, 154, 166], "partit": [20, 59, 60, 61, 72, 89, 101, 106, 113, 143], "pass": [0, 1, 10, 15, 19, 20, 21, 23, 28, 29, 30, 31, 32, 33, 35, 36, 37, 42, 56, 58, 62, 72, 74, 80, 89, 91, 92, 99, 101, 103, 105, 106, 109, 113, 114, 119, 120, 121, 123, 126, 127, 129, 130, 131, 134, 135, 142, 143, 144, 148, 149, 150, 151, 153, 160, 164, 167], "password": 9, "past": [0, 13, 15, 16, 72, 84, 88, 101], "past_key_valu": [129, 130], "past_key_value_length": 130, "past_key_values_length": 130, "past_kv_length": 134, "past_sequence_length": 134, "patch": [130, 134], "patch_siz": [130, 131], "path": [0, 1, 2, 9, 13, 15, 17, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 32, 34, 36, 37, 38, 42, 47, 52, 53, 54, 55, 56, 59, 60, 61, 62, 64, 65, 69, 72, 74, 80, 85, 90, 92, 94, 97, 99, 101, 108, 109, 112, 114, 119, 120, 121, 123, 129, 134, 140, 142, 143, 147, 148, 153, 156, 160, 162], "path_to_llama_from_hf": 150, "pathlib": [56, 69, 143], "pathlik": 131, "pathorn": 148, "pathsoffset": 1, "pattern": [8, 10, 12, 14, 15, 18, 19, 33, 57, 73, 79, 89, 91, 100, 129, 143, 148, 152, 159], "patternanalyz": 103, "patternrewrit": 103, "paus": [0, 15, 128, 167], "paused_request": 167, "pcie": [15, 22, 107], "pd": [15, 148], "pdf": [0, 100, 106], "pdl": [12, 19, 20, 148], "peak": [0, 2, 3, 4, 8, 12, 38], "peer": 15, "peft": 143, "peft_cache_config": [42, 85, 143, 148, 162], "peftcacheconfig": [0, 85, 143, 162], "peftcachemanag": [0, 148], "penal": [0, 102, 143], "penalti": [8, 91, 148, 165], "penalty_alpha": 102, "pend": [56, 167], "pending_load": 56, "pending_request": 167, "pennsylvania": 29, "peopl": [11, 23], "pep": 10, "per": [0, 1, 2, 3, 5, 6, 8, 10, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 37, 38, 57, 59, 60, 61, 72, 81, 83, 84, 88, 90, 98, 99, 101, 102, 104, 107, 109, 113, 115, 120, 121, 122, 123, 129, 130, 135, 137, 143, 148], "per_channel": 137, "per_group": 137, "per_token": 137, "per_token_scal": 129, "perceiv": 4, "percent": [0, 110], "percentag": [21, 37, 38, 106, 110, 120, 121], "percentil": [25, 28, 29, 30, 31, 37, 120, 148], "perf": [0, 2, 14, 16, 20, 38, 71, 93, 129, 143, 148, 156], "perf_best_practic": 148, "perf_metrics_max_request": 143, "perfect": [8, 15, 16], "perfectli": [11, 15], "perform": [0, 1, 3, 5, 6, 11, 13, 14, 22, 25, 26, 35, 37, 38, 42, 56, 63, 72, 78, 79, 80, 81, 83, 84, 86, 88, 89, 92, 93, 94, 98, 99, 101, 102, 103, 106, 108, 113, 114, 115, 120, 122, 125, 126, 128, 129, 134, 138, 142, 143, 144, 147, 148, 149, 151, 156, 158, 159, 160, 166], "performantli": 3, "perhap": 83, "period": [0, 8, 15, 83, 143], "permiss": 148, "permut": 129, "perplex": [79, 159], "persimmon": 148, "persist": [7, 8, 19, 56], "persistentkvcacheconnectorlead": 56, "persistentkvcacheconnectormetadata": 56, "persistentkvcacheconnectorwork": 56, "person": [58, 62, 116], "perspect": [8, 10, 19, 89], "pertain": [75, 154], "phase": [0, 3, 6, 8, 10, 11, 12, 13, 14, 15, 16, 19, 22, 81, 84, 88, 92, 98, 103, 109, 120, 124, 125, 126, 127, 128, 129, 135, 143, 146, 148, 151, 166], "phi": [75, 79, 90, 129, 136, 137, 138, 145, 148, 154, 159], "phi3config": 131, "phi3forcausallm": [131, 138, 145], "phi3model": 131, "phi4mmforcausallm": [138, 145], "phiconfig": 131, "phiforcausallm": 131, "phimodel": 131, "physic": [19, 129, 135], "pick": 126, "pickl": 148, "pictur": 29, "pie": 18, "piec": [1, 15, 88, 126], "piecewis": [143, 148], "pil": 21, "pin": [0, 1, 105], "ping": 148, "pinnedmemusag": 0, "pinnedpool": 1, "pip": [2, 9, 23, 28, 29, 30, 31, 78, 94, 96, 148, 158], "pip3": [78, 97, 158], "pipefail": 29, "pipelin": [0, 1, 3, 6, 20, 21, 22, 23, 26, 35, 38, 55, 75, 87, 93, 99, 102, 113, 120, 124, 125, 135, 143, 146, 148, 154, 167], "pipeline_parallel_s": [55, 122, 123, 143], "pipelineparallel": [0, 1, 102], "pipelineparallelismrank": 1, "pitfal": [105, 115], "pixart": 130, "pixartalphatextproject": 130, "pixel": 86, "pixel_valu": 131, "pkl5": 148, "pl": [37, 97, 120], "place": [1, 10, 15, 19, 22, 29, 83, 97, 100, 129, 144, 148, 149], "placemen": 15, "placement": [12, 15, 89], "plai": [20, 88, 126], "plan": [12, 15, 16, 19, 78, 94, 99, 101, 143, 148, 158], "planner": 148, "plateau": 88, "platform": [8, 15, 20, 31, 37, 44, 47, 52, 53, 55, 94, 97, 116, 117, 120, 140, 141, 142, 147, 148], "pleas": [2, 3, 5, 6, 7, 12, 14, 15, 16, 17, 20, 26, 28, 29, 30, 31, 37, 38, 42, 44, 51, 72, 75, 81, 83, 84, 86, 94, 95, 98, 101, 103, 107, 109, 111, 117, 120, 122, 124, 125, 129, 139, 140, 141, 142, 143, 148, 154, 167], "plot": [16, 18], "plu": [15, 16, 79, 107, 134, 159], "plug": 20, "plugin": [72, 80, 94, 101, 102, 103, 110, 112, 126, 129, 131, 135, 137, 139, 143, 148, 160], "plugin_config": [123, 127, 129, 131, 143], "plugin_namespac": 103, "plugin_typ": 103, "plugin_v2": 103, "plugin_v2_gemm_0": 139, "pluginconfig": [132, 143], "pluginconfigmeta": 132, "pluginfield": 148, "pluginv2build": 139, "pm": [2, 12, 37, 120], "pmi": 139, "pmi2_init": 139, "pmix": [26, 59, 60, 61, 113, 139], "png": [26, 40, 46, 67], "po": 130, "point": [1, 4, 7, 11, 15, 16, 19, 20, 23, 35, 38, 51, 55, 72, 78, 88, 90, 101, 113, 122, 128, 129, 137, 139, 143, 147, 148, 158], "pointer": [0, 1, 10, 15, 102, 114, 129, 134, 148], "pointerelementtyp": 1, "pointermemorymap": 1, "polar": [138, 145], "polici": [0, 1, 15, 18, 20, 21, 37, 49, 89, 120, 121, 135, 143], "poll": [0, 26], "polyhedr": 113, "pong": 148, "pool": [0, 1, 14, 15, 16, 18, 19, 72, 81, 83, 88, 92, 101, 129, 134, 143, 148, 166, 167], "pooled_project": [130, 131], "pooled_projection_dim": 130, "pooledpin": 0, "poor": 8, "popd": 139, "popfirstgentoken": 0, "popul": [1, 28, 29, 30, 31, 70, 72, 101, 113, 129, 143], "popular": [7, 11, 13, 101, 112, 115, 117, 142, 146], "port": [0, 9, 16, 17, 20, 26, 28, 29, 30, 31, 43, 81, 117, 147], "portfolio": 5, "portion": [89, 100, 122, 129, 135], "pos_emb_typ": 129, "pos_embd_param": [72, 151], "pos_embed_max_s": 130, "pos_embed_typ": 130, "pose": 127, "posit": [0, 1, 12, 13, 57, 109, 120, 129, 130, 134, 143, 148, 151], "position_embed": [129, 130], "position_embedding_typ": [72, 101, 112, 129, 130, 131], "position_encoding_2d": 131, "position_id": [72, 131, 134, 139, 144, 149, 151], "positionalembeddingparam": [72, 151], "positionembeddingtyp": [72, 101, 129, 130, 131], "positionid": [0, 1], "positionidsbas": 1, "positionidsdevic": 1, "positionidshost": 1, "positionidshostcopi": 1, "positionoffset": 1, "positionoffsetsdevic": 1, "positionoffsetshost": 1, "positionoffsetshostcopi": 1, "posix": 0, "posix_debug_fallback": 0, "possibl": [2, 10, 11, 13, 14, 15, 16, 19, 22, 28, 29, 30, 31, 34, 36, 38, 42, 72, 81, 88, 91, 94, 97, 98, 99, 101, 102, 105, 109, 113, 119, 120, 123, 126, 128, 129, 135, 139, 143, 148, 150], "possibli": [1, 104, 129], "post": [0, 4, 7, 8, 9, 10, 12, 13, 14, 15, 16, 19, 35, 36, 38, 81, 82, 112, 119, 129, 145, 147, 148, 161], "post_act_fn": 130, "post_attention_layernorm": [114, 144, 149], "post_input_id": 134, "post_layernorm": [111, 112, 114, 129, 139], "post_merg": 33, "post_pad": 129, "post_proc": 148, "post_prompt": 134, "post_strid": 129, "posterior_threshold": 143, "posterioralpha": 1, "posterioralphahost": 1, "posteriorthreshold": [0, 1], "posteriorthresholdhost": 1, "postproc_param": 143, "postproc_work": 143, "postprocess": [26, 130, 143, 148], "postprocess_tokenizer_dir": 143, "postprocessor": [0, 143], "postprocparam": 143, "postprocwork": 143, "potenti": [0, 1, 8, 10, 11, 14, 15, 18, 19, 22, 36, 37, 83, 104, 109, 119, 120, 123, 144, 149], "pow": 129, "power": [5, 7, 12, 14, 15, 19, 20, 58, 75, 83, 105, 113, 126, 148, 154], "pp": [0, 3, 6, 16, 21, 26, 37, 38, 81, 98, 102, 106, 120, 121, 129, 148], "pp1": 38, "pp2": [16, 38, 81, 120], "pp4": 38, "pp8": 38, "pp_communicate_final_output_id": 134, "pp_communicate_new_token": 134, "pp_reduce_scatt": [22, 127], "pp_size": [21, 23, 26, 38, 43, 112, 113, 120, 122, 133, 148], "ppreducescatt": 1, "pq": 143, "pr": [10, 12, 15, 19, 29], "practic": [4, 5, 12, 14, 15, 16, 73, 92, 93, 101, 104, 113, 135, 148, 152], "practition": [28, 29, 30, 31], "pre": [0, 1, 15, 16, 19, 20, 23, 25, 33, 34, 35, 37, 72, 81, 89, 91, 92, 94, 96, 97, 99, 101, 112, 120, 129, 135, 143, 146, 147, 148, 151], "pre_input_id": 134, "pre_layernorm": 129, "pre_merg": 33, "pre_onli": 130, "pre_pad": 129, "pre_prompt": 134, "pre_quant_scal": [112, 143], "pre_strid": 129, "prebuilt": 94, "preced": [113, 129], "precis": [1, 2, 3, 7, 11, 15, 19, 20, 21, 22, 28, 37, 90, 93, 102, 114, 120, 123, 127, 132, 135, 138, 143, 146, 148], "precompute_relative_attention_bia": 131, "precomputed_relative_attent": 130, "predefin": [72, 109, 144, 149, 151], "predict": [1, 8, 10, 12, 13, 15, 35, 101, 109, 148], "predictor": 109, "predictsdrafttoken": 1, "prefer": [7, 19, 94, 108, 156], "prefer_managed_weight": 130, "prefer_plugin": 129, "preferenti": 83, "prefetch": 12, "prefil": [0, 10, 14, 15, 16, 21, 56, 81, 82, 92, 93, 124, 125, 136, 143, 145, 146, 148, 161], "prefix": [11, 13, 18, 25, 28, 29, 30, 31, 33, 83, 92, 99, 109, 112, 121, 129, 132, 139, 142, 143], "prefix_token_ad": 58, "preliminari": [3, 5, 6, 15], "preload": 114, "prem": 20, "premis": 13, "prepar": [0, 12, 13, 15, 23, 28, 29, 30, 31, 35, 36, 60, 72, 81, 88, 91, 92, 98, 119, 126, 129, 131, 137, 148, 151, 165], "prepare_dataset": [2, 36, 37, 38, 60, 93, 119, 120, 121], "prepare_draft_token": 92, "prepare_input": [131, 135], "prepare_position_ids_for_cogvlm": 134, "prepare_recurrent_input": 131, "prepare_resourc": [92, 150, 166], "prepend": 139, "preprocess": [21, 80, 86, 114, 134, 137, 148, 160], "preprocess_weights_hook": 131, "preprocessed_dataset": 21, "preprocessor": [16, 37, 120], "preqrequisit": 97, "prequant_scaling_factor": 112, "prerequisit": [93, 97], "presenc": [102, 113], "presence_penalti": [134, 143, 148], "presencepenalti": [0, 1, 102], "present": [0, 8, 9, 10, 15, 16, 19, 25, 37, 38, 81, 84, 88, 92, 120, 126, 127, 137, 143, 148], "preserv": 123, "presid": [47, 48, 49, 97, 121, 128, 140, 142, 147], "press": 25, "pressur": [15, 19], "pretrained_config": [80, 144, 149, 160], "pretrained_model_name_or_path": 131, "pretrainedconfig": [111, 115, 131, 132, 143, 144, 149], "pretrainedmodel": [115, 131, 135], "pretrainedtoken": 58, "pretrainedtokenizerbas": 143, "prevdrafttokenslen": 1, "prevent": [8, 10, 11, 12, 14, 20, 21, 28, 29, 30, 31, 83, 84, 142, 148], "preview": 148, "previou": [1, 2, 4, 9, 11, 13, 15, 18, 19, 35, 37, 80, 83, 87, 95, 99, 100, 108, 109, 115, 120, 121, 122, 123, 126, 127, 128, 148], "previous": [1, 3, 18, 19, 35, 83, 88, 92, 108, 123, 126, 128, 148], "previous_batch": [35, 87], "previous_tensors_devic": [35, 87], "prevscor": 1, "price": [37, 120], "priem": 56, "primari": [0, 1, 7, 15, 19, 29, 32, 80, 83, 104, 135, 148, 160, 167], "primarili": [2, 19, 72, 151], "primit": [14, 15, 19, 113], "principl": [8, 15], "print": [1, 10, 11, 21, 33, 38, 42, 47, 48, 49, 51, 52, 53, 54, 55, 56, 57, 58, 62, 63, 64, 65, 66, 67, 68, 69, 70, 72, 97, 101, 120, 121, 128, 135, 139, 140, 142, 143, 147], "print_iter_log": [2, 20, 25, 60, 143], "printabl": 143, "prior": [22, 94, 97, 99], "priorit": [7, 8, 35, 83, 88, 126, 128], "prioriti": [0, 1, 75, 83, 104, 105, 114, 143, 154], "prioritytyp": 0, "priorityupd": 0, "privat": [0, 1, 17, 18, 102, 143], "privileg": 103, "prm": [138, 145], "prmreward": 11, "pro": [12, 18, 38, 146, 148], "prob": [129, 148, 165], "probabilist": 130, "probabl": [0, 1, 10, 12, 13, 20, 64, 91, 102, 105, 109, 129, 134, 143, 148], "probe_answ": 11, "probe_respons": 11, "probe_suffix": 11, "probe_suffix_token_num": 11, "probe_task": 11, "probe_text": 11, "probil": 1, "problem": [2, 8, 10, 11, 14, 19, 72, 89, 101, 139, 148], "problemat": [10, 92], "proc": [19, 114], "proccessed_weight": 114, "proccessed_zero": 114, "proce": [11, 16, 19, 81, 94], "procedur": [2, 92], "proceed": [8, 10, 113], "process": [0, 1, 2, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 21, 22, 23, 25, 28, 29, 30, 31, 34, 35, 36, 37, 38, 51, 55, 56, 57, 58, 59, 60, 61, 72, 78, 79, 80, 81, 84, 86, 87, 88, 89, 91, 98, 99, 101, 102, 104, 107, 109, 112, 113, 115, 119, 120, 121, 122, 126, 127, 128, 129, 134, 139, 142, 143, 144, 146, 147, 148, 149, 150, 151, 158, 159, 160, 167], "process_input": 134, "process_logits_including_draft": 134, "processor": [0, 16, 50, 72, 82, 86, 93, 101, 131, 134, 143, 145, 148, 161], "processorbatch": 0, "processormap": 0, "prod": 129, "produc": [0, 1, 10, 11, 20, 37, 42, 88, 91, 92, 99, 103, 113, 120, 121, 123, 126, 127, 129, 148], "product": [5, 8, 9, 15, 16, 18, 19, 20, 23, 56, 58, 72, 73, 75, 79, 81, 88, 89, 100, 101, 109, 113, 126, 127, 128, 129, 151, 152, 154, 159], "profession": 17, "profil": [19, 22, 45, 46, 93, 123, 126, 129, 134, 135, 139, 143, 148], "profiling_verbos": [22, 143], "profit": [37, 109, 120], "program": [10, 11, 19, 47, 52, 53, 55, 56, 97, 115, 128, 139, 140, 142, 147], "progress": [1, 8, 11, 12, 37, 78, 120, 129, 143, 158], "proj": [112, 114, 139], "project": [10, 14, 19, 72, 78, 94, 96, 97, 101, 106, 129, 130, 144, 149, 158, 166], "projector_hidden_act": 131, "prologu": [59, 60, 61], "promin": 109, "promis": [13, 16, 109, 115], "promot": 148, "prompt": [0, 2, 10, 11, 16, 18, 21, 22, 23, 25, 26, 28, 29, 30, 31, 32, 35, 37, 41, 42, 47, 48, 49, 51, 52, 53, 54, 55, 57, 58, 59, 62, 63, 64, 65, 68, 69, 72, 74, 75, 81, 83, 85, 88, 91, 92, 97, 99, 102, 105, 109, 111, 120, 121, 126, 127, 128, 130, 134, 140, 142, 143, 147, 148, 151, 153, 154, 162, 165], "prompt_a": 57, "prompt_adapter_request": [143, 148], "prompt_b": 57, "prompt_embedding_t": [130, 131, 134], "prompt_embedding_table_s": 131, "prompt_id": 58, "prompt_len": [72, 151], "prompt_logprob": [143, 148], "prompt_lookup": 148, "prompt_lookup_num_token": 102, "prompt_tabl": 134, "prompt_task": [131, 134], "prompt_token": [17, 20, 28, 29, 30, 31, 147], "prompt_token_id": [20, 28, 29, 30, 31, 42, 143], "prompt_vocab_s": [131, 134], "promptadapterrequest": 143, "promptinput": [143, 148], "promptlen": 0, "promptli": [10, 16, 81], "prompttableoffload": 0, "prompttuningconfig": 0, "prompttuningembed": 130, "prompttuningen": 1, "pronounc": [8, 15, 19, 109], "proof": 166, "prop": 1, "propag": [1, 105, 148], "propel": 8, "proper": [19, 21, 32, 37, 75, 98, 120, 154], "properli": [10, 15, 17, 19, 20, 28, 29, 30, 31, 32, 97, 114, 126, 128], "properti": [18, 34, 48, 51, 70, 91, 99, 129, 131, 132, 134, 143], "proport": [72, 101], "proportion": 18, "propos": [0, 8, 11, 12, 18, 92, 93], "proposer_task": 11, "proprietari": [80, 160], "protect": [1, 32, 55, 142, 148], "proto": 143, "protocol": [0, 10, 16, 19, 26, 51, 81], "prototyp": [19, 32, 93, 143, 158], "prototype_control": 11, "proud": [8, 12, 15, 16], "prove": [14, 109], "prover": [79, 159], "provid": [0, 1, 2, 3, 4, 7, 8, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 42, 48, 49, 51, 58, 62, 72, 73, 75, 77, 78, 79, 80, 81, 83, 84, 85, 88, 93, 94, 98, 99, 100, 101, 102, 103, 104, 105, 106, 108, 109, 111, 112, 115, 116, 119, 120, 121, 122, 123, 124, 125, 126, 127, 129, 134, 135, 138, 139, 142, 143, 144, 146, 147, 148, 149, 150, 151, 152, 154, 157, 158, 159, 160, 162], "provinc": 29, "proxy_dispatch_result_thread": 120, "prune": [10, 103, 109, 129], "pseudo": [72, 94, 101, 129, 137], "pt": [21, 56], "pth": [80, 114, 148, 160], "ptq": [7, 123, 148], "ptr": 1, "ptr_idx": 114, "ptrdiff_t": 1, "ptune": [33, 148], "ptuning_setup": 134, "ptuning_setup_fuyu": 134, "ptuning_setup_llava_next": 134, "ptuning_setup_phi3": 134, "ptuning_setup_pixtr": 134, "ptuningconfig": 0, "public": [0, 1, 7, 32, 117, 148], "publish": [2, 3, 6, 28, 29, 30, 31, 37, 38, 95, 120, 148], "pull": [2, 9, 17, 20, 33, 94, 148], "pull_pipe_addr": 143, "punctuat": 143, "puneeshkhanna": 148, "purchas": [37, 120], "pure": [20, 134], "purpos": [1, 8, 14, 15, 16, 25, 32, 72, 88, 92, 94, 101, 104, 121, 123, 126, 127], "pursu": [47, 52, 53, 55, 97, 140, 142, 147], "push": [14, 15, 93, 116], "push_pipe_addr": 143, "pushd": 139, "put": [1, 10, 12, 19, 29, 59, 60, 61, 88, 97, 112, 126, 142], "put_zcopi": [81, 98], "pwd": [2, 94], "py": [2, 12, 13, 14, 15, 25, 28, 29, 30, 31, 32, 34, 36, 37, 38, 48, 49, 56, 57, 59, 60, 64, 74, 78, 86, 92, 93, 94, 99, 100, 101, 103, 106, 108, 109, 110, 111, 112, 113, 114, 115, 119, 120, 121, 122, 123, 129, 132, 134, 139, 142, 144, 148, 149, 150, 153, 158, 166, 167], "py3": 148, "py_draft_token": 92, "py_executor_cr": 167, "py_rewind_len": 92, "pybind": [143, 148], "pybind_class": 143, "pybind_equ": 143, "pybind_inst": 143, "pybindmirror": 143, "pydant": [32, 75, 143, 148, 154], "pydantic_cor": 143, "pydanticserializationerror": 143, "pydanticundefin": 143, "pyexecutor": [32, 35, 56, 87, 92, 148, 166, 167], "pynvml": 148, "pypi": [94, 96, 148], "pytest": [32, 33], "python": [1, 2, 11, 13, 14, 16, 19, 21, 25, 26, 28, 29, 30, 31, 32, 33, 35, 36, 37, 38, 42, 53, 54, 57, 64, 74, 75, 78, 79, 85, 97, 101, 102, 103, 106, 109, 111, 113, 115, 119, 120, 121, 122, 137, 142, 143, 144, 146, 147, 148, 149, 150, 153, 154, 158, 159, 162, 166, 167], "python3": [2, 28, 29, 30, 31, 36, 37, 59, 60, 86, 94, 106, 108, 110, 112, 119, 120, 139], "python_bind": 2, "python_e2": 134, "python_plugin": 148, "pythonpath": [2, 60, 61], "pytorch": [2, 10, 11, 13, 14, 15, 20, 21, 23, 26, 33, 35, 38, 48, 49, 50, 56, 64, 72, 73, 79, 80, 81, 85, 86, 90, 91, 92, 93, 94, 97, 103, 109, 112, 129, 142, 143, 144, 148, 150, 151, 152, 159, 160, 162, 164, 165, 166, 167], "pytorch_cuda_alloc_conf": 28, "pytorch_extra_arg": 60, "pytorch_model": 139, "pytorch_model_registri": 166, "pytorchconfig": [72, 143, 148, 151], "pytorchmodelengin": [150, 166], "pzzzzz5142": 148, "q": [3, 9, 12, 14, 21, 37, 72, 81, 98, 101, 102, 106, 120, 129, 139, 144, 149, 151], "q_b_proj": 129, "q_dim": 129, "q_lora_rank": [129, 130], "q_proj": [37, 80, 85, 114, 120, 144, 149, 160, 162], "q_scale": [72, 101, 129, 130, 131], "qa": [23, 28, 29, 30, 31, 109], "qformat": [120, 133], "qgmma": 148, "qingquansong": 148, "qk_layernorm": [130, 131], "qk_nope_head_dim": [129, 130], "qk_norm": 130, "qk_rope_head_dim": [129, 130], "qkv": [103, 106, 112, 114, 129, 139, 148, 151], "qkv_bia": [129, 148], "qkv_dim": 129, "qkv_proj": [80, 144, 149, 160], "qo_indptr": [72, 151], "qpi": 107, "qserv": 148, "quadrat": [72, 101, 135], "qualifi": 33, "qualiti": [13, 15, 20, 91, 123, 127], "qualnam": [129, 131, 133, 143], "quant": [37, 90, 115, 120, 129, 143, 148, 164], "quant_algo": [37, 42, 85, 112, 114, 115, 120, 123, 131, 143, 162], "quant_config": [42, 72, 85, 115, 123, 131, 143, 151, 162], "quant_medusa_head": 133, "quant_mod": [115, 130, 131, 134, 143], "quantalgo": [42, 85, 93, 123, 131, 133, 143, 162], "quantconfig": [42, 72, 85, 93, 115, 123, 131, 143, 148, 151, 162], "quanticonfig": 115, "quantifi": [8, 72], "quantiz": [2, 3, 4, 12, 14, 19, 20, 21, 22, 28, 30, 31, 38, 47, 52, 72, 75, 78, 79, 93, 97, 101, 102, 107, 113, 114, 118, 121, 124, 125, 129, 130, 131, 134, 138, 140, 141, 142, 143, 144, 146, 147, 148, 149, 151, 154, 158, 159], "quantizaton": 120, "quantize_and_export": 133, "quantize_kwarg": 131, "quantize_lm_head": [133, 148], "quantized_valu": [72, 101], "quantizedkernel": 113, "quantizetensorplugin": 113, "quantmod": [1, 72, 101, 102, 129, 130, 131, 133, 134, 143], "quantmodewrapp": [129, 143], "quebec": 29, "queen": 29, "queri": [3, 14, 16, 17, 20, 26, 28, 29, 30, 31, 37, 63, 74, 81, 83, 93, 99, 102, 104, 109, 113, 120, 129, 135, 148, 151, 153, 166], "query_dim": 130, "query_key_valu": 114, "query_length": 130, "query_pre_attn_scalar": 131, "question": [11, 15, 18, 29, 37, 62, 120, 135, 139], "queu": [0, 38, 88, 126], "queue": [0, 10, 26, 33, 35, 143, 150], "quick": [27, 48, 49, 78, 101, 120, 121, 146, 151, 158], "quick_gelu": 129, "quicker": 122, "quickli": [19, 86, 115, 147], "quickstart": [48, 49, 121, 142], "quickstart_advanc": [13, 59], "quickstart_multimod": 86, "quit": [25, 103], "quot": [33, 148], "qweight": 114, "qwen": [11, 26, 37, 46, 57, 79, 86, 90, 114, 120, 129, 131, 137, 138, 145, 148, 159], "qwen1": [138, 148], "qwen2": [25, 26, 37, 40, 46, 67, 79, 86, 106, 120, 136, 138, 145, 148, 159], "qwen2_5_vlforconditionalgener": [138, 145], "qwen2_audio_7b_instruct": 33, "qwen2audio": 148, "qwen2forcausallm": [138, 145], "qwen2forprocessrewardmodel": [138, 145], "qwen2forrewardmodel": [138, 145], "qwen2forsequenceclassif": 148, "qwen2vl": 148, "qwen2vlforconditionalgener": [138, 145], "qwen3": [15, 57, 79, 89, 138, 145, 148, 159], "qwen3forcausallm": [138, 145], "qwen3mo": [138, 145], "qwen3moeforcausallm": [138, 145], "qwenforcausallm": [114, 131], "qwenforcausallmgenerationsess": 134, "qwenvl": 148, "qwq": [79, 138, 145, 159], "qychen": 106, "qzero": 114, "r": [1, 23, 26, 47, 48, 49, 51, 52, 53, 54, 55, 57, 58, 62, 97, 106, 121, 128, 129, 139, 140, 142, 143, 147, 148], "r1": [10, 11, 15, 19, 26, 27, 65, 71, 79, 89, 90, 92, 93, 146, 148, 159], "r1_in_tensorrt": [12, 148], "race": 148, "radix": [83, 166], "rais": [11, 58, 65, 115, 121, 139, 143, 148], "raise_error": 143, "ram": 57, "rand": [21, 37, 120, 129], "rand_data": 129, "rand_data_valid": 131, "randint": 10, "random": [0, 18, 21, 23, 25, 28, 29, 30, 31, 38, 45, 46, 64, 102, 129, 143, 148], "random_imag": 25, "random_se": [23, 131, 134, 143], "randomdatasampl": 1, "randomdatavalid": 1, "randomli": [37, 38, 120], "randomse": [1, 102, 148], "randomseedtyp": 0, "rang": [0, 1, 10, 11, 16, 17, 21, 36, 37, 56, 58, 78, 83, 102, 105, 109, 119, 120, 127, 129, 131, 135, 137, 138, 139, 142, 143, 144, 146, 149, 158], "rank": [0, 1, 2, 8, 14, 15, 19, 20, 22, 35, 37, 78, 81, 89, 93, 98, 99, 100, 102, 106, 115, 120, 129, 131, 134, 135, 139, 143, 148, 158], "rank0": 112, "rank1": 112, "rapid": [38, 109], "rapidli": 17, "rate": [0, 2, 10, 12, 13, 14, 16, 21, 25, 28, 29, 30, 31, 35, 37, 38, 45, 46, 57, 92, 120, 121, 148], "rather": [14, 15, 17, 72, 97, 101, 103, 109], "ratio": [14, 15, 16, 18, 83, 143], "ration": [16, 81], "rational": 14, "raw": [20, 26, 35, 80, 86, 160], "raw_audio": 134, "raw_imag": 134, "rc": [28, 29, 30, 31], "rcn": [28, 29, 30, 31], "rdma": [16, 81, 98], "re": [2, 7, 11, 15, 17, 28, 29, 30, 31, 72, 88, 92, 143, 146, 148, 151], "reach": [0, 11, 16, 19, 35, 72, 88, 101, 112, 120, 123, 128, 142, 143], "reachabl": 34, "react": 15, "read": [0, 2, 10, 12, 13, 15, 19, 22, 37, 62, 72, 84, 98, 99, 101, 109, 111, 113, 114, 120, 143, 148], "read_config_from_the_custom_training_checkpoint": 115, "readabl": [35, 37, 120], "reader": 129, "readi": [0, 8, 9, 10, 17, 20, 28, 29, 30, 31, 35, 78, 86, 94, 156, 158], "readm": [109, 121, 140, 142, 148], "real": [2, 8, 12, 15, 18, 19, 20, 21, 84, 89, 92, 94, 103, 108, 121, 123, 126, 127, 129, 139], "realist": [88, 156], "realiti": 126, "realiz": [105, 109], "rearrang": 129, "reason": [0, 8, 10, 11, 12, 13, 15, 17, 26, 29, 33, 35, 37, 71, 72, 79, 88, 92, 93, 101, 102, 113, 115, 120, 122, 126, 127, 129, 139, 143, 159], "reasoning_cont": [20, 29], "reasoning_pars": [26, 43, 143], "reassess": 11, "rebalanc": 15, "rebuild": [34, 88, 127, 129, 139, 143], "receiv": [0, 1, 15, 16, 19, 25, 28, 29, 30, 31, 81, 89, 98, 99, 100, 107, 109, 123, 129, 143, 148], "recent": [1, 4, 8, 11, 12, 34, 72, 83, 101, 108], "recept": [16, 81], "recip": [12, 14, 90, 93, 137, 143], "reclaim": 0, "recogn": [12, 15, 16, 37, 109, 120, 144, 149], "recommend": [2, 4, 7, 10, 14, 15, 16, 20, 23, 25, 26, 28, 29, 30, 31, 32, 37, 58, 72, 81, 88, 94, 98, 101, 102, 109, 111, 114, 120, 124, 125, 126, 128, 139, 143, 144, 148, 149, 151, 156], "recomput": 57, "recompute_scale_factor": 129, "reconfigur": [97, 99], "reconstruct": [72, 101, 129], "record": [1, 2, 10, 12, 13, 15, 37, 103, 143, 146], "record_cr": 143, "record_stat": 143, "recored": 0, "recov": [57, 143], "rectangular": 29, "recur": 57, "recurr": [10, 109], "recurrentgemma": [137, 138, 148], "recurrentgemmaforcausallm": 131, "recurs": [2, 17, 75, 94, 142, 154], "recv": [0, 15, 113, 129], "recvconnect": 0, "recvpollperiodm": 0, "recycl": [72, 88, 101, 166], "redesign": 148, "redirect": [103, 143], "redistribut": [15, 19, 89], "redownload": [20, 28, 29, 30, 31], "redraft": [129, 131, 134, 148], "redrafter_draft_len_per_beam": 134, "redrafter_num_beam": 134, "redrafterforllamalm": 131, "redrafterforqwenlm": 131, "redraftermixin": 131, "reduc": [2, 3, 6, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 22, 28, 29, 30, 31, 33, 35, 36, 37, 38, 58, 63, 72, 81, 83, 84, 87, 88, 89, 90, 92, 94, 98, 99, 100, 101, 105, 107, 109, 113, 119, 120, 121, 122, 126, 128, 129, 135, 139, 142, 143, 146, 148, 151, 163], "reduce_fus": [22, 120, 123, 127, 148], "reduce_scatt": [19, 129], "reduceoper": 129, "reducescatt": [19, 22, 127, 148], "reduct": [8, 11, 12, 107, 109, 128, 129, 148], "redund": [8, 12, 15, 83, 109, 148], "redux": 148, "ref_templ": 143, "refactor": [2, 92, 115, 148], "refer": [0, 1, 2, 9, 15, 16, 17, 19, 20, 25, 26, 28, 29, 30, 31, 32, 37, 38, 39, 40, 41, 42, 43, 45, 46, 47, 58, 66, 67, 68, 69, 70, 72, 74, 75, 78, 81, 86, 94, 95, 98, 99, 101, 102, 103, 104, 106, 109, 113, 115, 117, 120, 121, 122, 123, 124, 125, 127, 129, 138, 140, 142, 144, 146, 147, 148, 149, 151, 153, 154, 158], "referenc": 123, "reference_wrapp": [0, 99], "references_commit": 32, "refin": [19, 148], "refit": [22, 113, 143, 148], "refit_engin": 113, "reflect": [8, 19, 81, 98, 126], "refresh": [2, 120], "regard": [78, 129, 158], "regardless": [11, 19, 80, 139, 160], "regex": [23, 91, 99, 143], "region": [11, 29, 36, 119], "regist": [15, 19, 56, 80, 116, 139, 143, 144, 148, 149, 160], "register_auto_model": [144, 149], "register_checkpoint_load": [80, 160], "register_checkpoint_weight_load": [80, 160], "register_config_load": [80, 160], "register_error": 143, "register_kv_cach": 56, "register_mapp": [80, 160], "register_network_output": 139, "registerdesc": 0, "registermemori": 0, "regress": [28, 30, 92, 101, 102, 113, 148], "regular": [0, 12, 91, 99, 101, 129, 143], "regularli": 11, "reinforc": [124, 125], "reject": [0, 13, 92, 93], "rel": [3, 15, 72, 88, 105, 126, 128, 129, 148], "rel_attn_t": 130, "relat": [18, 29, 36, 78, 81, 95, 98, 100, 104, 114, 119, 129, 132, 135, 139, 141, 142, 144, 148, 149, 158, 166], "relationship": [8, 135], "relative_attent": [129, 130], "relative_attention_bia": 129, "relax": [8, 15, 88, 92, 93, 101], "relaxed_delta": [12, 13, 65, 92, 143], "relaxed_topk": [12, 13, 65, 92, 143], "releas": [1, 3, 6, 7, 9, 10, 15, 16, 19, 20, 25, 28, 29, 30, 31, 32, 72, 80, 81, 83, 93, 94, 96, 101, 102, 104, 115, 129, 131, 135, 137, 138, 143, 147, 160], "release_build": [17, 94], "release_run": 94, "releasepag": 1, "releasest": 0, "releasewithtag": 1, "relev": [34, 80, 94, 102, 166], "reli": [15, 16, 19, 36, 79, 84, 98, 101, 103, 115, 119, 137, 142, 159], "reliabl": [78, 158], "reload": [15, 99], "relu": [112, 113, 129, 139], "remain": [0, 1, 8, 12, 15, 19, 32, 33, 38, 75, 83, 88, 94, 103, 105, 109, 110, 121, 123, 126, 127, 129, 135, 143, 148, 154], "remaind": [56, 83, 123], "remaining_chunk": 56, "remaining_token": 56, "remark": [12, 13, 19], "remateri": 1, "remedi": 10, "rememb": 15, "remind": [101, 151], "remot": [1, 15, 21, 143], "remotenam": 0, "remov": [0, 1, 2, 10, 13, 16, 20, 22, 23, 26, 32, 33, 72, 81, 83, 88, 92, 94, 101, 102, 103, 104, 113, 114, 123, 129, 135, 143, 144, 148, 149], "remove_const_t": 1, "remove_cv_t": 0, "remove_duplicated_kv_head": 131, "remove_input_pad": [22, 88, 101, 106, 129, 130, 134], "remove_pointer_t": 1, "remove_reference_t": 1, "remove_sequ": 166, "removeprefix": 143, "removesuffix": 143, "renam": 148, "rendezv": [81, 98], "reopen": 34, "reorder": [129, 130], "reorder_kv_cache_for_beam_search": 134, "rep": [36, 119], "repeat": [0, 13, 14, 63, 83, 84, 101, 129, 143], "repeat_interleav": 129, "repeatedli": 109, "repetit": [0, 18, 58, 102, 129, 143], "repetition_penalti": [102, 134, 143, 148], "repetitionpenalti": [0, 1, 102], "replac": [1, 2, 9, 11, 14, 17, 19, 28, 30, 31, 37, 57, 88, 89, 103, 113, 114, 115, 120, 121, 123, 128, 129, 135, 143, 144, 149], "replace_add_with_sub": 103, "replace_all_uses_with": [103, 129], "replace_input_with": 103, "replace_output_uses_with": 103, "replace_outputs_uses_with": 103, "replai": [10, 15], "replic": [0, 12, 15, 99, 129], "replica": 89, "replit": [137, 138, 148], "repo": [9, 20, 115, 121, 139, 148], "repo_id": 62, "report": [13, 14, 15, 17, 21, 36, 37, 38, 104, 119, 120, 135, 148], "report_json": [20, 21], "report_load_statist": 15, "reportpluginerror": 139, "repositori": [2, 9, 23, 34, 92, 94, 109, 116, 142, 147], "repr": 143, "repres": [0, 1, 2, 3, 7, 8, 11, 12, 15, 18, 19, 49, 51, 62, 88, 89, 104, 108, 109, 120, 126, 129, 134, 143, 167], "represent": [35, 80, 103, 113, 143, 160], "reproduc": [10, 19, 21, 37, 93, 120, 148], "req": [2, 25, 28, 29, 30, 31, 37, 38, 56, 92, 120, 121, 123, 126, 127], "req_id": [58, 91], "req_stat": 167, "reqbeamwidth": 1, "reqid": 0, "reqpromptlength": 1, "request": [0, 1, 2, 4, 6, 8, 10, 11, 13, 14, 15, 16, 18, 19, 21, 22, 23, 25, 26, 28, 29, 30, 31, 33, 35, 36, 37, 38, 45, 46, 49, 56, 57, 60, 72, 73, 81, 84, 85, 86, 89, 92, 93, 98, 101, 102, 105, 106, 113, 119, 120, 121, 123, 126, 127, 128, 129, 135, 143, 146, 147, 148, 150, 151, 152, 156, 162, 166, 167], "request_finish": 56, "request_id": [42, 56, 72, 143, 151], "request_json": 21, "request_perf_metr": [143, 148], "request_stats_max_iter": 143, "request_timeout": 26, "request_typ": 143, "request_type_context_and_gener": 0, "request_type_context_onli": 0, "request_type_generation_onli": 0, "requesterror": [93, 143], "requestid": [0, 98, 99], "requestidtyp": 0, "requestlist": 167, "requestoutput": [42, 93, 143, 148], "requestperfmetr": [0, 143], "requestschedul": 167, "requeststag": 0, "requeststat": 0, "requeststatsmaxiter": 0, "requeststatsperit": 0, "requeststatsperiter": 0, "requeststatsvec": 0, "requesttoken": 99, "requesttyp": [0, 1, 143], "requesttypesdevic": 1, "requestvector": 1, "requir": [0, 2, 3, 7, 8, 9, 10, 11, 12, 14, 15, 16, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 32, 33, 34, 37, 38, 51, 62, 70, 72, 73, 78, 80, 81, 83, 84, 88, 92, 94, 97, 98, 101, 102, 105, 106, 109, 113, 114, 115, 120, 121, 122, 123, 127, 129, 130, 135, 138, 139, 142, 143, 148, 152, 158, 160, 166], "require_ln_f": 131, "requiresattentionmask": 1, "rerun": [20, 28, 29, 30, 31, 127], "rescale_output_factor": 130, "research": [13, 15, 30, 31, 47, 52, 53, 55, 97, 101, 117, 137, 140, 142, 147], "reserv": [0, 1, 19, 23, 26, 28, 29, 30, 31, 128, 134, 135, 143, 167], "reserved_block": 167, "reset": [0, 1, 37, 102, 120, 134, 143, 148], "resetspeculativedecodingmodul": 1, "reshap": [1, 129], "reshapebuff": 1, "reshapecacheindirectionbuff": 1, "reshapespeculativedecodingbuff": 1, "resid": [15, 89, 106], "residu": [8, 92, 129, 139], "residual_connect": 130, "residual_mlp": 131, "residual_multipli": 131, "residual_rms_norm": 129, "residual_rms_norm_out_quant_fp8": 129, "residual_rms_norm_out_quant_nvfp4": 129, "residual_rms_norm_quant_fp8": 129, "residual_rms_norm_quant_nvfp4": 129, "residual_rms_prepost_norm": 129, "residualadd": [22, 127, 148], "resiz": 1, "resolv": [10, 16, 19, 40, 67, 75, 81, 139, 143, 154], "resourc": [0, 10, 12, 14, 16, 19, 20, 30, 33, 35, 44, 72, 80, 81, 88, 92, 98, 101, 115, 146, 147, 150, 160, 166, 167], "resource_manag": [92, 143], "resourcemanag": 92, "respect": [10, 16, 17, 20, 34, 42, 128, 129, 134, 135, 137, 143, 144, 149, 167], "respond": 19, "respons": [0, 8, 10, 11, 16, 19, 20, 25, 26, 28, 29, 30, 31, 35, 37, 42, 64, 65, 66, 67, 68, 69, 70, 80, 81, 85, 87, 91, 98, 104, 120, 129, 143, 150, 160, 162, 163], "response_format": 70, "response_json": 70, "responsepostprocesswork": 143, "responsewithid": 0, "responsewrapp": 143, "rest": [1, 16, 72, 81, 92, 101, 123, 143], "restart": 0, "restor": 1, "restoremod": 1, "restrict": [0, 33, 94, 99, 102, 129, 143, 165], "result": [0, 1, 3, 4, 5, 7, 10, 11, 13, 14, 15, 16, 18, 20, 21, 22, 25, 26, 28, 29, 30, 31, 33, 35, 37, 42, 72, 81, 84, 87, 89, 91, 94, 100, 101, 107, 109, 113, 120, 122, 123, 124, 125, 126, 127, 129, 130, 143, 144, 148, 149, 151, 165, 167], "result_dir": [25, 28, 29, 30, 31], "result_handl": 143, "retail": [37, 120], "retain": [3, 5, 13, 20, 92], "retent": [0, 49, 143], "retentionprior": 0, "retentionpriorityanddur": 0, "rethink": 109, "rethrown": 1, "retri": 33, "retriev": [1, 11, 16, 26, 35, 38, 81, 114, 129, 143], "retrievebadhandl": 1, "return": [0, 1, 9, 11, 16, 17, 19, 20, 28, 29, 30, 31, 32, 35, 42, 56, 57, 58, 64, 80, 81, 88, 91, 92, 99, 103, 106, 109, 111, 113, 114, 115, 120, 126, 129, 130, 131, 134, 135, 139, 143, 148, 160, 166, 167], "return_all_generated_token": 134, "return_context_logit": 143, "return_dict": 134, "return_encoder_output": [134, 143], "return_generation_logit": 143, "return_perf_metr": 143, "returnallgeneratedtoken": [0, 99], "returncontextlogit": 0, "returnencoderoutput": 0, "returngenerationlogit": 0, "returnlogprob": 0, "returnperfmetr": 0, "reus": [0, 10, 11, 13, 19, 22, 23, 33, 34, 48, 49, 57, 81, 82, 86, 92, 93, 98, 99, 104, 118, 129, 134, 135, 136, 143, 144, 145, 146, 148, 149, 156, 161, 166], "reusabl": [11, 15, 83, 104, 105], "reusedblock": [0, 26], "reusedblocksperrequest": 0, "reveal": [8, 11, 12, 14, 19], "revers": [1, 129], "revert": [83, 129], "review": [15, 37, 73, 96, 97, 120, 152], "revis": 143, "revisit": 93, "reward": 11, "reward_control": 11, "reward_kwarg": 11, "rewind": [13, 148], "rewrit": [78, 129, 144, 148, 149, 158], "rewritepatternmanag": 103, "rewrt": 139, "rf": 139, "rfind": [56, 143], "rg_lru": 129, "rgc": [37, 120], "rh": [0, 1], "rich": 112, "right": [11, 16, 18, 19, 20, 123, 129, 139, 143], "rigor": [37, 120], "rindex": 143, "risk": [10, 75, 81, 98, 113, 123, 128, 154], "river": 29, "rjust": 143, "rm": [9, 20, 25, 28, 29, 30, 31, 94, 129, 138, 139, 144, 145, 147, 149], "rms_norm": [12, 129, 144, 149], "rmsnorm": [12, 106, 129, 130, 131, 144, 148, 149], "rndv": [81, 98], "rnn": [22, 148], "rnn_conv_dim_s": 134, "rnn_head_siz": 134, "rnn_hidden_s": 134, "rnn_state": 131, "rnnconfig": 1, "rnnconvdims": 1, "rnnheadsiz": 1, "rnnhiddens": 1, "ro": [2, 34], "roadmap": 93, "roberta": [138, 148], "robertaforquestionansw": 131, "robertaforsequenceclassif": 131, "robertamodel": 131, "robin": 16, "robot": [20, 64], "robust": [12, 15, 79, 148, 159], "rock": 129, "roi": 58, "role": [9, 16, 17, 20, 23, 26, 29, 39, 40, 51, 66, 67, 70, 81, 88, 113, 126, 147, 156], "roll": [1, 10], "rollback": 10, "rooflin": 14, "room": 19, "root": [2, 19, 20, 28, 29, 30, 31, 94, 112, 116, 121, 129, 142, 143, 148], "root_lay": 103, "rootless": 34, "rope": [12, 14, 129, 134, 143, 148, 151], "rope_gpt_neox": [72, 101, 129, 131], "rope_gptj": [72, 101, 129], "rope_local_base_freq": 131, "rope_scaling_config": 129, "rope_scaling_long_factor": 130, "rope_scaling_long_mscal": 130, "rope_scaling_short_factor": 130, "rope_scaling_short_mscal": 130, "ropeembeddingutil": 129, "rotari": [0, 12, 129, 134, 143, 144, 149, 151], "rotary_bas": 131, "rotary_cos_sin": 129, "rotary_dim": 131, "rotary_embed": [144, 149], "rotary_embedding_bas": [129, 130], "rotary_embedding_base_loc": 130, "rotary_embedding_beta_fast": 130, "rotary_embedding_beta_slow": 130, "rotary_embedding_dim": [72, 101, 129, 131], "rotary_embedding_long_m_scal": 129, "rotary_embedding_max_posit": 129, "rotary_embedding_mscal": 130, "rotary_embedding_mscale_all_dim": 130, "rotary_embedding_origin_max_posit": 130, "rotary_embedding_original_max_posit": 129, "rotary_embedding_percentag": 130, "rotary_embedding_sc": 130, "rotary_embedding_scal": 129, "rotary_embedding_scale_typ": 129, "rotary_embedding_short_m_scal": 129, "rotary_inv_freq": [129, 130], "rotary_inv_freq_loc": 130, "rotary_pct": 131, "rotary_sc": [130, 131], "rotaryembed": [144, 149], "rotaryembeddingdim": [0, 1], "rotaryscalingtyp": 129, "rotat": 148, "rotate_every_two": 129, "rotate_half": 129, "roug": 23, "rouge_path": 23, "roughli": [18, 29], "round": [11, 16, 18, 57, 129, 143], "round_robin": 16, "round_trip": 143, "rout": [14, 15, 16, 19, 81, 89, 98, 148], "router": [14, 15, 16, 81, 89, 100, 106, 148], "router_gemm": 12, "routin": [15, 92, 103], "routingkerneltopk": 12, "row": [10, 88, 106, 126, 129, 137, 148], "rowlinear": [106, 130], "rowwis": [90, 143, 148], "rpartit": 143, "rr": 148, "rslora": 148, "rsp": 143, "rsplit": 143, "rst": 99, "rstrip": 143, "rt": 20, "rtx": [38, 146, 148], "rubric": 129, "rule": [75, 101, 122, 139, 154], "run": [0, 1, 3, 7, 10, 11, 12, 14, 16, 17, 18, 19, 21, 22, 23, 24, 26, 33, 34, 48, 50, 53, 54, 56, 57, 63, 64, 72, 78, 79, 81, 87, 88, 89, 92, 93, 94, 96, 97, 98, 99, 101, 102, 105, 108, 109, 111, 112, 113, 116, 117, 122, 123, 126, 127, 128, 129, 134, 135, 137, 139, 143, 144, 148, 149, 150, 151, 156, 158, 159, 163, 166], "run_all_demonstr": 64, "run_cmd": 94, "run_dtm_ngram": 109, "run_eagle3": 65, "run_mtp": 65, "run_ngram": 65, "run_sqsh": 94, "run_task": 11, "runner": [0, 112, 134], "runningleon": 148, "runpod": 116, "runtim": [0, 12, 13, 15, 16, 18, 19, 20, 21, 22, 26, 32, 36, 37, 50, 58, 62, 72, 73, 74, 75, 78, 80, 88, 93, 99, 101, 109, 110, 119, 120, 121, 124, 126, 129, 130, 131, 139, 142, 143, 144, 146, 148, 149, 151, 152, 153, 154, 156, 158, 160, 167], "runtime_config": 42, "runtime_default": 131, "runtime_error": 1, "runtime_rank": 134, "runtimedefault": [0, 131], "runtimedefaultsin": 131, "runtimeerror": [139, 143], "runtimetensor": 134, "rw": [9, 20, 28, 29, 30, 31], "s0": [72, 88, 101], "s1": [72, 88, 101], "s2": [72, 88, 101], "sacrif": 12, "sad": 134, "saeyoonoh": 148, "safe": [1, 14, 19, 103, 127], "safer": 129, "safetensor": [80, 112, 114, 139, 148, 160], "sage_attn": 129, "sage_attn_k_block_s": 129, "sage_attn_k_quant_s": 129, "sage_attn_q_block_s": 129, "sage_attn_q_quant_s": 129, "sage_attn_v_block_s": 129, "sage_attn_v_quant_s": 129, "sageattent": 129, "sai": [15, 36, 88, 119, 121, 126], "said": 123, "sake": [19, 88, 126], "sale": [37, 58, 120], "salloc": 94, "salt": [0, 143], "same": [0, 1, 4, 8, 10, 11, 13, 14, 15, 16, 19, 20, 21, 22, 25, 28, 29, 30, 31, 33, 37, 38, 49, 59, 60, 61, 64, 72, 81, 83, 84, 89, 92, 93, 94, 98, 99, 101, 102, 103, 104, 105, 106, 107, 109, 110, 113, 115, 120, 123, 127, 128, 129, 130, 132, 134, 135, 142, 143, 148], "sampl": [0, 1, 2, 10, 12, 13, 18, 21, 23, 28, 29, 30, 31, 35, 36, 37, 38, 47, 50, 52, 53, 54, 55, 58, 62, 72, 87, 93, 97, 99, 101, 113, 118, 119, 120, 129, 130, 134, 140, 141, 142, 143, 146, 147, 148, 163], "sample_num": 11, "sample_proj_bia": 130, "sample_st": [35, 87], "sample_weight_strip": 148, "samplemod": 129, "sampler": [21, 29, 35, 82, 92, 143, 145, 148, 161], "sampler_opt": 21, "sampler_typ": [9, 143], "samplertyp": 143, "sampling_config": 134, "sampling_param": [11, 42, 47, 48, 49, 51, 52, 53, 54, 55, 56, 57, 58, 63, 64, 85, 91, 97, 121, 128, 140, 142, 143, 147, 148, 162, 165], "samplingconfig": [0, 42, 99, 102, 134, 148], "samplingparam": [42, 47, 48, 49, 51, 52, 53, 54, 55, 56, 57, 58, 63, 64, 65, 85, 91, 93, 97, 121, 128, 140, 142, 143, 147, 148, 162, 165], "sandbox": 94, "saniti": [97, 122, 123, 127], "santa": 56, "santacod": [137, 138], "satfinit": 137, "satisfi": [11, 15, 16, 102, 114, 148], "satur": [15, 19, 20, 88], "save": [2, 8, 10, 13, 14, 15, 20, 22, 25, 28, 29, 30, 31, 36, 56, 72, 83, 92, 101, 105, 109, 115, 116, 119, 120, 123, 127, 128, 135, 143, 148], "save_checkpoint": [115, 131], "save_config": [115, 131], "save_kv_lay": 56, "saw": 123, "sbatch": [59, 60, 61, 113], "sbsa": [97, 141, 148], "scaffold": [93, 144, 148, 149], "scaffolding_llm": 148, "scaffoldingllm": 148, "scalabl": [8, 15, 19, 20], "scalar": [102, 107, 129], "scalartyp": 148, "scale": [0, 11, 14, 16, 20, 22, 49, 81, 89, 90, 92, 93, 102, 106, 114, 123, 129, 130, 137, 143, 148], "scale_d0": 129, "scale_d1": 129, "scale_factor": 129, "scale_output": 129, "scale_qk": 130, "scale_typ": 129, "scalia": [47, 97, 140, 142, 147], "scaling_factor": 129, "scaling_long_factor": 129, "scaling_short_factor": 129, "scalingvecpoint": 1, "scan": 18, "scanreducetempstorag": 1, "scanreducetempstoragebyt": 1, "scantempstorag": 1, "scantempstoragebyt": 1, "scarc": 33, "scatter": [15, 103, 129], "scatter_nd": 129, "scenario": [2, 5, 7, 8, 10, 12, 14, 15, 16, 19, 20, 21, 22, 38, 57, 72, 84, 86, 88, 89, 92, 101, 107, 109, 112, 117, 120, 121, 123, 126, 127, 147, 148], "scene": 11, "scfg": 134, "schedul": [0, 2, 10, 11, 13, 14, 15, 18, 21, 22, 23, 26, 28, 29, 30, 31, 32, 37, 78, 79, 81, 82, 91, 92, 93, 98, 99, 105, 106, 120, 121, 127, 135, 141, 143, 145, 148, 158, 159, 161], "schedule_request": 167, "scheduled_batch": [35, 87], "scheduled_request": 167, "scheduledrequest": [10, 92], "scheduler_config": [128, 143], "scheduler_output": 56, "scheduler_polici": 21, "schedulerconfig": [0, 93, 128, 143, 148], "scheduleroutput": 56, "schedulerpolici": 148, "scheduling_param": 143, "schedulingparam": 143, "schema": [0, 10, 37, 51, 71, 91, 93, 99, 120, 143, 148], "schema_gener": 143, "schema_json": 143, "scheme": [0, 83, 143, 148], "scicod": 12, "scienc": [47, 52, 53, 55, 56, 97, 140, 142, 147], "scope": [11, 13, 148], "score": [14, 83, 102], "scout": [18, 27, 79, 93, 138, 145, 148, 159], "scratch": [15, 34, 57, 120, 121, 122, 127], "script": [2, 15, 16, 19, 21, 25, 28, 29, 30, 31, 33, 34, 36, 37, 38, 57, 59, 60, 61, 75, 78, 81, 86, 89, 90, 94, 106, 108, 109, 111, 113, 115, 116, 119, 120, 121, 132, 137, 139, 142, 144, 148, 149, 154, 158, 164], "sd3": 130, "sd35adalayernormzerox": 130, "sd3patchemb": 130, "sd3transformer2dmodel": 131, "sd3transformer2dmodelconfig": 131, "sdxl": 148, "se": 146, "seamless": [79, 93, 141, 146, 148, 159], "seamlessli": [77, 93, 142, 146, 157], "search": [0, 1, 6, 11, 21, 22, 23, 26, 33, 35, 42, 83, 84, 93, 99, 102, 109, 123, 126, 129, 143, 148, 150], "seashor": [40, 67], "seat": [47, 97, 140, 142, 147], "sec": [2, 4, 16, 37, 38, 93, 120, 121, 123, 126, 127], "second": [1, 2, 3, 5, 6, 8, 10, 11, 12, 15, 16, 17, 18, 20, 26, 56, 57, 80, 81, 83, 88, 99, 102, 105, 106, 109, 126, 129, 143, 160], "secondari": [0, 83, 104, 135, 143], "secondary_offload_min_prior": [83, 143], "secondaryoffloadminprior": 0, "secondli": [88, 126], "section": [2, 13, 14, 15, 19, 20, 25, 26, 28, 29, 30, 31, 33, 34, 37, 72, 77, 80, 81, 83, 88, 92, 94, 99, 102, 113, 114, 115, 120, 121, 123, 124, 125, 126, 127, 129, 138, 147, 148, 151, 157, 160], "section_s": 129, "secur": [51, 148], "securityprotocol": 51, "see": [0, 1, 2, 3, 5, 6, 7, 11, 13, 14, 15, 18, 19, 20, 21, 23, 25, 26, 28, 29, 30, 31, 32, 33, 34, 37, 38, 40, 44, 47, 56, 67, 72, 74, 77, 78, 83, 86, 88, 89, 92, 94, 95, 96, 97, 101, 102, 104, 109, 113, 114, 116, 117, 120, 121, 123, 126, 127, 128, 129, 130, 131, 135, 137, 139, 148, 153, 157, 158, 166], "seed": [0, 9, 21, 23, 45, 46, 102, 133, 138, 143, 148], "seek": [28, 29, 30, 31], "seem": [37, 62, 105, 120, 122], "seen": [2, 15, 18, 37, 109, 120], "segment": 148, "select": [0, 1, 7, 12, 14, 16, 18, 19, 22, 33, 35, 37, 80, 81, 89, 93, 100, 102, 120, 127, 129, 134, 135, 143, 150, 160, 167], "selectcontextid": 0, "selectgenidx": 0, "selective_scan": 129, "self": [0, 11, 32, 35, 56, 58, 72, 80, 87, 91, 92, 101, 103, 111, 113, 114, 120, 129, 131, 134, 139, 143, 144, 149, 160, 166, 167], "self_attent": 114, "self_attention_mask": 130, "self_attention_packed_mask": 130, "self_attn": [114, 144, 149], "selfidx": 0, "sell": [37, 120], "semicolon": 94, "senat": [47, 97, 140, 142, 147], "send": [0, 9, 11, 12, 15, 16, 19, 20, 26, 28, 29, 30, 31, 81, 86, 98, 113, 121, 122, 129, 147, 148], "sens": 123, "sensit": [8, 12, 15, 19, 123], "sent": [0, 14, 15, 16, 25, 28, 29, 30, 31, 57, 81, 109, 143], "sentenc": [0, 9, 28, 30, 58, 102, 143, 147, 156], "sep": 143, "separ": [8, 10, 15, 16, 19, 22, 33, 38, 72, 73, 75, 80, 81, 83, 89, 94, 107, 109, 120, 129, 134, 143, 146, 147, 151, 152, 154, 160], "separate_match_rewrit": 103, "seq": [1, 37, 72, 74, 101, 120, 129, 148, 153], "seq_idx": 134, "seq_len": [38, 72, 129, 130, 151], "seq_length": 129, "seq_lens_cuda": [72, 151], "seqlen": [0, 129], "seqslot": 1, "sequenc": [0, 1, 2, 3, 4, 5, 6, 8, 10, 11, 12, 13, 14, 15, 16, 18, 20, 21, 26, 28, 29, 30, 31, 35, 37, 38, 56, 58, 64, 72, 73, 74, 81, 83, 88, 89, 91, 92, 93, 99, 101, 102, 103, 104, 105, 109, 113, 120, 121, 124, 125, 128, 129, 130, 134, 135, 143, 146, 148, 151, 152, 153, 166], "sequence_length": [129, 130, 134, 139], "sequence_length_buff": 134, "sequence_limit_length": 134, "sequenceindex": [0, 99], "sequencelengthscba": 1, "sequencelimitlength": 1, "sequenti": [0, 8, 13, 57, 81, 98, 109, 135], "seri": [11, 29, 78, 79, 146, 148, 158, 159], "serial": [19, 21, 22, 129, 131, 134, 143], "serializ": 143, "serialize_as_ani": 143, "serialize_engin": 134, "serializeds": 0, "serializedst": 0, "serv": [0, 6, 7, 8, 9, 10, 15, 17, 18, 19, 23, 28, 29, 30, 31, 39, 40, 41, 43, 45, 46, 50, 66, 67, 68, 69, 70, 72, 82, 87, 88, 89, 93, 98, 99, 101, 104, 113, 127, 143, 145, 146, 148, 150, 151, 158, 161, 163], "server": [0, 4, 15, 19, 25, 38, 39, 40, 41, 43, 45, 46, 66, 67, 68, 69, 70, 93, 98, 105, 109, 113, 116, 146, 147, 148, 156], "server_rol": 26, "server_start_timeout": 26, "servic": [11, 16, 19, 20, 34, 58, 81, 148], "session": [94, 101, 120, 134, 143], "set": [0, 1, 2, 9, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22, 26, 28, 29, 30, 31, 33, 34, 36, 38, 42, 48, 49, 51, 57, 59, 60, 61, 72, 75, 80, 81, 83, 84, 87, 88, 90, 91, 92, 94, 97, 98, 99, 100, 101, 102, 103, 104, 107, 108, 109, 110, 112, 114, 115, 119, 121, 123, 126, 127, 128, 129, 130, 131, 132, 134, 135, 139, 142, 143, 147, 148, 154, 156, 160, 163, 167], "set_api_statu": 32, "set_attn_processor": 131, "set_default_max_input_len": 143, "set_from_opt": 1, "set_if_not_exist": 131, "set_input_shap": 134, "set_rank": 131, "set_rel_attn_t": 130, "set_runtime_knobs_from_build_config": 143, "set_shap": 134, "setadditionalmodeloutput": [0, 99], "setallottedtimem": 0, "setattentiondpeventsgatherperiodm": 0, "setattr": 11, "setbackend": 0, "setbackendtyp": 0, "setbadword": 0, "setbatchingtyp": 0, "setbeamsearchdiversityr": 0, "setbeamwidth": [0, 1], "setbeamwidtharrai": 0, "setbitto": 0, "setcachesaltid": 0, "setcachest": 0, "setcachetransceiverconfig": [0, 81], "setclientid": 0, "setcommst": 0, "setcommunicationmod": 0, "setcommunicationtyp": 0, "setcontextfmha": 1, "setcontextphaseparam": 0, "setcopyonpartialreus": 0, "setcrossattentionmask": 0, "setcrosskvcachefract": 0, "setcudagraphcaches": 0, "setcudagraphmod": 0, "setdatatyp": 1, "setdebugconfig": 0, "setdebuginputtensor": 0, "setdebugoutputtensor": 0, "setdebugtensornam": 0, "setdebugtensorsmaxiter": 0, "setdecodingconfig": 0, "setdecodingmod": 0, "setdeviceid": 0, "seteagleconfig": 0, "setearlystop": 0, "setembeddingbia": 0, "setenableblockreus": 0, "setenablechunkedcontext": 0, "setenablecontextfmhafp32acc": 0, "setenablepartialreus": 0, "setenabletrtoverlap": 0, "setencodedvocab": 0, "setencoderhiddens": 1, "setencoderinputfeatur": 0, "setencoderinputtokenid": 0, "setencoderoutputlength": 0, "setendid": 0, "seteventbuffermaxs": 0, "setexecutionconfig": 1, "setextendedruntimeperfknobconfig": 0, "setexternaldrafttokensconfig": 0, "setfailfastonattentionwindowtoolarg": 0, "setfreegpumemoryfract": 0, "setfrequencypenalti": 0, "setfrom": 0, "setfrominput": 1, "setgathergenerationlogit": 0, "setgemmallreducedtyp": 1, "setgenerationstep": 1, "setgpuweightsperc": [0, 110], "setguideddecodingconfig": 0, "setguideddecodingparam": 0, "sethostcaches": 0, "setinittozero": 1, "setisorchestr": 0, "setiterstatsmaxiter": 0, "setkvcacheconfig": 0, "setkvcacheretentionconfig": 0, "setkvcachetyp": 1, "setlanguageadapteruid": 0, "setlayertyp": 1, "setlengthpenalti": 0, "setlevel": 1, "setlogitsdtyp": 1, "setlogitspostprocessor": 0, "setlogitspostprocessorconfig": 0, "setlogitspostprocessornam": 0, "setlookaheadconfig": 0, "setlookaheaddecodingconfig": 0, "setloraconfig": 0, "setloramodul": 1, "setmanagedweightsmap": 1, "setmanageweightstyp": 1, "setmaxattentionwindowvec": 0, "setmaxbatchs": [0, 1], "setmaxbeamwidth": [0, 1], "setmaxdraftpathlen": 1, "setmaxdrafttoken": 1, "setmaxencoderlen": 1, "setmaxgputotalbyt": 0, "setmaxinputlen": 1, "setmaxlorarank": 1, "setmaxnumpath": 1, "setmaxnumtoken": [0, 1], "setmaxpagesperblock": 1, "setmaxpositionembed": 1, "setmaxpromptembeddingtables": 1, "setmaxqueues": 0, "setmaxseqidlemicrosecond": 0, "setmaxsequencelen": 1, "setmaxtoken": 0, "setmaxtokensinbuff": 0, "setmedusachoic": 0, "setmem": 1, "setmemorytyp": 1, "setminp": 0, "setmintoken": 0, "setmlphiddens": 1, "setmodelnam": 1, "setmodelvari": 1, "setmropeconfig": 0, "setmultiblockmod": 0, "setmultimodalembed": 0, "setmultimodalinput": 0, "setnbcrosskvhead": 1, "setnbkvhead": 1, "setnorepeatngrams": 0, "setnormalizelogprob": 0, "setnumcopystream": 1, "setnumdecodingenginetoken": 1, "setnumkvheadspercrosslay": 1, "setnumkvheadsperlay": 1, "setnumlanguag": 1, "setnumnod": 0, "setnumreturnsequ": 0, "setonboardblock": 0, "setorchestratorconfig": 0, "setorchleadercomm": 0, "setoutputconfig": 0, "setpadid": 0, "setpagedcontextfmha": 1, "setpagewidth": 1, "setparallelconfig": 0, "setparticipantid": 0, "setpath": 1, "setpeftcacheconfig": 0, "setpositionid": 0, "setppreducescatt": 1, "setpresencepenalti": 0, "setprior": 0, "setprocessorbatch": 0, "setprocessormap": 0, "setprompttableoffload": 0, "setprompttuningconfig": 0, "setquantmod": 1, "setrecvpollperiodm": 0, "setrepetitionpenalti": 0, "setrepl": [0, 99], "setrequeststatsmaxiter": 0, "setrequesttyp": 0, "setreturnallgeneratedtoken": 0, "setrnnconfig": 1, "setrotaryembeddingdim": 1, "setsamplingconfig": 0, "setschedulerconfig": 0, "setse": 0, "setsecondaryoffloadminprior": 0, "setsinktokenlength": 0, "setsizeperhead": 1, "setskipcrossattnblock": [0, 1], "setslotsperpag": 1, "setspawnprocess": 0, "setspecdecconfig": 0, "setspeculativedecodingmod": 1, "setspeculativedecodingmodul": 1, "setstoptokenid": 0, "setstopword": 0, "setstream": 0, "settemperatur": 0, "setter": [0, 102], "settokenizerstr": 0, "settokensperblock": 1, "settopk": 0, "settopp": 0, "settoppdecai": 0, "settoppmin": 0, "settoppresetid": 0, "settotalnumpag": 1, "setup": [1, 9, 16, 17, 22, 25, 28, 29, 30, 31, 51, 59, 60, 61, 78, 97, 101, 122, 123, 134, 135, 142, 146, 147, 148, 158], "setup_embedding_parallel_mod": 143, "setup_fake_prompt": 134, "setup_fake_prompts_qwen2vl": 134, "setup_fake_prompts_vila": 134, "setup_input": 134, "setupbuff": 1, "setupcacheindirect": 1, "setupcacheindirectionbuff": 1, "setupspeculativedecod": 1, "setupspeculativedecodingbuff": 1, "setuptool": [78, 97, 158], "setusecrossattent": 1, "setusegpudirectstorag": 0, "setusemrop": 1, "setusepositionembed": 1, "setuseshapeinfer": 1, "setusetokentypeembed": 1, "setuseuvm": 0, "setvirtualmemoryalloc": 1, "setworkerexecutablepath": 0, "setzero": [0, 1], "seven": 18, "sever": [0, 1, 8, 10, 11, 15, 16, 17, 18, 19, 20, 35, 38, 42, 57, 72, 81, 88, 101, 103, 109, 112, 123, 124, 125, 126, 127, 129, 135, 139, 146, 151], "sft": [62, 79, 159], "sglang": [10, 11, 15, 87, 163], "sh": [17, 25, 28, 29, 30, 31, 34, 90, 113, 116, 148, 164], "shah": 148, "shall": [1, 115, 135], "shape": [0, 1, 10, 12, 14, 72, 88, 101, 103, 106, 112, 113, 127, 129, 131, 134, 135, 137, 139, 143, 148, 151, 166], "shape_cast_dtyp": 129, "shapeequ": 1, "shard": [12, 21, 33, 74, 75, 77, 78, 79, 89, 114, 120, 124, 129, 130, 153, 154, 157, 158, 159], "shard_map": 114, "sharding_along_vocab": 143, "sharding_dim": [129, 130], "share": [1, 2, 7, 8, 10, 12, 13, 14, 15, 16, 17, 18, 20, 22, 23, 72, 80, 81, 83, 84, 92, 94, 98, 99, 101, 103, 104, 105, 106, 109, 115, 122, 123, 129, 130, 148, 160], "share_embed": 148, "share_weight": 130, "shared_embedding_t": 148, "shared_expert_output": 129, "shared_fc1": 14, "shared_fc2": 14, "shared_ptr": [0, 1], "sharedconstptr": 1, "sharedptr": 1, "shelf": [10, 78, 148, 158], "shell": [33, 34, 95], "sherlock113": 148, "shift": [13, 15, 107], "ship": [20, 115], "shm": [15, 139], "short": [8, 11, 15, 20, 64, 72, 88, 101, 120, 123, 126], "short_factor": 129, "short_mscal": [129, 130], "shorter": [38, 58, 72, 88, 101], "shortli": 26, "shot": [28, 30, 31, 148], "should": [0, 1, 2, 10, 11, 14, 15, 19, 21, 23, 28, 29, 30, 31, 33, 37, 38, 42, 47, 49, 51, 59, 60, 61, 62, 63, 72, 77, 80, 83, 88, 92, 94, 96, 97, 99, 103, 105, 106, 107, 115, 120, 121, 122, 127, 128, 129, 130, 132, 134, 135, 140, 142, 143, 144, 147, 148, 149, 151, 157, 160, 166, 167], "should_early_stop": 11, "should_skip_modul": [80, 160], "should_stop": 134, "shouldus": [72, 101], "show": [4, 8, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 26, 28, 29, 30, 31, 33, 38, 47, 57, 81, 83, 84, 88, 99, 113, 121, 126, 127, 135, 138, 141, 147, 156], "showcas": [50, 57, 123, 127], "shown": [5, 11, 13, 15, 16, 25, 28, 29, 30, 31, 37, 38, 80, 81, 88, 94, 107, 120, 121, 123, 126, 127, 129, 160], "shrunk": 129, "shuffl": 129, "shutdown": [0, 57, 142, 143], "shutdown_abort": 143, "si": [72, 101], "sibl": 113, "side": [15, 19, 35, 75, 99, 129, 143, 154], "side_stream_id": 129, "sidestreamidtyp": 129, "sigh": 62, "sigmoid": [113, 129], "signal": [0, 11, 19], "signatur": [10, 32, 103, 129], "signifi": [88, 126], "signific": [5, 8, 10, 11, 13, 14, 15, 19, 35, 62, 72, 81, 98, 99, 101, 104, 122, 123, 126, 127], "significantli": [7, 8, 12, 13, 14, 15, 16, 19, 25, 28, 29, 30, 31, 35, 57, 72, 75, 84, 87, 88, 89, 121, 122, 123, 126, 127, 135, 151, 154, 163], "silicon": [14, 20], "silu": [113, 129, 130], "similar": [0, 2, 3, 5, 8, 10, 11, 13, 15, 18, 19, 20, 25, 36, 37, 42, 72, 92, 101, 102, 103, 109, 119, 120, 128, 129, 147, 150, 167], "similarli": [10, 80, 81, 109, 160], "simpl": [9, 11, 15, 17, 18, 19, 20, 38, 47, 56, 73, 75, 79, 92, 94, 96, 103, 104, 109, 113, 140, 141, 142, 147, 152, 154, 156, 159], "simple_shard_onli": [77, 157], "simpler": [15, 109], "simpleschedul": 167, "simplest": [92, 95, 129], "simpli": [11, 37, 38, 75, 84, 88, 92, 101, 109, 120, 126, 139, 142, 144, 147, 149, 154], "simplic": [18, 115], "simplifi": [8, 10, 23, 35, 37, 56, 78, 79, 88, 101, 115, 120, 126, 129, 142, 148, 158, 159], "simul": 57, "simultan": [8, 85, 91, 109, 126, 162], "sin": [0, 129, 130], "sinc": [0, 1, 2, 8, 10, 13, 14, 15, 18, 19, 21, 25, 26, 28, 29, 30, 31, 37, 42, 57, 58, 72, 75, 84, 88, 92, 94, 100, 101, 103, 105, 109, 110, 115, 116, 120, 121, 122, 123, 126, 127, 129, 131, 135, 143, 150, 154, 166, 167], "sincer": [10, 11, 14], "sinco": 130, "singl": [0, 1, 2, 5, 6, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 26, 28, 29, 30, 31, 35, 36, 37, 40, 49, 57, 67, 72, 75, 78, 81, 83, 88, 89, 92, 98, 99, 100, 101, 102, 104, 109, 111, 113, 115, 119, 120, 123, 127, 129, 131, 134, 135, 137, 143, 144, 146, 147, 148, 149, 150, 151, 154, 156, 158, 166], "singleton": [103, 129], "sink": [0, 1, 72, 101, 134, 143], "sink_token_len": 134, "sink_token_length": [72, 83, 101, 134, 143], "sinktokenlength": [0, 1], "sinusoid": 130, "sit": [29, 62, 115], "situaiton": 38, "situat": [19, 62, 88, 109, 121, 126], "six": 13, "size": [0, 1, 2, 4, 5, 7, 8, 10, 12, 13, 14, 15, 18, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 35, 36, 37, 38, 42, 57, 59, 60, 61, 72, 73, 74, 75, 77, 81, 84, 85, 88, 89, 92, 93, 98, 101, 102, 104, 105, 106, 107, 109, 110, 119, 120, 121, 122, 123, 124, 127, 129, 130, 131, 134, 139, 143, 148, 151, 152, 153, 154, 157, 162, 167], "size_t": [0, 1], "size_typ": [0, 1], "sizeof": 1, "sizeperhead": [0, 1], "sizetype32": [0, 1], "sizetype64": [0, 1], "skip": [0, 1, 2, 10, 21, 33, 56, 57, 73, 74, 80, 81, 94, 103, 114, 117, 129, 143, 148, 152, 153, 160, 167], "skip_attn": [129, 130], "skip_cross_attn_block": [131, 134], "skip_cross_kv": [130, 134], "skip_encod": 134, "skip_loading_weight": [73, 77, 152, 156, 157], "skip_special_token": [143, 148], "skip_tokenizer_init": [42, 143], "skipcrossattnblock": [0, 1], "sku": [121, 123, 126, 127], "skywork": [137, 138, 148], "sleep": 117, "slice": [1, 89, 100, 114, 129, 143, 148], "slice_shap": 114, "sliceinputtyp": 129, "slicen": 1, "slide": [0, 18, 82, 93, 104, 128, 129, 134, 145, 148, 161], "slider": [2, 12, 37, 120], "sliding_window": 131, "sliding_window_caus": 129, "slight": [2, 13, 14, 123, 126, 127], "slighter": 10, "slightli": [0, 10, 20, 26, 90, 106, 107, 123, 127], "slope": [72, 101, 129], "slot": [0, 1, 10, 15, 89, 148], "slot_map": [129, 131], "slotid": 15, "slotidx": 1, "slotsperpag": 1, "slow": [19, 99, 105, 122, 143], "slowdown": 19, "slower": [14, 35, 57, 104, 115, 122], "slowest": [8, 72, 101], "slurm": [15, 19, 25, 93, 97, 113, 139, 148], "slurm_script": [19, 89], "sm": [20, 138, 148], "sm100": [90, 138, 145], "sm120": [90, 138, 148], "sm121": 148, "sm80": [138, 148], "sm86": [138, 148], "sm89": [138, 148], "sm90": [90, 138, 145, 148], "small": [7, 10, 12, 13, 14, 15, 18, 19, 20, 57, 58, 72, 85, 89, 101, 105, 107, 109, 113, 121, 123, 126, 127, 129, 135, 136, 138, 139, 148, 162], "smaller": [1, 2, 11, 13, 18, 19, 20, 22, 36, 37, 77, 84, 88, 91, 109, 119, 120, 122, 126, 127, 128, 129, 135, 148, 157], "smallest": [0, 1, 104, 129], "smart": [16, 81, 129, 148], "smaug": [138, 148], "smi": [2, 12, 17, 20, 28, 29, 30, 31, 37, 81, 98, 120, 135], "smile": 62, "smith": [47, 52, 53, 54, 55, 97, 140, 142, 147], "smooth": [115, 143, 148], "smoother": 2, "smoothli": 11, "smoothquant": [7, 103, 148], "smoothquant_v": 143, "snapshot": [10, 120], "snapshot_download": 62, "snip": [37, 120], "snippet": [10, 20, 120, 148, 167], "snshrivas10": 62, "so": [0, 1, 2, 10, 11, 12, 13, 14, 15, 19, 20, 26, 28, 29, 30, 31, 33, 37, 38, 42, 48, 56, 72, 83, 88, 91, 92, 94, 99, 101, 103, 106, 109, 115, 116, 120, 122, 123, 126, 127, 128, 129, 130, 131, 135, 138, 143, 144, 148, 149, 156, 166], "soc": 56, "socketst": 0, "softmax": [13, 14, 72, 101, 113, 129, 151], "softplu": 129, "softwar": [14, 15, 18, 20, 28, 29, 30, 31, 96, 97, 99, 101, 113, 148], "sol": 16, "sole": 23, "solid": [124, 125], "solut": [10, 11, 15, 19, 20, 83, 89, 139, 142, 150], "some": [0, 2, 8, 10, 11, 12, 13, 14, 15, 16, 19, 22, 23, 25, 26, 29, 33, 34, 38, 58, 62, 72, 75, 80, 81, 83, 86, 88, 97, 98, 99, 100, 101, 102, 103, 105, 109, 110, 112, 113, 115, 117, 123, 124, 125, 127, 128, 129, 132, 135, 139, 142, 143, 144, 148, 149, 150, 154, 167], "some_uri": 34, "someth": [11, 42, 113], "sometim": [15, 16, 33, 37, 81, 120], "song": [37, 120], "soon": [0, 3, 4, 5, 6, 7, 19, 42, 83], "sophist": [8, 19, 75, 154], "sora": [40, 67], "sort": [0, 1, 8, 99, 102, 129], "sota": 148, "sourc": [2, 3, 6, 10, 11, 12, 14, 15, 21, 22, 26, 28, 29, 30, 31, 34, 39, 40, 41, 43, 45, 46, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 75, 78, 80, 95, 96, 97, 108, 111, 112, 114, 115, 129, 130, 131, 132, 133, 134, 143, 146, 148, 154, 158, 160], "source_dir": 34, "source_param1": [80, 160], "source_param2": [80, 160], "source_root": [59, 60, 61], "sourcetaskvalu": 1, "south": [26, 29], "southeast": 29, "southern": 29, "southwest": 29, "soyer": [111, 113, 139], "sp": [75, 154], "sp_kwarg": [75, 154], "space": [15, 16, 19, 57, 78, 81, 83, 86, 88, 92, 94, 106, 126, 135, 143, 158, 166], "spaces_between_special_token": [143, 148], "span": [12, 13, 15, 16, 84, 115], "spars": [14, 19, 89, 109, 129, 148], "sparse_fc1": 14, "sparse_fc2": 14, "sparsiti": [15, 19, 20, 22, 143], "spatial_norm_dim": 130, "spawn": [55, 121, 139, 142], "spawnprocess": 0, "speakleash": 138, "spec": [15, 22, 148], "spec_config": [18, 65], "spec_dec_mod": 143, "spec_decode_algo": [13, 18], "spec_decode_nextn": 13, "spec_decoding_generation_length": [129, 130, 131], "spec_decoding_is_generation_length_vari": [129, 130, 131], "spec_decoding_max_generation_length": [129, 130], "spec_decoding_packed_mask": [129, 130, 131], "spec_decoding_param": [130, 131], "spec_decoding_position_offset": [129, 130, 131], "spec_decoding_us": [129, 130], "specconfig": [92, 148], "specdec": 0, "specdecconfig": 0, "specdecfastlogitsinfo": 0, "specdecodinggenerationlength": 1, "specdecodinggenerationlengthshost": 1, "specdecodingpackedmask": 1, "specdecodingparam": 130, "specdecodingpositionoffset": 1, "specdecodingstat": 0, "special": [3, 10, 11, 13, 22, 72, 80, 81, 98, 101, 106, 113, 114, 143, 148, 160], "specif": [0, 1, 4, 7, 8, 10, 11, 12, 14, 15, 16, 17, 19, 20, 21, 25, 26, 28, 29, 30, 31, 32, 33, 37, 56, 58, 64, 75, 80, 81, 83, 85, 89, 91, 94, 97, 100, 102, 103, 104, 106, 107, 108, 109, 112, 115, 120, 122, 123, 127, 129, 142, 143, 144, 146, 147, 148, 149, 150, 154, 160, 162], "specifi": [0, 1, 2, 10, 11, 15, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 33, 35, 36, 37, 38, 42, 51, 58, 62, 70, 72, 73, 74, 75, 76, 78, 79, 81, 83, 84, 92, 94, 99, 101, 102, 103, 104, 106, 109, 114, 115, 119, 120, 122, 123, 126, 128, 129, 131, 132, 134, 135, 139, 142, 143, 147, 148, 151, 152, 153, 154, 155, 158, 159, 160], "specmetadata": 92, "spectrum": [8, 146], "specul": [0, 1, 12, 15, 16, 17, 35, 50, 78, 91, 93, 99, 118, 120, 121, 129, 131, 140, 142, 143, 146, 148, 158, 165], "speculative_config": [2, 9, 12, 13, 28, 65, 92, 143], "speculative_decod": 148, "speculative_decoding_draft_tokens_extern": 131, "speculative_decoding_mod": [22, 120, 143], "speculative_model": 92, "speculative_model_dir": [9, 17, 65, 143], "speculative_model_format": 143, "speculativeconfig": 143, "speculativedecod": 0, "speculativedecodingconfig": 0, "speculativedecodingfastlogitsinfo": 0, "speculativedecodingmetr": 0, "speculativedecodingmod": [92, 131, 143, 148], "speculativedecodingmodul": 148, "speculativedecodingoutput": 1, "speed": [4, 12, 13, 14, 15, 17, 22, 37, 38, 92, 113, 120, 127, 148], "speedup": [2, 4, 6, 7, 8, 10, 11, 12, 14, 16, 17, 20], "spent": 0, "spirit": 15, "split": [1, 18, 37, 72, 84, 88, 89, 100, 101, 106, 113, 120, 122, 123, 129, 135, 143, 146, 148], "split_input_id": 134, "split_prompt_by_imag": 134, "split_siz": 129, "split_size_or_sect": 129, "splitlin": 143, "splittransposecpu": 1, "splittransposecpuinn": 1, "splitwis": 98, "spot": [15, 19, 126], "sq": [7, 137, 148], "sqrt": [72, 101, 129], "sqsh": 94, "sqsh_path": 94, "squar": [19, 25, 88, 126, 129], "squared_relu": 129, "squash": 94, "squeez": [1, 129, 134], "src": [1, 113, 129], "src_seq_len": 129, "srcdesc": 0, "srctype": 1, "srun": [26, 59, 60, 61, 97, 113, 139], "ssd": [9, 17, 20], "ssh": 34, "sshd": 116, "ssid": 51, "ssm": [78, 129, 143, 158], "ssm_state": 131, "stabil": [8, 12, 15, 78, 93, 108, 143, 158], "stabl": [15, 22, 32, 72, 88, 101, 114, 121, 126, 127, 129, 143, 148], "stack": [9, 11, 12, 17, 20, 25, 28, 29, 30, 31, 79, 94, 114, 129, 147, 159], "stacklevel": 32, "stackoverflow": 34, "stage": [0, 11, 13, 16, 17, 21, 32, 38, 72, 75, 78, 81, 86, 92, 93, 101, 103, 135, 148, 151, 154, 158], "stage_list": 33, "stai": [4, 7, 15, 122, 127], "stall": 15, "stand": 113, "standalon": 115, "standard": [0, 3, 10, 11, 15, 17, 20, 21, 38, 73, 75, 78, 79, 86, 109, 113, 129, 152, 154, 158, 159], "starcod": [138, 148], "starcoder1": 137, "starcoder2": [79, 137, 148, 159], "starrickliu": 148, "start": [0, 2, 11, 13, 15, 19, 20, 22, 27, 32, 33, 34, 38, 39, 40, 41, 43, 45, 46, 48, 49, 61, 62, 66, 67, 68, 69, 70, 81, 88, 89, 94, 99, 101, 103, 105, 116, 117, 120, 121, 122, 126, 128, 129, 131, 133, 134, 135, 143, 146, 148], "start_dim": 129, "start_load_kv": 56, "started_loading_req_id": 56, "startswith": 143, "startup": [25, 28, 29, 30, 31, 81, 98, 139], "stat": [0, 143, 148], "state": [0, 1, 2, 11, 12, 13, 15, 16, 19, 22, 28, 29, 30, 31, 37, 38, 47, 48, 49, 57, 72, 78, 80, 83, 88, 89, 92, 93, 97, 99, 100, 101, 103, 104, 105, 109, 120, 121, 126, 128, 129, 140, 142, 143, 147, 148, 158, 160, 167], "state_dict": [80, 160], "state_dtyp": 134, "state_or_ptr": 129, "state_s": 134, "statement": [11, 142], "staten": 29, "stateptr": 0, "states": 1, "static": [0, 1, 14, 19, 22, 74, 83, 99, 108, 109, 129, 130, 131, 134, 143, 148, 153], "static_batch": [128, 143], "static_cast": [1, 137], "staticbatchingstat": 0, "statist": [0, 17, 21, 26, 37, 57, 89, 99, 109, 120, 143, 148], "statu": [1, 9, 15, 20, 28, 29, 30, 31, 32, 93, 139, 143, 148], "std": [0, 1, 15, 99], "stddev": [45, 46], "stderr": [28, 30, 31], "stdev": [2, 21, 36, 37, 38, 60, 119, 120, 121], "stdin": 21, "stdit": 148, "stdout": [2, 21, 36, 37, 38, 60, 119, 120, 121], "steadi": 38, "steady_clock": 0, "stem": 19, "step": [0, 1, 3, 10, 11, 12, 13, 20, 25, 32, 35, 37, 38, 57, 64, 72, 73, 87, 88, 90, 92, 93, 97, 101, 102, 103, 105, 109, 112, 113, 115, 117, 120, 121, 129, 134, 139, 143, 148, 150, 151, 152, 163, 166, 167], "still": [2, 8, 10, 11, 12, 13, 14, 15, 16, 19, 20, 32, 37, 84, 101, 114, 115, 120, 121, 123, 129, 134, 135, 148], "stine": 28, "stoica": 11, "stop": [0, 1, 11, 15, 20, 29, 35, 56, 58, 87, 88, 91, 99, 102, 103, 109, 120, 126, 134, 143, 146, 147, 148, 163, 165], "stop_reason": [17, 20, 28, 29, 30, 31, 143, 147, 148], "stop_token_id": [99, 143], "stop_words_data": 134, "stop_words_list": 134, "stopping_criteria": 134, "stoppingcriteria": [134, 148], "stoppingcriterialist": 134, "stoptokenid": [0, 99], "stopword": [0, 102], "stopwordslen": 1, "stopwordslist": 1, "stopwordsptr": 1, "storag": [0, 1, 8, 9, 17, 20, 80, 104, 106, 142, 143, 160], "store": [0, 1, 4, 10, 12, 13, 15, 17, 21, 26, 32, 33, 35, 37, 49, 57, 72, 80, 83, 84, 88, 89, 92, 101, 104, 105, 106, 113, 120, 128, 129, 131, 135, 137, 143, 144, 148, 149, 151, 160, 166], "store_tru": 57, "stori": [62, 64], "str": [11, 32, 53, 54, 56, 58, 62, 63, 64, 65, 69, 80, 81, 112, 115, 129, 130, 131, 134, 143, 160], "straight": 94, "straightforward": [10, 13, 19, 20], "strateg": 8, "strategi": [0, 2, 7, 13, 15, 16, 19, 35, 37, 42, 75, 77, 81, 91, 93, 107, 109, 120, 124, 129, 131, 135, 142, 143, 146, 148, 154, 157], "stream": [0, 1, 9, 10, 11, 14, 15, 20, 21, 22, 32, 36, 37, 42, 45, 46, 50, 56, 58, 81, 98, 99, 113, 119, 129, 134, 135, 139, 143, 148], "stream_interv": [19, 20, 28, 29, 143], "stream_ptr": [58, 91], "streaming_llm": 148, "streamingllm": [22, 148], "streamlin": [35, 37, 78, 79, 120, 142, 147, 158, 159], "streamptr": [0, 1, 99], "street": 62, "strenum": [133, 143], "stretch": 29, "strict": [12, 13, 15, 28, 30, 31, 75, 143, 154], "strict_bound": 129, "strict_dtyp": [129, 130], "strictbasemodel": 143, "stricter": 12, "strictli": [10, 37, 120, 143], "stride": [1, 129, 130], "strike": [15, 83, 109], "string": [0, 1, 19, 33, 37, 51, 70, 83, 99, 112, 120, 129, 134, 143], "string_valu": 105, "string_view": 1, "stringptrmap": 1, "stringvec": 0, "strip": [22, 143, 148], "strip_plan": 22, "strive": [78, 146, 158], "strong": [15, 18], "strongli": 123, "strongly_typ": [143, 148], "struct": [0, 1, 104], "structur": [0, 10, 14, 18, 20, 23, 37, 78, 83, 91, 92, 100, 103, 104, 109, 120, 129, 135, 143, 148, 158], "structural_tag": [91, 143], "struggl": 62, "student": [47, 52, 53, 55, 97, 140, 142, 147], "studi": [14, 88, 93, 121, 123, 124, 125, 127], "studio": 34, "style": [12, 72, 92, 101, 109, 148], "sub": [11, 15, 112, 115, 129, 143], "subclass": [1, 11, 58, 92, 115, 143, 144, 149], "subcommad": 120, "subcommand": [21, 38, 148], "subcompon": [80, 160], "subdirectori": [37, 120], "subgraph": [103, 129], "subject": [3, 5, 6, 7, 32, 72, 78, 81, 98, 129, 141, 143, 158], "submiss": 120, "submit": [11, 25, 28, 29, 30, 31, 32, 49, 106, 120, 143], "submit_sync": 143, "submittransferrequest": 0, "submodul": [2, 17, 92, 94, 144, 148, 149], "suboptim": [20, 113], "subscript": 129, "subsequ": [8, 10, 11, 13, 19, 33, 35, 56, 94, 105, 106, 109, 121, 148], "subset": [0, 13, 37, 89, 91, 92, 99, 102, 113, 115, 120, 129, 143, 165], "substanti": [8, 12, 14, 16, 81, 98, 105, 109], "substitut": [34, 143], "substr": [33, 143], "subsystem": 148, "subtract": 103, "succe": [1, 8, 135, 146, 148], "succeed": 134, "success": [1, 4, 10, 12, 19, 25, 28, 29, 30, 31, 33, 38, 99, 143], "successfulli": [1, 57, 109, 117, 123], "suddenli": 11, "sudo": [2, 12, 37, 78, 97, 120, 158], "suffer": [12, 15, 19], "suffici": [11, 19, 88, 122, 123], "suffix": [11, 28, 29, 30, 31, 92, 143], "sugar": 11, "suggest": [7, 11, 15, 58, 62, 101, 123, 142], "suit": [8, 15, 16, 20, 23, 35, 37, 38, 83, 101, 120], "suitabl": [15, 16, 34, 56, 81, 86, 143], "sum": [1, 103, 111, 129, 166], "sum_": 8, "sum_of_token": 129, "summar": [5, 7, 15, 23, 38, 84, 85, 101, 109, 110, 111, 112, 120, 128, 135, 162], "summari": [9, 11, 15, 104, 109], "summat": 129, "sunjiabin17": 148, "sunset": 64, "super": [11, 56, 79, 80, 103, 111, 114, 115, 138, 139, 144, 145, 149, 159, 160, 167], "superchip": 138, "superior": 8, "superjomn": 64, "supplementari": 130, "suppli": [33, 58, 92, 106, 156], "support": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 14, 15, 20, 21, 22, 23, 25, 26, 30, 31, 33, 34, 35, 38, 42, 51, 59, 60, 61, 62, 64, 72, 75, 77, 80, 83, 84, 88, 89, 91, 92, 93, 96, 97, 98, 99, 100, 101, 102, 104, 105, 106, 107, 108, 109, 110, 112, 115, 116, 118, 121, 123, 126, 127, 128, 129, 130, 132, 139, 141, 142, 143, 144, 147, 148, 149, 150, 151, 154, 156, 157, 160, 164, 165, 166, 167], "supports_backend": 143, "supportsinflightbatch": 1, "suppos": [92, 144, 149], "suprem": [47, 97, 140, 142, 147], "sure": [2, 10, 13, 15, 17, 20, 28, 29, 30, 31, 37, 56, 75, 81, 88, 92, 94, 97, 98, 115, 117, 120, 128, 129, 142, 148, 154], "surfac": 32, "surpass": [72, 84, 101], "surprisingli": 10, "surround": [72, 101, 148], "survei": 10, "swa": 104, "swap": [15, 104], "swapcas": 143, "sweep": [4, 16, 29, 113, 126], "sweet": 126, "swept": [5, 10], "swiftli": [15, 18], "swiglu": [22, 129, 148], "switch": [4, 7, 12, 14, 16, 23, 25, 81, 85, 94, 100, 105, 107, 108, 128, 135, 148, 162], "sxm": [4, 22, 38, 121, 123, 124, 125], "sy": [19, 56, 148], "symbol": 0, "sync": 134, "sync_quant_config_with_kv_cache_config_dtyp": 143, "synchron": [1, 8, 10, 15, 19, 35, 56, 99, 113, 139, 143, 148], "syncmessag": 0, "syntact": 11, "syntax": [93, 129, 147], "synthet": [2, 21, 25, 37, 38, 45, 46, 120], "synthetic_128_128": [37, 73, 120, 152], "synthetic_2048_2048": 121, "synthetic_2048_2048_1000": 121, "synthetic_lora_data": [37, 120], "system": [2, 4, 8, 10, 11, 13, 14, 15, 17, 19, 20, 23, 25, 28, 29, 30, 31, 33, 38, 39, 40, 48, 49, 51, 56, 57, 59, 60, 61, 66, 67, 70, 75, 80, 87, 89, 93, 94, 104, 105, 113, 122, 138, 141, 146, 147, 148, 154, 156, 163], "system_prompt": 23, "systemat": [12, 15, 16], "t": [0, 1, 8, 9, 11, 12, 14, 15, 19, 26, 28, 29, 30, 31, 35, 36, 37, 42, 56, 58, 59, 60, 61, 65, 72, 81, 84, 88, 89, 92, 97, 98, 101, 109, 113, 115, 116, 119, 120, 122, 126, 127, 129, 131, 134, 139, 143, 148, 156], "t4": 20, "t5": [90, 101, 102, 137, 138, 148], "t_": 13, "t_2": 13, "t_5": 13, "tab": 143, "tabl": [0, 4, 7, 22, 38, 89, 92, 102, 105, 120, 129, 130, 134, 138, 139, 143, 145, 148], "tabsiz": 143, "tackl": 14, "tactic": [14, 22], "tag": [0, 1, 11, 28, 29, 30, 31, 33, 34, 91, 94, 97, 116, 143, 148], "tagentrymap": 1, "tail": 8, "tailor": [7, 123, 127], "take": [0, 1, 10, 11, 13, 15, 16, 17, 18, 20, 28, 29, 30, 31, 35, 62, 72, 81, 83, 88, 92, 98, 101, 102, 103, 105, 107, 112, 115, 120, 121, 122, 126, 129, 130, 143, 148, 166], "taken": [3, 4, 15, 114, 129], "talk": [15, 62], "tanh": [129, 130], "tar": 23, "target": [0, 1, 2, 8, 14, 15, 16, 18, 21, 22, 42, 58, 75, 80, 94, 114, 120, 127, 128, 143, 148, 154, 160], "target_input_len": 21, "target_model": 92, "target_module_nam": [80, 160], "target_output_len": 21, "targetcach": 1, "targetpageid": 1, "targetprob": 1, "targettaskvalu": 1, "tarot": 62, "tarot_lora_dir": 62, "task": [0, 1, 8, 10, 11, 15, 18, 19, 20, 21, 23, 28, 30, 31, 35, 37, 53, 54, 58, 59, 60, 61, 84, 85, 87, 105, 106, 109, 111, 112, 120, 130, 134, 137, 142, 143, 148, 162, 163, 166], "task_collect": 11, "task_handl": 11, "task_id": [37, 106, 120], "task_vocab_s": 130, "taskid": [0, 1], "taskidtyp": 1, "tasklayermoduleconfig": 1, "tasklayermoduleconfigbind": 1, "tasklayermoduleconfiglistptr": 1, "taskshost": 1, "taskstatu": 11, "taskvalu": 1, "taskvalueptr": 1, "taslid": 1, "tayef": 148, "tb": 143, "tconstptr": 1, "tcp": 117, "tdp": 38, "team": [8, 10, 11, 12, 13, 14, 15, 16, 19, 28, 29, 30, 31, 33, 81, 112, 115, 117, 138, 148], "teardown": 1, "tech": [13, 15, 16, 81, 148], "technic": [8, 13, 14, 15, 19, 89, 104], "techniqu": [3, 8, 10, 12, 13, 14, 15, 16, 50, 72, 83, 84, 85, 88, 90, 101, 103, 109, 113, 122, 123, 124, 125, 128, 137, 148, 162], "technologi": [12, 17, 47, 52, 53, 55, 56, 58, 97, 140, 142, 146, 147], "tediou": 33, "tee": 57, "tekit_2025": 120, "tell": [28, 29, 30, 31, 40, 56, 62, 64, 67, 92, 127, 147, 156], "temb": 130, "temp": [64, 134], "temperatur": [0, 1, 9, 11, 17, 20, 28, 30, 31, 37, 39, 40, 41, 42, 47, 52, 53, 54, 55, 58, 63, 64, 70, 75, 81, 91, 97, 102, 120, 121, 128, 134, 140, 142, 143, 147, 148, 154, 165], "tempfil": 56, "templat": [0, 1, 23, 25, 33, 80, 113, 114, 143, 160], "tempor": [8, 134], "temporari": [80, 81, 98, 160], "temporarydirectori": 56, "ten": [7, 10, 13, 15, 109], "tenant": 83, "tend": [11, 18, 128], "tensor": [1, 2, 3, 4, 5, 6, 10, 12, 13, 14, 15, 17, 18, 19, 20, 21, 23, 26, 28, 29, 30, 31, 38, 55, 56, 58, 77, 80, 81, 88, 90, 91, 98, 102, 107, 112, 113, 114, 120, 123, 124, 125, 127, 129, 130, 131, 134, 137, 139, 143, 144, 146, 148, 149, 151, 157, 160], "tensor_dict": 134, "tensor_input": 103, "tensor_parallel_s": [55, 59, 60, 61, 89, 121, 122, 123, 127, 128, 143], "tensor_shap": 114, "tensorconstptr": 1, "tensorflow": 20, "tensorinfo": 134, "tensorloc": 129, "tensormap": 1, "tensorparallel": [0, 1, 102], "tensorptr": [0, 1], "tensorrt": [1, 3, 6, 8, 12, 14, 21, 22, 23, 25, 26, 27, 32, 33, 34, 35, 36, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 72, 73, 75, 79, 80, 81, 83, 84, 88, 92, 95, 96, 97, 99, 101, 102, 103, 104, 106, 107, 110, 111, 118, 119, 123, 124, 125, 127, 128, 129, 134, 137, 139, 141, 142, 143, 144, 147, 149, 150, 151, 152, 154, 159, 164, 166, 167], "tensorrt_llm": [0, 1, 2, 10, 17, 25, 26, 28, 29, 30, 31, 32, 34, 35, 37, 38, 42, 47, 48, 49, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 72, 75, 77, 78, 80, 85, 88, 90, 91, 92, 94, 95, 97, 99, 101, 102, 103, 106, 108, 110, 111, 113, 114, 115, 116, 117, 120, 121, 123, 127, 128, 129, 130, 131, 132, 133, 134, 139, 140, 141, 142, 143, 144, 147, 148, 149, 150, 151, 154, 157, 158, 160, 162, 164, 165, 166], "tensorrt_llm_gpt": 113, "tensorrt_llm_rouge1_threshold": 112, "tensorrtllm_backend": [106, 148], "tensortrt": 94, "tep": [81, 98], "tep4": 16, "term": [11, 15, 16, 19, 33, 96, 97, 113, 128, 129, 142], "termin": [0, 9, 11, 28, 29, 30, 31, 38, 94, 105, 117, 147, 148], "test": [1, 7, 11, 12, 13, 16, 18, 19, 21, 23, 37, 38, 40, 67, 72, 77, 78, 79, 89, 92, 93, 94, 97, 101, 120, 121, 123, 124, 125, 126, 127, 128, 138, 143, 148, 156, 157, 158, 159, 166], "test_beam_search_larg": 148, "test_cli_flow": 148, "test_e2": 148, "test_generate_with_se": 148, "test_gpt_ib_ptun": 33, "test_graph_rewrit": 103, "test_list": 33, "test_llm_api": 32, "test_llm_openai_triton_1gpu": 33, "test_llm_qwen2audio_single_gpu": 33, "test_openai": 33, "test_qwen2audio": 33, "test_text": 56, "test_triton": 33, "test_trt_llm": [110, 111, 112], "testb": 8, "testgpt2": 148, "texec": [0, 81], "text": [0, 11, 16, 18, 21, 22, 26, 28, 30, 31, 32, 33, 35, 37, 38, 40, 42, 47, 48, 49, 50, 55, 56, 57, 63, 64, 65, 67, 72, 86, 91, 97, 99, 101, 102, 105, 120, 121, 128, 134, 139, 140, 142, 143, 147, 148], "text0": 56, "text1": 56, "text_complet": [28, 30, 31], "text_diff": 143, "text_hidden_s": 131, "text_to_token": 58, "textattack": [138, 145], "textprompt": 143, "tg_group": 129, "tgt": [113, 129], "tgt_len": [129, 130], "tgt_seq_len": 129, "th": [1, 13, 112, 129], "than": [0, 1, 3, 4, 5, 7, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 35, 37, 38, 57, 72, 81, 83, 84, 88, 89, 91, 94, 98, 99, 101, 102, 103, 105, 109, 113, 120, 121, 122, 123, 126, 128, 129, 134, 135, 139, 143, 148, 151], "thank": [8, 10, 13, 15, 148], "thankfulli": 10, "thecodewrangl": 148, "theft": 83, "thei": [0, 1, 10, 11, 12, 13, 14, 15, 19, 20, 25, 29, 33, 34, 37, 72, 80, 83, 86, 88, 92, 94, 99, 101, 102, 106, 113, 114, 115, 120, 121, 123, 126, 127, 128, 129, 131, 137, 143, 148], "them": [0, 2, 8, 11, 12, 13, 14, 15, 16, 18, 19, 20, 28, 35, 36, 56, 59, 60, 61, 64, 75, 80, 81, 88, 92, 99, 100, 103, 109, 110, 119, 120, 122, 124, 125, 126, 128, 129, 134, 135, 143, 144, 149, 154, 160], "themselv": 33, "theoret": [10, 15, 93, 135], "theori": [20, 128], "therebi": [81, 98, 128], "therefor": [2, 11, 18, 19, 38, 102, 110, 115, 129, 139, 166], "thermal": [37, 120], "theta": 129, "thi": [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 42, 44, 47, 48, 49, 51, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 70, 72, 75, 77, 78, 79, 80, 81, 83, 84, 85, 87, 88, 89, 90, 91, 92, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 134, 135, 137, 139, 140, 141, 142, 143, 144, 147, 148, 149, 150, 151, 154, 156, 157, 158, 159, 160, 162, 163, 166, 167], "thin": 115, "thing": [16, 47, 52, 53, 55, 88, 97, 102, 117, 126, 127, 140, 142, 147], "think": [11, 12, 13, 14, 92, 124, 125], "third": [10, 11, 16, 18, 33, 83, 96, 97, 99, 143, 148], "this_modul": 56, "thorough": [15, 20], "those": [2, 12, 13, 14, 15, 19, 22, 26, 33, 36, 72, 75, 78, 84, 88, 89, 99, 100, 101, 102, 112, 113, 119, 121, 127, 129, 130, 137, 143, 154, 158], "though": [10, 11, 13, 15, 16, 81, 88, 115, 126, 135], "thought": [11, 93], "thread": [0, 1, 10, 15, 19, 42, 72, 101, 107, 120, 134, 142, 143, 148], "three": [7, 8, 11, 12, 14, 16, 21, 72, 86, 89, 99, 112, 128, 129, 137, 143, 144, 149, 150, 151], "threshold": [0, 8, 12, 13, 32, 83, 129, 134, 143], "threw": 1, "throttl": [37, 120], "through": [0, 1, 2, 8, 10, 11, 12, 15, 16, 17, 19, 20, 22, 26, 29, 32, 72, 78, 80, 83, 88, 89, 92, 94, 101, 102, 103, 107, 108, 109, 113, 114, 120, 121, 122, 123, 126, 127, 130, 146, 147, 148, 158, 160], "throughout": [8, 19, 121, 124, 125], "throughput": [0, 3, 4, 5, 9, 10, 13, 15, 16, 17, 19, 25, 28, 29, 30, 31, 35, 36, 60, 63, 72, 73, 81, 85, 86, 87, 88, 89, 93, 98, 99, 101, 119, 123, 126, 127, 128, 148, 151, 152, 162, 163], "throw": [0, 1], "thrown": 1, "thu": [2, 10, 12, 14, 15, 19, 34, 92, 94, 105, 115, 129, 135], "thumb": [101, 122, 139], "ti": [13, 20, 72, 101], "tightli": 20, "tiiuae": [37, 120], "tile": 14, "tilen": 148, "time": [0, 1, 2, 5, 7, 8, 10, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23, 25, 28, 29, 30, 31, 35, 36, 37, 38, 47, 52, 53, 54, 55, 62, 81, 83, 84, 87, 89, 92, 93, 94, 97, 98, 99, 101, 105, 106, 107, 109, 110, 113, 119, 120, 121, 123, 124, 126, 128, 129, 134, 139, 140, 142, 143, 146, 147, 148, 163, 166], "time_": 8, "time_embed_dim": 130, "time_encod": 134, "time_i": 8, "time_point": 0, "timedelta": 143, "timedout": 0, "timelin": [10, 16, 81, 112, 148], "timeout": [0, 8, 15, 26, 42, 81, 98, 143, 148], "timeout_it": [8, 29, 143], "timepoint": 0, "timestamp": 0, "timestep": [130, 131], "timestepembed": 130, "timingmetr": 0, "tini": 62, "tinyllama": [16, 26, 35, 39, 41, 45, 47, 48, 49, 51, 52, 53, 54, 55, 58, 62, 63, 64, 66, 68, 70, 74, 75, 78, 79, 81, 97, 140, 142, 147, 153, 154, 158, 159], "tip": 93, "titl": [51, 91, 143], "titlecas": 143, "tle": 110, "tllm": [82, 143, 145, 161], "tllm_benchmark_req_queues_s": 19, "tllm_checkpoint_16gpu_tp8_pp2": 122, "tllm_ckpt_dir": 111, "tllm_engine_dir": 111, "tllm_kei": [114, 130], "tllm_llmapi_build_cach": 148, "tllm_llmapi_enable_nvtx": [36, 119], "tllm_log_level": [57, 139], "tllm_nvtx_debug": [19, 36, 119], "tllm_override_layer_num": 148, "tllm_profile_record_gc": [19, 36, 119], "tllm_profile_start_stop": [19, 36, 119], "tllm_to_externel_key_dict": 114, "tllm_torch_profile_trac": [36, 119], "tllm_trace_model_forward": 148, "tllm_weight": 114, "tllmruntim": [1, 102, 139], "tlntin": 148, "tma": 148, "tmp": [25, 28, 29, 30, 31, 36, 37, 60, 73, 106, 110, 119, 120, 122, 152], "tmp9so41y3r": [37, 120], "tmpowsrb_f4": [37, 120], "tmpxhdvasex": [37, 120], "to_arrai": 129, "to_dict": [131, 143], "to_json": 143, "to_json_fil": 131, "to_layer_quant_config": 131, "to_legacy_set": 132, "to_python": 143, "to_str": [0, 1, 99], "to_trt": 131, "tobyt": 1, "todo": [1, 29, 64, 129], "togeth": [3, 10, 11, 12, 16, 17, 20, 22, 64, 72, 88, 99, 101, 102, 106, 113, 134, 137, 146, 148], "toggl": [36, 57, 83, 119], "toi": [88, 126], "toitensor": 0, "tojsonstr": 0, "tok": [3, 5, 6, 16, 25, 28, 29, 30, 31, 93, 127], "token": [0, 1, 2, 3, 6, 7, 9, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 35, 36, 37, 38, 45, 46, 49, 51, 56, 57, 58, 60, 64, 72, 73, 74, 75, 81, 83, 84, 86, 88, 89, 92, 93, 99, 100, 101, 102, 104, 105, 109, 113, 116, 119, 120, 121, 123, 124, 127, 129, 130, 134, 135, 137, 143, 144, 148, 149, 150, 151, 152, 153, 154], "token_count": 58, "token_drop": 130, "token_end": 143, "token_id": [28, 30, 31, 42, 91, 143], "token_ids_diff": 143, "token_norm_dist": 93, "token_range_retention_config": [49, 143], "token_start": 143, "token_type_id": [131, 134], "token_unif_dist": 93, "tokenend": 0, "tokenextraid": 1, "tokenextraidtyp": 1, "tokenid": 1, "tokenidtyp": [0, 1], "tokenization_utils_bas": 143, "tokenized_request": [28, 30, 31], "tokenizer_dir": [111, 113, 139, 143], "tokenizer_image_token": 134, "tokenizer_max_seq_length": [123, 131, 133, 143], "tokenizer_mod": 143, "tokenizer_revis": 143, "tokenizer_str": [0, 99], "tokenizerbas": 143, "tokenizerstr": [0, 99], "tokenlogprob": 143, "tokenrangeretentionconfig": [0, 49, 83, 143], "tokenrangeretentionprior": 0, "tokens_": 8, "tokens_i": 8, "tokens_per_block": [22, 56, 57, 77, 84, 104, 105, 134, 143, 148, 157, 166], "tokensperblock": [0, 1, 26, 102], "tokensperstep": 1, "tokensprompt": 143, "tokenstart": 0, "tokyo": [40, 67], "toler": [7, 15, 148], "toml": [80, 160], "tomodulenam": 1, "tomoduletyp": 1, "tonylek": 148, "too": [0, 2, 10, 14, 15, 19, 26, 48, 81, 92, 98, 99, 101, 121, 126, 134, 139, 143], "took": 121, "tool": [2, 10, 11, 14, 15, 19, 20, 21, 28, 29, 30, 31, 34, 83, 86, 89, 112, 120, 146, 147, 148], "tool_cal": [17, 20, 29, 147], "toolcal": 11, "toolkit": [7, 12, 17, 20, 28, 29, 30, 31, 97, 115, 150], "toolset": 148, "top": [0, 8, 9, 10, 13, 14, 15, 16, 30, 31, 33, 35, 64, 72, 81, 89, 91, 92, 100, 101, 102, 109, 113, 129, 143, 148, 165], "top1": 12, "top_k": [9, 64, 75, 91, 102, 134, 143, 148, 154, 165], "top_k_valu": 64, "top_p": [9, 11, 20, 29, 47, 52, 53, 54, 55, 58, 63, 64, 91, 97, 102, 121, 128, 134, 140, 142, 143, 147, 165], "top_p_decai": [134, 143], "top_p_min": [134, 143], "top_p_reset_id": [134, 143], "top_p_valu": 64, "topenkoff": 148, "topic": [15, 25, 127], "topk": [0, 1, 12, 14, 19, 100, 102, 109, 129, 148], "topk_logit": 99, "topklastdim": 129, "topklogit": 99, "topkmedusahead": 1, "topktopp": [0, 102], "topmodelmixin": [115, 131], "topn": 12, "topologi": [15, 19], "topp": [0, 1, 102, 148], "toppdecai": [0, 1, 102], "toppmin": [0, 1, 102, 143], "toppresetid": [0, 1, 102], "torch": [10, 56, 58, 72, 73, 74, 75, 77, 78, 79, 80, 82, 91, 94, 97, 101, 114, 120, 129, 134, 139, 143, 144, 145, 148, 149, 152, 153, 154, 156, 157, 158, 159, 160, 161], "torch_compile_config": [143, 148], "torchaudio": 97, "torchcompileconfig": [93, 143], "torchllmarg": [25, 28, 29, 30, 31, 56, 93, 143, 148], "torchsampl": [9, 92, 143], "torchvis": 97, "toronto": 29, "tostr": [0, 1], "total": [0, 1, 2, 8, 10, 13, 15, 16, 19, 21, 22, 23, 25, 26, 28, 29, 30, 31, 33, 37, 38, 72, 88, 100, 101, 102, 109, 112, 114, 120, 121, 122, 135, 143, 166], "total_lat": [3, 6], "total_token": [17, 20, 28, 29, 30, 31, 147], "totalaccepteddrafttoken": 0, "totaldrafttoken": 0, "totalgentoken": 1, "totalnumpag": 1, "totensor": 0, "touch": [116, 144, 149], "toward": [15, 87, 92, 163], "tp": [0, 2, 3, 4, 5, 6, 7, 12, 13, 14, 15, 16, 17, 20, 21, 26, 37, 38, 60, 73, 81, 98, 100, 102, 106, 113, 120, 121, 129, 143, 146, 148, 152], "tp1": [3, 4, 5, 10, 38], "tp2": [16, 38, 81, 120, 148], "tp4": [10, 12, 29, 38], "tp4ep2": 12, "tp8": [5, 12, 14, 29, 38], "tp8ep2": 12, "tp_1_pp_1": 120, "tp_dim": [114, 130], "tp_group": [129, 130], "tp_rank": [114, 129, 130], "tp_size": [9, 17, 20, 21, 23, 25, 26, 38, 43, 59, 61, 100, 106, 112, 113, 114, 115, 120, 122, 129, 130, 133, 148], "tp_split_dim": 130, "tpot": [2, 6, 8, 16, 25, 28, 29, 30, 31, 38, 81], "tprank": 1, "tps_": 8, "tpsize": 1, "tqdm": [114, 143, 148], "trace": [15, 21, 22, 23, 26, 36, 93, 115, 119, 139], "track": [1, 15, 32, 34, 72, 78, 88, 101, 104, 129, 143, 158], "trade": [2, 14, 20, 35, 92, 105], "tradeoff": [7, 12, 13, 93, 123], "tradit": [0, 8, 11, 89], "traffic": [15, 16, 73, 81, 83, 89, 152, 156], "trail": 143, "train": [4, 7, 11, 13, 18, 29, 92, 109, 111, 112, 113, 115, 120, 129, 139, 144, 149], "trainabl": [85, 162], "trait": 148, "trampolin": 10, "transa": 129, "transb": 129, "transceiv": [0, 143], "transfer": [0, 14, 15, 16, 19, 81, 98, 113, 143, 148], "transfer_mod": [83, 143], "transferdesc": 0, "transfermod": 0, "transferop": 0, "transferrequest": 0, "transferstatu": 0, "transform": [0, 19, 21, 22, 23, 26, 35, 42, 58, 72, 75, 78, 79, 80, 86, 88, 100, 101, 109, 111, 112, 113, 114, 131, 135, 138, 139, 143, 144, 148, 149, 150, 154, 158, 159, 160, 166], "transformerstoken": 143, "transit": [8, 11], "translat": [19, 20, 34, 78, 85, 128, 143, 148, 158, 162], "transmiss": [16, 19, 81, 98, 107], "transmit": [81, 98, 107], "transpar": [10, 15, 18, 19], "transparent_hugepag": 19, "transpos": [1, 112, 129], "transposit": 129, "travel": 29, "travers": [10, 113], "treat": [12, 19, 72, 84, 101, 129], "tree": [0, 9, 10, 11, 21, 29, 83, 92, 95, 120, 134, 139, 166], "tremend": 10, "trend": 18, "tri": [14, 167], "tricki": 131, "trigger": [10, 15, 19, 22, 32, 42, 72, 80, 93, 101, 103, 113, 142, 143], "trigger_completion_at_end": 129, "trim": 1, "trimpool": 1, "triton": [10, 33, 75, 77, 79, 93, 105, 106, 109, 113, 143, 146, 148, 154, 157, 159], "triton_serv": 33, "tritonserv": 148, "trivial": 113, "troubleshoot": [93, 148], "trt": [0, 4, 11, 21, 26, 44, 72, 77, 78, 79, 81, 88, 92, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 113, 114, 116, 120, 126, 129, 131, 133, 134, 135, 139, 148, 151, 154, 157, 158, 159], "trt_ckpt": [106, 110, 112, 139], "trt_engin": [106, 110, 112, 139], "trt_llm_data": 34, "trt_llm_disable_load_weights_in_parallel": 17, "trt_root": 2, "trt_tensor": [113, 129], "trtdatatyp": 1, "trtgptmodel": 135, "trtgptmodeloptionalparam": 148, "trtgptmodelv1": 148, "trtllm": [2, 9, 11, 13, 15, 17, 19, 20, 28, 29, 30, 31, 34, 37, 38, 39, 40, 41, 42, 43, 45, 46, 50, 59, 66, 67, 68, 69, 70, 74, 79, 80, 84, 89, 93, 98, 105, 106, 110, 111, 112, 113, 115, 120, 123, 124, 125, 126, 127, 135, 139, 143, 148, 153, 158, 159, 160], "trtllm_deep_ep_token_limit": 148, "trtllm_dg_jit_use_nvcc": 2, "trtllm_disable_kv_cache_transfer_overlap": [81, 98], "trtllm_disable_unified_convert": 114, "trtllm_enable_kvcache_receive_parallel": [81, 98], "trtllm_enable_mmha_multi_block_debug": 120, "trtllm_enable_pdl": [2, 9, 12, 13, 17, 20, 120], "trtllm_force_xqa": [72, 101], "trtllm_kvcache_send_max_concurrency_num": [81, 98], "trtllm_kvcache_transfer_buffer_s": [81, 98], "trtllm_kvcache_transfer_use_async_buff": [81, 98], "trtllm_llama_eager_fusion_dis": 148, "trtllm_mmha_blocks_per_sequ": 120, "trtllm_mmha_kernel_block_s": 120, "trtllm_model": 114, "trtllm_modules_to_hf_modul": [37, 85, 120, 134, 162], "trtllm_pdl_overlap_ratio": 120, "trtllm_precompiled_loc": 94, "trtllm_prefetch_ratio": 120, "trtllm_request_kv_cache_concurr": [81, 98], "trtllm_try_zcopy_for_kvcache_transf": [81, 98], "trtllm_use_precompil": 94, "trtllmarg": [93, 143], "trtllmattent": [93, 151], "trtllmattentionwrapp": 148, "trtllmsampler": 143, "trtllmworker": 11, "trtlmmdatatyp": 0, "true": [0, 1, 2, 8, 9, 11, 12, 13, 14, 15, 17, 18, 20, 25, 26, 28, 29, 30, 31, 36, 38, 42, 54, 56, 57, 58, 60, 63, 64, 65, 72, 74, 75, 81, 84, 87, 89, 91, 92, 99, 102, 103, 105, 109, 112, 119, 120, 123, 127, 129, 130, 131, 132, 134, 135, 139, 143, 148, 153, 154, 156, 163], "true_output_valu": 129, "true_valu": 129, "truli": 19, "truncat": [143, 148], "truncate_prompt_token": [143, 148], "truncation_sid": 11, "trust": [14, 21, 143], "trust_remote_cod": [9, 11, 17, 20, 21, 23, 25, 26, 143, 148], "truth": [78, 158], "try": [0, 1, 15, 17, 18, 19, 20, 28, 29, 30, 31, 38, 48, 62, 70, 83, 86, 95, 99, 111, 115, 123, 126, 127, 128, 135, 139, 141, 142, 143, 147, 164], "tsuji": [37, 120], "ttensor": 1, "ttft": [8, 16, 25, 28, 29, 30, 31, 37, 38, 81, 88, 123, 126, 127, 128, 148], "ttim": 148, "ttl": 12, "tunabl": [124, 125], "tune": [0, 4, 7, 8, 12, 14, 15, 16, 20, 21, 22, 25, 37, 38, 85, 88, 89, 92, 93, 99, 109, 120, 123, 125, 127, 130, 131, 134, 135, 143, 146, 148, 156, 162], "tuner": 0, "tupl": [0, 1, 56, 129, 130, 134, 143, 167], "turn": [10, 14, 16, 20, 84, 94, 101, 102, 105, 109, 123, 134, 135, 148], "turn1": 18, "turn2": 18, "turnaround": 33, "tushar": 148, "tutori": 25, "tweak": 128, "twice": [10, 113], "two": [0, 4, 8, 9, 11, 12, 13, 14, 15, 16, 18, 19, 21, 22, 26, 29, 32, 33, 37, 40, 57, 67, 72, 80, 81, 82, 83, 84, 88, 89, 91, 93, 94, 98, 99, 100, 101, 102, 103, 105, 106, 107, 109, 110, 112, 113, 115, 120, 121, 123, 126, 128, 129, 130, 132, 142, 143, 145, 147, 148, 150, 160, 161, 165, 166, 167], "twofold": 109, "twoshot": [107, 129, 143], "txt": [2, 20, 21, 23, 33, 36, 37, 60, 73, 115, 119, 120, 121, 148, 152], "type": [1, 4, 7, 9, 11, 14, 16, 17, 20, 21, 22, 25, 26, 28, 29, 30, 31, 33, 37, 39, 40, 41, 45, 46, 51, 56, 58, 62, 63, 64, 65, 67, 70, 72, 74, 79, 80, 81, 83, 88, 90, 91, 92, 93, 98, 99, 101, 102, 103, 106, 112, 113, 120, 123, 127, 129, 131, 133, 134, 137, 138, 139, 143, 144, 147, 148, 149, 150, 151, 153, 156, 159, 160, 166], "typedef": [0, 1], "typeerror": 143, "typenam": [0, 1, 113], "typetrait": 0, "typic": [0, 7, 13, 14, 15, 16, 19, 25, 26, 28, 29, 30, 31, 81, 86, 98, 103, 111, 113, 115, 122, 123, 127, 128, 132, 134, 135, 142, 144, 148, 149], "typo": 148, "u": [1, 8, 14, 15, 16, 28, 29, 30, 31, 37, 38, 47, 52, 53, 54, 55, 97, 103, 116, 120, 140, 142, 143, 147, 148], "ub": [107, 129, 143], "ub_oneshot": 120, "ub_tp_siz": 120, "ubuntu": [97, 141, 148], "uc_handl": 1, "uc_ptr": 1, "uc_va": 1, "ucsd": 11, "ucx": [0, 16, 81, 98, 143, 148], "ucx_cuda_ipc_enable_mnnvl": [81, 98], "ucx_max_rndv_rail": [81, 98], "ucx_net_devic": [81, 98], "ucx_rndv_schem": [81, 98], "ue8m0": 90, "uid": [0, 34, 134], "uint16_t": 0, "uint32": 1, "uint32_t": [0, 1, 129], "uint64": [1, 105], "uint64_t": [0, 1], "uint8": 1, "uint8_t": [0, 1], "uintptr_t": [0, 1], "uk": 14, "uk_bgemm": 12, "ulimit": [9, 17, 20, 25, 94, 139, 147], "ultim": 122, "ultra": [79, 159], "ulyss": 148, "unabl": 126, "unaccept": 123, "unaffect": 19, "unari": 129, "unaryoper": 129, "unbind": 129, "unblock": [10, 19], "uncas": [138, 143, 145], "uncertain_word": 11, "uncertainti": 109, "unchang": [15, 33, 75, 109, 127, 129, 154], "uncom": 34, "uncommon": 113, "undefin": 129, "under": [0, 7, 8, 9, 16, 18, 19, 20, 22, 30, 31, 32, 33, 37, 38, 78, 81, 84, 86, 94, 120, 139, 142, 143, 148, 158], "undergo": [78, 79, 158, 159], "underli": [0, 1, 15, 16, 80, 81, 103, 109, 143, 160], "underlying_type_t": 1, "underlyingtyp": [0, 1], "underscor": 123, "understand": [8, 15, 33, 36, 37, 75, 88, 89, 94, 119, 154], "understood": [126, 143], "underutil": [19, 109], "underwai": 16, "uneven": [8, 148], "unevenli": 12, "unexpect": [10, 19, 139, 143, 148], "unfinish": 0, "unfortun": 19, "unfus": [129, 148], "unfuse_qkv_project": 131, "ungath": 1, "unguid": 51, "unicast": 1, "unicastconfigur": 1, "unicod": 143, "unicodeencodeerror": 143, "unif": 148, "unifi": [7, 23, 78, 80, 112, 115, 148, 158, 160], "uniform": [8, 21, 37, 38, 120, 129], "uniformli": 8, "uniniti": [72, 151], "union": [129, 143], "uniqu": [0, 1, 22, 37, 80, 84, 101, 102, 104, 106, 109, 112, 120, 143, 160], "unique_ptr": [0, 1], "uniqueconstptr": 1, "uniqueptr": 1, "uniquetoken": [0, 1], "unit": [1, 14, 17, 20, 28, 29, 30, 31, 32, 37, 47, 48, 49, 56, 93, 94, 97, 104, 114, 120, 121, 128, 140, 142, 147, 148], "unittest": [32, 33], "univers": [8, 47, 52, 53, 55, 97, 140, 142, 147], "unknown": [1, 21, 143], "unleash": 18, "unless": [0, 32, 42, 88, 122, 127, 128, 143], "unlik": [13, 75, 91, 105, 109, 154], "unlock": 15, "unnecessari": [10, 11, 103, 144, 148, 149, 167], "unneed": [12, 72, 101], "unordered_map": [0, 1, 99], "unpack": 10, "unpatchifi": 131, "unpredict": 8, "unsaf": [81, 98], "unsaferemov": 1, "unsatisfactori": 19, "unschedul": [88, 126], "unset": [15, 81, 98, 128], "unsign": 1, "unspecifi": [22, 23, 26, 129], "unsqueez": [1, 129], "unstabl": [115, 143], "unsupport": [33, 148], "untest": [82, 145, 161], "until": [0, 1, 8, 10, 11, 15, 25, 28, 29, 30, 31, 83, 92, 99, 102, 105, 109, 143], "untouch": [129, 143], "unus": [0, 37, 120], "up": [0, 2, 4, 5, 8, 9, 10, 12, 13, 14, 15, 16, 17, 19, 20, 21, 22, 26, 28, 29, 30, 31, 35, 37, 51, 56, 58, 72, 80, 81, 88, 92, 93, 98, 101, 102, 106, 109, 120, 126, 127, 143, 147, 148, 160, 166], "up_proj": 114, "upcast": 129, "upcast_attent": 130, "upcast_softmax": 130, "upcom": [7, 78, 158, 166], "updat": [0, 2, 5, 10, 13, 14, 15, 17, 18, 22, 35, 78, 80, 86, 87, 89, 94, 104, 109, 113, 114, 115, 116, 129, 134, 139, 143, 158, 160, 163, 166], "update_forward_ref": 143, "update_from_dict": 143, "update_key_map": 114, "update_kv_cache_typ": 143, "update_output_ids_by_offset": 134, "update_resourc": [150, 166], "update_state_after_alloc": 56, "update_strategi": 129, "updatenumreturnbeam": 0, "updatespositionid": 1, "upgrad": [78, 97, 148, 158], "uplift": [123, 126, 127], "upon": [1, 10, 17, 19, 35, 38, 80, 109, 127, 139, 148], "upper": [8, 65, 120, 129, 135, 143], "uppercas": 143, "uq_qr_gemm": 12, "url": [16, 26, 40, 45, 46, 67, 81, 94, 97, 148], "us": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 19, 21, 22, 23, 25, 26, 28, 29, 30, 31, 32, 35, 36, 37, 38, 42, 44, 47, 49, 50, 51, 54, 56, 57, 59, 60, 61, 62, 64, 72, 73, 74, 75, 76, 77, 78, 79, 81, 83, 85, 86, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 104, 105, 107, 108, 112, 113, 114, 115, 116, 117, 119, 120, 121, 122, 123, 124, 125, 126, 129, 130, 131, 132, 134, 137, 139, 140, 141, 143, 144, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 162, 165, 166, 167], "usabl": [11, 19, 141, 143], "usag": [0, 3, 6, 14, 16, 17, 19, 20, 21, 22, 26, 28, 29, 30, 31, 32, 47, 72, 83, 84, 89, 93, 95, 98, 101, 103, 104, 113, 115, 120, 127, 128, 129, 140, 142, 143, 147, 148, 151], "use_beam_hyp": 134, "use_beam_search": [91, 143, 148], "use_cach": [129, 130, 131], "use_context_fmha_for_gener": 148, "use_custom_all_reduc": 148, "use_diff_of_squar": 129, "use_dynamic_tre": 143, "use_embedding_shar": 148, "use_fast": 11, "use_fp32_acc": 129, "use_fp8": 130, "use_fp8_context_fmha": [22, 101, 120, 148], "use_fused_mlp": [22, 120, 148], "use_gemm_allreduce_plugin": 134, "use_gpt_attention_plugin": 134, "use_gpu_direct_storag": 134, "use_implicit_relative_attent": 130, "use_kv_cach": [130, 134, 148], "use_logn_sc": 130, "use_lora": 131, "use_lora_plugin": 134, "use_low_precision_moe_combin": 143, "use_mamba_conv1d_plugin": 134, "use_meta_recip": 143, "use_modelopt_quant": 115, "use_mrop": 143, "use_mtp_vanilla": 143, "use_one_more_block": 134, "use_paged_context_fmha": [22, 72, 101, 105, 120, 123, 127], "use_parallel_embed": [112, 113, 131], "use_preload": 131, "use_prompt_tun": [131, 148], "use_py_sess": 139, "use_refit": 143, "use_relaxed_acceptance_for_think": [12, 13, 65, 92, 143], "use_runtime_default": 134, "use_safetensors_load": 131, "use_strip_plan": 143, "use_torch_sampl": 9, "use_tqdm": 143, "use_uvm": [83, 143], "use_variable_beam_width_search": 134, "usebantoken": 0, "usebanword": 0, "usecrossattent": 1, "usedefaultvalu": 1, "usednumblock": [0, 26], "usedraftlogit": 1, "usedraftlogitshost": 1, "usedynamictre": 0, "usedynamictreehost": 1, "useexpliciteosstop": 0, "usefrequencypenalti": 0, "usegemmallreduceplugin": 1, "usegptattentionplugin": [1, 102], "usegpudirectstorag": 0, "uselanguageadapt": 1, "useloraplugin": 1, "usemambaconv1dplugin": 1, "usemaxlengthstop": 0, "useminlen": 0, "useminlength": 0, "useminp": 0, "usemrop": 1, "usenorepeatngrams": 0, "useoccurrencepenalti": 0, "usepackedinput": 1, "usepagedst": 1, "usepenalti": 0, "usepositionembed": 1, "usepresencepenalti": 0, "useprogthread": 0, "useprompttun": 1, "user": [0, 2, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 25, 26, 28, 29, 30, 31, 32, 34, 35, 36, 37, 38, 39, 40, 51, 66, 67, 70, 72, 75, 80, 81, 83, 84, 88, 89, 94, 96, 98, 99, 101, 102, 103, 105, 106, 107, 108, 113, 114, 115, 116, 119, 120, 126, 127, 128, 129, 131, 135, 137, 139, 142, 143, 146, 147, 148, 154, 156, 160], "user_buff": [22, 123], "user_provid": [131, 143], "userandomacceptancethreshold": 1, "userbuff": [143, 148], "userepetitionpenalti": 0, "usernam": [9, 17], "userprovideddecodingconfig": [92, 93, 143], "useshapeinfer": 1, "usespecdecod": 1, "usestopword": 0, "usetemp": 0, "usetemperatur": 0, "usetokentypeembed": 1, "useuvm": 0, "usevariablebeamwidthsearch": 0, "using_oss_cutlass_": 108, "using_oss_cutlass_low_latency_gemm": 108, "using_oss_cutlass_moe_gemm": 108, "usr": [2, 29, 39, 40, 41, 43, 45, 46, 112, 120], "usual": [13, 38, 84, 92, 97, 113, 115, 121, 127, 129, 143, 166], "utf": 143, "utf8": 143, "util": [0, 1, 2, 3, 8, 11, 12, 14, 15, 16, 17, 20, 22, 23, 26, 28, 29, 30, 31, 35, 36, 37, 47, 63, 72, 73, 75, 81, 84, 86, 87, 88, 92, 101, 102, 109, 113, 119, 120, 123, 127, 128, 135, 143, 146, 148, 151, 152, 154, 163], "uv": 14, "uv_gemm": 12, "uvicorn": 25, "uvm": [0, 1, 19, 143], "v": [1, 2, 3, 4, 7, 9, 11, 12, 14, 17, 18, 20, 28, 29, 30, 31, 32, 34, 72, 88, 93, 101, 102, 106, 129, 134, 137, 138, 139, 144, 145, 149, 151], "v0": [3, 4, 5, 6, 37, 38, 79, 87, 88, 106, 120, 138, 145, 148, 159, 163], "v1": [9, 16, 17, 20, 26, 28, 29, 30, 31, 35, 38, 39, 40, 41, 45, 47, 48, 49, 51, 52, 53, 54, 55, 58, 62, 63, 64, 66, 67, 68, 69, 70, 74, 75, 78, 79, 81, 85, 97, 138, 140, 142, 145, 147, 148, 153, 154, 156, 158, 159, 162], "v10": 148, "v100": 148, "v12": 148, "v2": [7, 14, 32, 79, 80, 90, 137, 138, 148, 159, 160], "v3": [8, 13, 15, 26, 36, 89, 92, 119, 137, 138, 145, 148], "v9": 5, "v_dim": 129, "v_head_dim": [129, 130], "v_proj": [37, 80, 85, 114, 120, 144, 149, 160, 162], "vacat": [47, 97, 140, 142, 147], "valid": [0, 1, 8, 10, 13, 15, 19, 28, 29, 30, 31, 38, 75, 79, 80, 81, 84, 89, 92, 99, 109, 129, 134, 143, 148, 154, 156, 159, 160], "validate_and_init_token": 143, "validate_attention_dp_config": 143, "validate_auto_parallel": 143, "validate_batch_wait_max_tokens_ratio": 143, "validate_batch_wait_timeout_it": 143, "validate_batch_wait_timeout_m": 143, "validate_build_config_remain": 143, "validate_build_config_with_runtime_param": 143, "validate_capture_num_token": 143, "validate_checkpoint_format": 143, "validate_cuda_graph_config": 143, "validate_cuda_graph_max_batch_s": 143, "validate_dtyp": 143, "validate_enable_build_cach": 143, "validate_gpus_per_nod": 143, "validate_kv_cache_dtyp": 143, "validate_load_balanc": 143, "validate_lora_config_consist": 143, "validate_max_attention_window": 143, "validate_max_gpu_total_byt": 143, "validate_model": 143, "validate_model_format_misc": 143, "validate_parallel_config": 143, "validate_peft_cache_config": 143, "validate_positive_valu": 143, "validate_quant_config": 143, "validate_runtime_arg": 143, "validate_speculative_config": 143, "validate_stream_interv": 143, "validate_torch_compile_max_num_stream": 143, "validatevec": 1, "validationerror": 143, "validmpiconfig": 1, "valu": [0, 1, 2, 3, 4, 8, 10, 11, 13, 14, 16, 18, 19, 20, 21, 22, 23, 25, 26, 28, 29, 30, 31, 32, 33, 35, 37, 42, 57, 58, 64, 72, 74, 75, 81, 83, 84, 86, 88, 92, 98, 101, 102, 104, 105, 106, 107, 110, 112, 113, 114, 120, 121, 123, 126, 128, 129, 131, 132, 133, 134, 135, 137, 139, 143, 148, 151, 153, 154, 166, 167], "valuabl": [8, 12, 15, 16], "value_typ": 0, "valueerror": [65, 143], "valuestatu": 1, "vanilla": [72, 101, 143, 148, 151], "vanillaattent": [72, 151], "var": 129, "vari": [5, 8, 11, 15, 16, 17, 19, 25, 28, 29, 30, 31, 81, 126, 127, 145, 166], "variabl": [0, 1, 2, 5, 12, 15, 16, 19, 28, 34, 36, 37, 59, 60, 61, 76, 83, 93, 97, 102, 104, 114, 119, 120, 139, 142, 143, 148, 155], "variabledraftlength": 1, "varianc": [8, 14, 123, 126, 127, 129], "variant": [0, 3, 10, 13, 14, 20, 38, 72, 92, 99, 101, 115, 129, 148, 151], "variat": 8, "varieti": [37, 79, 86, 90, 120, 121, 148, 159], "variou": [11, 15, 16, 19, 21, 25, 34, 37, 38, 64, 72, 74, 78, 79, 80, 86, 94, 101, 109, 120, 123, 126, 142, 143, 146, 147, 148, 153, 158, 159], "varnam": 1, "vartyp": 1, "vastli": 8, "vboost": [2, 12, 37, 120], "vbw": 148, "ve": [12, 62], "vec": [0, 1], "vec2": 129, "veclogprob": 0, "vectoken": 0, "vectokenextraid": [0, 1], "vector": [0, 1, 14, 99, 101, 102, 104, 106, 129], "vecuniquetoken": [0, 1], "vehicl": 17, "verbatim": 131, "verbos": [21, 22, 23, 26, 37, 76, 120, 143, 155], "veri": [7, 13, 15, 17, 18, 19, 20, 28, 29, 30, 31, 57, 72, 88, 92, 101, 112, 113, 121, 122, 123, 148], "verif": [0, 10, 11, 13, 18, 109, 143], "verifi": [9, 10, 13, 17, 18, 19, 20, 23, 33, 57, 109, 127, 129, 148], "verification_batch": 18, "verificationsets": 0, "vermont": 29, "versa": [14, 105], "version": [0, 1, 2, 10, 12, 14, 15, 19, 20, 23, 26, 28, 30, 31, 32, 34, 37, 42, 72, 81, 83, 88, 92, 94, 95, 98, 101, 102, 112, 114, 115, 120, 121, 129, 139, 141, 143, 148], "versu": 11, "vertic": 129, "vertical_strid": 130, "vgqa": 104, "via": [0, 10, 12, 15, 16, 19, 20, 25, 28, 29, 30, 31, 33, 37, 59, 60, 61, 62, 75, 77, 78, 79, 81, 92, 94, 96, 107, 108, 109, 120, 123, 124, 125, 127, 128, 129, 141, 143, 148, 154, 157, 158, 159], "vice": [14, 105], "vicuna": 109, "video": [21, 26, 37, 40, 67, 120, 134, 138, 145, 148], "video_grid_thw": 134, "video_path": 134, "video_preprocess": 134, "video_url": [26, 40, 67], "view": [1, 10, 13, 15, 17, 75, 129, 134, 154], "vila": [40, 67, 90, 136, 137, 138, 145, 148], "vinyl": [37, 120], "violat": [10, 148], "virtual": [0, 1, 130], "virtualmemorymanagertest": 1, "vision": [20, 21, 78, 86, 90, 134, 137, 138, 143, 145, 148, 158], "vision_grid_thw": 134, "vision_length": 129, "vision_model_typ": 131, "vision_start": 129, "vision_token_mask": 130, "visit": [12, 78, 109, 148, 158], "visual": [8, 17, 34, 88, 126, 143, 148], "visual_engine_dir": 134, "visual_featur": 134, "visualize_network": [22, 143, 148], "vit": [143, 148], "vital": [7, 103], "vl": [25, 26, 37, 40, 46, 67, 86, 120, 136, 138, 148], "vllm": [10, 11, 148], "vlm": [78, 138, 148, 158], "vocab": [92, 129, 134], "vocab_embed": [111, 114], "vocab_s": [0, 10, 112, 114, 130, 131, 134, 143, 144, 149], "vocab_size_pad": 134, "vocabs": [1, 102], "vocabsizepad": [0, 1], "vocabulari": [0, 1, 10, 38, 102, 105, 109, 130, 134], "void": [0, 1, 99, 113], "volatil": 8, "volta": 148, "volum": [1, 8, 37, 93, 94, 107, 120], "volumenonneg": 1, "vonjackustc": 148, "vote": [47, 97, 140, 142, 147], "vram": 57, "vswa": 104, "vulner": 148, "vultureprim": 148, "w": [1, 2, 6, 9, 10, 12, 14, 17, 18, 20, 21, 26, 28, 29, 30, 31, 81, 129, 131, 137, 138, 148], "w1": 129, "w4a": [137, 148], "w4a16": [7, 21, 90, 112, 131, 143], "w4a16_awq": [21, 42, 112, 115, 143], "w4a16_gptq": [21, 112, 143], "w4a16_mxfp4": 143, "w4a8": [7, 90, 148], "w4a8_awq": [21, 112, 115, 143], "w4a8_mxfp4_fp8": [143, 148], "w4a8_mxfp4_mxfp8": 143, "w4a8_nvfp4_fp8": 143, "w4a8_qserve_per_channel": 143, "w4a8_qserve_per_group": 143, "w4aint8": 148, "w8a": 137, "w8a16": [7, 21, 112, 131, 143], "w8a16_gptq": 143, "w8a8": [4, 7], "w8a8_sq_per_channel": [112, 143], "w8a8_sq_per_channel_per_tensor_plugin": [131, 143], "w8a8_sq_per_channel_per_token_plugin": [131, 143], "w8a8_sq_per_tensor_per_token_plugin": [131, 143], "w8a8_sq_per_tensor_plugin": [131, 143], "wa": [0, 1, 8, 10, 13, 14, 15, 19, 29, 37, 38, 57, 72, 83, 88, 95, 97, 99, 101, 102, 112, 120, 121, 123, 126, 127, 128, 130, 137, 139, 143, 144, 148, 149, 167], "wai": [10, 11, 12, 13, 14, 15, 16, 18, 19, 37, 55, 56, 58, 72, 81, 88, 92, 95, 96, 101, 102, 103, 107, 120, 121, 123, 129, 135, 142, 148], "wait": [0, 1, 10, 11, 14, 15, 19, 25, 35, 42, 87, 99, 115, 120, 129, 143, 146, 163], "wait_for_layer_load": 56, "wait_for_sav": 56, "waiv": 93, "walk": [17, 20, 40, 62, 67, 121, 122, 123], "wang1120": 148, "wangkuiyi": 148, "want": [2, 11, 12, 13, 15, 17, 19, 20, 28, 29, 30, 31, 36, 37, 56, 72, 80, 84, 88, 94, 101, 109, 115, 117, 119, 120, 123, 126, 128, 129, 139, 143, 144, 148, 149], "war": 1, "warm": [21, 81, 98, 166], "warmup": [2, 15, 19, 20, 21, 36, 72, 119, 120, 121, 148, 151, 166], "warn": [21, 22, 23, 25, 26, 32, 38, 58, 72, 76, 101, 120, 135, 143, 155], "warn_on_unstable_feature_usag": 143, "warp": [107, 148], "washington": 29, "wast": [11, 14, 19, 33, 35], "watch": 127, "watt": 20, "wdkv": 12, "wdq": 12, "we": [1, 2, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 25, 26, 28, 29, 30, 31, 36, 37, 38, 47, 52, 53, 55, 56, 62, 78, 79, 81, 83, 84, 88, 92, 94, 97, 98, 100, 102, 103, 106, 107, 108, 109, 110, 112, 115, 116, 117, 119, 120, 121, 122, 123, 126, 127, 129, 134, 139, 140, 142, 143, 144, 147, 148, 149, 158, 159], "web": 117, "weekli": [28, 29, 30, 31], "weig": 129, "weight": [0, 1, 3, 4, 7, 9, 12, 13, 15, 17, 18, 20, 21, 22, 23, 26, 28, 29, 30, 31, 38, 55, 73, 74, 79, 85, 89, 90, 100, 106, 115, 121, 122, 123, 129, 130, 131, 134, 143, 146, 148, 152, 153, 159, 162], "weight_index": 129, "weight_load": [80, 130, 160], "weight_mapp": [80, 160], "weight_only_groupwise_quant_matmul": 137, "weight_only_precis": 148, "weight_spars": [22, 143], "weight_stream": [22, 110, 143], "weightonlygroupwisequantmatmulplugin": 137, "weights_dict": [80, 115, 160], "weights_scaling_factor": [112, 114], "weightsinpoint": 1, "weightsoutpoint": 1, "welcom": [11, 15, 78, 158], "well": [4, 10, 15, 18, 19, 20, 36, 42, 72, 83, 88, 98, 101, 102, 113, 119, 126, 127, 137, 138, 143, 147], "were": [0, 1, 3, 7, 10, 14, 16, 19, 38, 88, 92, 108, 109, 112, 115, 122, 126, 143, 148], "weren": 97, "west": 29, "wget": [23, 139], "what": [11, 14, 15, 20, 26, 29, 33, 36, 37, 40, 62, 64, 65, 67, 81, 83, 85, 88, 93, 94, 98, 99, 119, 120, 121, 123, 126, 127, 143, 162], "whatev": 1, "wheel": [94, 96, 97, 148], "when": [0, 1, 2, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 25, 26, 28, 29, 30, 31, 32, 33, 34, 36, 37, 42, 48, 49, 57, 58, 72, 75, 81, 83, 84, 88, 89, 91, 92, 94, 97, 98, 99, 100, 101, 102, 104, 105, 106, 107, 109, 113, 114, 115, 116, 119, 120, 121, 123, 126, 127, 128, 129, 130, 131, 134, 135, 137, 139, 143, 144, 146, 148, 149, 151, 154, 165, 166], "whenev": [1, 72], "where": [0, 1, 3, 7, 8, 10, 12, 13, 14, 15, 16, 19, 21, 28, 29, 30, 31, 35, 37, 38, 39, 41, 42, 49, 57, 62, 66, 68, 72, 73, 81, 83, 84, 89, 92, 98, 101, 102, 104, 105, 107, 108, 109, 112, 113, 120, 123, 126, 128, 129, 134, 137, 143, 147, 148, 152, 156, 167], "wherea": [0, 16, 81, 112, 126], "whether": [0, 1, 8, 11, 15, 16, 18, 22, 23, 72, 74, 81, 83, 99, 101, 106, 122, 123, 127, 129, 130, 134, 143, 146, 150, 151, 153], "which": [0, 1, 3, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 48, 57, 58, 63, 64, 72, 74, 75, 78, 79, 80, 81, 83, 84, 86, 88, 89, 90, 91, 92, 94, 98, 99, 100, 101, 102, 103, 105, 106, 109, 112, 113, 114, 115, 119, 120, 121, 123, 126, 127, 128, 129, 131, 132, 134, 135, 137, 142, 143, 147, 148, 150, 151, 153, 154, 158, 159, 160, 164, 165, 167], "while": [0, 1, 3, 4, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 28, 29, 30, 31, 33, 35, 37, 57, 72, 78, 80, 81, 84, 87, 97, 98, 100, 103, 104, 105, 107, 108, 109, 113, 115, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 135, 137, 142, 148, 151, 158, 160], "whisper": [137, 138, 148], "whisperencod": 131, "whitespac": 143, "whl": [2, 94, 97], "who": [10, 11, 13, 142], "whole": [1, 11, 84, 129, 143], "whose": [8, 10, 12, 15, 16, 18, 33, 64, 81, 98, 105, 112, 130, 143], "why": [0, 14, 17, 20, 33, 88, 113, 123, 126, 127, 129, 135, 143], "wide": [0, 13, 17, 79, 93, 121, 146, 148, 159], "wide_ep": [19, 89], "wideep": [28, 143], "widespread": 100, "width": [0, 1, 25, 46, 72, 79, 81, 91, 101, 102, 130, 134, 135, 143, 148, 159], "wildcard": 33, "win": 143, "window": [0, 1, 8, 18, 22, 26, 37, 82, 93, 104, 109, 120, 129, 134, 143, 145, 148, 161], "window_s": 101, "windows": 0, "wip": [12, 82, 156, 161], "wireless": 51, "wirelessaccesspoint": 51, "wise": [15, 19, 89, 90, 103, 129, 143, 148], "wish": [80, 94, 105], "with_ssh": 116, "with_traceback": 143, "within": [8, 10, 11, 14, 15, 19, 20, 28, 29, 30, 31, 37, 64, 72, 81, 83, 86, 93, 98, 101, 104, 107, 109, 113, 120, 122, 123, 126, 127, 129, 143, 147, 166], "without": [0, 1, 2, 7, 10, 11, 12, 15, 16, 18, 19, 22, 28, 29, 32, 33, 35, 42, 57, 58, 72, 78, 79, 84, 85, 87, 88, 99, 101, 107, 109, 113, 114, 120, 123, 127, 129, 131, 143, 144, 148, 149, 151, 156, 158, 159, 162, 163], "wkr": 12, "wo": [12, 114, 148], "wo_gemm": [12, 19], "won": [19, 29, 122], "word": [0, 11, 91, 99, 101, 102, 129, 134, 143, 146, 148, 165], "word_dict": 134, "word_embed": 114, "word_embeddings_layernorm": 114, "work": [2, 8, 10, 15, 18, 20, 21, 29, 35, 37, 38, 42, 56, 59, 60, 61, 65, 72, 80, 81, 83, 84, 85, 93, 94, 95, 97, 98, 101, 102, 103, 104, 107, 109, 113, 115, 124, 125, 129, 134, 137, 139, 143, 144, 148, 149, 162], "workaround": [2, 20, 28, 114, 142, 148], "workdir": [26, 59, 60, 61, 94], "worker": [16, 22, 26, 35, 81, 92, 113, 120, 135, 143, 148], "worker_cl": 11, "worker_not_support": 11, "worker_tag": 11, "workerexecutablepath": 0, "workertag": 11, "workflow": [2, 11, 13, 15, 16, 23, 38, 42, 72, 73, 75, 78, 79, 81, 93, 101, 102, 111, 112, 121, 123, 124, 125, 129, 139, 142, 143, 148, 152, 154, 158, 159], "workload": [8, 10, 11, 14, 15, 16, 17, 18, 19, 22, 25, 36, 37, 38, 73, 81, 88, 89, 100, 107, 113, 119, 120, 121, 123, 124, 125, 126, 127, 146, 152], "workspac": [1, 15, 21, 22, 26, 34, 37, 85, 120, 129, 135, 143, 148, 162], "workstat": 4, "world": [0, 2, 8, 13, 15, 20, 22, 29, 37, 59, 60, 61, 74, 75, 84, 85, 103, 120, 121, 122, 123, 129, 146, 153, 154, 162], "world_config": 134, "world_siz": [75, 77, 112, 115, 129, 148, 154, 157], "worldconfig": [0, 102, 134], "worldsiz": 1, "worri": 11, "wors": [19, 22, 109, 123], "worst": [15, 88, 126, 127], "worth": [101, 104, 123, 127], "would": [0, 10, 13, 15, 37, 81, 88, 92, 103, 109, 120, 121, 123, 126, 128, 129, 143, 144, 149], "wpa2": 51, "wqr": 12, "wrap": [0, 1, 22, 113, 121, 129, 132, 134, 142, 148], "wrapped_properti": 143, "wrapper": [1, 10, 15, 28, 29, 30, 31, 72, 79, 103, 115, 151, 159], "write": [0, 1, 11, 12, 15, 22, 56, 64, 105, 114, 129, 139, 148, 156], "written": [11, 21, 113, 120, 129], "wrong": [109, 148], "wsl": 148, "wuk": 12, "wuq": 12, "wuv": 12, "www": 148, "x": [0, 1, 9, 10, 15, 25, 26, 28, 29, 30, 31, 38, 83, 94, 95, 99, 102, 106, 110, 120, 129, 130, 131, 137, 143, 147, 148], "x64": 20, "x86": 105, "x86_64": [97, 138], "xcomposer2": 148, "xf": 23, "xgrammar": [0, 10, 51, 70, 91, 99, 143, 148], "xl": 148, "xml": 99, "xmlcharrefreplac": 143, "xor": 129, "xqa": [93, 148], "xxx": [114, 115, 139], "xxx_plugin": 132, "xy": 129, "y": [2, 6, 10, 11, 15, 37, 78, 90, 94, 95, 97, 99, 116, 120, 129, 131, 137, 143, 158], "y_bia": 129, "yaml": [9, 15, 16, 17, 19, 20, 21, 23, 25, 26, 32, 33, 37, 38, 44, 70, 80, 81, 89, 92, 120, 148, 160], "yarn": [129, 148], "ye": [81, 82, 98, 129, 135, 136, 145, 161], "yeah": 62, "yelp": [138, 145], "yen": [37, 120], "yet": [0, 4, 12, 15, 18, 26, 81, 95, 102, 115, 129, 156, 167], "yield": [8, 11, 14, 18, 42, 105, 123, 126], "yiyixu": [40, 67], "yml": [2, 13, 25, 26, 28, 29, 30, 31, 33, 34, 38, 43, 81, 120], "york": [28, 29, 30, 31, 39, 41, 66, 68, 147, 156], "you": [2, 7, 9, 11, 12, 13, 14, 15, 16, 17, 19, 20, 22, 23, 25, 26, 28, 29, 30, 31, 33, 37, 38, 39, 40, 42, 44, 48, 49, 51, 59, 60, 61, 62, 65, 66, 67, 70, 72, 73, 74, 75, 78, 80, 81, 83, 84, 85, 86, 88, 90, 91, 92, 93, 94, 95, 97, 99, 100, 101, 102, 103, 105, 106, 108, 109, 112, 113, 115, 116, 117, 120, 122, 123, 124, 125, 126, 127, 128, 129, 134, 135, 139, 141, 142, 143, 144, 147, 148, 149, 151, 152, 153, 154, 156, 158, 160, 162, 164], "your": [2, 7, 9, 11, 13, 15, 20, 22, 25, 28, 29, 30, 31, 32, 33, 36, 37, 38, 42, 44, 48, 62, 64, 72, 73, 74, 75, 78, 79, 80, 81, 88, 89, 91, 94, 97, 98, 105, 106, 107, 109, 115, 116, 117, 119, 120, 121, 122, 123, 124, 125, 126, 127, 139, 142, 143, 144, 147, 149, 151, 152, 153, 154, 158, 159, 160, 166], "your_data_path": [2, 13], "your_dockerhub_usernam": [116, 117], "your_model_dir": 13, "your_model_path": [2, 15], "your_public_kei": 117, "your_work_path": 2, "yourself": [73, 152, 164], "yuhuili": 65, "yyi": 139, "z": [11, 94, 95, 129], "zars19": 148, "zero": [0, 1, 10, 57, 87, 99, 114, 129, 130, 137, 139, 142, 143, 163], "zero_is_placehold": 129, "zfill": 143, "zhang": 11, "zhuang": 11, "zip": 56, "zjli2013": 148, "zoo": [20, 58, 148], "zoom": [8, 15], "\u00b5": 19, "\u7f8e\u56fd\u7684\u9996\u90fd\u5728\u54ea\u91cc": 69}, "titles": ["Executor", "Runtime", "How to get best performance on DeepSeek-R1 in TensorRT LLM", "Falcon-180B on a single H200 GPU with INT4 AWQ, and 6.7x faster Llama-70B over A100", "H100 has 4.6x A100 Performance in TensorRT LLM, achieving 10,000 tok/s at 100ms to first token", "H200 achieves nearly 12,000 tokens/sec on Llama2-13B with TensorRT LLM", "New XQA-kernel provides 2.4x more Llama-70B throughput within the same latency budget", "Speed up inference with SOTA quantization techniques in TRT-LLM", "ADP Balance Strategy", "Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM)", "Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly", "Inference Time Compute Implementation in TensorRT LLM", "Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs", "DeepSeek R1 MTP Implementation and Optimization", "Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers", "Scaling Expert Parallelism in TensorRT LLM (Part 1: Design and Implementation of Large-scale EP)", "Disaggregated Serving in TensorRT LLM", "How to launch Llama4 Maverick + Eagle3 TensorRT LLM server", "N-Gram\u202fSpeculative\u202fDecoding\u202fin TensorRT LLM", "Scaling Expert Parallelism in TensorRT LLM (Part 2: Performance Status and Optimization)", "Running a High Performance GPT-OSS-120B Inference Server with TensorRT LLM", "trtllm-bench", "trtllm-build", "trtllm-eval", "trtllm-serve", "Run benchmarking with trtllm-serve auto_deploy pip auto_deploy (c++ struct)": [[0, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsIbEE", false]], "tensorrt_llm::executor::typetraits::value (c++ member)": [[0, "_CPPv4N12tensorrt_llm8executor10TypeTraitsIbE5valueE", false]], "tensorrt_llm::executor::typetraits (c++ struct)": [[0, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsIfEE", false]], "tensorrt_llm::executor::typetraits::value (c++ member)": [[0, "_CPPv4N12tensorrt_llm8executor10TypeTraitsIfE5valueE", false]], "tensorrt_llm::executor::typetraits (c++ struct)": [[0, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsI4halfEE", false]], "tensorrt_llm::executor::typetraits::value (c++ member)": [[0, "_CPPv4N12tensorrt_llm8executor10TypeTraitsI4halfE5valueE", false]], "tensorrt_llm::executor::typetraits (c++ struct)": [[0, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsINSt7int32_tEEE", false]], "tensorrt_llm::executor::typetraits::value (c++ member)": [[0, "_CPPv4N12tensorrt_llm8executor10TypeTraitsINSt7int32_tEE5valueE", false]], "tensorrt_llm::executor::typetraits (c++ struct)": [[0, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsINSt7int64_tEEE", false]], "tensorrt_llm::executor::typetraits::value (c++ member)": [[0, "_CPPv4N12tensorrt_llm8executor10TypeTraitsINSt7int64_tEE5valueE", false]], "tensorrt_llm::executor::typetraits (c++ struct)": [[0, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsINSt6int8_tEEE", false]], "tensorrt_llm::executor::typetraits::value (c++ member)": [[0, "_CPPv4N12tensorrt_llm8executor10TypeTraitsINSt6int8_tEE5valueE", false]], "tensorrt_llm::executor::typetraits (c++ struct)": [[0, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsINSt7uint8_tEEE", false]], "tensorrt_llm::executor::typetraits::value (c++ member)": [[0, "_CPPv4N12tensorrt_llm8executor10TypeTraitsINSt7uint8_tEE5valueE", false]], "tensorrt_llm::executor::typetraits (c++ struct)": [[0, "_CPPv4I0EN12tensorrt_llm8executor10TypeTraitsIP1TEE", false]], "tensorrt_llm::executor::typetraits::value (c++ member)": [[0, "_CPPv4N12tensorrt_llm8executor10TypeTraitsIP1TE5valueE", false]], "tensorrt_llm::executor::veclogprobs (c++ type)": [[0, "_CPPv4N12tensorrt_llm8executor11VecLogProbsE", false]], "tensorrt_llm::executor::vectokenextraids (c++ type)": [[0, "_CPPv4N12tensorrt_llm8executor16VecTokenExtraIdsE", false]], "tensorrt_llm::executor::vectokens (c++ type)": [[0, "_CPPv4N12tensorrt_llm8executor9VecTokensE", false]], "tensorrt_llm::executor::version (c++ function)": [[0, "_CPPv4N12tensorrt_llm8executor7versionEv", false]], "tensorrt_llm::layers (c++ type)": [[1, "_CPPv4N12tensorrt_llm6layersE", false]], "tensorrt_llm::mpi (c++ type)": [[0, "_CPPv4N12tensorrt_llm3mpiE", false]], "tensorrt_llm::runtime (c++ type)": [[0, "_CPPv4N12tensorrt_llm7runtimeE", false], [1, "_CPPv4N12tensorrt_llm7runtimeE", false]], "tensorrt_llm::runtime::allreducebuffers (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffersE", false]], "tensorrt_llm::runtime::allreducebuffers::allreducebuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers16AllReduceBuffersE10SizeType3210SizeType3210SizeType3210SizeType32RK13BufferManagerRK11WorldConfigKb", false]], "tensorrt_llm::runtime::allreducebuffers::mallreducecommptrs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers18mAllReduceCommPtrsE", false]], "tensorrt_llm::runtime::allreducebuffers::mflagptrs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers9mFlagPtrsE", false]], "tensorrt_llm::runtime::allreducebuffers::mipcmemoryhandles (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers17mIpcMemoryHandlesE", false]], "tensorrt_llm::runtime::allreducebuffers::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers9TensorPtrE", false]], "tensorrt_llm::runtime::buffercast (c++ function)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime10bufferCastEP1TR7IBuffer", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime10bufferCastEPK1TRK7IBuffer", false]], "tensorrt_llm::runtime::buffercastornull (c++ function)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKN7IBuffer9SharedPtrE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKN7ITensor9SharedPtrE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKNSt8optionalIN7IBuffer9SharedPtrEEE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKNSt8optionalIN7ITensor9SharedPtrEEE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKN7IBuffer14SharedConstPtrE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKN7ITensor14SharedConstPtrE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKNSt8optionalIN7IBuffer14SharedConstPtrEEE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKNSt8optionalIN7ITensor14SharedConstPtrEEE", false]], "tensorrt_llm::runtime::bufferdatatype (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime14BufferDataTypeE", false]], "tensorrt_llm::runtime::bufferdatatype::bufferdatatype (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType14BufferDataTypeEN8nvinfer18DataTypeEbb", false]], "tensorrt_llm::runtime::bufferdatatype::getdatatype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType11getDataTypeEv", false]], "tensorrt_llm::runtime::bufferdatatype::getsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType7getSizeEv", false]], "tensorrt_llm::runtime::bufferdatatype::getsizeinbits (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType13getSizeInBitsEv", false]], "tensorrt_llm::runtime::bufferdatatype::ispointer (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType9isPointerEv", false]], "tensorrt_llm::runtime::bufferdatatype::isunsigned (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType10isUnsignedEv", false]], "tensorrt_llm::runtime::bufferdatatype::ktrtpointertype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType15kTrtPointerTypeE", false]], "tensorrt_llm::runtime::bufferdatatype::mdatatype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType9mDataTypeE", false]], "tensorrt_llm::runtime::bufferdatatype::mpointer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType8mPointerE", false]], "tensorrt_llm::runtime::bufferdatatype::munsigned (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType9mUnsignedE", false]], "tensorrt_llm::runtime::bufferdatatype::operator nvinfer1::datatype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataTypecvN8nvinfer18DataTypeEEv", false]], "tensorrt_llm::runtime::buffermanager (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManagerE", false]], "tensorrt_llm::runtime::buffermanager::allocate (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeN8nvinfer14DimsEN8nvinfer18DataTypeE", false], [1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeNSt6size_tEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::buffermanager (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager13BufferManagerE13CudaStreamPtrb", false]], "tensorrt_llm::runtime::buffermanager::copy (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyEPKvR7IBuffer", false], [1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyEPKvR7IBuffer10MemoryType", false], [1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferPv", false], [1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferPv10MemoryType", false], [1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferR7IBuffer", false]], "tensorrt_llm::runtime::buffermanager::copyfrom (c++ function)": [[1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10IBufferPtrRKNSt6vectorI1TEE10MemoryType", false], [1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrP1TN8nvinfer14DimsE10MemoryType", false], [1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrRKNSt6vectorI1TEEN8nvinfer14DimsE10MemoryType", false], [1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8copyFromERK7IBuffer10MemoryType", false], [1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8copyFromERK7ITensor10MemoryType", false]], "tensorrt_llm::runtime::buffermanager::cpu (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager3cpuEN8nvinfer14DimsEN8nvinfer18DataTypeE", false], [1, "_CPPv4N12tensorrt_llm7runtime13BufferManager3cpuENSt6size_tEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::cudamempoolptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager14CudaMemPoolPtrE", false]], "tensorrt_llm::runtime::buffermanager::cudastreamptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager13CudaStreamPtrE", false]], "tensorrt_llm::runtime::buffermanager::emptybuffer (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager11emptyBufferE10MemoryTypeN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::emptytensor (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager11emptyTensorE10MemoryTypeN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::getstream (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager9getStreamEv", false]], "tensorrt_llm::runtime::buffermanager::gpu (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager3gpuEN8nvinfer14DimsEN8nvinfer18DataTypeE", false], [1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager3gpuENSt6size_tEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::gpusync (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7gpuSyncEN8nvinfer14DimsEN8nvinfer18DataTypeE", false], [1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7gpuSyncENSt6size_tEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::ibufferptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10IBufferPtrE", false]], "tensorrt_llm::runtime::buffermanager::ipcnvls (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7ipcNvlsENSt3setIiEEN8nvinfer14DimsEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::itensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10ITensorPtrE", false]], "tensorrt_llm::runtime::buffermanager::kbyte_type (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10kBYTE_TYPEE", false]], "tensorrt_llm::runtime::buffermanager::managed (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7managedEN8nvinfer14DimsEN8nvinfer18DataTypeE", false], [1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7managedENSt6size_tEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::memorypoolfree (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager14memoryPoolFreeEv", false]], "tensorrt_llm::runtime::buffermanager::memorypoolreserved (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager18memoryPoolReservedEv", false]], "tensorrt_llm::runtime::buffermanager::memorypooltrimto (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager16memoryPoolTrimToENSt6size_tE", false]], "tensorrt_llm::runtime::buffermanager::memorypoolused (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager14memoryPoolUsedEv", false]], "tensorrt_llm::runtime::buffermanager::mpool (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager5mPoolE", false]], "tensorrt_llm::runtime::buffermanager::mstream (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7mStreamE", false]], "tensorrt_llm::runtime::buffermanager::mtrimpool (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager9mTrimPoolE", false]], "tensorrt_llm::runtime::buffermanager::pinned (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager6pinnedEN8nvinfer14DimsEN8nvinfer18DataTypeE", false], [1, "_CPPv4N12tensorrt_llm7runtime13BufferManager6pinnedENSt6size_tEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::pinnedpool (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10pinnedPoolEN8nvinfer14DimsEN8nvinfer18DataTypeE", false], [1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10pinnedPoolENSt6size_tEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::buffermanager::setmem (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager6setMemER7IBuffer7int32_t", false]], "tensorrt_llm::runtime::buffermanager::setzero (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager7setZeroER7IBuffer", false]], "tensorrt_llm::runtime::buffermanager::~buffermanager (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13BufferManagerD0Ev", false]], "tensorrt_llm::runtime::bufferrange (c++ class)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime11BufferRangeE", false]], "tensorrt_llm::runtime::bufferrange::base (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime11BufferRange4BaseE", false]], "tensorrt_llm::runtime::bufferrange::bufferrange (c++ function)": [[1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI1UEEbEEEN12tensorrt_llm7runtime11BufferRange11BufferRangeERK7IBuffer", false], [1, "_CPPv4I0_NSt11enable_if_tIXntNSt10is_const_vI1UEEEbEEEN12tensorrt_llm7runtime11BufferRange11BufferRangeER7IBuffer", false], [1, "_CPPv4N12tensorrt_llm7runtime11BufferRange11BufferRangeEP1T9size_type", false]], "tensorrt_llm::runtime::cachesaltidtype (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime15CacheSaltIDTypeE", false]], "tensorrt_llm::runtime::canaccesspeer (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13canAccessPeerERK11WorldConfig", false]], "tensorrt_llm::runtime::clearvirtualmemoryallocator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime27clearVirtualMemoryAllocatorEv", false]], "tensorrt_llm::runtime::constpointercast (c++ function)": [[1, "_CPPv4I00EN12tensorrt_llm7runtime16constPointerCastENSt10shared_ptrINSt14remove_const_tI1TEEEERRNSt10unique_ptrI1T1DEE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime16constPointerCastENSt10shared_ptrINSt14remove_const_tI1TEEEERKNSt10shared_ptrI1TEE", false]], "tensorrt_llm::runtime::cudaevent (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime9CudaEventE", false]], "tensorrt_llm::runtime::cudaevent::cudaevent (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent9CudaEventE7pointerb", false], [1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent9CudaEventEj", false]], "tensorrt_llm::runtime::cudaevent::deleter (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7DeleterE", false]], "tensorrt_llm::runtime::cudaevent::deleter::deleter (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7Deleter7DeleterEb", false], [1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7Deleter7DeleterEv", false]], "tensorrt_llm::runtime::cudaevent::deleter::mownsevent (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7Deleter10mOwnsEventE", false]], "tensorrt_llm::runtime::cudaevent::deleter::operator() (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9CudaEvent7DeleterclE7pointer", false]], "tensorrt_llm::runtime::cudaevent::element_type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent12element_typeE", false]], "tensorrt_llm::runtime::cudaevent::eventptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent8EventPtrE", false]], "tensorrt_llm::runtime::cudaevent::get (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9CudaEvent3getEv", false]], "tensorrt_llm::runtime::cudaevent::mevent (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent6mEventE", false]], "tensorrt_llm::runtime::cudaevent::pointer (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7pointerE", false]], "tensorrt_llm::runtime::cudaevent::synchronize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9CudaEvent11synchronizeEv", false]], "tensorrt_llm::runtime::cudastream (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime10CudaStreamE", false]], "tensorrt_llm::runtime::cudastream::cudastream (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamE12cudaStream_t", false], [1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamE12cudaStream_tib", false], [1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamEji", false]], "tensorrt_llm::runtime::cudastream::deleter (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7DeleterE", false]], "tensorrt_llm::runtime::cudastream::deleter::deleter (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7Deleter7DeleterEb", false], [1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7Deleter7DeleterEv", false]], "tensorrt_llm::runtime::cudastream::deleter::mownsstream (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7Deleter11mOwnsStreamE", false]], "tensorrt_llm::runtime::cudastream::deleter::operator() (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream7DeleterclE12cudaStream_t", false]], "tensorrt_llm::runtime::cudastream::get (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream3getEv", false]], "tensorrt_llm::runtime::cudastream::getdevice (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream9getDeviceEv", false]], "tensorrt_llm::runtime::cudastream::mdevice (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7mDeviceE", false]], "tensorrt_llm::runtime::cudastream::mstream (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7mStreamE", false]], "tensorrt_llm::runtime::cudastream::record (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream6recordEN9CudaEvent7pointerE", false], [1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream6recordERK9CudaEvent", false]], "tensorrt_llm::runtime::cudastream::streamptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime10CudaStream9StreamPtrE", false]], "tensorrt_llm::runtime::cudastream::synchronize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream11synchronizeEv", false]], "tensorrt_llm::runtime::cudastream::wait (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream4waitEN9CudaEvent7pointerE", false], [1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream4waitERK9CudaEvent", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocatorE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::allocate (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator8allocateEP7PointerNSt6size_tEi", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13ConfigurationE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::backgroundconfiguration (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration23backgroundConfigurationE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::configuration (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtr", false], [1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtrb", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::mbackground (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration11mBackgroundE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::mbackstream (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration11mBackStreamE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::mmanager (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration8mManagerE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::mmode (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration5mModeE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::mpagesize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration9mPageSizeE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::mtag (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration4mTagE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::pagealigned (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration11pageAlignedENSt6size_tE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::configuration::setvirtualmemoryallocator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration25setVirtualMemoryAllocatorERKNSt6stringE11RestoreModeNSt10shared_ptrI10CudaStreamEE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::cudastreamptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13CudaStreamPtrE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::cudavirtualmemoryallocator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator26CudaVirtualMemoryAllocatorENSt10shared_ptrI13ConfigurationEE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::deallocate (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator10deallocateE7PointerNSt6size_tE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::mconfig (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator7mConfigE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::operator bool (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocatorcvbEv", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::pointer (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator7PointerE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::restoremode (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreModeE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::restoremode::cpu (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode3CPUE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::restoremode::memset (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode6MEMSETE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::restoremode::none (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode4NONEE", false]], "tensorrt_llm::runtime::cudavirtualmemoryallocator::restoremode::pinned (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode6PINNEDE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunkE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::_release (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk8_releaseEb", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::configurator (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12ConfiguratorE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::configurator::configurator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator12ConfiguratorERK12Configurator", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator12ConfiguratorERR12Configurator", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator12ConfiguratorEv", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::configurator::operator= (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12ConfiguratoraSERK12Configurator", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12ConfiguratoraSERR12Configurator", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::configurator::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator5setupE28CUmemGenericAllocationHandle", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::configurator::teardown (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator8teardownE28CUmemGenericAllocationHandleb", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::configurator::~configurator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12ConfiguratorD0Ev", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::configuratorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk15ConfiguratorPtrE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::configurators (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk13ConfiguratorsE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::creator (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7CreatorE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::creator::create (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator6createEv", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::creator::creator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7CreatorERK7Creator", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7CreatorERR7Creator", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7CreatorEv", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::creator::operator= (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7CreatoraSERK7Creator", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7CreatoraSERR7Creator", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::creator::release (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7releaseE28CUmemGenericAllocationHandleb", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::creator::~creator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7CreatorD0Ev", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::creatorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk10CreatorPtrE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::cudavirtualmemorychunk (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkERK22CUDAVirtualMemoryChunk", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkERR10CreatorPtrRR13Configurators", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkERR22CUDAVirtualMemoryChunk", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkEv", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::invalid_state (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk13INVALID_STATEE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::materialize (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk11materializeEv", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::mconfigurators (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk14mConfiguratorsE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::mcreator (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk8mCreatorE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::mhandle (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7mHandleE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::mstate (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6mStateE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::operator bool (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime22CUDAVirtualMemoryChunkcvbEv", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::operator= (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunkaSERK22CUDAVirtualMemoryChunk", false], [1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunkaSERR22CUDAVirtualMemoryChunk", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::release (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7releaseEv", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::status (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6StatusE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::status (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6statusEv", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::status::errored (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status7ERROREDE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::status::invalid (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status7INVALIDE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::status::materialized (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status12MATERIALIZEDE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::status::released (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status8RELEASEDE", false]], "tensorrt_llm::runtime::cudavirtualmemorychunk::~cudavirtualmemorychunk (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunkD0Ev", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManagerE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::add (c++ function)": [[1, "_CPPv4IDpEN12tensorrt_llm7runtime24CudaVirtualMemoryManager3addEv9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrEDpRR13Configurators", false], [1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERR22CUDAVirtualMemoryChunk", false], [1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrERRN22CUDAVirtualMemoryChunk13ConfiguratorsE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::addbadhandle (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager12addBadHandleE9uintptr_t", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::entry (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager5EntryE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::entry::mentryit (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager5Entry8mEntryItE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::entry::mmemory (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager5Entry7mMemoryE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::materializewithtag (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager18materializeWithTagERKNSt6stringE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::mbadhandles (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager11mBadHandlesE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::mentries (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager8mEntriesE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::mmemories (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager9mMemoriesE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::mmutex (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager6mMutexE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::pointermemorymap (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager16PointerMemoryMapE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::releasewithtag (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager14releaseWithTagERKNSt6stringE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::remove (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager6removeE9uintptr_t", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::retrievebadhandles (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager18retrieveBadHandlesEv", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::tagentrymap (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager11TagEntryMapE", false]], "tensorrt_llm::runtime::cudavirtualmemorymanager::unsaferemove (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager12unsafeRemoveE9uintptr_t", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4I_N8nvinfer18DataTypeE_b_bEN12tensorrt_llm7runtime14DataTypeTraitsE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4I_N8nvinfer18DataTypeE_bEN12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEE4typeE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4I_bEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kBOOLE9kUnsignedEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kBOOLE9kUnsignedE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kBOOLE9kUnsignedE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kBOOLE9kUnsignedE4typeE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kFLOATEEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kFLOATEE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kFLOATEE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kFLOATEE4typeE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kHALFEEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kHALFEE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kHALFEE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kHALFEE4typeE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EXL1EEEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EXL1EEE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EXL1EEE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EXL1EEE4typeE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EE4typeE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EXL1EEEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EXL1EEE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EXL1EEE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EXL1EEE4typeE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EE4typeE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kINT8EEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kINT8EE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kINT8EE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kINT8EE4typeE", false]], "tensorrt_llm::runtime::datatypetraits (c++ struct)": [[1, "_CPPv4I_bEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kUINT8E9kUnsignedEE", false]], "tensorrt_llm::runtime::datatypetraits::name (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kUINT8E9kUnsignedE4nameE", false]], "tensorrt_llm::runtime::datatypetraits::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kUINT8E9kUnsignedE4sizeE", false]], "tensorrt_llm::runtime::datatypetraits::type (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kUINT8E9kUnsignedE4typeE", false]], "tensorrt_llm::runtime::decoder (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoderE", false]], "tensorrt_llm::runtime::decoder::beamsearchbuffers (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffersE", false]], "tensorrt_llm::runtime::decoder::beamsearchbuffers::beamsearchbuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers17BeamSearchBuffersERK13BufferManager", false]], "tensorrt_llm::runtime::decoder::beamsearchbuffers::mcumlogprobstmp (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers15mCumLogProbsTmpE", false]], "tensorrt_llm::runtime::decoder::beamsearchbuffers::mnumsms (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers7mNumSMsE", false]], "tensorrt_llm::runtime::decoder::beamsearchbuffers::moutputbeamhypotheses (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers21mOutputBeamHypothesesE", false]], "tensorrt_llm::runtime::decoder::beamsearchbuffers::reshape (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers7reshapeE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::decoder::decoderstate (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderStateE", false]], "tensorrt_llm::runtime::decoder::decoderstate::decoderstate (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12DecoderStateEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::decodinginputptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState16DecodingInputPtrE", false]], "tensorrt_llm::runtime::decoder::decoderstate::decodingoutputptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState17DecodingOutputPtrE", false]], "tensorrt_llm::runtime::decoder::decoderstate::disablelookahead (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState16disableLookaheadERK13RequestVector", false]], "tensorrt_llm::runtime::decoder::decoderstate::getacceptedlengthscumsum (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState24getAcceptedLengthsCumSumEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getacceptedpackedpaths (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState22getAcceptedPackedPathsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getallnewtokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState15getAllNewTokensEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getbeamsearchbuffers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState20getBeamSearchBuffersEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getcacheindirectioninput (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState24getCacheIndirectionInputEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getcacheindirectionoutput (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState25getCacheIndirectionOutputEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getcumlogprobs (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getCumLogProbsE10SizeType32", false], [1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getCumLogProbsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::geteaglebuffers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState15getEagleBuffersEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getexplicitdrafttokensbuffers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState29getExplicitDraftTokensBuffersEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getfinishedsum (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getFinishedSumEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getfinishreasons (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState16getFinishReasonsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getgatheredids (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getGatheredIdsE10SizeType32", false], [1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getGatheredIdsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getgenerationsteps (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getGenerationStepsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getids (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState6getIdsE10SizeType32", false], [1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState6getIdsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getjointdecodinginput (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState21getJointDecodingInputEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getjointdecodingoutput (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState22getJointDecodingOutputEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getlogprobs (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState11getLogProbsE10SizeType32", false], [1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState11getLogProbsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getlookaheadbuffers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState19getLookaheadBuffersEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getmaxbeamwidth (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState15getMaxBeamWidthEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getmaxdecodingdecodertokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState27getMaxDecodingDecoderTokensEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getmaxdecodingenginetokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState26getMaxDecodingEngineTokensEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getmaxnumsequences (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getMaxNumSequencesEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getmaxsequencelength (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState20getMaxSequenceLengthEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getnextdrafttokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getNextDraftTokensEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getnextdrafttokenslengths (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState25getNextDraftTokensLengthsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getnumdecodingenginetokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState26getNumDecodingEngineTokensE10SizeType32", false], [1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState26getNumDecodingEngineTokensEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getparentids (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState12getParentIdsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getprevdrafttokenslengths (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState25getPrevDraftTokensLengthsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getsequencelengths (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getSequenceLengthsE10SizeType32", false], [1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getSequenceLengthsEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::getspeculativedecodingmode (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState26getSpeculativeDecodingModeEv", false]], "tensorrt_llm::runtime::decoder::decoderstate::llmrequestptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState13LlmRequestPtrE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mbeamsearchbuffers (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState18mBeamSearchBuffersE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mjointdecodinginput (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState19mJointDecodingInputE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mjointdecodingoutput (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState20mJointDecodingOutputE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mmaxbeamwidth (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState13mMaxBeamWidthE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mmaxdecodingdecodertokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState25mMaxDecodingDecoderTokensE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mmaxdecodingenginetokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24mMaxDecodingEngineTokensE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mmaxnumsequences (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState16mMaxNumSequencesE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mmaxsequencelength (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState18mMaxSequenceLengthE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mnumdecodingenginetokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24mNumDecodingEngineTokensE", false]], "tensorrt_llm::runtime::decoder::decoderstate::mspeculativedecodingmode (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24mSpeculativeDecodingModeE", false]], "tensorrt_llm::runtime::decoder::decoderstate::requestvector (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState13RequestVectorE", false]], "tensorrt_llm::runtime::decoder::decoderstate::reshapebuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", false]], "tensorrt_llm::runtime::decoder::decoderstate::reshapecacheindirectionbuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState30reshapeCacheIndirectionBuffersE10SizeType3210SizeType3210SizeType32", false]], "tensorrt_llm::runtime::decoder::decoderstate::reshapespeculativedecodingbuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState33reshapeSpeculativeDecodingBuffersERK23SpeculativeDecodingMode10SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", false]], "tensorrt_llm::runtime::decoder::decoderstate::setbeamwidth (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12setBeamWidthE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::decoder::decoderstate::setgenerationsteps (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState18setGenerationStepsERKNSt6vectorI10SizeType32EE", false]], "tensorrt_llm::runtime::decoder::decoderstate::setnumdecodingenginetokens (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState26setNumDecodingEngineTokensE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::decoder::decoderstate::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", false]], "tensorrt_llm::runtime::decoder::decoderstate::setupbuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12setupBuffersEN8nvinfer18DataTypeERK13BufferManager", false]], "tensorrt_llm::runtime::decoder::decoderstate::setupcacheindirection (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState21setupCacheIndirectionE10SizeType3210SizeType3210SizeType32RK13BufferManager", false]], "tensorrt_llm::runtime::decoder::decoderstate::setupcacheindirectionbuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState28setupCacheIndirectionBuffersERK13BufferManager", false]], "tensorrt_llm::runtime::decoder::decoderstate::setupspeculativedecoding (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24setupSpeculativeDecodingERK23SpeculativeDecodingMode10SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", false]], "tensorrt_llm::runtime::decoder::decoderstate::setupspeculativedecodingbuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState31setupSpeculativeDecodingBuffersE23SpeculativeDecodingModeN8nvinfer18DataTypeERK13BufferManager", false]], "tensorrt_llm::runtime::decoder::decoderstate::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState9TensorPtrE", false]], "tensorrt_llm::runtime::decoder_batch (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime13decoder_batchE", false]], "tensorrt_llm::runtime::decoder_batch::input (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5InputE", false]], "tensorrt_llm::runtime::decoder_batch::input::batchslots (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input10batchSlotsE", false]], "tensorrt_llm::runtime::decoder_batch::input::input (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input5InputERKNSt6vectorI14TensorConstPtrEE", false], [1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input5InputERKNSt6vectorINSt6vectorI14TensorConstPtrEEEE10SizeType32", false]], "tensorrt_llm::runtime::decoder_batch::input::logits (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input6logitsE", false]], "tensorrt_llm::runtime::decoder_batch::input::maxdecodersteps (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input15maxDecoderStepsE", false]], "tensorrt_llm::runtime::decoder_batch::input::tensorconstptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input14TensorConstPtrE", false]], "tensorrt_llm::runtime::decoder_batch::input::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input9TensorPtrE", false]], "tensorrt_llm::runtime::decodinginput (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInputE", false]], "tensorrt_llm::runtime::decodinginput::badwordslens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12badWordsLensE", false]], "tensorrt_llm::runtime::decodinginput::badwordslists (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13badWordsListsE", false]], "tensorrt_llm::runtime::decodinginput::badwordsptrs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12badWordsPtrsE", false]], "tensorrt_llm::runtime::decodinginput::batchsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput9batchSizeE", false]], "tensorrt_llm::runtime::decodinginput::batchslots (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput10batchSlotsE", false]], "tensorrt_llm::runtime::decodinginput::beamwidths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput10beamWidthsE", false]], "tensorrt_llm::runtime::decodinginput::cacheindirection (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput16cacheIndirectionE", false]], "tensorrt_llm::runtime::decodinginput::decodinginput (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13DecodingInputEv", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11eagleInputsE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputsE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::acceptedlens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs12acceptedLensE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::acceptedpathids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs15acceptedPathIdsE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::acceptedtokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs14acceptedTokensE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::chunkedcontextnexttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs24chunkedContextNextTokensE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::lastdraftlens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs13lastDraftLensE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::lastdraftpaths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs14lastDraftPathsE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::lastdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs15lastDraftTokensE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::nextdraftlens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs13nextDraftLensE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::nextdraftpaths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs14nextDraftPathsE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::nextdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs15nextDraftTokensE", false]], "tensorrt_llm::runtime::decodinginput::eagleinputs::seqslots (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs8seqSlotsE", false]], "tensorrt_llm::runtime::decodinginput::embeddingbias (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13embeddingBiasE", false]], "tensorrt_llm::runtime::decodinginput::endids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput6endIdsE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputsE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25explicitDraftTokensInputsE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::bestpathindices (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs15bestPathIndicesE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::bestpathlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs15bestPathLengthsE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::lastdraftindices (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs16lastDraftIndicesE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::lastdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs15lastDraftTokensE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::lastgenerationlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs21lastGenerationLengthsE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::lastpositionidsbase (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs19lastPositionIdsBaseE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::masks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs5masksE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::maxgenlengthdevice (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs18maxGenLengthDeviceE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::nextdraftindices (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs16nextDraftIndicesE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::nextdraftprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs14nextDraftProbsE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::nextdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs15nextDraftTokensE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::nextflattokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs14nextFlatTokensE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::nextgenerationlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs21nextGenerationLengthsE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::packedpositionids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs17packedPositionIdsE", false]], "tensorrt_llm::runtime::decodinginput::explicitdrafttokensinputs::seqslots (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs8seqSlotsE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputsE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25externalDraftTokensInputsE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::constantthreshold (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs17constantThresholdE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::draftlogits (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs11draftLogitsE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::draftlogitshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs15draftLogitsHostE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::draftprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs10draftProbsE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::drafttokenids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs13draftTokenIdsE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::drafttokenidshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs17draftTokenIdsHostE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::numdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs14numDraftTokensE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::numdrafttokenshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs18numDraftTokensHostE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::step (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs4stepE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::targetprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs11targetProbsE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::usedraftlogits (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs14useDraftLogitsE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::usedraftlogitshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs18useDraftLogitsHostE", false]], "tensorrt_llm::runtime::decodinginput::externaldrafttokensinputs::userandomacceptancethreshold (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs28useRandomAcceptanceThresholdE", false]], "tensorrt_llm::runtime::decodinginput::finishreasons (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13finishReasonsE", false]], "tensorrt_llm::runtime::decodinginput::generationsteps (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15generationStepsE", false]], "tensorrt_llm::runtime::decodinginput::lengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput7lengthsE", false]], "tensorrt_llm::runtime::decodinginput::logitsvec (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput9logitsVecE", false]], "tensorrt_llm::runtime::decodinginput::lookaheadinputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15lookaheadInputsE", false]], "tensorrt_llm::runtime::decodinginput::lookaheadinputs (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15LookaheadInputsE", false]], "tensorrt_llm::runtime::decodinginput::lookaheadinputs::tokensperstep (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15LookaheadInputs13tokensPerStepE", false]], "tensorrt_llm::runtime::decodinginput::maxattentionwindow (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput18maxAttentionWindowE", false]], "tensorrt_llm::runtime::decodinginput::maxbadwordslen (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput14maxBadWordsLenE", false]], "tensorrt_llm::runtime::decodinginput::maxlength (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput9maxLengthE", false]], "tensorrt_llm::runtime::decodinginput::maxstopwordslen (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15maxStopWordsLenE", false]], "tensorrt_llm::runtime::decodinginput::medusainputs (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputsE", false]], "tensorrt_llm::runtime::decodinginput::medusainputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12medusaInputsE", false]], "tensorrt_llm::runtime::decodinginput::medusainputs::medusacurtokensperstep (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs22medusaCurTokensPerStepE", false]], "tensorrt_llm::runtime::decodinginput::medusainputs::medusalogits (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs12medusaLogitsE", false]], "tensorrt_llm::runtime::decodinginput::medusainputs::medusapaths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs11medusaPathsE", false]], "tensorrt_llm::runtime::decodinginput::medusainputs::medusatargettokensperstep (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs25medusaTargetTokensPerStepE", false]], "tensorrt_llm::runtime::decodinginput::medusainputs::medusatreeids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs13medusaTreeIdsE", false]], "tensorrt_llm::runtime::decodinginput::norepeatngramsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput17noRepeatNgramSizeE", false]], "tensorrt_llm::runtime::decodinginput::sequencelimitlength (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput19sequenceLimitLengthE", false]], "tensorrt_llm::runtime::decodinginput::sinktokenlength (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15sinkTokenLengthE", false]], "tensorrt_llm::runtime::decodinginput::step (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput4stepE", false]], "tensorrt_llm::runtime::decodinginput::stopwordslens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13stopWordsLensE", false]], "tensorrt_llm::runtime::decodinginput::stopwordslists (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput14stopWordsListsE", false]], "tensorrt_llm::runtime::decodinginput::stopwordsptrs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13stopWordsPtrsE", false]], "tensorrt_llm::runtime::decodinginput::tensorconstptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput14TensorConstPtrE", false]], "tensorrt_llm::runtime::decodinginput::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput9TensorPtrE", false]], "tensorrt_llm::runtime::decodingoutput (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutputE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypothesesE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14beamHypothesesE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::batchdones (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses10batchDonesE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::cumlogprobscba (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses14cumLogProbsCBAE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::empty (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses5emptyERK13BufferManager", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::init (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses4initERK13BufferManager11TokenIdType", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::logprobscba (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses11logProbsCBAE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::minnormedscorescba (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses18minNormedScoresCBAE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::normedscorescba (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses15normedScoresCBAE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::numbeamscba (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses11numBeamsCBAE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::outputidscba (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses12outputIdsCBAE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::release (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses7releaseEv", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::reshape (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses7reshapeE10SizeType3210SizeType3210SizeType32", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::sequencelengthscba (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses18sequenceLengthsCBAE", false]], "tensorrt_llm::runtime::decodingoutput::beamhypotheses::slice (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses5sliceE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::decodingoutput::cacheindirection (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput16cacheIndirectionE", false]], "tensorrt_llm::runtime::decodingoutput::cumlogprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput11cumLogProbsE", false]], "tensorrt_llm::runtime::decodingoutput::decodingoutput (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14DecodingOutputEv", false]], "tensorrt_llm::runtime::decodingoutput::eaglebuffers (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput12eagleBuffersE", false]], "tensorrt_llm::runtime::decodingoutput::explicitdrafttokensbuffers (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26explicitDraftTokensBuffersE", false]], "tensorrt_llm::runtime::decodingoutput::finishedsum (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput11finishedSumE", false]], "tensorrt_llm::runtime::decodingoutput::finishreasons (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput13finishReasonsE", false]], "tensorrt_llm::runtime::decodingoutput::gatheredids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput11gatheredIdsE", false]], "tensorrt_llm::runtime::decodingoutput::ids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput3idsE", false]], "tensorrt_llm::runtime::decodingoutput::knegativeinfinity (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput17kNegativeInfinityE", false]], "tensorrt_llm::runtime::decodingoutput::lengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput7lengthsE", false]], "tensorrt_llm::runtime::decodingoutput::logprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput8logProbsE", false]], "tensorrt_llm::runtime::decodingoutput::logprobstiled (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput13logProbsTiledE", false]], "tensorrt_llm::runtime::decodingoutput::lookaheadoutputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput16lookaheadOutputsE", false]], "tensorrt_llm::runtime::decodingoutput::newtokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput9newTokensE", false]], "tensorrt_llm::runtime::decodingoutput::newtokenssteps (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14newTokensStepsE", false]], "tensorrt_llm::runtime::decodingoutput::newtokensvec (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput12newTokensVecE", false]], "tensorrt_llm::runtime::decodingoutput::parentids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput9parentIdsE", false]], "tensorrt_llm::runtime::decodingoutput::speculativedecodingoutputs (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputsE", false]], "tensorrt_llm::runtime::decodingoutput::speculativedecodingoutputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26speculativeDecodingOutputsE", false]], "tensorrt_llm::runtime::decodingoutput::speculativedecodingoutputs::acceptedlengthscumsum (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs21acceptedLengthsCumSumE", false]], "tensorrt_llm::runtime::decodingoutput::speculativedecodingoutputs::acceptedtokenslen (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs17acceptedTokensLenE", false]], "tensorrt_llm::runtime::decodingoutput::speculativedecodingoutputs::nextdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs15nextDraftTokensE", false]], "tensorrt_llm::runtime::decodingoutput::speculativedecodingoutputs::nextdrafttokenslen (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs18nextDraftTokensLenE", false]], "tensorrt_llm::runtime::decodingoutput::speculativedecodingoutputs::pathsoffsets (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs12pathsOffsetsE", false]], "tensorrt_llm::runtime::decodingoutput::speculativedecodingoutputs::prevdrafttokenslen (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs18prevDraftTokensLenE", false]], "tensorrt_llm::runtime::decodingoutput::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput9TensorPtrE", false]], "tensorrt_llm::runtime::deviceallocationnvls (c++ class)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime20DeviceAllocationNvlsE", false]], "tensorrt_llm::runtime::deviceallocationnvls::_capacity (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls9_capacityE", false]], "tensorrt_llm::runtime::deviceallocationnvls::_handle (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls7_handleE", false]], "tensorrt_llm::runtime::deviceallocationnvls::deviceallocationnvls (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls20DeviceAllocationNvlsEv", false]], "tensorrt_llm::runtime::deviceallocationnvls::free (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls4freeEv", false]], "tensorrt_llm::runtime::deviceallocationnvls::getcapacity (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime20DeviceAllocationNvls11getCapacityEv", false]], "tensorrt_llm::runtime::deviceallocationnvls::getipcunicastpointers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls21getIpcUnicastPointersEv", false]], "tensorrt_llm::runtime::deviceallocationnvls::getmulticastpointer (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime20DeviceAllocationNvls19getMulticastPointerEv", false]], "tensorrt_llm::runtime::deviceallocationnvls::getunicastpointer (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime20DeviceAllocationNvls17getUnicastPointerEv", false]], "tensorrt_llm::runtime::deviceallocationnvls::reset (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls5resetE6size_tNSt3setIiEE", false]], "tensorrt_llm::runtime::deviceallocationnvls::~deviceallocationnvls (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvlsD0Ev", false]], "tensorrt_llm::runtime::eaglebuffers (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffersE", false]], "tensorrt_llm::runtime::eaglebuffers::bufferptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers9BufferPtrE", false]], "tensorrt_llm::runtime::eaglebuffers::chunkedcontextnexttokenshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers28chunkedContextNextTokensHostE", false]], "tensorrt_llm::runtime::eaglebuffers::cumsumgenerationlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers23cumSumGenerationLengthsE", false]], "tensorrt_llm::runtime::eaglebuffers::eaglebuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12EagleBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN8executor14DecodingConfigE", false]], "tensorrt_llm::runtime::eaglebuffers::engineinputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12engineInputsE", false]], "tensorrt_llm::runtime::eaglebuffers::engineoutputs (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputsE", false]], "tensorrt_llm::runtime::eaglebuffers::engineoutputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13engineOutputsE", false]], "tensorrt_llm::runtime::eaglebuffers::engineoutputs::acceptedlens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs12acceptedLensE", false]], "tensorrt_llm::runtime::eaglebuffers::engineoutputs::acceptedpaths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs13acceptedPathsE", false]], "tensorrt_llm::runtime::eaglebuffers::engineoutputs::acceptedtokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs14acceptedTokensE", false]], "tensorrt_llm::runtime::eaglebuffers::engineoutputs::chunkedcontextnexttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs24chunkedContextNextTokensE", false]], "tensorrt_llm::runtime::eaglebuffers::engineoutputs::nextdraftlens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs13nextDraftLensE", false]], "tensorrt_llm::runtime::eaglebuffers::engineoutputs::nextdraftpaths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs14nextDraftPathsE", false]], "tensorrt_llm::runtime::eaglebuffers::engineoutputs::nextdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs15nextDraftTokensE", false]], "tensorrt_llm::runtime::eaglebuffers::greedysamplinghost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers18greedySamplingHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6InputsE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::alllayersdrafttokenids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs22allLayersDraftTokenIdsE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::alllayersdrafttokenidspredecessor (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs33allLayersDraftTokenIdsPredecessorE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::alllayersscores (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs15allLayersScoresE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::chunkedcontextnexttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs24chunkedContextNextTokensE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::create (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs6createE10SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfig", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::currentexpandindices (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs20currentExpandIndicesE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::draftlens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs9draftLensE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::draftpaths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs10draftPathsE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::draftpathshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs14draftPathsHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::drafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs11draftTokensE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::dynamictreemaxtopkhost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs22dynamicTreeMaxTopKHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::eaglenetctxcontextlengthshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs29eagleNetCtxContextLengthsHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::eaglenetctxpastkeyvaluelengthshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs34eagleNetCtxPastKeyValueLengthsHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::eaglenetctxrequesttypeshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs27eagleNetCtxRequestTypesHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::eaglenetgencontextlengthshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs29eagleNetGenContextLengthsHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::eaglenetgenpastkeyvaluelengthshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs34eagleNetGenPastKeyValueLengthsHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::eaglenetgenrequesttypeshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs27eagleNetGenRequestTypesHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::inputgentokenshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs18inputGenTokensHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::posterioralpha (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs14posteriorAlphaE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::posteriorthreshold (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs18posteriorThresholdE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::prevscores (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs10prevScoresE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::randomdatasample (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs16randomDataSampleE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::randomdatavalidation (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs20randomDataValidationE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::specdecodinggenerationlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs29specDecodingGenerationLengthsE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::specdecodinggenerationlengthshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs33specDecodingGenerationLengthsHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::specdecodingpackedmasks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs23specDecodingPackedMasksE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::specdecodingpositionoffsets (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs27specDecodingPositionOffsetsE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::temperatures (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs12temperaturesE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::usedynamictreehost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs18useDynamicTreeHostE", false]], "tensorrt_llm::runtime::eaglebuffers::inputs::usespecdecoding (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs15useSpecDecodingE", false]], "tensorrt_llm::runtime::eaglebuffers::insertinputtensors (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", false]], "tensorrt_llm::runtime::eaglebuffers::itensor (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers7ITensorE", false]], "tensorrt_llm::runtime::eaglebuffers::llmrequestptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13LlmRequestPtrE", false]], "tensorrt_llm::runtime::eaglebuffers::maxgenerationlength (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers19maxGenerationLengthE", false]], "tensorrt_llm::runtime::eaglebuffers::mdefaultposteriorthreshold (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers26mDefaultPosteriorThresholdE", false]], "tensorrt_llm::runtime::eaglebuffers::mdogreedysampling (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers17mDoGreedySamplingE", false]], "tensorrt_llm::runtime::eaglebuffers::posterioralphahost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers18posteriorAlphaHostE", false]], "tensorrt_llm::runtime::eaglebuffers::posteriorthresholdhost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers22posteriorThresholdHostE", false]], "tensorrt_llm::runtime::eaglebuffers::requestvector (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13RequestVectorE", false]], "tensorrt_llm::runtime::eaglebuffers::reshape (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", false]], "tensorrt_llm::runtime::eaglebuffers::scanreducetempstorage (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers21scanReduceTempStorageE", false]], "tensorrt_llm::runtime::eaglebuffers::scanreducetempstoragebytes (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers26scanReduceTempStorageBytesE", false]], "tensorrt_llm::runtime::eaglebuffers::setfrominputs (c++ function)": [[1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", false], [1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", false]], "tensorrt_llm::runtime::eaglebuffers::sizetype32 (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers10SizeType32E", false]], "tensorrt_llm::runtime::eaglebuffers::tensormap (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers9TensorMapE", false]], "tensorrt_llm::runtime::eaglebuffers::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers9TensorPtrE", false]], "tensorrt_llm::runtime::eaglemodule (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime11EagleModuleE", false]], "tensorrt_llm::runtime::eaglemodule::eaglemodule (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11EagleModule11EagleModuleE10SizeType3210SizeType3210SizeType3210SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime11EagleModule11EagleModuleEv", false]], "tensorrt_llm::runtime::eaglemodule::getdefaulteaglechoices (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11EagleModule22getDefaultEagleChoicesEv", false]], "tensorrt_llm::runtime::eaglemodule::getmaxnonleafnodesperlayer (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11EagleModule26getMaxNonLeafNodesPerLayerEv", false]], "tensorrt_llm::runtime::eaglemodule::getnumtransformerlayers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11EagleModule23getNumTransformerLayersEv", false]], "tensorrt_llm::runtime::eaglemodule::mdefaulteaglechoices (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11EagleModule20mDefaultEagleChoicesE", false]], "tensorrt_llm::runtime::eaglemodule::mmaxnonleafnodesperlayer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11EagleModule24mMaxNonLeafNodesPerLayerE", false]], "tensorrt_llm::runtime::eaglemodule::mnumtransformerslayer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11EagleModule21mNumTransformersLayerE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffersE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::bufferptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers9BufferPtrE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::cumsumgenerationlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers23cumSumGenerationLengthsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineinputs (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers12EngineInputsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineinputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers12engineInputsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineinputs::positionoffsets (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers12EngineInputs15positionOffsetsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineinputs::requesttypesdevice (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers12EngineInputs18requestTypesDeviceE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13engineOutputsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::bestpathindices (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs15bestPathIndicesE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::bestpathlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs15bestPathLengthsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::masks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs5masksE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::maxgentoken (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs11maxGenTokenE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::nextdraftindices (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs16nextDraftIndicesE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::nextdraftprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs14nextDraftProbsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::nextdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs15nextDraftTokensE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::nextflattokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs14nextFlatTokensE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::nextgenerationlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs21nextGenerationLengthsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::nextpositionoffsets (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs19nextPositionOffsetsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::packedpositionids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs17packedPositionIdsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::engineoutputs::totalgentoken (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs13totalGenTokenE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::explicitdrafttokensbuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers26ExplicitDraftTokensBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6InputsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::create (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs6createE10SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::draftindices (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs12draftIndicesE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::draftprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs10draftProbsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::drafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs11draftTokensE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::generationlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs17generationLengthsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::generationlengthshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs21generationLengthsHostE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::maxgenlengthhost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs16maxGenLengthHostE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::packedmasks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs11packedMasksE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::positionids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs11positionIdsE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::positionidsbase (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs15positionIdsBaseE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::randomdatasample (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs16randomDataSampleE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::randomdatavalidation (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs20randomDataValidationE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::temperatures (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs12temperaturesE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::inputs::usespecdecoding (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs15useSpecDecodingE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::insertinputtensors (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::itensor (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers7ITensorE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::reshape (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::scantempstorage (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers15scanTempStorageE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::scantempstoragebytes (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers20scanTempStorageBytesE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::setfrominputs (c++ function)": [[1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", false], [1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::sizetype32 (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers10SizeType32E", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::tensormap (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers9TensorMapE", false]], "tensorrt_llm::runtime::explicitdrafttokensbuffers::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers9TensorPtrE", false]], "tensorrt_llm::runtime::genericprompttuningparams (c++ class)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime25GenericPromptTuningParamsE", false]], "tensorrt_llm::runtime::genericprompttuningparams::embeddingtable (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams14embeddingTableE", false]], "tensorrt_llm::runtime::genericprompttuningparams::genericprompttuningparams (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams25GenericPromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", false]], "tensorrt_llm::runtime::genericprompttuningparams::prompttuningenabled (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams19promptTuningEnabledE", false]], "tensorrt_llm::runtime::genericprompttuningparams::sizetype32 (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams10SizeType32E", false]], "tensorrt_llm::runtime::genericprompttuningparams::tasks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams5tasksE", false]], "tensorrt_llm::runtime::genericprompttuningparams::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams9TensorPtrE", false]], "tensorrt_llm::runtime::genericprompttuningparams::vocabsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams9vocabSizeE", false]], "tensorrt_llm::runtime::getdefaultbatchslots (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20getDefaultBatchSlotsEN7runtime10SizeType32E", false]], "tensorrt_llm::runtime::getvirtualmemoryallocator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25getVirtualMemoryAllocatorEv", false]], "tensorrt_llm::runtime::getvirtualmemorymanager (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23getVirtualMemoryManagerEv", false]], "tensorrt_llm::runtime::gptdecoder (c++ class)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime10GptDecoderE", false]], "tensorrt_llm::runtime::gptdecoder::cudastreamptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder13CudaStreamPtrE", false]], "tensorrt_llm::runtime::gptdecoder::disablelookahead (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", false]], "tensorrt_llm::runtime::gptdecoder::forwardasync (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder12forwardAsyncER14DecodingOutputRK13DecodingInput", false]], "tensorrt_llm::runtime::gptdecoder::forwardsync (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder11forwardSyncER14DecodingOutputRK13DecodingInput", false]], "tensorrt_llm::runtime::gptdecoder::getsamplingconfig (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder17getSamplingConfigEv", false]], "tensorrt_llm::runtime::gptdecoder::gptdecoder (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10GptDecoderERKN8executor12DecodingModeE6size_t6size_t6size_t6size_tRK13CudaStreamPtrNSt10shared_ptrIK25SpeculativeDecodingModuleEE", false]], "tensorrt_llm::runtime::gptdecoder::mdecodinglayerworkspace (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder23mDecodingLayerWorkspaceE", false]], "tensorrt_llm::runtime::gptdecoder::mdecodingmode (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder13mDecodingModeE", false]], "tensorrt_llm::runtime::gptdecoder::mdynamicdecodelayer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder19mDynamicDecodeLayerE", false]], "tensorrt_llm::runtime::gptdecoder::mmanager (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder8mManagerE", false]], "tensorrt_llm::runtime::gptdecoder::mmaxnumsequences (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder16mMaxNumSequencesE", false]], "tensorrt_llm::runtime::gptdecoder::msamplingconfig (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder15mSamplingConfigE", false]], "tensorrt_llm::runtime::gptdecoder::mvocabsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10mVocabSizeE", false]], "tensorrt_llm::runtime::gptdecoder::mvocabsizepadded (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder16mVocabSizePaddedE", false]], "tensorrt_llm::runtime::gptdecoder::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", false]], "tensorrt_llm::runtime::gptdecoder::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder9TensorPtrE", false]], "tensorrt_llm::runtime::gptdecoderbatched (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatchedE", false]], "tensorrt_llm::runtime::gptdecoderbatched::cudastreamptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched13CudaStreamPtrE", false]], "tensorrt_llm::runtime::gptdecoderbatched::disablelookahead (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched16disableLookaheadERK13RequestVectorRK9TensorPtr", false]], "tensorrt_llm::runtime::gptdecoderbatched::finalize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", false]], "tensorrt_llm::runtime::gptdecoderbatched::forward (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched7forwardERKN7decoder12DecoderStateERKN13decoder_batch5InputE", false]], "tensorrt_llm::runtime::gptdecoderbatched::forwardasync (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched12forwardAsyncERKN7decoder12DecoderStateERKN13decoder_batch5InputE", false]], "tensorrt_llm::runtime::gptdecoderbatched::forwarddispatch (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched15forwardDispatchERKN7decoder12DecoderStateERKN13decoder_batch5InputE", false]], "tensorrt_llm::runtime::gptdecoderbatched::getbuffermanager (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched16getBufferManagerEv", false]], "tensorrt_llm::runtime::gptdecoderbatched::getdecoderstream (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched16getDecoderStreamEv", false]], "tensorrt_llm::runtime::gptdecoderbatched::getunderlyingdecoder (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched20getUnderlyingDecoderEv", false]], "tensorrt_llm::runtime::gptdecoderbatched::gptdecoderbatched (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched17GptDecoderBatchedE13CudaStreamPtr", false]], "tensorrt_llm::runtime::gptdecoderbatched::gptdecoderptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched13GptDecoderPtrE", false]], "tensorrt_llm::runtime::gptdecoderbatched::llmrequestptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched13LlmRequestPtrE", false]], "tensorrt_llm::runtime::gptdecoderbatched::mbuffermanager (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched14mBufferManagerE", false]], "tensorrt_llm::runtime::gptdecoderbatched::mdecoder (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched8mDecoderE", false]], "tensorrt_llm::runtime::gptdecoderbatched::mdecoderstream (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched14mDecoderStreamE", false]], "tensorrt_llm::runtime::gptdecoderbatched::mruntimestream (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched14mRuntimeStreamE", false]], "tensorrt_llm::runtime::gptdecoderbatched::requestvector (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched13RequestVectorE", false]], "tensorrt_llm::runtime::gptdecoderbatched::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", false]], "tensorrt_llm::runtime::gptdecoderbatched::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched9TensorPtrE", false]], "tensorrt_llm::runtime::gptjsonconfig (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfigE", false]], "tensorrt_llm::runtime::gptjsonconfig::enginefilename (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14engineFilenameERK11WorldConfig", false], [1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14engineFilenameERK11WorldConfigRKNSt6stringE", false]], "tensorrt_llm::runtime::gptjsonconfig::getcontextparallelism (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig21getContextParallelismEv", false]], "tensorrt_llm::runtime::gptjsonconfig::getgpuspernode (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14getGpusPerNodeEv", false]], "tensorrt_llm::runtime::gptjsonconfig::getmodelconfig (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14getModelConfigEv", false]], "tensorrt_llm::runtime::gptjsonconfig::getmodelconfigmutable (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig21getModelConfigMutableEv", false]], "tensorrt_llm::runtime::gptjsonconfig::getname (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig7getNameEv", false]], "tensorrt_llm::runtime::gptjsonconfig::getpipelineparallelism (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig22getPipelineParallelismEv", false]], "tensorrt_llm::runtime::gptjsonconfig::getprecision (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig12getPrecisionEv", false]], "tensorrt_llm::runtime::gptjsonconfig::getruntimedefaults (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig18getRuntimeDefaultsEv", false]], "tensorrt_llm::runtime::gptjsonconfig::gettensorparallelism (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig20getTensorParallelismEv", false]], "tensorrt_llm::runtime::gptjsonconfig::getversion (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig10getVersionEv", false]], "tensorrt_llm::runtime::gptjsonconfig::getworldsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig12getWorldSizeEv", false]], "tensorrt_llm::runtime::gptjsonconfig::gptjsonconfig (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", false]], "tensorrt_llm::runtime::gptjsonconfig::mcontextparallelism (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig19mContextParallelismE", false]], "tensorrt_llm::runtime::gptjsonconfig::mgpuspernode (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig12mGpusPerNodeE", false]], "tensorrt_llm::runtime::gptjsonconfig::mmodelconfig (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig12mModelConfigE", false]], "tensorrt_llm::runtime::gptjsonconfig::mname (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5mNameE", false]], "tensorrt_llm::runtime::gptjsonconfig::mpipelineparallelism (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig20mPipelineParallelismE", false]], "tensorrt_llm::runtime::gptjsonconfig::mprecision (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig10mPrecisionE", false]], "tensorrt_llm::runtime::gptjsonconfig::mruntimedefaults (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig16mRuntimeDefaultsE", false]], "tensorrt_llm::runtime::gptjsonconfig::mtensorparallelism (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig18mTensorParallelismE", false]], "tensorrt_llm::runtime::gptjsonconfig::mversion (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig8mVersionE", false]], "tensorrt_llm::runtime::gptjsonconfig::parse (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5parseERKNSt10filesystem4pathE", false], [1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5parseERKNSt6stringE", false], [1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5parseERNSt7istreamE", false]], "tensorrt_llm::runtime::ibuffer (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBufferE", false]], "tensorrt_llm::runtime::ibuffer::data (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4dataENSt6size_tE", false], [1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4dataEv", false], [1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer4dataENSt6size_tE", false], [1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer4dataEv", false]], "tensorrt_llm::runtime::ibuffer::datatype (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer8DataTypeE", false]], "tensorrt_llm::runtime::ibuffer::getcapacity (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer11getCapacityEv", false]], "tensorrt_llm::runtime::ibuffer::getdatatype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer11getDataTypeEv", false]], "tensorrt_llm::runtime::ibuffer::getdatatypename (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer15getDataTypeNameE8DataType", false], [1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer15getDataTypeNameEv", false]], "tensorrt_llm::runtime::ibuffer::getmemorytype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer13getMemoryTypeEv", false]], "tensorrt_llm::runtime::ibuffer::getmemorytypename (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer17getMemoryTypeNameEv", false]], "tensorrt_llm::runtime::ibuffer::getsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer7getSizeEv", false]], "tensorrt_llm::runtime::ibuffer::getsizeinbytes (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer14getSizeInBytesEv", false]], "tensorrt_llm::runtime::ibuffer::ibuffer (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer7IBufferERK7IBuffer", false], [1, "_CPPv4N12tensorrt_llm7runtime7IBuffer7IBufferEv", false]], "tensorrt_llm::runtime::ibuffer::memorytype (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer10memoryTypeEPKv", false]], "tensorrt_llm::runtime::ibuffer::operator= (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBufferaSERK7IBuffer", false]], "tensorrt_llm::runtime::ibuffer::release (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer7releaseEv", false]], "tensorrt_llm::runtime::ibuffer::resize (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer6resizeENSt6size_tE", false]], "tensorrt_llm::runtime::ibuffer::sharedconstptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer14SharedConstPtrE", false]], "tensorrt_llm::runtime::ibuffer::sharedptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer9SharedPtrE", false]], "tensorrt_llm::runtime::ibuffer::slice (c++ function)": [[1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", false], [1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", false], [1, "_CPPv4N12tensorrt_llm7runtime7IBuffer5sliceE9SharedPtrNSt6size_tE", false], [1, "_CPPv4N12tensorrt_llm7runtime7IBuffer5sliceE9SharedPtrNSt6size_tENSt6size_tE", false]], "tensorrt_llm::runtime::ibuffer::tobytes (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer7toBytesENSt6size_tE", false]], "tensorrt_llm::runtime::ibuffer::uniqueconstptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer14UniqueConstPtrE", false]], "tensorrt_llm::runtime::ibuffer::uniqueptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBuffer9UniquePtrE", false]], "tensorrt_llm::runtime::ibuffer::view (c++ function)": [[1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer4viewE14UniqueConstPtrRR9TConstPtrNSt6size_tE", false], [1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4viewE9SharedPtr", false], [1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4viewE9SharedPtrNSt6size_tE", false]], "tensorrt_llm::runtime::ibuffer::wrap (c++ function)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tENSt6size_tE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrRNSt6vectorI1TEE", false], [1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tE", false], [1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tENSt6size_tE", false]], "tensorrt_llm::runtime::ibuffer::~ibuffer (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7IBufferD0Ev", false]], "tensorrt_llm::runtime::igptdecoder (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoderE", false]], "tensorrt_llm::runtime::igptdecoder::create (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", false]], "tensorrt_llm::runtime::igptdecoder::disablelookahead (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", false]], "tensorrt_llm::runtime::igptdecoder::forwardasync (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder12forwardAsyncER14DecodingOutputRK13DecodingInput", false]], "tensorrt_llm::runtime::igptdecoder::forwardsync (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder11forwardSyncER14DecodingOutputRK13DecodingInput", false]], "tensorrt_llm::runtime::igptdecoder::getsamplingconfig (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder17getSamplingConfigEv", false]], "tensorrt_llm::runtime::igptdecoder::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", false]], "tensorrt_llm::runtime::igptdecoder::tensorconstptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder14TensorConstPtrE", false]], "tensorrt_llm::runtime::igptdecoder::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder9TensorPtrE", false]], "tensorrt_llm::runtime::igptdecoder::~igptdecoder (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoderD0Ev", false]], "tensorrt_llm::runtime::igptdecoderbatched (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatchedE", false]], "tensorrt_llm::runtime::igptdecoderbatched::cudastreamptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched13CudaStreamPtrE", false]], "tensorrt_llm::runtime::igptdecoderbatched::disablelookahead (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched16disableLookaheadERK13RequestVectorRK9TensorPtr", false]], "tensorrt_llm::runtime::igptdecoderbatched::finalize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime18IGptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", false]], "tensorrt_llm::runtime::igptdecoderbatched::forward (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched7forwardERKN7decoder12DecoderStateERKN13decoder_batch5InputE", false]], "tensorrt_llm::runtime::igptdecoderbatched::forwardasync (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched12forwardAsyncERKN7decoder12DecoderStateERKN13decoder_batch5InputE", false]], "tensorrt_llm::runtime::igptdecoderbatched::igptdecoderbatched (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched18IGptDecoderBatchedEv", false]], "tensorrt_llm::runtime::igptdecoderbatched::llmrequestptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched13LlmRequestPtrE", false]], "tensorrt_llm::runtime::igptdecoderbatched::requestvector (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched13RequestVectorE", false]], "tensorrt_llm::runtime::igptdecoderbatched::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", false]], "tensorrt_llm::runtime::igptdecoderbatched::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched9TensorPtrE", false]], "tensorrt_llm::runtime::igptdecoderbatched::~igptdecoderbatched (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatchedD0Ev", false]], "tensorrt_llm::runtime::ipcmemory (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemoryE", false]], "tensorrt_llm::runtime::ipcmemory::allocateipcmemory (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory17allocateIpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfig", false]], "tensorrt_llm::runtime::ipcmemory::bufferptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9BufferPtrE", false]], "tensorrt_llm::runtime::ipcmemory::destroyipcmemory (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory16destroyIpcMemoryEv", false]], "tensorrt_llm::runtime::ipcmemory::flags_size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory10FLAGS_SIZEE", false]], "tensorrt_llm::runtime::ipcmemory::getcommptrs (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9IpcMemory11getCommPtrsEv", false]], "tensorrt_llm::runtime::ipcmemory::ipcmemory (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfigb", false], [1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryERK9IpcMemory", false], [1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryERR9IpcMemory", false]], "tensorrt_llm::runtime::ipcmemory::mbuffer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory7mBufferE", false]], "tensorrt_llm::runtime::ipcmemory::mcommptrs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9mCommPtrsE", false]], "tensorrt_llm::runtime::ipcmemory::mopenipc (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory8mOpenIpcE", false]], "tensorrt_llm::runtime::ipcmemory::mtprank (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory7mTpRankE", false]], "tensorrt_llm::runtime::ipcmemory::operator= (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemoryaSERK9IpcMemory", false], [1, "_CPPv4N12tensorrt_llm7runtime9IpcMemoryaSERR9IpcMemory", false]], "tensorrt_llm::runtime::ipcmemory::~ipcmemory (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9IpcMemoryD0Ev", false]], "tensorrt_llm::runtime::ipcnvlsallocate (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime15ipcNvlsAllocateE6size_tNSt3setIiEE", false]], "tensorrt_llm::runtime::ipcnvlsfree (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ipcNvlsFreeEP13IpcNvlsHandle", false]], "tensorrt_llm::runtime::ipcnvlshandle (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandleE", false]], "tensorrt_llm::runtime::ipcnvlshandle::ipc_uc_handles (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle14ipc_uc_handlesE", false]], "tensorrt_llm::runtime::ipcnvlshandle::ipc_uc_ptrs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle11ipc_uc_ptrsE", false]], "tensorrt_llm::runtime::ipcnvlshandle::ipc_uc_vas (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle10ipc_uc_vasE", false]], "tensorrt_llm::runtime::ipcnvlshandle::mc_handle (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle9mc_handleE", false]], "tensorrt_llm::runtime::ipcnvlshandle::mc_ptr (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle6mc_ptrE", false]], "tensorrt_llm::runtime::ipcnvlshandle::mc_va (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle5mc_vaE", false]], "tensorrt_llm::runtime::ipcnvlshandle::size (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle4sizeE", false]], "tensorrt_llm::runtime::ipcnvlshandle::uc_handle (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle9uc_handleE", false]], "tensorrt_llm::runtime::ipcnvlshandle::uc_ptr (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle6uc_ptrE", false]], "tensorrt_llm::runtime::ipcnvlshandle::uc_va (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle5uc_vaE", false]], "tensorrt_llm::runtime::ipcnvlssupported (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime16ipcNvlsSupportedEv", false]], "tensorrt_llm::runtime::itensor (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensorE", false]], "tensorrt_llm::runtime::itensor::at (c++ function)": [[1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atE14UniqueConstPtrRR9TConstPtrRK5Shape", false], [1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atEN7ITensor14UniqueConstPtrERR9TConstPtrRKNSt16initializer_listI9DimType64EE", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor2atE9SharedPtrRK5Shape", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor2atE9SharedPtrRKNSt16initializer_listI9DimType64EE", false]], "tensorrt_llm::runtime::itensor::castsize (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor8castSizeE6size_t", false]], "tensorrt_llm::runtime::itensor::dimtype64 (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor9DimType64E", false]], "tensorrt_llm::runtime::itensor::flattenn (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor8flattenNE9SharedPtrNSt7int64_tE", false]], "tensorrt_llm::runtime::itensor::getdimension (c++ function)": [[1, "_CPPv4I_10SizeType32ENK12tensorrt_llm7runtime7ITensor12getDimensionE9DimType64v", false]], "tensorrt_llm::runtime::itensor::getshape (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime7ITensor8getShapeEv", false]], "tensorrt_llm::runtime::itensor::itensor (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor7ITensorERK7ITensor", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor7ITensorEv", false]], "tensorrt_llm::runtime::itensor::makeshape (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor9makeShapeERKNSt16initializer_listI9DimType64EE", false]], "tensorrt_llm::runtime::itensor::operator= (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensoraSERK7ITensor", false]], "tensorrt_llm::runtime::itensor::reshape (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor7reshapeERK5Shape", false]], "tensorrt_llm::runtime::itensor::resize (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor6resizeENSt6size_tE", false]], "tensorrt_llm::runtime::itensor::shape (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor5ShapeE", false]], "tensorrt_llm::runtime::itensor::shapeequals (c++ function)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor11shapeEqualsEbRK5ShapePK1T10SizeType32", false], [1, "_CPPv4I0ENK12tensorrt_llm7runtime7ITensor11shapeEqualsEbPK1T10SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor11shapeEqualsERK5ShapeRK5Shape", false], [1, "_CPPv4NK12tensorrt_llm7runtime7ITensor11shapeEqualsERK5Shape", false], [1, "_CPPv4NK12tensorrt_llm7runtime7ITensor11shapeEqualsERKNSt16initializer_listI10SizeType32EE", false]], "tensorrt_llm::runtime::itensor::sharedconstptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor14SharedConstPtrE", false]], "tensorrt_llm::runtime::itensor::sharedptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor9SharedPtrE", false]], "tensorrt_llm::runtime::itensor::slice (c++ function)": [[1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", false], [1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", false], [1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5Shape", false], [1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5ShapeNSt6size_tE", false], [1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EE", false], [1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EENSt6size_tE", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrNSt6size_tE", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrNSt6size_tENSt6size_tE", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRK5Shape", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRK5Shape9DimType64", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRKNSt16initializer_listI9DimType64EE", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRKNSt16initializer_listI9DimType64EE9DimType64", false]], "tensorrt_llm::runtime::itensor::squeeze (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor7squeezeE10SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor7squeezeERK5Shape10SizeType32", false]], "tensorrt_llm::runtime::itensor::strides (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor7stridesERK5Shape", false]], "tensorrt_llm::runtime::itensor::tensormap (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor9TensorMapE", false]], "tensorrt_llm::runtime::itensor::tostring (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor8toStringERK5Shape", false]], "tensorrt_llm::runtime::itensor::uniqueconstptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor14UniqueConstPtrE", false]], "tensorrt_llm::runtime::itensor::uniqueptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor9UniquePtrE", false]], "tensorrt_llm::runtime::itensor::unsqueeze (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor9unsqueezeE10SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor9unsqueezeERK5Shape10SizeType32", false]], "tensorrt_llm::runtime::itensor::view (c++ function)": [[1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor4viewE14UniqueConstPtrRR9TConstPtrRK5Shape", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor4viewE9SharedPtr", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor4viewEN7IBuffer9SharedPtrERK5Shape", false]], "tensorrt_llm::runtime::itensor::volume (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor6volumeERK5Shape", false]], "tensorrt_llm::runtime::itensor::volumenonnegative (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensor17volumeNonNegativeERK5Shape", false]], "tensorrt_llm::runtime::itensor::wrap (c++ function)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5Shape", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5ShapeNSt6size_tE", false], [1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrRNSt6vectorI1TEERK5Shape", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5Shape", false], [1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5ShapeNSt6size_tE", false]], "tensorrt_llm::runtime::itensor::~itensor (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime7ITensorD0Ev", false]], "tensorrt_llm::runtime::lamportinitializeall (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20lamportInitializeAllEPvPvPv6size_t", false]], "tensorrt_llm::runtime::localcreator (c++ struct)": [[1, "_CPPv4I_bEN12tensorrt_llm7runtime12LocalCreatorE", false]], "tensorrt_llm::runtime::localcreator::create (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator6createEv", false]], "tensorrt_llm::runtime::localcreator::localcreator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator12LocalCreatorERK19CUmemAllocationProp6size_t", false]], "tensorrt_llm::runtime::localcreator::mprop (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator5mPropE", false]], "tensorrt_llm::runtime::localcreator::msize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator5mSizeE", false]], "tensorrt_llm::runtime::localcreator::release (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator7releaseE28CUmemGenericAllocationHandleb", false]], "tensorrt_llm::runtime::lookaheaddecodingbuffers (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffersE", false]], "tensorrt_llm::runtime::lookaheaddecodingbuffers::generationlengths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers17generationLengthsE", false]], "tensorrt_llm::runtime::lookaheaddecodingbuffers::lookaheaddecodingbuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers24LookaheadDecodingBuffersE10SizeType3210SizeType32RK13BufferManager", false]], "tensorrt_llm::runtime::lookaheaddecodingbuffers::packedmasks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers11packedMasksE", false]], "tensorrt_llm::runtime::lookaheaddecodingbuffers::positionids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers11positionIdsE", false]], "tensorrt_llm::runtime::lookaheaddecodingbuffers::positionoffsets (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers15positionOffsetsE", false]], "tensorrt_llm::runtime::lookaheaddecodingbuffers::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers9TensorPtrE", false]], "tensorrt_llm::runtime::lookaheadmodule (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModuleE", false]], "tensorrt_llm::runtime::lookaheadmodule::getexecutionconfig (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime15LookaheadModule18getExecutionConfigEv", false]], "tensorrt_llm::runtime::lookaheadmodule::lookaheadmodule (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule15LookaheadModuleE10SizeType3210SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule15LookaheadModuleEv", false]], "tensorrt_llm::runtime::lookaheadmodule::mexecutionconfig (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule16mExecutionConfigE", false]], "tensorrt_llm::runtime::lookaheadmodule::setexecutionconfig (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule18setExecutionConfigERKN8executor23LookaheadDecodingConfigE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffersE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::batchslotshostcopy (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers18batchSlotsHostCopyE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::cumsumlength (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers12cumSumLengthE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::disablelookaheaddecoding (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers24disableLookaheadDecodingEv", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::enablelookaheaddecoding (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23enableLookaheadDecodingE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::generationlengthsdevice (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23generationLengthsDeviceE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::generationlengthshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers21generationLengthsHostE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::generationlengthshostcopy (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers25generationLengthsHostCopyE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::insertinputtensors (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers18insertInputTensorsER9TensorMapR9TensorMapRK11WorldConfig", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::lookaheadruntimebuffers (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23LookaheadRuntimeBuffersE10SizeType3210SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfigRKN8executor14DecodingConfigERK11TllmRuntime", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::packedmaskhost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers14packedMaskHostE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::packedmaskhostcopy (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers18packedMaskHostCopyE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::packedmasksdevice (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers17packedMasksDeviceE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::positionidsdevice (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers17positionIdsDeviceE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::positionidshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers15positionIdsHostE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::positionidshostcopy (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers19positionIdsHostCopyE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::positionoffsetsdevice (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers21positionOffsetsDeviceE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::positionoffsetshost (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers19positionOffsetsHostE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::positionoffsetshostcopy (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23positionOffsetsHostCopyE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::reshape (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers7reshapeE10SizeType3210SizeType3210SizeType32", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::setfrominputs (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::tensormap (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers9TensorMapE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers9TensorPtrE", false]], "tensorrt_llm::runtime::lookaheadruntimebuffers::usespecdecoding (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers15useSpecDecodingE", false]], "tensorrt_llm::runtime::loracache (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCacheE", false]], "tensorrt_llm::runtime::loracache::bump (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache4bumpE10TaskIdType", false]], "tensorrt_llm::runtime::loracache::bumptaskinprogress (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache18bumpTaskInProgressE10TaskIdType", false]], "tensorrt_llm::runtime::loracache::claimpageswithevict (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache19claimPagesWithEvictE10SizeType32", false]], "tensorrt_llm::runtime::loracache::copytask (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache8copyTaskE10TaskIdTypeR9LoraCacheb", false]], "tensorrt_llm::runtime::loracache::copytaskmappages (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache16copyTaskMapPagesER9TaskValueRK9TaskValueRKNSt6vectorI6size_tEERK9LoraCache", false]], "tensorrt_llm::runtime::loracache::copytopages (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", false]], "tensorrt_llm::runtime::loracache::determinenumpages (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache17determineNumPagesE10TaskIdType", false], [1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache17determineNumPagesE9TensorPtr", false]], "tensorrt_llm::runtime::loracache::fits (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache4fitsE9TensorPtr", false]], "tensorrt_llm::runtime::loracache::get (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache3getE10TaskIdType", false]], "tensorrt_llm::runtime::loracache::getnumpages (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache11getNumPagesEv", false]], "tensorrt_llm::runtime::loracache::getpageptr (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache10getPagePtrE6size_t", false]], "tensorrt_llm::runtime::loracache::getstatus (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache9getStatusE10TaskIdType", false]], "tensorrt_llm::runtime::loracache::has (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache3hasE10TaskIdType", false]], "tensorrt_llm::runtime::loracache::isdone (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache6isDoneE10TaskIdType", false]], "tensorrt_llm::runtime::loracache::isloaded (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache8isLoadedE10TaskIdType", false]], "tensorrt_llm::runtime::loracache::loadweights (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsE10TaskIdType9TensorPtr9TensorPtr", false], [1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsER9TaskValue9TensorPtr9TensorPtr", false]], "tensorrt_llm::runtime::loracache::loracache (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9LoraCacheERK26LoraCachePageManagerConfigRK11ModelConfigRK11WorldConfigRK13BufferManager", false]], "tensorrt_llm::runtime::loracache::markalldone (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11markAllDoneEv", false]], "tensorrt_llm::runtime::loracache::marktaskdone (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache12markTaskDoneE10TaskIdType", false]], "tensorrt_llm::runtime::loracache::mbuffermanager (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache14mBufferManagerE", false]], "tensorrt_llm::runtime::loracache::mcachemap (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9mCacheMapE", false]], "tensorrt_llm::runtime::loracache::mcachemutex (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11mCacheMutexE", false]], "tensorrt_llm::runtime::loracache::mcachepagemanager (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17mCachePageManagerE", false]], "tensorrt_llm::runtime::loracache::mdevicebuffermanagers (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21mDeviceBufferManagersE", false]], "tensorrt_llm::runtime::loracache::mdonetasks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache10mDoneTasksE", false]], "tensorrt_llm::runtime::loracache::minprogresstasks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache16mInProgressTasksE", false]], "tensorrt_llm::runtime::loracache::mmodelconfig (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache12mModelConfigE", false]], "tensorrt_llm::runtime::loracache::mmoduleidtomodule (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17mModuleIdToModuleE", false]], "tensorrt_llm::runtime::loracache::mpagemanagerconfig (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache18mPageManagerConfigE", false]], "tensorrt_llm::runtime::loracache::mpagesmutex (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11mPagesMutexE", false]], "tensorrt_llm::runtime::loracache::mworldconfig (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache12mWorldConfigE", false]], "tensorrt_llm::runtime::loracache::put (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache3putE10TaskIdType9TensorPtr9TensorPtrb", false]], "tensorrt_llm::runtime::loracache::splittransposecpu (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17splitTransposeCpuER7ITensorRK7ITensor10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::loracache::splittransposecpuinner (c++ function)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime9LoraCache22splitTransposeCpuInnerEvR7ITensorRK7ITensor10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::loracache::taskidtype (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache10TaskIdTypeE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfigE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::adaptersize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig11adapterSizeE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::insize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig6inSizeE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::layerid (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig7layerIdE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::moduleid (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig8moduleIdE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::numslots (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig8numSlotsE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::operator== (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfigeqERKN9LoraCache21TaskLayerModuleConfigE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::outsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig7outSizeE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::pageid (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig6pageIdE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::scalingvecpointer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig17scalingVecPointerE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::slotidx (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig7slotIdxE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::tostring (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig8toStringEv", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::weightsinpointer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig16weightsInPointerE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfig::weightsoutpointer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig17weightsOutPointerE", false]], "tensorrt_llm::runtime::loracache::tasklayermoduleconfiglistptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache28TaskLayerModuleConfigListPtrE", false]], "tensorrt_llm::runtime::loracache::taskvalue (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValueE", false]], "tensorrt_llm::runtime::loracache::taskvalue::configs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue7configsE", false]], "tensorrt_llm::runtime::loracache::taskvalue::done (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue4doneE", false]], "tensorrt_llm::runtime::loracache::taskvalue::inprogress (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue10inProgressE", false]], "tensorrt_llm::runtime::loracache::taskvalue::it (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue2itE", false]], "tensorrt_llm::runtime::loracache::taskvalue::loaded (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue6loadedE", false]], "tensorrt_llm::runtime::loracache::taskvalue::loadinprogress (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue14loadInProgressE", false]], "tensorrt_llm::runtime::loracache::taskvalue::operator= (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValueaSERR9TaskValue", false]], "tensorrt_llm::runtime::loracache::taskvalue::pageids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue7pageIdsE", false]], "tensorrt_llm::runtime::loracache::taskvalue::taskvalue (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERKNSt6vectorINSt6size_tEEERK28TaskLayerModuleConfigListPtrNSt4listI10TaskIdTypeE8iteratorEbbbb", false], [1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERR9TaskValue", false], [1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueEv", false]], "tensorrt_llm::runtime::loracache::taskvalue::~taskvalue (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValueD0Ev", false]], "tensorrt_llm::runtime::loracache::taskvalueptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache12TaskValuePtrE", false]], "tensorrt_llm::runtime::loracache::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TensorPtrE", false]], "tensorrt_llm::runtime::loracache::valuestatus (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatusE", false]], "tensorrt_llm::runtime::loracache::valuestatus::kvalue_status_loaded (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatus20kVALUE_STATUS_LOADEDE", false]], "tensorrt_llm::runtime::loracache::valuestatus::kvalue_status_missing (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatus21kVALUE_STATUS_MISSINGE", false]], "tensorrt_llm::runtime::loracache::valuestatus::kvalue_status_processing (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatus24kVALUE_STATUS_PROCESSINGE", false]], "tensorrt_llm::runtime::loracachefullexception (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime22LoraCacheFullExceptionE", false]], "tensorrt_llm::runtime::loracachefullexception::loracachefullexception (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22LoraCacheFullException22LoraCacheFullExceptionERKNSt6stringE", false]], "tensorrt_llm::runtime::loracachefullexception::~loracachefullexception (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime22LoraCacheFullExceptionD0Ev", false]], "tensorrt_llm::runtime::loracachepagemanager (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManagerE", false]], "tensorrt_llm::runtime::loracachepagemanager::blockptr (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime20LoraCachePageManager8blockPtrE10SizeType32", false]], "tensorrt_llm::runtime::loracachepagemanager::claimpages (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager10claimPagesE10SizeType32", false]], "tensorrt_llm::runtime::loracachepagemanager::initialize (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager10initializeERK13BufferManager", false]], "tensorrt_llm::runtime::loracachepagemanager::loracachepagemanager (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager20LoraCachePageManagerERK26LoraCachePageManagerConfigRK13BufferManager", false]], "tensorrt_llm::runtime::loracachepagemanager::mconfig (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager7mConfigE", false]], "tensorrt_llm::runtime::loracachepagemanager::mfreepageids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager12mFreePageIdsE", false]], "tensorrt_llm::runtime::loracachepagemanager::mispagefree (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager11mIsPageFreeE", false]], "tensorrt_llm::runtime::loracachepagemanager::mpageblocks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager11mPageBlocksE", false]], "tensorrt_llm::runtime::loracachepagemanager::mutablepageptr (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager14mutablePagePtrENSt6size_tE", false]], "tensorrt_llm::runtime::loracachepagemanager::numavailablepages (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime20LoraCachePageManager17numAvailablePagesEv", false]], "tensorrt_llm::runtime::loracachepagemanager::pageptr (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime20LoraCachePageManager7pagePtrENSt6size_tE", false]], "tensorrt_llm::runtime::loracachepagemanager::releasepages (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager12releasePagesERKNSt6vectorINSt6size_tEEE", false]], "tensorrt_llm::runtime::loracachepagemanager::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager9TensorPtrE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfigE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::getdatatype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig11getDataTypeEv", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::getinittozero (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig13getInitToZeroEv", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::getmaxpagesperblock (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig19getMaxPagesPerBlockEv", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::getmemorytype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig13getMemoryTypeEv", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::getnumcopystreams (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig17getNumCopyStreamsEv", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::getpagewidth (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig12getPageWidthEv", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::getslotsperpage (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig15getSlotsPerPageEv", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::gettotalnumpages (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig16getTotalNumPagesEv", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::loracachepagemanagerconfig (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig26LoraCachePageManagerConfigEN7runtime10MemoryTypeEN8nvinfer18DataTypeE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::mdatatype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig9mDataTypeE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::minittozero (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig11mInitToZeroE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::mmaxpagesperblock (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig17mMaxPagesPerBlockE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::mmemorytype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig11mMemoryTypeE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::mnumcopystreams (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig15mNumCopyStreamsE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::mpagewidth (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig10mPageWidthE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::mslotsperpage (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig13mSlotsPerPageE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::mtotalnumpages (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig14mTotalNumPagesE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::setdatatype (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig11setDataTypeERKN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::setinittozero (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig13setInitToZeroEb", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::setmaxpagesperblock (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig19setMaxPagesPerBlockERK10SizeType32", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::setmemorytype (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig13setMemoryTypeERKN7runtime10MemoryTypeE", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::setnumcopystreams (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig17setNumCopyStreamsE10SizeType32", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::setpagewidth (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig12setPageWidthERK10SizeType32", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::setslotsperpage (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig15setSlotsPerPageERK10SizeType32", false]], "tensorrt_llm::runtime::loracachepagemanagerconfig::settotalnumpage (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig15setTotalNumPageERK10SizeType32", false]], "tensorrt_llm::runtime::loraexpectedexception (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime21LoraExpectedExceptionE", false]], "tensorrt_llm::runtime::loraexpectedexception::loraexpectedexception (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime21LoraExpectedException21LoraExpectedExceptionERKNSt6stringE", false]], "tensorrt_llm::runtime::loraexpectedexception::~loraexpectedexception (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime21LoraExpectedExceptionD0Ev", false]], "tensorrt_llm::runtime::loramodule (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModuleE", false]], "tensorrt_llm::runtime::loramodule::createloramodules (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", false]], "tensorrt_llm::runtime::loramodule::flattenedinoutsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule18flattenedInOutSizeE10SizeType32b", false]], "tensorrt_llm::runtime::loramodule::indim (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule5inDimEv", false]], "tensorrt_llm::runtime::loramodule::indimfirst (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule10inDimFirstEv", false]], "tensorrt_llm::runtime::loramodule::insize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule6inSizeE10SizeType32", false]], "tensorrt_llm::runtime::loramodule::intpsplitdim (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule12inTpSplitDimEv", false]], "tensorrt_llm::runtime::loramodule::localinadaptersize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule18localInAdapterSizeE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::loramodule::localindim (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule10localInDimE10SizeType32", false]], "tensorrt_llm::runtime::loramodule::localinoutsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule14localInOutSizeE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::loramodule::localinsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule11localInSizeE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::loramodule::localoutadaptersize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule19localOutAdapterSizeE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::loramodule::localoutdim (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule11localOutDimE10SizeType32", false]], "tensorrt_llm::runtime::loramodule::localoutsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule12localOutSizeE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::loramodule::localscalessize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule15localScalesSizeE10SizeType32b", false]], "tensorrt_llm::runtime::loramodule::localtotalsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule14localTotalSizeE10SizeType3210SizeType32b", false]], "tensorrt_llm::runtime::loramodule::loramodule (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10LoraModule", false], [1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10ModuleType10SizeType3210SizeType32bb10SizeType3210SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleEv", false]], "tensorrt_llm::runtime::loramodule::mindim (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule6mInDimE", false]], "tensorrt_llm::runtime::loramodule::mindimfirst (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule11mInDimFirstE", false]], "tensorrt_llm::runtime::loramodule::mintpsplitdim (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule13mInTpSplitDimE", false]], "tensorrt_llm::runtime::loramodule::moduletype (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleTypeE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kattn_dense (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType11kATTN_DENSEE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kattn_k (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType7kATTN_KE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kattn_q (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType7kATTN_QE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kattn_qkv (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType9kATTN_QKVE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kattn_v (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType7kATTN_VE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kcross_attn_dense (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType17kCROSS_ATTN_DENSEE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kcross_attn_k (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType13kCROSS_ATTN_KE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kcross_attn_q (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType13kCROSS_ATTN_QE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kcross_attn_qkv (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType15kCROSS_ATTN_QKVE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kcross_attn_v (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType13kCROSS_ATTN_VE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kinvalid (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType8kINVALIDE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kmlp_4h_to_h (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMLP_4H_TO_HE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kmlp_gate (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType9kMLP_GATEE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kmlp_gate_up (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMLP_GATE_UPE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kmlp_h_to_4h (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMLP_H_TO_4HE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kmlp_router (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType11kMLP_ROUTERE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kmoe_4h_to_h (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMOE_4H_TO_HE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kmoe_gate (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType9kMOE_GATEE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kmoe_h_to_4h (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMOE_H_TO_4HE", false]], "tensorrt_llm::runtime::loramodule::moduletype::kmoe_router (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType11kMOE_ROUTERE", false]], "tensorrt_llm::runtime::loramodule::moutdim (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule7mOutDimE", false]], "tensorrt_llm::runtime::loramodule::moutdimfirst (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12mOutDimFirstE", false]], "tensorrt_llm::runtime::loramodule::mouttpsplitdim (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule14mOutTpSplitDimE", false]], "tensorrt_llm::runtime::loramodule::mtype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule5mTypeE", false]], "tensorrt_llm::runtime::loramodule::name (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule4nameEv", false]], "tensorrt_llm::runtime::loramodule::operator= (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModuleaSERK10LoraModule", false]], "tensorrt_llm::runtime::loramodule::outdim (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule6outDimEv", false]], "tensorrt_llm::runtime::loramodule::outdimfirst (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule11outDimFirstEv", false]], "tensorrt_llm::runtime::loramodule::outsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule7outSizeE10SizeType32", false]], "tensorrt_llm::runtime::loramodule::outtpsplitdim (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule13outTpSplitDimEv", false]], "tensorrt_llm::runtime::loramodule::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule9TensorPtrE", false]], "tensorrt_llm::runtime::loramodule::tomodulename (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12toModuleNameE10ModuleType", false], [1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12toModuleNameE10SizeType32", false]], "tensorrt_llm::runtime::loramodule::tomoduletype (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12toModuleTypeERKNSt11string_viewE", false]], "tensorrt_llm::runtime::loramodule::value (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule5valueEv", false]], "tensorrt_llm::runtime::lorataskidtype (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14LoraTaskIdTypeE", false]], "tensorrt_llm::runtime::medusamodule (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime12MedusaModuleE", false]], "tensorrt_llm::runtime::medusamodule::getmedusachoices (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime12MedusaModule16getMedusaChoicesEv", false]], "tensorrt_llm::runtime::medusamodule::mdefaultmedusachoices (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule21mDefaultMedusaChoicesE", false]], "tensorrt_llm::runtime::medusamodule::medusachoices (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule13MedusaChoicesE", false]], "tensorrt_llm::runtime::medusamodule::medusamodule (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule12MedusaModuleE10SizeType3210SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule12MedusaModuleEv", false]], "tensorrt_llm::runtime::medusamodule::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule9TensorPtrE", false]], "tensorrt_llm::runtime::memorycounters (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCountersE", false]], "tensorrt_llm::runtime::memorycounters::allocate (c++ function)": [[1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime14MemoryCounters8allocateEv10SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8allocateE10MemoryType10SizeType32", false]], "tensorrt_llm::runtime::memorycounters::bytestostring (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters13bytesToStringE10SizeType32i", false], [1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters13bytesToStringE8DiffTypei", false]], "tensorrt_llm::runtime::memorycounters::deallocate (c++ function)": [[1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime14MemoryCounters10deallocateEv10SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters10deallocateE10MemoryType10SizeType32", false]], "tensorrt_llm::runtime::memorycounters::difftype (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8DiffTypeE", false]], "tensorrt_llm::runtime::memorycounters::getcpu (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters6getCpuEv", false]], "tensorrt_llm::runtime::memorycounters::getcpudiff (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters10getCpuDiffEv", false]], "tensorrt_llm::runtime::memorycounters::getgpu (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters6getGpuEv", false]], "tensorrt_llm::runtime::memorycounters::getgpudiff (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters10getGpuDiffEv", false]], "tensorrt_llm::runtime::memorycounters::getinstance (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters11getInstanceEv", false]], "tensorrt_llm::runtime::memorycounters::getpinned (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters9getPinnedEv", false]], "tensorrt_llm::runtime::memorycounters::getpinneddiff (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters13getPinnedDiffEv", false]], "tensorrt_llm::runtime::memorycounters::getpinnedpool (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters13getPinnedPoolEv", false]], "tensorrt_llm::runtime::memorycounters::getpinnedpooldiff (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters17getPinnedPoolDiffEv", false]], "tensorrt_llm::runtime::memorycounters::getuvm (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters6getUVMEv", false]], "tensorrt_llm::runtime::memorycounters::getuvmdiff (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters10getUVMDiffEv", false]], "tensorrt_llm::runtime::memorycounters::mcpu (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters4mCpuE", false]], "tensorrt_llm::runtime::memorycounters::mcpudiff (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8mCpuDiffE", false]], "tensorrt_llm::runtime::memorycounters::memorycounters (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters14MemoryCountersEv", false]], "tensorrt_llm::runtime::memorycounters::mgpu (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters4mGpuE", false]], "tensorrt_llm::runtime::memorycounters::mgpudiff (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8mGpuDiffE", false]], "tensorrt_llm::runtime::memorycounters::mpinned (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters7mPinnedE", false]], "tensorrt_llm::runtime::memorycounters::mpinneddiff (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters11mPinnedDiffE", false]], "tensorrt_llm::runtime::memorycounters::mpinnedpool (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters11mPinnedPoolE", false]], "tensorrt_llm::runtime::memorycounters::mpinnedpooldiff (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters15mPinnedPoolDiffE", false]], "tensorrt_llm::runtime::memorycounters::muvm (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters4mUVME", false]], "tensorrt_llm::runtime::memorycounters::muvmdiff (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8mUVMDiffE", false]], "tensorrt_llm::runtime::memorycounters::sizetype32 (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters10SizeType32E", false]], "tensorrt_llm::runtime::memorycounters::tostring (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters8toStringEv", false]], "tensorrt_llm::runtime::memorytype (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime10MemoryTypeE", false]], "tensorrt_llm::runtime::memorytype::kcpu (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10MemoryType4kCPUE", false]], "tensorrt_llm::runtime::memorytype::kgpu (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10MemoryType4kGPUE", false]], "tensorrt_llm::runtime::memorytype::kpinned (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10MemoryType7kPINNEDE", false]], "tensorrt_llm::runtime::memorytype::kpinnedpool (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10MemoryType11kPINNEDPOOLE", false]], "tensorrt_llm::runtime::memorytype::kuvm (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime10MemoryType4kUVME", false]], "tensorrt_llm::runtime::memorytypestring (c++ struct)": [[1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime16MemoryTypeStringE", false]], "tensorrt_llm::runtime::memorytypestring (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kCPUEEE", false]], "tensorrt_llm::runtime::memorytypestring::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kCPUEE5valueE", false]], "tensorrt_llm::runtime::memorytypestring (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kGPUEEE", false]], "tensorrt_llm::runtime::memorytypestring::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kGPUEE5valueE", false]], "tensorrt_llm::runtime::memorytypestring (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType7kPINNEDEEE", false]], "tensorrt_llm::runtime::memorytypestring::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType7kPINNEDEE5valueE", false]], "tensorrt_llm::runtime::memorytypestring (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType11kPINNEDPOOLEEE", false]], "tensorrt_llm::runtime::memorytypestring::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType11kPINNEDPOOLEE5valueE", false]], "tensorrt_llm::runtime::memorytypestring (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kUVMEEE", false]], "tensorrt_llm::runtime::memorytypestring::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kUVMEE5valueE", false]], "tensorrt_llm::runtime::memsetconfigurator (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfiguratorE", false]], "tensorrt_llm::runtime::memsetconfigurator::maddress (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator8mAddressE", false]], "tensorrt_llm::runtime::memsetconfigurator::memsetconfigurator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator18MemsetConfiguratorE11CUdeviceptr6size_t7uint8_t8CUstream", false]], "tensorrt_llm::runtime::memsetconfigurator::mfirsttime (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator10mFirstTimeE", false]], "tensorrt_llm::runtime::memsetconfigurator::msize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator5mSizeE", false]], "tensorrt_llm::runtime::memsetconfigurator::mstream (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator7mStreamE", false]], "tensorrt_llm::runtime::memsetconfigurator::mvalue (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator6mValueE", false]], "tensorrt_llm::runtime::memsetconfigurator::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator5setupE28CUmemGenericAllocationHandle", false]], "tensorrt_llm::runtime::memsetconfigurator::teardown (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator8teardownE28CUmemGenericAllocationHandleb", false]], "tensorrt_llm::runtime::modelconfig (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfigE", false]], "tensorrt_llm::runtime::modelconfig::computecontextlogits (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20computeContextLogitsEb", false], [1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20computeContextLogitsEv", false]], "tensorrt_llm::runtime::modelconfig::computegenerationlogits (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig23computeGenerationLogitsEb", false], [1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig23computeGenerationLogitsEv", false]], "tensorrt_llm::runtime::modelconfig::countlocallayers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16countLocalLayersE9LayerType10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::modelconfig::countlowerranklayers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20countLowerRankLayersE9LayerType10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::modelconfig::disableseamlesslookaheaddecoding (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig32disableSeamlessLookaheadDecodingEv", false]], "tensorrt_llm::runtime::modelconfig::enableseamlesslookaheaddecoding (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig31enableSeamlessLookaheadDecodingE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::getcontextfmha (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getContextFMHAEv", false]], "tensorrt_llm::runtime::modelconfig::getdatatype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig11getDataTypeEv", false]], "tensorrt_llm::runtime::modelconfig::getencoderhiddensize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getEncoderHiddenSizeEv", false]], "tensorrt_llm::runtime::modelconfig::getfirstlocallayer (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18getFirstLocalLayerE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::modelconfig::getgemmallreducedtype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21getGemmAllReduceDtypeEv", false]], "tensorrt_llm::runtime::modelconfig::gethiddensize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13getHiddenSizeEv", false]], "tensorrt_llm::runtime::modelconfig::getkvcachetype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getKVCacheTypeEv", false]], "tensorrt_llm::runtime::modelconfig::getkvdatatype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13getKvDataTypeEv", false]], "tensorrt_llm::runtime::modelconfig::getlayertypes (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13getLayerTypesEv", false]], "tensorrt_llm::runtime::modelconfig::getlogitsdtype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getLogitsDtypeEv", false]], "tensorrt_llm::runtime::modelconfig::getloramodules (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getLoraModulesEv", false]], "tensorrt_llm::runtime::modelconfig::getmanageweightstype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getManageWeightsTypeEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxbatchsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getMaxBatchSizeEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxbeamwidth (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getMaxBeamWidthEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxdecodingdrafttokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig25getMaxDecodingDraftTokensEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxdecodingtokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getMaxDecodingTokensEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxencoderlen (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16getMaxEncoderLenEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxinputlen (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getMaxInputLenEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxlorarank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getMaxLoraRankEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxnumtokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getMaxNumTokensEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxpositionembeddings (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig24getMaxPositionEmbeddingsEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxpromptembeddingtablesize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig30getMaxPromptEmbeddingTableSizeEv", false]], "tensorrt_llm::runtime::modelconfig::getmaxsequencelen (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig17getMaxSequenceLenEv", false]], "tensorrt_llm::runtime::modelconfig::getmlphiddensize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16getMlpHiddenSizeEv", false]], "tensorrt_llm::runtime::modelconfig::getmodelname (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getModelNameEv", false]], "tensorrt_llm::runtime::modelconfig::getmodelvariant (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getModelVariantEv", false]], "tensorrt_llm::runtime::modelconfig::getnbattentionlayers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getNbAttentionLayersE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::modelconfig::getnbheads (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig10getNbHeadsEv", false]], "tensorrt_llm::runtime::modelconfig::getnbkvheads (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getNbKvHeadsE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::getnblayers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig11getNbLayersE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::modelconfig::getnbrnnlayers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getNbRnnLayersE10SizeType3210SizeType32", false]], "tensorrt_llm::runtime::modelconfig::getnumkvheadsforgivenlayers (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig27getNumKvHeadsForGivenLayersERKNSt6vectorI10SizeType32EEb", false]], "tensorrt_llm::runtime::modelconfig::getnumkvheadsperlayer (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21getNumKvHeadsPerLayerEv", false]], "tensorrt_llm::runtime::modelconfig::getnumkvheadsperlayerlocalrange (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig31getNumKvHeadsPerLayerLocalRangeE10SizeType3210SizeType32b", false]], "tensorrt_llm::runtime::modelconfig::getnumlanguages (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getNumLanguagesEv", false]], "tensorrt_llm::runtime::modelconfig::getoptprofilessplitpoints (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig25getOptProfilesSplitPointsEv", false]], "tensorrt_llm::runtime::modelconfig::getpagedcontextfmha (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig19getPagedContextFMHAEv", false]], "tensorrt_llm::runtime::modelconfig::getppreducescatter (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18getPpReduceScatterEv", false]], "tensorrt_llm::runtime::modelconfig::getquantmode (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getQuantModeEv", false]], "tensorrt_llm::runtime::modelconfig::getrnnconfig (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getRnnConfigEv", false]], "tensorrt_llm::runtime::modelconfig::getrotaryembeddingdim (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21getRotaryEmbeddingDimEv", false]], "tensorrt_llm::runtime::modelconfig::getsizeperhead (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getSizePerHeadEv", false]], "tensorrt_llm::runtime::modelconfig::getspeculativedecodingmode (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig26getSpeculativeDecodingModeEv", false]], "tensorrt_llm::runtime::modelconfig::getspeculativedecodingmodule (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig28getSpeculativeDecodingModuleEv", false]], "tensorrt_llm::runtime::modelconfig::getspeculativedecodingmoduleptr (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig31getSpeculativeDecodingModulePtrEv", false], [1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig31getSpeculativeDecodingModulePtrEv", false]], "tensorrt_llm::runtime::modelconfig::gettokensperblock (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig17getTokensPerBlockEv", false]], "tensorrt_llm::runtime::modelconfig::getvocabsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getVocabSizeEv", false]], "tensorrt_llm::runtime::modelconfig::getvocabsizepadded (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18getVocabSizePaddedE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::hasrnnconfig (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12hasRnnConfigEv", false]], "tensorrt_llm::runtime::modelconfig::hasspeculativedecodingmodule (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig28hasSpeculativeDecodingModuleEv", false]], "tensorrt_llm::runtime::modelconfig::iscontinuouskvcache (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig19isContinuousKVCacheEv", false]], "tensorrt_llm::runtime::modelconfig::iskvcacheenabled (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16isKVCacheEnabledEv", false]], "tensorrt_llm::runtime::modelconfig::ismultimodal (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12isMultiModalEv", false]], "tensorrt_llm::runtime::modelconfig::ispagedkvcache (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14isPagedKVCacheEv", false]], "tensorrt_llm::runtime::modelconfig::isrnnbased (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig10isRnnBasedEv", false]], "tensorrt_llm::runtime::modelconfig::istransformerbased (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18isTransformerBasedEv", false]], "tensorrt_llm::runtime::modelconfig::iswhisper (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig9isWhisperEv", false]], "tensorrt_llm::runtime::modelconfig::kdefault_num_tokens_per_block (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig29kDEFAULT_NUM_TOKENS_PER_BLOCKE", false]], "tensorrt_llm::runtime::modelconfig::kopt_profiles_split_points (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26kOPT_PROFILES_SPLIT_POINTSE", false]], "tensorrt_llm::runtime::modelconfig::kvcachetype (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11KVCacheTypeE", false]], "tensorrt_llm::runtime::modelconfig::kvcachetype::kcontinuous (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11KVCacheType11kCONTINUOUSE", false]], "tensorrt_llm::runtime::modelconfig::kvcachetype::kdisabled (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11KVCacheType9kDISABLEDE", false]], "tensorrt_llm::runtime::modelconfig::kvcachetype::kpaged (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11KVCacheType6kPAGEDE", false]], "tensorrt_llm::runtime::modelconfig::kvcachetypefromstring (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21KVCacheTypeFromStringENSt6stringE", false]], "tensorrt_llm::runtime::modelconfig::layertype (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerTypeE", false]], "tensorrt_llm::runtime::modelconfig::layertype::kattention (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerType10kATTENTIONE", false]], "tensorrt_llm::runtime::modelconfig::layertype::klinear (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerType7kLINEARE", false]], "tensorrt_llm::runtime::modelconfig::layertype::knoop (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerType5kNOOPE", false]], "tensorrt_llm::runtime::modelconfig::layertype::krecurrent (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerType10kRECURRENTE", false]], "tensorrt_llm::runtime::modelconfig::manageweightstype (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17ManageWeightsTypeE", false]], "tensorrt_llm::runtime::modelconfig::manageweightstype::kdisabled (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17ManageWeightsType9kDisabledE", false]], "tensorrt_llm::runtime::modelconfig::manageweightstype::kenabled (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17ManageWeightsType8kEnabledE", false]], "tensorrt_llm::runtime::modelconfig::mcomputecontextlogits (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21mComputeContextLogitsE", false]], "tensorrt_llm::runtime::modelconfig::mcomputegenerationlogits (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24mComputeGenerationLogitsE", false]], "tensorrt_llm::runtime::modelconfig::mcontextfmha (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mContextFMHAE", false]], "tensorrt_llm::runtime::modelconfig::mdatatype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9mDataTypeE", false]], "tensorrt_llm::runtime::modelconfig::mencoderhiddensize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mEncoderHiddenSizeE", false]], "tensorrt_llm::runtime::modelconfig::mgemmallreducedtype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig19mGemmAllReduceDtypeE", false]], "tensorrt_llm::runtime::modelconfig::mhiddensize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11mHiddenSizeE", false]], "tensorrt_llm::runtime::modelconfig::minputpacked (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mInputPackedE", false]], "tensorrt_llm::runtime::modelconfig::mkvcachetype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mKVCacheTypeE", false]], "tensorrt_llm::runtime::modelconfig::mlayertypes (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11mLayerTypesE", false]], "tensorrt_llm::runtime::modelconfig::mlogitsdtype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mLogitsDtypeE", false]], "tensorrt_llm::runtime::modelconfig::mloramodules (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mLoraModulesE", false]], "tensorrt_llm::runtime::modelconfig::mmanageweightstype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mManageWeightsTypeE", false]], "tensorrt_llm::runtime::modelconfig::mmaxbatchsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mMaxBatchSizeE", false]], "tensorrt_llm::runtime::modelconfig::mmaxbeamwidth (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mMaxBeamWidthE", false]], "tensorrt_llm::runtime::modelconfig::mmaxencoderlen (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14mMaxEncoderLenE", false]], "tensorrt_llm::runtime::modelconfig::mmaxinputlen (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mMaxInputLenE", false]], "tensorrt_llm::runtime::modelconfig::mmaxlorarank (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mMaxLoraRankE", false]], "tensorrt_llm::runtime::modelconfig::mmaxnumtokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mMaxNumTokensE", false]], "tensorrt_llm::runtime::modelconfig::mmaxpositionembeddings (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22mMaxPositionEmbeddingsE", false]], "tensorrt_llm::runtime::modelconfig::mmaxpromptembeddingtablesize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig28mMaxPromptEmbeddingTableSizeE", false]], "tensorrt_llm::runtime::modelconfig::mmaxsequencelen (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15mMaxSequenceLenE", false]], "tensorrt_llm::runtime::modelconfig::mmlphiddensize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14mMlpHiddenSizeE", false]], "tensorrt_llm::runtime::modelconfig::mmodelname (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig10mModelNameE", false]], "tensorrt_llm::runtime::modelconfig::mmodelvariant (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mModelVariantE", false]], "tensorrt_llm::runtime::modelconfig::mnbattentionlayers (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mNbAttentionLayersE", false]], "tensorrt_llm::runtime::modelconfig::mnbheads (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig8mNbHeadsE", false]], "tensorrt_llm::runtime::modelconfig::mnblayers (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9mNbLayersE", false]], "tensorrt_llm::runtime::modelconfig::mnbrnnlayers (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mNbRnnLayersE", false]], "tensorrt_llm::runtime::modelconfig::mnumkvheadsperattentionlayer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig28mNumKvHeadsPerAttentionLayerE", false]], "tensorrt_llm::runtime::modelconfig::mnumkvheadspercrossattentionlayer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig33mNumKvHeadsPerCrossAttentionLayerE", false]], "tensorrt_llm::runtime::modelconfig::mnumlanguages (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mNumLanguagesE", false]], "tensorrt_llm::runtime::modelconfig::modelconfig (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11ModelConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::modelconfig::modelvariant (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariantE", false]], "tensorrt_llm::runtime::modelconfig::modelvariant::kchatglm (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant8kChatGlmE", false]], "tensorrt_llm::runtime::modelconfig::modelvariant::kencdec (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant7kEncDecE", false]], "tensorrt_llm::runtime::modelconfig::modelvariant::kglm (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant4kGlmE", false]], "tensorrt_llm::runtime::modelconfig::modelvariant::kgpt (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant4kGptE", false]], "tensorrt_llm::runtime::modelconfig::modelvariant::kmamba (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant6kMambaE", false]], "tensorrt_llm::runtime::modelconfig::modelvariant::krecurrentgemma (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant15kRecurrentGemmaE", false]], "tensorrt_llm::runtime::modelconfig::mpagedcontextfmha (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17mPagedContextFMHAE", false]], "tensorrt_llm::runtime::modelconfig::mpagedstate (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11mPagedStateE", false]], "tensorrt_llm::runtime::modelconfig::mppreducescatter (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig16mPpReduceScatterE", false]], "tensorrt_llm::runtime::modelconfig::mquantmode (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig10mQuantModeE", false]], "tensorrt_llm::runtime::modelconfig::mrnnconfig (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig10mRnnConfigE", false]], "tensorrt_llm::runtime::modelconfig::mrotaryembeddingdim (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig19mRotaryEmbeddingDimE", false]], "tensorrt_llm::runtime::modelconfig::msizeperhead (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mSizePerHeadE", false]], "tensorrt_llm::runtime::modelconfig::mskipcrossattnblocks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20mSkipCrossAttnBlocksE", false]], "tensorrt_llm::runtime::modelconfig::mspeculativedecodingmode (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24mSpeculativeDecodingModeE", false]], "tensorrt_llm::runtime::modelconfig::mspeculativedecodingmodule (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26mSpeculativeDecodingModuleE", false]], "tensorrt_llm::runtime::modelconfig::mtokensperblock (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15mTokensPerBlockE", false]], "tensorrt_llm::runtime::modelconfig::musecrossattention (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mUseCrossAttentionE", false]], "tensorrt_llm::runtime::modelconfig::musegemmallreduceplugin (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig23mUseGemmAllReducePluginE", false]], "tensorrt_llm::runtime::modelconfig::musegptattentionplugin (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22mUseGptAttentionPluginE", false]], "tensorrt_llm::runtime::modelconfig::museloraplugin (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14mUseLoraPluginE", false]], "tensorrt_llm::runtime::modelconfig::musemambaconv1dplugin (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21mUseMambaConv1dPluginE", false]], "tensorrt_llm::runtime::modelconfig::musemrope (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9mUseMropeE", false]], "tensorrt_llm::runtime::modelconfig::musepositionembedding (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21mUsePositionEmbeddingE", false]], "tensorrt_llm::runtime::modelconfig::museshapeinference (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mUseShapeInferenceE", false]], "tensorrt_llm::runtime::modelconfig::musetokentypeembedding (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22mUseTokenTypeEmbeddingE", false]], "tensorrt_llm::runtime::modelconfig::mvocabsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig10mVocabSizeE", false]], "tensorrt_llm::runtime::modelconfig::resetspeculativedecodingmodule (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig30resetSpeculativeDecodingModuleEv", false]], "tensorrt_llm::runtime::modelconfig::rnnconfig (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfigE", false]], "tensorrt_llm::runtime::modelconfig::rnnconfig::convkernel (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig10convKernelE", false]], "tensorrt_llm::runtime::modelconfig::rnnconfig::rnnconvdimsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig14rnnConvDimSizeE", false]], "tensorrt_llm::runtime::modelconfig::rnnconfig::rnnheadsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig11rnnHeadSizeE", false]], "tensorrt_llm::runtime::modelconfig::rnnconfig::rnnhiddensize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig13rnnHiddenSizeE", false]], "tensorrt_llm::runtime::modelconfig::rnnconfig::statesize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig9stateSizeE", false]], "tensorrt_llm::runtime::modelconfig::setcontextfmha (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setContextFMHAEb", false]], "tensorrt_llm::runtime::modelconfig::setencoderhiddensize (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setEncoderHiddenSizeE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setgemmallreducedtype (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21setGemmAllReduceDtypeEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::modelconfig::setkvcachetype (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setKVCacheTypeE11KVCacheType", false]], "tensorrt_llm::runtime::modelconfig::setlayertypes (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13setLayerTypesERKNSt6vectorI9LayerTypeEE", false]], "tensorrt_llm::runtime::modelconfig::setlogitsdtype (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setLogitsDtypeEN8nvinfer18DataTypeE", false]], "tensorrt_llm::runtime::modelconfig::setloramodules (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setLoraModulesERKNSt6vectorI10LoraModuleEE", false]], "tensorrt_llm::runtime::modelconfig::setmanageweightstype (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setManageWeightsTypeEK17ManageWeightsType", false]], "tensorrt_llm::runtime::modelconfig::setmaxbatchsize (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setMaxBatchSizeE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setmaxbeamwidth (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setMaxBeamWidthE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setmaxencoderlen (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig16setMaxEncoderLenE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setmaxinputlen (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setMaxInputLenE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setmaxlorarank (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setMaxLoraRankE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setmaxnumtokens (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setMaxNumTokensENSt8optionalI10SizeType32EE", false]], "tensorrt_llm::runtime::modelconfig::setmaxpositionembeddings (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24setMaxPositionEmbeddingsE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setmaxpromptembeddingtablesize (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig30setMaxPromptEmbeddingTableSizeE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setmaxsequencelen (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17setMaxSequenceLenE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setmlphiddensize (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig16setMlpHiddenSizeE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setmodelname (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setModelNameERKNSt6stringE", false]], "tensorrt_llm::runtime::modelconfig::setmodelvariant (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setModelVariantE12ModelVariant", false]], "tensorrt_llm::runtime::modelconfig::setnbcrosskvheads (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17setNbCrossKvHeadsE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setnbkvheads (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setNbKvHeadsE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setnumkvheadspercrosslayer (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26setNumKvHeadsPerCrossLayerERKNSt6vectorI10SizeType32EE", false]], "tensorrt_llm::runtime::modelconfig::setnumkvheadsperlayer (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21setNumKvHeadsPerLayerERKNSt6vectorI10SizeType32EE", false]], "tensorrt_llm::runtime::modelconfig::setnumlanguages (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setNumLanguagesENSt8optionalI10SizeType32EE", false]], "tensorrt_llm::runtime::modelconfig::setpagedcontextfmha (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig19setPagedContextFMHAEb", false]], "tensorrt_llm::runtime::modelconfig::setppreducescatter (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18setPpReduceScatterEb", false]], "tensorrt_llm::runtime::modelconfig::setquantmode (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setQuantModeEN6common9QuantModeE", false]], "tensorrt_llm::runtime::modelconfig::setrnnconfig (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setRnnConfigERK9RnnConfig", false]], "tensorrt_llm::runtime::modelconfig::setrotaryembeddingdim (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21setRotaryEmbeddingDimE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setsizeperhead (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setSizePerHeadE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setskipcrossattnblocks (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22setSkipCrossAttnBlocksEb", false]], "tensorrt_llm::runtime::modelconfig::setspeculativedecodingmode (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26setSpeculativeDecodingModeE23SpeculativeDecodingMode", false]], "tensorrt_llm::runtime::modelconfig::setspeculativedecodingmodule (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig28setSpeculativeDecodingModuleERKNSt10shared_ptrI25SpeculativeDecodingModuleEE", false]], "tensorrt_llm::runtime::modelconfig::settokensperblock (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17setTokensPerBlockE10SizeType32", false]], "tensorrt_llm::runtime::modelconfig::setusecrossattention (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setUseCrossAttentionEb", false]], "tensorrt_llm::runtime::modelconfig::setusemrope (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11setUseMropeEb", false]], "tensorrt_llm::runtime::modelconfig::setusepositionembedding (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig23setUsePositionEmbeddingEb", false]], "tensorrt_llm::runtime::modelconfig::setuseshapeinference (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setUseShapeInferenceEb", false]], "tensorrt_llm::runtime::modelconfig::setusetokentypeembedding (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24setUseTokenTypeEmbeddingEb", false]], "tensorrt_llm::runtime::modelconfig::skipcrossattnblocks (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig19skipCrossAttnBlocksEv", false]], "tensorrt_llm::runtime::modelconfig::supportsinflightbatching (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig24supportsInflightBatchingEv", false]], "tensorrt_llm::runtime::modelconfig::usecrossattention (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig17useCrossAttentionEv", false]], "tensorrt_llm::runtime::modelconfig::usegemmallreduceplugin (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22useGemmAllReducePluginEb", false], [1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig22useGemmAllReducePluginEv", false]], "tensorrt_llm::runtime::modelconfig::usegptattentionplugin (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21useGptAttentionPluginEb", false], [1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21useGptAttentionPluginEv", false]], "tensorrt_llm::runtime::modelconfig::uselanguageadapter (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18useLanguageAdapterEv", false]], "tensorrt_llm::runtime::modelconfig::useloraplugin (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13useLoraPluginEb", false], [1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13useLoraPluginEv", false]], "tensorrt_llm::runtime::modelconfig::usemambaconv1dplugin (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20useMambaConv1dPluginEb", false], [1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20useMambaConv1dPluginEv", false]], "tensorrt_llm::runtime::modelconfig::usemrope (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig8useMropeEv", false]], "tensorrt_llm::runtime::modelconfig::usepackedinput (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14usePackedInputEb", false], [1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14usePackedInputEv", false]], "tensorrt_llm::runtime::modelconfig::usepagedstate (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13usePagedStateEb", false], [1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13usePagedStateEv", false]], "tensorrt_llm::runtime::modelconfig::usepositionembedding (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20usePositionEmbeddingEv", false]], "tensorrt_llm::runtime::modelconfig::useprompttuning (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15usePromptTuningEv", false]], "tensorrt_llm::runtime::modelconfig::useshapeinference (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig17useShapeInferenceEv", false]], "tensorrt_llm::runtime::modelconfig::usetokentypeembedding (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21useTokenTypeEmbeddingEv", false]], "tensorrt_llm::runtime::mpi_group_barrier (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime17MPI_group_barrierENSt3setIiEE", false]], "tensorrt_llm::runtime::multicastconfigurator (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfiguratorE", false]], "tensorrt_llm::runtime::multicastconfigurator::mbindoffset (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator11mBindOffsetE", false]], "tensorrt_llm::runtime::multicastconfigurator::mdevice (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator7mDeviceE", false]], "tensorrt_llm::runtime::multicastconfigurator::mmulticast (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator10mMulticastE", false]], "tensorrt_llm::runtime::multicastconfigurator::msize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator5mSizeE", false]], "tensorrt_llm::runtime::multicastconfigurator::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator5setupE28CUmemGenericAllocationHandle", false]], "tensorrt_llm::runtime::multicastconfigurator::teardown (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator8teardownE28CUmemGenericAllocationHandleb", false]], "tensorrt_llm::runtime::offloadconfigurator (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfiguratorE", false]], "tensorrt_llm::runtime::offloadconfigurator::maddress (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator8mAddressE", false]], "tensorrt_llm::runtime::offloadconfigurator::mbackedstorage (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator14mBackedStorageE", false]], "tensorrt_llm::runtime::offloadconfigurator::mbacktype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator9mBackTypeE", false]], "tensorrt_llm::runtime::offloadconfigurator::mondemand (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator9mOndemandE", false]], "tensorrt_llm::runtime::offloadconfigurator::msize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator5mSizeE", false]], "tensorrt_llm::runtime::offloadconfigurator::mstream (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator7mStreamE", false]], "tensorrt_llm::runtime::offloadconfigurator::offloadconfigurator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator19OffloadConfiguratorE11CUdeviceptr6size_t10MemoryType8CUstreamb", false]], "tensorrt_llm::runtime::offloadconfigurator::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator5setupE28CUmemGenericAllocationHandle", false]], "tensorrt_llm::runtime::offloadconfigurator::teardown (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator8teardownE28CUmemGenericAllocationHandleb", false]], "tensorrt_llm::runtime::operator<< (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK10LoraModule", false], [1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK26LoraCachePageManagerConfig", false], [1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK7IBuffer", false], [1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK7ITensor", false], [1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERKN7ITensor5ShapeE", false], [1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERKN9LoraCache21TaskLayerModuleConfigE", false]], "tensorrt_llm::runtime::pointerelementtype (c++ type)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime18PointerElementTypeE", false]], "tensorrt_llm::runtime::prompttuningparams (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParamsE", false]], "tensorrt_llm::runtime::prompttuningparams::filltaskstensor (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams15fillTasksTensorE9TensorPtr10SizeType3210SizeType32RKNSt6vectorI10SizeType32EERKNSt6vectorI10SizeType32EERK13BufferManagerb", false]], "tensorrt_llm::runtime::prompttuningparams::prompttuningparams (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams18PromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", false]], "tensorrt_llm::runtime::prompttuningparams::sizetype32 (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams10SizeType32E", false]], "tensorrt_llm::runtime::prompttuningparams::tensorptr (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams9TensorPtrE", false]], "tensorrt_llm::runtime::rawengine (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngineE", false]], "tensorrt_llm::runtime::rawengine::getaddress (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine10getAddressEv", false]], "tensorrt_llm::runtime::rawengine::gethostmemory (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine13getHostMemoryEv", false]], "tensorrt_llm::runtime::rawengine::getmanagedweightsmapopt (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine23getManagedWeightsMapOptEv", false]], "tensorrt_llm::runtime::rawengine::getpath (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine7getPathEv", false]], "tensorrt_llm::runtime::rawengine::getpathopt (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine10getPathOptEv", false]], "tensorrt_llm::runtime::rawengine::getsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine7getSizeEv", false]], "tensorrt_llm::runtime::rawengine::gettype (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine7getTypeEv", false]], "tensorrt_llm::runtime::rawengine::mengineaddr (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine11mEngineAddrE", false]], "tensorrt_llm::runtime::rawengine::menginebuffer (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine13mEngineBufferE", false]], "tensorrt_llm::runtime::rawengine::menginepath (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine11mEnginePathE", false]], "tensorrt_llm::runtime::rawengine::menginesize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine11mEngineSizeE", false]], "tensorrt_llm::runtime::rawengine::mmanagedweightsmap (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine18mManagedWeightsMapE", false]], "tensorrt_llm::runtime::rawengine::mtype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine5mTypeE", false]], "tensorrt_llm::runtime::rawengine::rawengine (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineENSt10filesystem4pathE", false], [1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineEPKN8nvinfer111IHostMemoryE", false], [1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineEPKvNSt6size_tE", false]], "tensorrt_llm::runtime::rawengine::setmanagedweightsmap (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine20setManagedWeightsMapENSt3mapINSt6stringEN12tensorrt_llm8executor6TensorEEE", false]], "tensorrt_llm::runtime::rawengine::setpath (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine7setPathENSt10filesystem4pathE", false]], "tensorrt_llm::runtime::rawengine::type (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4TypeE", false]], "tensorrt_llm::runtime::rawengine::type::addresswithsize (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4Type15AddressWithSizeE", false]], "tensorrt_llm::runtime::rawengine::type::filepath (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4Type8FilePathE", false]], "tensorrt_llm::runtime::rawengine::type::hostmemory (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4Type10HostMemoryE", false]], "tensorrt_llm::runtime::requesttype (c++ enum)": [[1, "_CPPv4N12tensorrt_llm7runtime11RequestTypeE", false]], "tensorrt_llm::runtime::requesttype::kcontext (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11RequestType8kCONTEXTE", false]], "tensorrt_llm::runtime::requesttype::kgeneration (c++ enumerator)": [[1, "_CPPv4N12tensorrt_llm7runtime11RequestType11kGENERATIONE", false]], "tensorrt_llm::runtime::runtimedefaults (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaultsE", false]], "tensorrt_llm::runtime::runtimedefaults::maxattentionwindowvec (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults21maxAttentionWindowVecE", false]], "tensorrt_llm::runtime::runtimedefaults::runtimedefaults (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults15RuntimeDefaultsENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalI10SizeType32EE", false], [1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults15RuntimeDefaultsEv", false]], "tensorrt_llm::runtime::runtimedefaults::sinktokenlength (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults15sinkTokenLengthE", false]], "tensorrt_llm::runtime::samplingconfig (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfigE", false]], "tensorrt_llm::runtime::samplingconfig::beamsearchdiversityrate (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig23beamSearchDiversityRateE", false]], "tensorrt_llm::runtime::samplingconfig::beamwidth (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig9beamWidthE", false]], "tensorrt_llm::runtime::samplingconfig::beamwidtharray (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14beamWidthArrayE", false]], "tensorrt_llm::runtime::samplingconfig::cumlogprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig11cumLogProbsE", false]], "tensorrt_llm::runtime::samplingconfig::draftacceptancethreshold (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig24draftAcceptanceThresholdE", false]], "tensorrt_llm::runtime::samplingconfig::earlystopping (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig13earlyStoppingE", false]], "tensorrt_llm::runtime::samplingconfig::floattype (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig9FloatTypeE", false]], "tensorrt_llm::runtime::samplingconfig::frequencypenalty (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig16frequencyPenaltyE", false]], "tensorrt_llm::runtime::samplingconfig::fusevalues (c++ function)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig10fuseValuesE6OptVecI1TERKNSt6vectorI14SamplingConfigEENSt8functionIF6OptVecI1TE6size_tEEE1T", false]], "tensorrt_llm::runtime::samplingconfig::getmaxbeamwidth (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14SamplingConfig15getMaxBeamWidthEv", false]], "tensorrt_llm::runtime::samplingconfig::getnumreturnbeams (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14SamplingConfig17getNumReturnBeamsEv", false]], "tensorrt_llm::runtime::samplingconfig::lengthpenalty (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig13lengthPenaltyE", false]], "tensorrt_llm::runtime::samplingconfig::minlength (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig9minLengthE", false]], "tensorrt_llm::runtime::samplingconfig::minp (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig4minPE", false]], "tensorrt_llm::runtime::samplingconfig::norepeatngramsize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig17noRepeatNgramSizeE", false]], "tensorrt_llm::runtime::samplingconfig::normalizelogprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig17normalizeLogProbsE", false]], "tensorrt_llm::runtime::samplingconfig::numreturnsequences (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig18numReturnSequencesE", false]], "tensorrt_llm::runtime::samplingconfig::operator== (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime14SamplingConfigeqERK14SamplingConfig", false]], "tensorrt_llm::runtime::samplingconfig::optvec (c++ type)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig6OptVecE", false]], "tensorrt_llm::runtime::samplingconfig::originaltemperature (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig19originalTemperatureE", false]], "tensorrt_llm::runtime::samplingconfig::outputlogprobs (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14outputLogProbsE", false]], "tensorrt_llm::runtime::samplingconfig::presencepenalty (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig15presencePenaltyE", false]], "tensorrt_llm::runtime::samplingconfig::randomseed (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig10randomSeedE", false]], "tensorrt_llm::runtime::samplingconfig::repetitionpenalty (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig17repetitionPenaltyE", false]], "tensorrt_llm::runtime::samplingconfig::samplingconfig (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigE10SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigERKN8executor14SamplingConfigERKNSt8optionalIN8executor25ExternalDraftTokensConfigEEE", false], [1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigERKNSt6vectorI14SamplingConfigEE", false]], "tensorrt_llm::runtime::samplingconfig::temperature (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig11temperatureE", false]], "tensorrt_llm::runtime::samplingconfig::topk (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig4topKE", false]], "tensorrt_llm::runtime::samplingconfig::topkmedusaheads (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig15topKMedusaHeadsE", false]], "tensorrt_llm::runtime::samplingconfig::topp (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig4topPE", false]], "tensorrt_llm::runtime::samplingconfig::toppdecay (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig9topPDecayE", false]], "tensorrt_llm::runtime::samplingconfig::toppmin (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig7topPMinE", false]], "tensorrt_llm::runtime::samplingconfig::toppresetids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig12topPResetIdsE", false]], "tensorrt_llm::runtime::samplingconfig::usedefaultvalues (c++ function)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig16useDefaultValuesEbRK6OptVecI1TE1T", false]], "tensorrt_llm::runtime::samplingconfig::validate (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig8validateEv", false]], "tensorrt_llm::runtime::samplingconfig::validatevec (c++ function)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig11validateVecEbNSt6stringERK6OptVecI1TE1TNSt8optionalI1TEE", false]], "tensorrt_llm::runtime::setvirtualmemoryallocator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25setVirtualMemoryAllocatorERKNSt6stringEN26CudaVirtualMemoryAllocator11RestoreModeENSt10shared_ptrI10CudaStreamEE", false]], "tensorrt_llm::runtime::sizetype32 (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime10SizeType32E", false]], "tensorrt_llm::runtime::sizetype64 (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime10SizeType64E", false]], "tensorrt_llm::runtime::speculativedecodingmode (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingModeE", false]], "tensorrt_llm::runtime::speculativedecodingmode::allbitset (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode9allBitSetE14UnderlyingType", false]], "tensorrt_llm::runtime::speculativedecodingmode::anybitset (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode9anyBitSetE14UnderlyingType", false]], "tensorrt_llm::runtime::speculativedecodingmode::drafttokensexternal (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode19DraftTokensExternalEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::eagle (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode5EagleEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::explicitdrafttokens (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode19ExplicitDraftTokensEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::hasdraftlogits (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode14hasDraftLogitsEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::isdrafttokensexternal (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode21isDraftTokensExternalEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::iseagle (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode7isEagleEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::isexplicitdrafttokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode21isExplicitDraftTokensEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::islookaheaddecoding (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode19isLookaheadDecodingEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::ismedusa (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode8isMedusaEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::isnone (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode6isNoneEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::kdrafttokensexternal (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode20kDraftTokensExternalE", false]], "tensorrt_llm::runtime::speculativedecodingmode::keagle (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode6kEagleE", false]], "tensorrt_llm::runtime::speculativedecodingmode::kexplicitdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode20kExplicitDraftTokensE", false]], "tensorrt_llm::runtime::speculativedecodingmode::klookaheaddecoding (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode18kLookaheadDecodingE", false]], "tensorrt_llm::runtime::speculativedecodingmode::kmedusa (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode7kMedusaE", false]], "tensorrt_llm::runtime::speculativedecodingmode::knone (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode5kNoneE", false]], "tensorrt_llm::runtime::speculativedecodingmode::lookaheaddecoding (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode17LookaheadDecodingEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::medusa (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode6MedusaEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::mstate (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode6mStateE", false]], "tensorrt_llm::runtime::speculativedecodingmode::needsdecoderprologue (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode20needsDecoderPrologueEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::needskvcacherewind (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode18needsKVCacheRewindEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::none (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode4NoneEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::operator== (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingModeeqERK23SpeculativeDecodingMode", false]], "tensorrt_llm::runtime::speculativedecodingmode::predictsdrafttokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode19predictsDraftTokensEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::requiresattentionmask (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode21requiresAttentionMaskEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::speculativedecodingmode (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode23SpeculativeDecodingModeE14UnderlyingType", false]], "tensorrt_llm::runtime::speculativedecodingmode::underlyingtype (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode14UnderlyingTypeE", false]], "tensorrt_llm::runtime::speculativedecodingmode::updatespositionids (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode18updatesPositionIdsEv", false]], "tensorrt_llm::runtime::speculativedecodingmode::variabledraftlength (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode19variableDraftLengthEv", false]], "tensorrt_llm::runtime::speculativedecodingmodule (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModuleE", false]], "tensorrt_llm::runtime::speculativedecodingmodule::computenumpackedmasks (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule21computeNumPackedMasksEv", false]], "tensorrt_llm::runtime::speculativedecodingmodule::getmaxdecodingdrafttokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule25getMaxDecodingDraftTokensEv", false]], "tensorrt_llm::runtime::speculativedecodingmodule::getmaxdecodingtokens (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule20getMaxDecodingTokensEv", false]], "tensorrt_llm::runtime::speculativedecodingmodule::getmaxdraftpathlen (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule18getMaxDraftPathLenEv", false]], "tensorrt_llm::runtime::speculativedecodingmodule::getmaxnumpaths (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule14getMaxNumPathsEv", false]], "tensorrt_llm::runtime::speculativedecodingmodule::getmaxpathlen (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule13getMaxPathLenEv", false]], "tensorrt_llm::runtime::speculativedecodingmodule::getnumpackedmasks (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule17getNumPackedMasksEv", false]], "tensorrt_llm::runtime::speculativedecodingmodule::mmaxdecodingdrafttokens (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule23mMaxDecodingDraftTokensE", false]], "tensorrt_llm::runtime::speculativedecodingmodule::mmaxdraftpathlen (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule16mMaxDraftPathLenE", false]], "tensorrt_llm::runtime::speculativedecodingmodule::mmaxnumpackedmasks (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule18mMaxNumPackedMasksE", false]], "tensorrt_llm::runtime::speculativedecodingmodule::mmaxnumpaths (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule12mMaxNumPathsE", false]], "tensorrt_llm::runtime::speculativedecodingmodule::operator= (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModuleaSERK25SpeculativeDecodingModule", false]], "tensorrt_llm::runtime::speculativedecodingmodule::setmaxdraftpathlen (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule18setMaxDraftPathLenE10SizeType32", false]], "tensorrt_llm::runtime::speculativedecodingmodule::setmaxdrafttokens (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule17setMaxDraftTokensE10SizeType32", false]], "tensorrt_llm::runtime::speculativedecodingmodule::setmaxnumpaths (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule14setMaxNumPathsE10SizeType32", false]], "tensorrt_llm::runtime::speculativedecodingmodule::speculativedecodingmodule (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleE10SizeType3210SizeType3210SizeType32", false], [1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleERK25SpeculativeDecodingModule", false], [1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleEv", false]], "tensorrt_llm::runtime::speculativedecodingmodule::~speculativedecodingmodule (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModuleD0Ev", false]], "tensorrt_llm::runtime::stringptrmap (c++ type)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime12StringPtrMapE", false]], "tensorrt_llm::runtime::tllmlogger (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime10TllmLoggerE", false]], "tensorrt_llm::runtime::tllmlogger::getlevel (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10TllmLogger8getLevelEv", false]], "tensorrt_llm::runtime::tllmlogger::log (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10TllmLogger3logE8SeverityPKN8nvinfer19AsciiCharE", false]], "tensorrt_llm::runtime::tllmlogger::setlevel (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime10TllmLogger8setLevelE8Severity", false]], "tensorrt_llm::runtime::to_string (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime9to_stringERK26LoraCachePageManagerConfig", false], [1, "_CPPv4N12tensorrt_llm7runtime9to_stringERKN9LoraCache21TaskLayerModuleConfigE", false]], "tensorrt_llm::runtime::tokenextraidtype (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime16TokenExtraIdTypeE", false]], "tensorrt_llm::runtime::tokenidtype (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime11TokenIdTypeE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4I0_bEN12tensorrt_llm7runtime11TRTDataTypeE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIbEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIbE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIfEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIfE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeI4halfEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeI4halfE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIN7kernels13FinishedStateEEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIN7kernels13FinishedStateEE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIN7kernels12KVCacheIndexEEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIN7kernels12KVCacheIndexEE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIN7runtime11RequestTypeEEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIN7runtime11RequestTypeEE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt7int32_tEEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt7int32_tEE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt7int64_tEEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt7int64_tEE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt6int8_tEEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt6int8_tEE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt8uint32_tEEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt8uint32_tEE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt8uint64_tEEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt8uint64_tEE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt7uint8_tEEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt7uint8_tEE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4I0EN12tensorrt_llm7runtime11TRTDataTypeIP1TEE", false]], "tensorrt_llm::runtime::trtdatatype::kunderlyingtype (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIP1TE15kUnderlyingTypeE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIP1TE5valueE", false]], "tensorrt_llm::runtime::trtdatatype (c++ struct)": [[1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIPvEE", false]], "tensorrt_llm::runtime::trtdatatype::value (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIPvE5valueE", false]], "tensorrt_llm::runtime::unicastconfigurator (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfiguratorE", false]], "tensorrt_llm::runtime::unicastconfigurator::maddress (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator8mAddressE", false]], "tensorrt_llm::runtime::unicastconfigurator::mdesc (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator5mDescE", false]], "tensorrt_llm::runtime::unicastconfigurator::msize (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator5mSizeE", false]], "tensorrt_llm::runtime::unicastconfigurator::setup (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator5setupE28CUmemGenericAllocationHandle", false]], "tensorrt_llm::runtime::unicastconfigurator::teardown (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator8teardownE28CUmemGenericAllocationHandleb", false]], "tensorrt_llm::runtime::unicastconfigurator::unicastconfigurator (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator19UnicastConfiguratorE11CUdeviceptr6size_tRK15CUmemAccessDesc", false]], "tensorrt_llm::runtime::uniquetoken (c++ struct)": [[1, "_CPPv4N12tensorrt_llm7runtime11UniqueTokenE", false]], "tensorrt_llm::runtime::uniquetoken::operator== (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11UniqueTokeneqERK11UniqueToken", false]], "tensorrt_llm::runtime::uniquetoken::tokenextraid (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11UniqueToken12tokenExtraIdE", false]], "tensorrt_llm::runtime::uniquetoken::tokenid (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11UniqueToken7tokenIdE", false]], "tensorrt_llm::runtime::vectokenextraids (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime16VecTokenExtraIdsE", false]], "tensorrt_llm::runtime::vecuniquetokens (c++ type)": [[1, "_CPPv4N12tensorrt_llm7runtime15VecUniqueTokensE", false]], "tensorrt_llm::runtime::worldconfig (c++ class)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfigE", false]], "tensorrt_llm::runtime::worldconfig::enableattentiondp (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig17enableAttentionDPEv", false]], "tensorrt_llm::runtime::worldconfig::getcontextparallelgroup (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig23getContextParallelGroupEv", false]], "tensorrt_llm::runtime::worldconfig::getcontextparallelism (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig21getContextParallelismEv", false]], "tensorrt_llm::runtime::worldconfig::getcontextparallelrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig22getContextParallelRankEv", false]], "tensorrt_llm::runtime::worldconfig::getdevice (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig9getDeviceEv", false]], "tensorrt_llm::runtime::worldconfig::getdeviceof (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig11getDeviceOfE10SizeType32", false]], "tensorrt_llm::runtime::worldconfig::getgpuspergroup (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig15getGpusPerGroupEv", false]], "tensorrt_llm::runtime::worldconfig::getgpuspernode (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig14getGpusPerNodeEv", false]], "tensorrt_llm::runtime::worldconfig::getlastrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig11getLastRankEv", false]], "tensorrt_llm::runtime::worldconfig::getlocalrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig12getLocalRankEv", false]], "tensorrt_llm::runtime::worldconfig::getnoderank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig11getNodeRankEv", false]], "tensorrt_llm::runtime::worldconfig::getnoderankof (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig13getNodeRankOfE10SizeType32", false]], "tensorrt_llm::runtime::worldconfig::getpipelineparallelgroup (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig24getPipelineParallelGroupEv", false]], "tensorrt_llm::runtime::worldconfig::getpipelineparallelism (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig22getPipelineParallelismEv", false]], "tensorrt_llm::runtime::worldconfig::getpipelineparallelrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig23getPipelineParallelRankEv", false]], "tensorrt_llm::runtime::worldconfig::getrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig7getRankEv", false]], "tensorrt_llm::runtime::worldconfig::getsize (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig7getSizeEv", false]], "tensorrt_llm::runtime::worldconfig::gettensorparallelgroup (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig22getTensorParallelGroupEv", false]], "tensorrt_llm::runtime::worldconfig::gettensorparallelism (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig20getTensorParallelismEv", false]], "tensorrt_llm::runtime::worldconfig::gettensorparallelrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig21getTensorParallelRankEv", false]], "tensorrt_llm::runtime::worldconfig::iscontextparallel (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig17isContextParallelEv", false]], "tensorrt_llm::runtime::worldconfig::isfirstcontextparallelrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig26isFirstContextParallelRankEv", false]], "tensorrt_llm::runtime::worldconfig::isfirstpipelineparallelrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig27isFirstPipelineParallelRankEv", false]], "tensorrt_llm::runtime::worldconfig::isfirsttensorparallelrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig25isFirstTensorParallelRankEv", false]], "tensorrt_llm::runtime::worldconfig::islastpipelineparallelrank (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig26isLastPipelineParallelRankEv", false]], "tensorrt_llm::runtime::worldconfig::ispipelineparallel (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig18isPipelineParallelEv", false]], "tensorrt_llm::runtime::worldconfig::istensorparallel (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig16isTensorParallelEv", false]], "tensorrt_llm::runtime::worldconfig::kdefaultgpuspernode (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig19kDefaultGpusPerNodeE", false]], "tensorrt_llm::runtime::worldconfig::mcontextparallelism (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig19mContextParallelismE", false]], "tensorrt_llm::runtime::worldconfig::mdeviceids (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig10mDeviceIdsE", false]], "tensorrt_llm::runtime::worldconfig::menableattentiondp (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig18mEnableAttentionDPE", false]], "tensorrt_llm::runtime::worldconfig::mgpuspernode (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig12mGpusPerNodeE", false]], "tensorrt_llm::runtime::worldconfig::mpi (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig3mpiE10SizeType32NSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEEb", false]], "tensorrt_llm::runtime::worldconfig::mpipelineparallelism (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig20mPipelineParallelismE", false]], "tensorrt_llm::runtime::worldconfig::mrank (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig5mRankE", false]], "tensorrt_llm::runtime::worldconfig::mtensorparallelism (c++ member)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig18mTensorParallelismE", false]], "tensorrt_llm::runtime::worldconfig::validmpiconfig (c++ function)": [[1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig14validMpiConfigEv", false]], "tensorrt_llm::runtime::worldconfig::worldconfig (c++ function)": [[1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig11WorldConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalINSt6vectorI10SizeType32EEEEb", false]], "text (tensorrt_llm.llmapi.completionoutput attribute)": [[147, "tensorrt_llm.llmapi.CompletionOutput.text", false]], "text_diff (tensorrt_llm.llmapi.completionoutput attribute)": [[147, "tensorrt_llm.llmapi.CompletionOutput.text_diff", false]], "text_diff (tensorrt_llm.llmapi.completionoutput property)": [[147, "id4", false]], "timeout_iters (tensorrt_llm.llmapi.attentiondpconfig attribute)": [[147, "tensorrt_llm.llmapi.AttentionDpConfig.timeout_iters", false]], "timestepembedding (class in tensorrt_llm.layers.embedding)": [[134, "tensorrt_llm.layers.embedding.TimestepEmbedding", false]], "timesteps (class in tensorrt_llm.layers.embedding)": [[134, "tensorrt_llm.layers.embedding.Timesteps", false]], "title() (tensorrt_llm.llmapi.batchingtype method)": [[147, "tensorrt_llm.llmapi.BatchingType.title", false]], "title() (tensorrt_llm.llmapi.capacityschedulerpolicy method)": [[147, "tensorrt_llm.llmapi.CapacitySchedulerPolicy.title", false]], "title() (tensorrt_llm.llmapi.contextchunkingpolicy method)": [[147, "tensorrt_llm.llmapi.ContextChunkingPolicy.title", false]], "title() (tensorrt_llm.llmapi.quantalgo method)": [[147, "tensorrt_llm.llmapi.QuantAlgo.title", false]], "to_dict() (tensorrt_llm.llmapi.buildconfig method)": [[147, "tensorrt_llm.llmapi.BuildConfig.to_dict", false]], "to_dict() (tensorrt_llm.llmapi.calibconfig method)": [[147, "tensorrt_llm.llmapi.CalibConfig.to_dict", false]], "to_dict() (tensorrt_llm.llmapi.quantconfig method)": [[147, "tensorrt_llm.llmapi.QuantConfig.to_dict", false]], "to_dict() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.to_dict", false]], "to_dict() (tensorrt_llm.llmapi.trtllmargs method)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.to_dict", false]], "to_dict() (tensorrt_llm.models.chatglmconfig method)": [[135, "tensorrt_llm.models.ChatGLMConfig.to_dict", false]], "to_dict() (tensorrt_llm.models.cogvlmconfig method)": [[135, "tensorrt_llm.models.CogVLMConfig.to_dict", false]], "to_dict() (tensorrt_llm.models.dbrxconfig method)": [[135, "tensorrt_llm.models.DbrxConfig.to_dict", false]], "to_dict() (tensorrt_llm.models.falconconfig method)": [[135, "tensorrt_llm.models.FalconConfig.to_dict", false]], "to_dict() (tensorrt_llm.models.gemmaconfig method)": [[135, "tensorrt_llm.models.GemmaConfig.to_dict", false]], "to_dict() (tensorrt_llm.models.gptconfig method)": [[135, "tensorrt_llm.models.GPTConfig.to_dict", false]], "to_dict() (tensorrt_llm.models.gptjconfig method)": [[135, "tensorrt_llm.models.GPTJConfig.to_dict", false]], "to_dict() (tensorrt_llm.models.llamaconfig method)": [[135, "tensorrt_llm.models.LLaMAConfig.to_dict", false]], "to_dict() (tensorrt_llm.models.medusaconfig method)": [[135, "tensorrt_llm.models.MedusaConfig.to_dict", false]], "to_dict() (tensorrt_llm.models.pretrainedconfig method)": [[135, "tensorrt_llm.models.PretrainedConfig.to_dict", false]], "to_json_file() (tensorrt_llm.models.pretrainedconfig method)": [[135, "tensorrt_llm.models.PretrainedConfig.to_json_file", false]], "to_layer_quant_config() (tensorrt_llm.models.pretrainedconfig method)": [[135, "tensorrt_llm.models.PretrainedConfig.to_layer_quant_config", false]], "to_legacy_setting() (tensorrt_llm.plugin.pluginconfig method)": [[136, "tensorrt_llm.plugin.PluginConfig.to_legacy_setting", false]], "token_drop() (tensorrt_llm.layers.embedding.labelembedding method)": [[134, "tensorrt_llm.layers.embedding.LabelEmbedding.token_drop", false]], "token_end (tensorrt_llm.llmapi.kvcacheretentionconfig.tokenrangeretentionconfig property)": [[147, "tensorrt_llm.llmapi.KvCacheRetentionConfig.TokenRangeRetentionConfig.token_end", false]], "token_ids (tensorrt_llm.llmapi.completionoutput attribute)": [[147, "tensorrt_llm.llmapi.CompletionOutput.token_ids", false]], "token_ids_diff (tensorrt_llm.llmapi.completionoutput attribute)": [[147, "tensorrt_llm.llmapi.CompletionOutput.token_ids_diff", false]], "token_ids_diff (tensorrt_llm.llmapi.completionoutput property)": [[147, "id5", false]], "token_range_retention_configs (tensorrt_llm.llmapi.kvcacheretentionconfig property)": [[147, "tensorrt_llm.llmapi.KvCacheRetentionConfig.token_range_retention_configs", false]], "token_start (tensorrt_llm.llmapi.kvcacheretentionconfig.tokenrangeretentionconfig property)": [[147, "tensorrt_llm.llmapi.KvCacheRetentionConfig.TokenRangeRetentionConfig.token_start", false]], "tokenizer (tensorrt_llm.llmapi.llm attribute)": [[147, "tensorrt_llm.llmapi.LLM.tokenizer", false]], "tokenizer (tensorrt_llm.llmapi.llm property)": [[147, "id1", false]], "tokenizer (tensorrt_llm.llmapi.multimodalencoder property)": [[147, "tensorrt_llm.llmapi.MultimodalEncoder.tokenizer", false]], "tokenizer (tensorrt_llm.llmapi.torchllmargs attribute)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.tokenizer", false]], "tokenizer (tensorrt_llm.llmapi.trtllmargs attribute)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.tokenizer", false]], "tokenizer_image_token() (tensorrt_llm.runtime.multimodalmodelrunner static method)": [[138, "tensorrt_llm.runtime.MultimodalModelRunner.tokenizer_image_token", false]], "tokenizer_max_seq_length (tensorrt_llm.llmapi.calibconfig attribute)": [[147, "tensorrt_llm.llmapi.CalibConfig.tokenizer_max_seq_length", false]], "tokenizer_mode (tensorrt_llm.llmapi.torchllmargs attribute)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.tokenizer_mode", false]], "tokenizer_mode (tensorrt_llm.llmapi.trtllmargs attribute)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.tokenizer_mode", false]], "tokenizer_revision (tensorrt_llm.llmapi.torchllmargs attribute)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.tokenizer_revision", false]], "tokenizer_revision (tensorrt_llm.llmapi.trtllmargs attribute)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.tokenizer_revision", false]], "tokens_per_block (tensorrt_llm.llmapi.kvcacheconfig attribute)": [[147, "tensorrt_llm.llmapi.KvCacheConfig.tokens_per_block", false]], "tokens_per_block (tensorrt_llm.plugin.pluginconfig attribute)": [[136, "tensorrt_llm.plugin.PluginConfig.tokens_per_block", false]], "tokens_per_block (tensorrt_llm.runtime.generationsession property)": [[138, "tensorrt_llm.runtime.GenerationSession.tokens_per_block", false]], "tokens_per_block (tensorrt_llm.runtime.modelconfig attribute)": [[138, "tensorrt_llm.runtime.ModelConfig.tokens_per_block", false]], "top_k (tensorrt_llm.llmapi.samplingparams attribute)": [[147, "tensorrt_llm.llmapi.SamplingParams.top_k", false]], "top_k (tensorrt_llm.runtime.samplingconfig attribute)": [[138, "tensorrt_llm.runtime.SamplingConfig.top_k", false]], "top_p (tensorrt_llm.llmapi.samplingparams attribute)": [[147, "tensorrt_llm.llmapi.SamplingParams.top_p", false]], "top_p (tensorrt_llm.runtime.samplingconfig attribute)": [[138, "tensorrt_llm.runtime.SamplingConfig.top_p", false]], "top_p_decay (tensorrt_llm.llmapi.samplingparams attribute)": [[147, "tensorrt_llm.llmapi.SamplingParams.top_p_decay", false]], "top_p_decay (tensorrt_llm.runtime.samplingconfig attribute)": [[138, "tensorrt_llm.runtime.SamplingConfig.top_p_decay", false]], "top_p_min (tensorrt_llm.llmapi.samplingparams attribute)": [[147, "tensorrt_llm.llmapi.SamplingParams.top_p_min", false]], "top_p_min (tensorrt_llm.runtime.samplingconfig attribute)": [[138, "tensorrt_llm.runtime.SamplingConfig.top_p_min", false]], "top_p_reset_ids (tensorrt_llm.llmapi.samplingparams attribute)": [[147, "tensorrt_llm.llmapi.SamplingParams.top_p_reset_ids", false]], "top_p_reset_ids (tensorrt_llm.runtime.samplingconfig attribute)": [[138, "tensorrt_llm.runtime.SamplingConfig.top_p_reset_ids", false]], "topk (tensorrt_llm.llmapi.rocketsparseattentionconfig attribute)": [[147, "tensorrt_llm.llmapi.RocketSparseAttentionConfig.topk", false]], "topk() (in module tensorrt_llm.functional)": [[133, "tensorrt_llm.functional.topk", false]], "topr (tensorrt_llm.llmapi.rocketsparseattentionconfig attribute)": [[147, "tensorrt_llm.llmapi.RocketSparseAttentionConfig.topr", false]], "torch_compile_config (tensorrt_llm.llmapi.torchllmargs attribute)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.torch_compile_config", false]], "torchcompileconfig (class in tensorrt_llm.llmapi)": [[147, "tensorrt_llm.llmapi.TorchCompileConfig", false]], "torchcompileconfig.config (class in tensorrt_llm.llmapi)": [[147, "tensorrt_llm.llmapi.TorchCompileConfig.Config", false]], "torchllmargs (class in tensorrt_llm.llmapi)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs", false]], "torchllmargs.config (class in tensorrt_llm.llmapi)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.Config", false]], "tp_split_dim() (tensorrt_llm.layers.linear.linear class method)": [[134, "tensorrt_llm.layers.linear.Linear.tp_split_dim", false]], "tp_split_dim() (tensorrt_llm.layers.linear.linearbase class method)": [[134, "tensorrt_llm.layers.linear.LinearBase.tp_split_dim", false]], "tp_split_dim() (tensorrt_llm.layers.linear.rowlinear class method)": [[134, "tensorrt_llm.layers.linear.RowLinear.tp_split_dim", false]], "transfer_mode (tensorrt_llm.llmapi.kvcacheretentionconfig property)": [[147, "tensorrt_llm.llmapi.KvCacheRetentionConfig.transfer_mode", false]], "translate() (tensorrt_llm.llmapi.batchingtype method)": [[147, "tensorrt_llm.llmapi.BatchingType.translate", false]], "translate() (tensorrt_llm.llmapi.capacityschedulerpolicy method)": [[147, "tensorrt_llm.llmapi.CapacitySchedulerPolicy.translate", false]], "translate() (tensorrt_llm.llmapi.contextchunkingpolicy method)": [[147, "tensorrt_llm.llmapi.ContextChunkingPolicy.translate", false]], "translate() (tensorrt_llm.llmapi.quantalgo method)": [[147, "tensorrt_llm.llmapi.QuantAlgo.translate", false]], "transpose() (in module tensorrt_llm.functional)": [[133, "tensorrt_llm.functional.transpose", false]], "transpose() (tensorrt_llm.functional.tensor method)": [[133, "tensorrt_llm.functional.Tensor.transpose", false]], "trtllm-bench command line option": [[22, "cmdoption-trtllm-bench-log_level", false], [22, "cmdoption-trtllm-bench-m", false], [22, "cmdoption-trtllm-bench-model_path", false], [22, "cmdoption-trtllm-bench-w", false]], "trtllm-bench-build command line option": [[22, "cmdoption-trtllm-bench-build-dataset", false], [22, "cmdoption-trtllm-bench-build-max_batch_size", false], [22, "cmdoption-trtllm-bench-build-max_num_tokens", false], [22, "cmdoption-trtllm-bench-build-max_seq_len", false], [22, "cmdoption-trtllm-bench-build-no_weights_loading", false], [22, "cmdoption-trtllm-bench-build-pp", false], [22, "cmdoption-trtllm-bench-build-q", false], [22, "cmdoption-trtllm-bench-build-target_input_len", false], [22, "cmdoption-trtllm-bench-build-target_output_len", false], [22, "cmdoption-trtllm-bench-build-tp", false], [22, "cmdoption-trtllm-bench-build-trust_remote_code", false]], "trtllm-bench-latency command line option": [[22, "cmdoption-trtllm-bench-latency-backend", false], [22, "cmdoption-trtllm-bench-latency-beam_width", false], [22, "cmdoption-trtllm-bench-latency-concurrency", false], [22, "cmdoption-trtllm-bench-latency-dataset", false], [22, "cmdoption-trtllm-bench-latency-engine_dir", false], [22, "cmdoption-trtllm-bench-latency-ep", false], [22, "cmdoption-trtllm-bench-latency-extra_llm_api_options", false], [22, "cmdoption-trtllm-bench-latency-iteration_log", false], [22, "cmdoption-trtllm-bench-latency-kv_cache_free_gpu_mem_fraction", false], [22, "cmdoption-trtllm-bench-latency-max_input_len", false], [22, "cmdoption-trtllm-bench-latency-max_seq_len", false], [22, "cmdoption-trtllm-bench-latency-medusa_choices", false], [22, "cmdoption-trtllm-bench-latency-modality", false], [22, "cmdoption-trtllm-bench-latency-num_requests", false], [22, "cmdoption-trtllm-bench-latency-pp", false], [22, "cmdoption-trtllm-bench-latency-report_json", false], [22, "cmdoption-trtllm-bench-latency-sampler_options", false], [22, "cmdoption-trtllm-bench-latency-tp", false], [22, "cmdoption-trtllm-bench-latency-warmup", false]], "trtllm-bench-throughput command line option": [[22, "cmdoption-trtllm-bench-throughput-backend", false], [22, "cmdoption-trtllm-bench-throughput-beam_width", false], [22, "cmdoption-trtllm-bench-throughput-cluster_size", false], [22, "cmdoption-trtllm-bench-throughput-concurrency", false], [22, "cmdoption-trtllm-bench-throughput-custom_module_dirs", false], [22, "cmdoption-trtllm-bench-throughput-data_device", false], [22, "cmdoption-trtllm-bench-throughput-dataset", false], [22, "cmdoption-trtllm-bench-throughput-enable_chunked_context", false], [22, "cmdoption-trtllm-bench-throughput-engine_dir", false], [22, "cmdoption-trtllm-bench-throughput-eos_id", false], [22, "cmdoption-trtllm-bench-throughput-ep", false], [22, "cmdoption-trtllm-bench-throughput-extra_llm_api_options", false], [22, "cmdoption-trtllm-bench-throughput-image_data_format", false], [22, "cmdoption-trtllm-bench-throughput-iteration_log", false], [22, "cmdoption-trtllm-bench-throughput-kv_cache_free_gpu_mem_fraction", false], [22, "cmdoption-trtllm-bench-throughput-max_batch_size", false], [22, "cmdoption-trtllm-bench-throughput-max_input_len", false], [22, "cmdoption-trtllm-bench-throughput-max_num_tokens", false], [22, "cmdoption-trtllm-bench-throughput-max_seq_len", false], [22, "cmdoption-trtllm-bench-throughput-modality", false], [22, "cmdoption-trtllm-bench-throughput-no_skip_tokenizer_init", false], [22, "cmdoption-trtllm-bench-throughput-num_requests", false], [22, "cmdoption-trtllm-bench-throughput-output_json", false], [22, "cmdoption-trtllm-bench-throughput-pp", false], [22, "cmdoption-trtllm-bench-throughput-report_json", false], [22, "cmdoption-trtllm-bench-throughput-request_json", false], [22, "cmdoption-trtllm-bench-throughput-sampler_options", false], [22, "cmdoption-trtllm-bench-throughput-scheduler_policy", false], [22, "cmdoption-trtllm-bench-throughput-streaming", false], [22, "cmdoption-trtllm-bench-throughput-target_input_len", false], [22, "cmdoption-trtllm-bench-throughput-target_output_len", false], [22, "cmdoption-trtllm-bench-throughput-tp", false], [22, "cmdoption-trtllm-bench-throughput-warmup", false]], "trtllm-eval command line option": [[24, "cmdoption-trtllm-eval-backend", false], [24, "cmdoption-trtllm-eval-disable_kv_cache_reuse", false], [24, "cmdoption-trtllm-eval-ep_size", false], [24, "cmdoption-trtllm-eval-extra_llm_api_options", false], [24, "cmdoption-trtllm-eval-gpus_per_node", false], [24, "cmdoption-trtllm-eval-kv_cache_free_gpu_memory_fraction", false], [24, "cmdoption-trtllm-eval-log_level", false], [24, "cmdoption-trtllm-eval-max_batch_size", false], [24, "cmdoption-trtllm-eval-max_beam_width", false], [24, "cmdoption-trtllm-eval-max_num_tokens", false], [24, "cmdoption-trtllm-eval-max_seq_len", false], [24, "cmdoption-trtllm-eval-model", false], [24, "cmdoption-trtllm-eval-pp_size", false], [24, "cmdoption-trtllm-eval-tokenizer", false], [24, "cmdoption-trtllm-eval-tp_size", false], [24, "cmdoption-trtllm-eval-trust_remote_code", false]], "trtllm-eval-cnn_dailymail command line option": [[24, "cmdoption-trtllm-eval-cnn_dailymail-apply_chat_template", false], [24, "cmdoption-trtllm-eval-cnn_dailymail-dataset_path", false], [24, "cmdoption-trtllm-eval-cnn_dailymail-max_input_length", false], [24, "cmdoption-trtllm-eval-cnn_dailymail-max_output_length", false], [24, "cmdoption-trtllm-eval-cnn_dailymail-num_samples", false], [24, "cmdoption-trtllm-eval-cnn_dailymail-random_seed", false], [24, "cmdoption-trtllm-eval-cnn_dailymail-rouge_path", false], [24, "cmdoption-trtllm-eval-cnn_dailymail-system_prompt", false]], "trtllm-eval-gpqa_diamond command line option": [[24, "cmdoption-trtllm-eval-gpqa_diamond-apply_chat_template", false], [24, "cmdoption-trtllm-eval-gpqa_diamond-chat_template_kwargs", false], [24, "cmdoption-trtllm-eval-gpqa_diamond-dataset_path", false], [24, "cmdoption-trtllm-eval-gpqa_diamond-max_input_length", false], [24, "cmdoption-trtllm-eval-gpqa_diamond-max_output_length", false], [24, "cmdoption-trtllm-eval-gpqa_diamond-num_samples", false], [24, "cmdoption-trtllm-eval-gpqa_diamond-random_seed", false], [24, "cmdoption-trtllm-eval-gpqa_diamond-system_prompt", false]], "trtllm-eval-gpqa_extended command line option": [[24, "cmdoption-trtllm-eval-gpqa_extended-apply_chat_template", false], [24, "cmdoption-trtllm-eval-gpqa_extended-chat_template_kwargs", false], [24, "cmdoption-trtllm-eval-gpqa_extended-dataset_path", false], [24, "cmdoption-trtllm-eval-gpqa_extended-max_input_length", false], [24, "cmdoption-trtllm-eval-gpqa_extended-max_output_length", false], [24, "cmdoption-trtllm-eval-gpqa_extended-num_samples", false], [24, "cmdoption-trtllm-eval-gpqa_extended-random_seed", false], [24, "cmdoption-trtllm-eval-gpqa_extended-system_prompt", false]], "trtllm-eval-gpqa_main command line option": [[24, "cmdoption-trtllm-eval-gpqa_main-apply_chat_template", false], [24, "cmdoption-trtllm-eval-gpqa_main-chat_template_kwargs", false], [24, "cmdoption-trtllm-eval-gpqa_main-dataset_path", false], [24, "cmdoption-trtllm-eval-gpqa_main-max_input_length", false], [24, "cmdoption-trtllm-eval-gpqa_main-max_output_length", false], [24, "cmdoption-trtllm-eval-gpqa_main-num_samples", false], [24, "cmdoption-trtllm-eval-gpqa_main-random_seed", false], [24, "cmdoption-trtllm-eval-gpqa_main-system_prompt", false]], "trtllm-eval-gsm8k command line option": [[24, "cmdoption-trtllm-eval-gsm8k-apply_chat_template", false], [24, "cmdoption-trtllm-eval-gsm8k-chat_template_kwargs", false], [24, "cmdoption-trtllm-eval-gsm8k-dataset_path", false], [24, "cmdoption-trtllm-eval-gsm8k-fewshot_as_multiturn", false], [24, "cmdoption-trtllm-eval-gsm8k-max_input_length", false], [24, "cmdoption-trtllm-eval-gsm8k-max_output_length", false], [24, "cmdoption-trtllm-eval-gsm8k-num_samples", false], [24, "cmdoption-trtllm-eval-gsm8k-random_seed", false], [24, "cmdoption-trtllm-eval-gsm8k-system_prompt", false]], "trtllm-eval-json_mode_eval command line option": [[24, "cmdoption-trtllm-eval-json_mode_eval-dataset_path", false], [24, "cmdoption-trtllm-eval-json_mode_eval-max_input_length", false], [24, "cmdoption-trtllm-eval-json_mode_eval-max_output_length", false], [24, "cmdoption-trtllm-eval-json_mode_eval-num_samples", false], [24, "cmdoption-trtllm-eval-json_mode_eval-random_seed", false], [24, "cmdoption-trtllm-eval-json_mode_eval-system_prompt", false]], "trtllm-eval-mmlu command line option": [[24, "cmdoption-trtllm-eval-mmlu-accuracy_threshold", false], [24, "cmdoption-trtllm-eval-mmlu-apply_chat_template", false], [24, "cmdoption-trtllm-eval-mmlu-chat_template_kwargs", false], [24, "cmdoption-trtllm-eval-mmlu-check_accuracy", false], [24, "cmdoption-trtllm-eval-mmlu-dataset_path", false], [24, "cmdoption-trtllm-eval-mmlu-max_input_length", false], [24, "cmdoption-trtllm-eval-mmlu-max_output_length", false], [24, "cmdoption-trtllm-eval-mmlu-num_fewshot", false], [24, "cmdoption-trtllm-eval-mmlu-num_samples", false], [24, "cmdoption-trtllm-eval-mmlu-random_seed", false], [24, "cmdoption-trtllm-eval-mmlu-system_prompt", false]], "trtllm-eval-mmmu command line option": [[24, "cmdoption-trtllm-eval-mmmu-chat_template_kwargs", false], [24, "cmdoption-trtllm-eval-mmmu-dataset_path", false], [24, "cmdoption-trtllm-eval-mmmu-max_input_length", false], [24, "cmdoption-trtllm-eval-mmmu-max_output_length", false], [24, "cmdoption-trtllm-eval-mmmu-num_samples", false], [24, "cmdoption-trtllm-eval-mmmu-random_seed", false], [24, "cmdoption-trtllm-eval-mmmu-system_prompt", false]], "trtllm-serve-disaggregated command line option": [[27, "cmdoption-trtllm-serve-disaggregated-c", false], [27, "cmdoption-trtllm-serve-disaggregated-l", false], [27, "cmdoption-trtllm-serve-disaggregated-m", false], [27, "cmdoption-trtllm-serve-disaggregated-metrics-log-interval", false], [27, "cmdoption-trtllm-serve-disaggregated-r", false], [27, "cmdoption-trtllm-serve-disaggregated-t", false]], "trtllm-serve-disaggregated_mpi_worker command line option": [[27, "cmdoption-trtllm-serve-disaggregated_mpi_worker-c", false], [27, "cmdoption-trtllm-serve-disaggregated_mpi_worker-log_level", false]], "trtllm-serve-mm_embedding_serve command line option": [[27, "cmdoption-trtllm-serve-mm_embedding_serve-arg-MODEL", false], [27, "cmdoption-trtllm-serve-mm_embedding_serve-extra_encoder_options", false], [27, "cmdoption-trtllm-serve-mm_embedding_serve-gpus_per_node", false], [27, "cmdoption-trtllm-serve-mm_embedding_serve-host", false], [27, "cmdoption-trtllm-serve-mm_embedding_serve-log_level", false], [27, "cmdoption-trtllm-serve-mm_embedding_serve-max_batch_size", false], [27, "cmdoption-trtllm-serve-mm_embedding_serve-max_num_tokens", false], [27, "cmdoption-trtllm-serve-mm_embedding_serve-metadata_server_config_file", false], [27, "cmdoption-trtllm-serve-mm_embedding_serve-port", false], [27, "cmdoption-trtllm-serve-mm_embedding_serve-trust_remote_code", false]], "trtllm-serve-serve command line option": [[27, "cmdoption-trtllm-serve-serve-arg-MODEL", false], [27, "cmdoption-trtllm-serve-serve-backend", false], [27, "cmdoption-trtllm-serve-serve-cluster_size", false], [27, "cmdoption-trtllm-serve-serve-enable_chunked_prefill", false], [27, "cmdoption-trtllm-serve-serve-ep_size", false], [27, "cmdoption-trtllm-serve-serve-extra_llm_api_options", false], [27, "cmdoption-trtllm-serve-serve-fail_fast_on_attention_window_too_large", false], [27, "cmdoption-trtllm-serve-serve-gpus_per_node", false], [27, "cmdoption-trtllm-serve-serve-host", false], [27, "cmdoption-trtllm-serve-serve-kv_cache_free_gpu_memory_fraction", false], [27, "cmdoption-trtllm-serve-serve-log_level", false], [27, "cmdoption-trtllm-serve-serve-max_batch_size", false], [27, "cmdoption-trtllm-serve-serve-max_beam_width", false], [27, "cmdoption-trtllm-serve-serve-max_num_tokens", false], [27, "cmdoption-trtllm-serve-serve-max_seq_len", false], [27, "cmdoption-trtllm-serve-serve-metadata_server_config_file", false], [27, "cmdoption-trtllm-serve-serve-num_postprocess_workers", false], [27, "cmdoption-trtllm-serve-serve-port", false], [27, "cmdoption-trtllm-serve-serve-pp_size", false], [27, "cmdoption-trtllm-serve-serve-reasoning_parser", false], [27, "cmdoption-trtllm-serve-serve-server_role", false], [27, "cmdoption-trtllm-serve-serve-tokenizer", false], [27, "cmdoption-trtllm-serve-serve-tp_size", false], [27, "cmdoption-trtllm-serve-serve-trust_remote_code", false]], "trtllm_modules_to_hf_modules (tensorrt_llm.runtime.modelconfig attribute)": [[138, "tensorrt_llm.runtime.ModelConfig.trtllm_modules_to_hf_modules", false]], "trtllmargs (class in tensorrt_llm.llmapi)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs", false]], "trtllmargs.config (class in tensorrt_llm.llmapi)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.Config", false]], "truncate_prompt_tokens (tensorrt_llm.llmapi.samplingparams attribute)": [[147, "tensorrt_llm.llmapi.SamplingParams.truncate_prompt_tokens", false]], "trust_remote_code (tensorrt_llm.llmapi.torchllmargs attribute)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.trust_remote_code", false]], "trust_remote_code (tensorrt_llm.llmapi.trtllmargs attribute)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.trust_remote_code", false]], "twoshot (tensorrt_llm.functional.allreducestrategy attribute)": [[133, "tensorrt_llm.functional.AllReduceStrategy.TWOSHOT", false]], "ub (tensorrt_llm.functional.allreducestrategy attribute)": [[133, "tensorrt_llm.functional.AllReduceStrategy.UB", false]], "unary() (in module tensorrt_llm.functional)": [[133, "tensorrt_llm.functional.unary", false]], "unbind() (in module tensorrt_llm.functional)": [[133, "tensorrt_llm.functional.unbind", false]], "unbind() (tensorrt_llm.functional.tensor method)": [[133, "tensorrt_llm.functional.Tensor.unbind", false]], "unfuse_qkv_projections() (tensorrt_llm.models.sd3transformer2dmodel method)": [[135, "tensorrt_llm.models.SD3Transformer2DModel.unfuse_qkv_projections", false]], "unpatchify() (tensorrt_llm.models.dit method)": [[135, "tensorrt_llm.models.DiT.unpatchify", false]], "unsqueeze() (in module tensorrt_llm.functional)": [[133, "tensorrt_llm.functional.unsqueeze", false]], "unsqueeze() (tensorrt_llm.functional.tensor method)": [[133, "tensorrt_llm.functional.Tensor.unsqueeze", false]], "update() (tensorrt_llm.llmapi.buildconfig method)": [[147, "tensorrt_llm.llmapi.BuildConfig.update", false]], "update() (tensorrt_llm.runtime.samplingconfig method)": [[138, "tensorrt_llm.runtime.SamplingConfig.update", false]], "update_forward_refs() (tensorrt_llm.llmapi.attentiondpconfig class method)": [[147, "tensorrt_llm.llmapi.AttentionDpConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.autodecodingconfig class method)": [[147, "tensorrt_llm.llmapi.AutoDecodingConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.cachetransceiverconfig class method)": [[147, "tensorrt_llm.llmapi.CacheTransceiverConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.calibconfig class method)": [[147, "tensorrt_llm.llmapi.CalibConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.cudagraphconfig class method)": [[147, "tensorrt_llm.llmapi.CudaGraphConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.drafttargetdecodingconfig class method)": [[147, "tensorrt_llm.llmapi.DraftTargetDecodingConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.dynamicbatchconfig class method)": [[147, "tensorrt_llm.llmapi.DynamicBatchConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.eagledecodingconfig class method)": [[147, "tensorrt_llm.llmapi.EagleDecodingConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.extendedruntimeperfknobconfig class method)": [[147, "tensorrt_llm.llmapi.ExtendedRuntimePerfKnobConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.kvcacheconfig class method)": [[147, "tensorrt_llm.llmapi.KvCacheConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.lookaheaddecodingconfig class method)": [[147, "tensorrt_llm.llmapi.LookaheadDecodingConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.medusadecodingconfig class method)": [[147, "tensorrt_llm.llmapi.MedusaDecodingConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.moeconfig class method)": [[147, "tensorrt_llm.llmapi.MoeConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.mtpdecodingconfig class method)": [[147, "tensorrt_llm.llmapi.MTPDecodingConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.ngramdecodingconfig class method)": [[147, "tensorrt_llm.llmapi.NGramDecodingConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.rocketsparseattentionconfig class method)": [[147, "tensorrt_llm.llmapi.RocketSparseAttentionConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.savehiddenstatesdecodingconfig class method)": [[147, "tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.schedulerconfig class method)": [[147, "tensorrt_llm.llmapi.SchedulerConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.torchcompileconfig class method)": [[147, "tensorrt_llm.llmapi.TorchCompileConfig.update_forward_refs", false]], "update_forward_refs() (tensorrt_llm.llmapi.userprovideddecodingconfig class method)": [[147, "tensorrt_llm.llmapi.UserProvidedDecodingConfig.update_forward_refs", false]], "update_from_dict() (tensorrt_llm.llmapi.buildconfig method)": [[147, "tensorrt_llm.llmapi.BuildConfig.update_from_dict", false]], "update_kv_cache_type() (tensorrt_llm.llmapi.buildconfig method)": [[147, "tensorrt_llm.llmapi.BuildConfig.update_kv_cache_type", false]], "update_output_ids_by_offset() (tensorrt_llm.runtime.generationsession method)": [[138, "tensorrt_llm.runtime.GenerationSession.update_output_ids_by_offset", false]], "update_strategy() (tensorrt_llm.functional.allreduceparams method)": [[133, "tensorrt_llm.functional.AllReduceParams.update_strategy", false]], "upper() (tensorrt_llm.llmapi.batchingtype method)": [[147, "tensorrt_llm.llmapi.BatchingType.upper", false]], "upper() (tensorrt_llm.llmapi.capacityschedulerpolicy method)": [[147, "tensorrt_llm.llmapi.CapacitySchedulerPolicy.upper", false]], "upper() (tensorrt_llm.llmapi.contextchunkingpolicy method)": [[147, "tensorrt_llm.llmapi.ContextChunkingPolicy.upper", false]], "upper() (tensorrt_llm.llmapi.quantalgo method)": [[147, "tensorrt_llm.llmapi.QuantAlgo.upper", false]], "use_beam_hyps (tensorrt_llm.runtime.samplingconfig attribute)": [[138, "tensorrt_llm.runtime.SamplingConfig.use_beam_hyps", false]], "use_beam_search (tensorrt_llm.llmapi.samplingparams attribute)": [[147, "tensorrt_llm.llmapi.SamplingParams.use_beam_search", false]], "use_dynamic_tree (tensorrt_llm.llmapi.eagledecodingconfig attribute)": [[147, "tensorrt_llm.llmapi.EagleDecodingConfig.use_dynamic_tree", false]], "use_fp8_context_fmha (tensorrt_llm.plugin.pluginconfig attribute)": [[136, "tensorrt_llm.plugin.PluginConfig.use_fp8_context_fmha", false]], "use_fused_mlp (tensorrt_llm.plugin.pluginconfig attribute)": [[136, "tensorrt_llm.plugin.PluginConfig.use_fused_mlp", false]], "use_gemm_allreduce_plugin (tensorrt_llm.runtime.generationsession property)": [[138, "tensorrt_llm.runtime.GenerationSession.use_gemm_allreduce_plugin", false]], "use_gpt_attention_plugin (tensorrt_llm.runtime.generationsession property)": [[138, "tensorrt_llm.runtime.GenerationSession.use_gpt_attention_plugin", false]], "use_kv_cache (tensorrt_llm.runtime.generationsession property)": [[138, "tensorrt_llm.runtime.GenerationSession.use_kv_cache", false]], "use_lora() (tensorrt_llm.models.decodermodel method)": [[135, "tensorrt_llm.models.DecoderModel.use_lora", false]], "use_lora() (tensorrt_llm.models.encodermodel method)": [[135, "tensorrt_llm.models.EncoderModel.use_lora", false]], "use_lora() (tensorrt_llm.models.gemmaforcausallm method)": [[135, "tensorrt_llm.models.GemmaForCausalLM.use_lora", false]], "use_lora() (tensorrt_llm.models.gptforcausallm method)": [[135, "tensorrt_llm.models.GPTForCausalLM.use_lora", false]], "use_lora() (tensorrt_llm.models.llamaforcausallm method)": [[135, "tensorrt_llm.models.LLaMAForCausalLM.use_lora", false]], "use_lora() (tensorrt_llm.models.mllamaforcausallm method)": [[135, "tensorrt_llm.models.MLLaMAForCausalLM.use_lora", false]], "use_lora() (tensorrt_llm.models.phi3forcausallm method)": [[135, "tensorrt_llm.models.Phi3ForCausalLM.use_lora", false]], "use_lora() (tensorrt_llm.models.phiforcausallm method)": [[135, "tensorrt_llm.models.PhiForCausalLM.use_lora", false]], "use_lora_plugin (tensorrt_llm.runtime.generationsession property)": [[138, "tensorrt_llm.runtime.GenerationSession.use_lora_plugin", false]], "use_lora_plugin (tensorrt_llm.runtime.modelrunner property)": [[138, "tensorrt_llm.runtime.ModelRunner.use_lora_plugin", false]], "use_low_precision_moe_combine (tensorrt_llm.llmapi.moeconfig attribute)": [[147, "tensorrt_llm.llmapi.MoeConfig.use_low_precision_moe_combine", false]], "use_mamba_conv1d_plugin (tensorrt_llm.runtime.generationsession property)": [[138, "tensorrt_llm.runtime.GenerationSession.use_mamba_conv1d_plugin", false]], "use_meta_recipe (tensorrt_llm.llmapi.quantconfig attribute)": [[147, "tensorrt_llm.llmapi.QuantConfig.use_meta_recipe", false]], "use_mrope (tensorrt_llm.llmapi.buildconfig attribute)": [[147, "tensorrt_llm.llmapi.BuildConfig.use_mrope", false]], "use_mtp_vanilla (tensorrt_llm.llmapi.mtpdecodingconfig attribute)": [[147, "tensorrt_llm.llmapi.MTPDecodingConfig.use_mtp_vanilla", false]], "use_paged_context_fmha (tensorrt_llm.plugin.pluginconfig attribute)": [[136, "tensorrt_llm.plugin.PluginConfig.use_paged_context_fmha", false]], "use_prompt_tuning() (tensorrt_llm.models.encodermodel method)": [[135, "tensorrt_llm.models.EncoderModel.use_prompt_tuning", false]], "use_refit (tensorrt_llm.llmapi.buildconfig attribute)": [[147, "tensorrt_llm.llmapi.BuildConfig.use_refit", false]], "use_relaxed_acceptance_for_thinking (tensorrt_llm.llmapi.mtpdecodingconfig attribute)": [[147, "tensorrt_llm.llmapi.MTPDecodingConfig.use_relaxed_acceptance_for_thinking", false]], "use_strip_plan (tensorrt_llm.llmapi.buildconfig attribute)": [[147, "tensorrt_llm.llmapi.BuildConfig.use_strip_plan", false]], "use_uvm (tensorrt_llm.llmapi.kvcacheconfig attribute)": [[147, "tensorrt_llm.llmapi.KvCacheConfig.use_uvm", false]], "user_buffer (tensorrt_llm.plugin.pluginconfig attribute)": [[136, "tensorrt_llm.plugin.PluginConfig.user_buffer", false]], "user_provided (tensorrt_llm.models.speculativedecodingmode attribute)": [[135, "tensorrt_llm.models.SpeculativeDecodingMode.USER_PROVIDED", false]], "userprovideddecodingconfig (class in tensorrt_llm.llmapi)": [[147, "tensorrt_llm.llmapi.UserProvidedDecodingConfig", false]], "userprovideddecodingconfig.config (class in tensorrt_llm.llmapi)": [[147, "tensorrt_llm.llmapi.UserProvidedDecodingConfig.Config", false]], "validate() (tensorrt_llm.llmapi.attentiondpconfig class method)": [[147, "tensorrt_llm.llmapi.AttentionDpConfig.validate", false]], "validate() (tensorrt_llm.llmapi.autodecodingconfig method)": [[147, "tensorrt_llm.llmapi.AutoDecodingConfig.validate", false]], "validate() (tensorrt_llm.llmapi.cachetransceiverconfig class method)": [[147, "tensorrt_llm.llmapi.CacheTransceiverConfig.validate", false]], "validate() (tensorrt_llm.llmapi.calibconfig class method)": [[147, "tensorrt_llm.llmapi.CalibConfig.validate", false]], "validate() (tensorrt_llm.llmapi.cudagraphconfig class method)": [[147, "tensorrt_llm.llmapi.CudaGraphConfig.validate", false]], "validate() (tensorrt_llm.llmapi.drafttargetdecodingconfig method)": [[147, "tensorrt_llm.llmapi.DraftTargetDecodingConfig.validate", false]], "validate() (tensorrt_llm.llmapi.dynamicbatchconfig class method)": [[147, "tensorrt_llm.llmapi.DynamicBatchConfig.validate", false]], "validate() (tensorrt_llm.llmapi.eagledecodingconfig method)": [[147, "tensorrt_llm.llmapi.EagleDecodingConfig.validate", false]], "validate() (tensorrt_llm.llmapi.extendedruntimeperfknobconfig class method)": [[147, "tensorrt_llm.llmapi.ExtendedRuntimePerfKnobConfig.validate", false]], "validate() (tensorrt_llm.llmapi.kvcacheconfig class method)": [[147, "tensorrt_llm.llmapi.KvCacheConfig.validate", false]], "validate() (tensorrt_llm.llmapi.lookaheaddecodingconfig method)": [[147, "tensorrt_llm.llmapi.LookaheadDecodingConfig.validate", false]], "validate() (tensorrt_llm.llmapi.medusadecodingconfig method)": [[147, "tensorrt_llm.llmapi.MedusaDecodingConfig.validate", false]], "validate() (tensorrt_llm.llmapi.moeconfig class method)": [[147, "tensorrt_llm.llmapi.MoeConfig.validate", false]], "validate() (tensorrt_llm.llmapi.mtpdecodingconfig method)": [[147, "tensorrt_llm.llmapi.MTPDecodingConfig.validate", false]], "validate() (tensorrt_llm.llmapi.ngramdecodingconfig method)": [[147, "tensorrt_llm.llmapi.NGramDecodingConfig.validate", false]], "validate() (tensorrt_llm.llmapi.rocketsparseattentionconfig class method)": [[147, "tensorrt_llm.llmapi.RocketSparseAttentionConfig.validate", false]], "validate() (tensorrt_llm.llmapi.savehiddenstatesdecodingconfig method)": [[147, "tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig.validate", false]], "validate() (tensorrt_llm.llmapi.schedulerconfig class method)": [[147, "tensorrt_llm.llmapi.SchedulerConfig.validate", false]], "validate() (tensorrt_llm.llmapi.torchcompileconfig class method)": [[147, "tensorrt_llm.llmapi.TorchCompileConfig.validate", false]], "validate() (tensorrt_llm.llmapi.userprovideddecodingconfig method)": [[147, "tensorrt_llm.llmapi.UserProvidedDecodingConfig.validate", false]], "validate() (tensorrt_llm.plugin.pluginconfig method)": [[136, "tensorrt_llm.plugin.PluginConfig.validate", false]], "validate_and_init_tokenizer() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_and_init_tokenizer", false]], "validate_and_init_tokenizer() (tensorrt_llm.llmapi.trtllmargs method)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.validate_and_init_tokenizer", false]], "validate_attention_dp_config() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_attention_dp_config", false]], "validate_auto_parallel() (tensorrt_llm.llmapi.trtllmargs method)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.validate_auto_parallel", false]], "validate_batch_wait_max_tokens_ratio() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_batch_wait_max_tokens_ratio", false]], "validate_batch_wait_timeout_iters() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_batch_wait_timeout_iters", false]], "validate_batch_wait_timeout_ms() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_batch_wait_timeout_ms", false]], "validate_build_config_remaining() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_build_config_remaining", false]], "validate_build_config_remaining() (tensorrt_llm.llmapi.trtllmargs method)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.validate_build_config_remaining", false]], "validate_build_config_with_runtime_params() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_build_config_with_runtime_params", false]], "validate_build_config_with_runtime_params() (tensorrt_llm.llmapi.trtllmargs method)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.validate_build_config_with_runtime_params", false]], "validate_capture_num_tokens() (tensorrt_llm.llmapi.torchcompileconfig class method)": [[147, "tensorrt_llm.llmapi.TorchCompileConfig.validate_capture_num_tokens", false]], "validate_checkpoint_format() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_checkpoint_format", false]], "validate_cuda_graph_config() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_cuda_graph_config", false]], "validate_cuda_graph_max_batch_size() (tensorrt_llm.llmapi.cudagraphconfig class method)": [[147, "tensorrt_llm.llmapi.CudaGraphConfig.validate_cuda_graph_max_batch_size", false]], "validate_dtype() (tensorrt_llm.llmapi.torchllmargs class method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_dtype", false]], "validate_dtype() (tensorrt_llm.llmapi.trtllmargs class method)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.validate_dtype", false]], "validate_dtype_not_auto() (tensorrt_llm.plugin.pluginconfig class method)": [[136, "tensorrt_llm.plugin.PluginConfig.validate_dtype_not_auto", false]], "validate_enable_build_cache() (tensorrt_llm.llmapi.trtllmargs method)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.validate_enable_build_cache", false]], "validate_gpus_per_node() (tensorrt_llm.llmapi.torchllmargs class method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_gpus_per_node", false]], "validate_gpus_per_node() (tensorrt_llm.llmapi.trtllmargs class method)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.validate_gpus_per_node", false]], "validate_kv_cache_dtype() (tensorrt_llm.llmapi.trtllmargs method)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.validate_kv_cache_dtype", false]], "validate_load_balancer() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_load_balancer", false]], "validate_lora_config_consistency() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_lora_config_consistency", false]], "validate_lora_config_consistency() (tensorrt_llm.llmapi.trtllmargs method)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.validate_lora_config_consistency", false]], "validate_max_attention_window() (tensorrt_llm.llmapi.kvcacheconfig class method)": [[147, "tensorrt_llm.llmapi.KvCacheConfig.validate_max_attention_window", false]], "validate_max_gpu_total_bytes() (tensorrt_llm.llmapi.kvcacheconfig class method)": [[147, "tensorrt_llm.llmapi.KvCacheConfig.validate_max_gpu_total_bytes", false]], "validate_model() (tensorrt_llm.llmapi.torchllmargs class method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_model", false]], "validate_model() (tensorrt_llm.llmapi.trtllmargs class method)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.validate_model", false]], "validate_model_format_misc() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_model_format_misc", false]], "validate_model_format_misc() (tensorrt_llm.llmapi.trtllmargs method)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.validate_model_format_misc", false]], "validate_parallel_config() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_parallel_config", false]], "validate_parallel_config() (tensorrt_llm.llmapi.trtllmargs method)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.validate_parallel_config", false]], "validate_peft_cache_config() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_peft_cache_config", false]], "validate_peft_cache_config() (tensorrt_llm.llmapi.trtllmargs method)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.validate_peft_cache_config", false]], "validate_positive_values() (tensorrt_llm.llmapi.lookaheaddecodingconfig class method)": [[147, "tensorrt_llm.llmapi.LookaheadDecodingConfig.validate_positive_values", false]], "validate_quant_config() (tensorrt_llm.llmapi.trtllmargs class method)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.validate_quant_config", false]], "validate_runtime_args() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_runtime_args", false]], "validate_runtime_args() (tensorrt_llm.llmapi.trtllmargs method)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.validate_runtime_args", false]], "validate_speculative_config() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_speculative_config", false]], "validate_speculative_config() (tensorrt_llm.llmapi.trtllmargs method)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.validate_speculative_config", false]], "validate_stream_interval() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.validate_stream_interval", false]], "validate_torch_compile_max_num_streams() (tensorrt_llm.llmapi.torchcompileconfig class method)": [[147, "tensorrt_llm.llmapi.TorchCompileConfig.validate_torch_compile_max_num_streams", false]], "verbatim (tensorrt_llm.models.gemmaconfig attribute)": [[135, "tensorrt_llm.models.GemmaConfig.VERBATIM", false]], "video_preprocess() (tensorrt_llm.runtime.multimodalmodelrunner method)": [[138, "tensorrt_llm.runtime.MultimodalModelRunner.video_preprocess", false]], "view() (in module tensorrt_llm.functional)": [[133, "tensorrt_llm.functional.view", false]], "view() (tensorrt_llm.functional.tensor method)": [[133, "tensorrt_llm.functional.Tensor.view", false]], "view() (tensorrt_llm.runtime.tensorinfo method)": [[138, "tensorrt_llm.runtime.TensorInfo.view", false]], "visual_engine_dir (tensorrt_llm.runtime.multimodalmodelrunner property)": [[138, "tensorrt_llm.runtime.MultimodalModelRunner.visual_engine_dir", false]], "visualize_network (tensorrt_llm.llmapi.buildconfig attribute)": [[147, "tensorrt_llm.llmapi.BuildConfig.visualize_network", false]], "vocab_size (tensorrt_llm.runtime.generationsession property)": [[138, "tensorrt_llm.runtime.GenerationSession.vocab_size", false]], "vocab_size (tensorrt_llm.runtime.modelconfig attribute)": [[138, "tensorrt_llm.runtime.ModelConfig.vocab_size", false]], "vocab_size (tensorrt_llm.runtime.modelrunner property)": [[138, "tensorrt_llm.runtime.ModelRunner.vocab_size", false]], "vocab_size (tensorrt_llm.runtime.modelrunnercpp property)": [[138, "tensorrt_llm.runtime.ModelRunnerCpp.vocab_size", false]], "vocab_size_padded (tensorrt_llm.runtime.modelrunner property)": [[138, "tensorrt_llm.runtime.ModelRunner.vocab_size_padded", false]], "vocab_size_padded (tensorrt_llm.runtime.modelrunnercpp property)": [[138, "tensorrt_llm.runtime.ModelRunnerCpp.vocab_size_padded", false]], "w4a16 (tensorrt_llm.llmapi.quantalgo attribute)": [[147, "tensorrt_llm.llmapi.QuantAlgo.W4A16", false]], "w4a16_awq (tensorrt_llm.llmapi.quantalgo attribute)": [[147, "tensorrt_llm.llmapi.QuantAlgo.W4A16_AWQ", false]], "w4a16_gptq (tensorrt_llm.llmapi.quantalgo attribute)": [[147, "tensorrt_llm.llmapi.QuantAlgo.W4A16_GPTQ", false]], "w4a16_mxfp4 (tensorrt_llm.llmapi.quantalgo attribute)": [[147, "tensorrt_llm.llmapi.QuantAlgo.W4A16_MXFP4", false]], "w4a8_awq (tensorrt_llm.llmapi.quantalgo attribute)": [[147, "tensorrt_llm.llmapi.QuantAlgo.W4A8_AWQ", false]], "w4a8_mxfp4_fp8 (tensorrt_llm.llmapi.quantalgo attribute)": [[147, "tensorrt_llm.llmapi.QuantAlgo.W4A8_MXFP4_FP8", false]], "w4a8_mxfp4_mxfp8 (tensorrt_llm.llmapi.quantalgo attribute)": [[147, "tensorrt_llm.llmapi.QuantAlgo.W4A8_MXFP4_MXFP8", false]], "w4a8_nvfp4_fp8 (tensorrt_llm.llmapi.quantalgo attribute)": [[147, "tensorrt_llm.llmapi.QuantAlgo.W4A8_NVFP4_FP8", false]], "w4a8_qserve_per_channel (tensorrt_llm.llmapi.quantalgo attribute)": [[147, "tensorrt_llm.llmapi.QuantAlgo.W4A8_QSERVE_PER_CHANNEL", false]], "w4a8_qserve_per_group (tensorrt_llm.llmapi.quantalgo attribute)": [[147, "tensorrt_llm.llmapi.QuantAlgo.W4A8_QSERVE_PER_GROUP", false]], "w8a16 (tensorrt_llm.llmapi.quantalgo attribute)": [[147, "tensorrt_llm.llmapi.QuantAlgo.W8A16", false]], "w8a16_gptq (tensorrt_llm.llmapi.quantalgo attribute)": [[147, "tensorrt_llm.llmapi.QuantAlgo.W8A16_GPTQ", false]], "w8a8_sq_per_channel (tensorrt_llm.llmapi.quantalgo attribute)": [[147, "tensorrt_llm.llmapi.QuantAlgo.W8A8_SQ_PER_CHANNEL", false]], "w8a8_sq_per_channel_per_tensor_plugin (tensorrt_llm.llmapi.quantalgo attribute)": [[147, "tensorrt_llm.llmapi.QuantAlgo.W8A8_SQ_PER_CHANNEL_PER_TENSOR_PLUGIN", false]], "w8a8_sq_per_channel_per_token_plugin (tensorrt_llm.llmapi.quantalgo attribute)": [[147, "tensorrt_llm.llmapi.QuantAlgo.W8A8_SQ_PER_CHANNEL_PER_TOKEN_PLUGIN", false]], "w8a8_sq_per_tensor_per_token_plugin (tensorrt_llm.llmapi.quantalgo attribute)": [[147, "tensorrt_llm.llmapi.QuantAlgo.W8A8_SQ_PER_TENSOR_PER_TOKEN_PLUGIN", false]], "w8a8_sq_per_tensor_plugin (tensorrt_llm.llmapi.quantalgo attribute)": [[147, "tensorrt_llm.llmapi.QuantAlgo.W8A8_SQ_PER_TENSOR_PLUGIN", false]], "warn_on_unstable_feature_usage() (tensorrt_llm.llmapi.torchllmargs method)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.warn_on_unstable_feature_usage", false]], "weight_loader() (tensorrt_llm.layers.attention.deepseekv2attention method)": [[134, "tensorrt_llm.layers.attention.DeepseekV2Attention.weight_loader", false]], "weight_loader() (tensorrt_llm.layers.embedding.embedding method)": [[134, "tensorrt_llm.layers.embedding.Embedding.weight_loader", false]], "weight_loader() (tensorrt_llm.layers.linear.linearbase method)": [[134, "tensorrt_llm.layers.linear.LinearBase.weight_loader", false]], "weight_only_groupwise_quant_matmul_plugin (tensorrt_llm.plugin.pluginconfig attribute)": [[136, "tensorrt_llm.plugin.PluginConfig.weight_only_groupwise_quant_matmul_plugin", false]], "weight_only_quant_matmul_plugin (tensorrt_llm.plugin.pluginconfig attribute)": [[136, "tensorrt_llm.plugin.PluginConfig.weight_only_quant_matmul_plugin", false]], "weight_sparsity (tensorrt_llm.llmapi.buildconfig attribute)": [[147, "tensorrt_llm.llmapi.BuildConfig.weight_sparsity", false]], "weight_streaming (tensorrt_llm.llmapi.buildconfig attribute)": [[147, "tensorrt_llm.llmapi.BuildConfig.weight_streaming", false]], "where() (in module tensorrt_llm.functional)": [[133, "tensorrt_llm.functional.where", false]], "whisperencoder (class in tensorrt_llm.models)": [[135, "tensorrt_llm.models.WhisperEncoder", false]], "window_size (tensorrt_llm.llmapi.rocketsparseattentionconfig attribute)": [[147, "tensorrt_llm.llmapi.RocketSparseAttentionConfig.window_size", false]], "with_traceback() (tensorrt_llm.llmapi.requesterror method)": [[147, "tensorrt_llm.llmapi.RequestError.with_traceback", false]], "workspace (tensorrt_llm.llmapi.trtllmargs attribute)": [[147, "tensorrt_llm.llmapi.TrtLlmArgs.workspace", false]], "wrapped_property (tensorrt_llm.llmapi.torchllmargs attribute)": [[147, "tensorrt_llm.llmapi.TorchLlmArgs.wrapped_property", false]], "wrapped_property (tensorrt_llm.llmapi.trtllmargs attribute)": [[147, "id17", false], [147, "id20", false], [147, "tensorrt_llm.llmapi.TrtLlmArgs.wrapped_property", false]], "write_interval (tensorrt_llm.llmapi.savehiddenstatesdecodingconfig attribute)": [[147, "tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig.write_interval", false]], "yarn (tensorrt_llm.functional.positionembeddingtype attribute)": [[133, "tensorrt_llm.functional.PositionEmbeddingType.yarn", false]], "yarn (tensorrt_llm.functional.rotaryscalingtype attribute)": [[133, "tensorrt_llm.functional.RotaryScalingType.yarn", false]], "zfill() (tensorrt_llm.llmapi.batchingtype method)": [[147, "tensorrt_llm.llmapi.BatchingType.zfill", false]], "zfill() (tensorrt_llm.llmapi.capacityschedulerpolicy method)": [[147, "tensorrt_llm.llmapi.CapacitySchedulerPolicy.zfill", false]], "zfill() (tensorrt_llm.llmapi.contextchunkingpolicy method)": [[147, "tensorrt_llm.llmapi.ContextChunkingPolicy.zfill", false]], "zfill() (tensorrt_llm.llmapi.quantalgo method)": [[147, "tensorrt_llm.llmapi.QuantAlgo.zfill", false]]}, "objects": {"": [[1, 0, 1, "c.FMT_DIM", "FMT_DIM"], [1, 0, 1, "c.SET_FROM_OPTIONAL", "SET_FROM_OPTIONAL"], [1, 1, 1, "_CPPv48nvinfer1", "nvinfer1"], [0, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [0, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [0, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [0, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [0, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [0, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [0, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [0, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [1, 1, 1, "_CPPv412tensorrt_llm", "tensorrt_llm"], [0, 1, 1, "_CPPv4N12tensorrt_llm13batch_managerE", "tensorrt_llm::batch_manager"], [1, 1, 1, "_CPPv4N12tensorrt_llm13batch_managerE", "tensorrt_llm::batch_manager"], [1, 1, 1, "_CPPv4N12tensorrt_llm13batch_managerE", "tensorrt_llm::batch_manager"], [1, 1, 1, "_CPPv4N12tensorrt_llm13batch_managerE", "tensorrt_llm::batch_manager"], [1, 1, 1, "_CPPv4N12tensorrt_llm13batch_managerE", "tensorrt_llm::batch_manager"], [0, 1, 1, "_CPPv4N12tensorrt_llm13batch_manager16kv_cache_managerE", "tensorrt_llm::batch_manager::kv_cache_manager"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executorE", "tensorrt_llm::executor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executorE", "tensorrt_llm::executor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executorE", "tensorrt_llm::executor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executorE", "tensorrt_llm::executor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executorE", "tensorrt_llm::executor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executorE", "tensorrt_llm::executor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executorE", "tensorrt_llm::executor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executorE", "tensorrt_llm::executor"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor21AdditionalModelOutputE", "tensorrt_llm::executor::AdditionalModelOutput"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor21AdditionalModelOutput21AdditionalModelOutputENSt6stringEb", "tensorrt_llm::executor::AdditionalModelOutput::AdditionalModelOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor21AdditionalModelOutput21AdditionalModelOutputENSt6stringEb", "tensorrt_llm::executor::AdditionalModelOutput::AdditionalModelOutput::gatherContext"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor21AdditionalModelOutput21AdditionalModelOutputENSt6stringEb", "tensorrt_llm::executor::AdditionalModelOutput::AdditionalModelOutput::name"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor21AdditionalModelOutput13gatherContextE", "tensorrt_llm::executor::AdditionalModelOutput::gatherContext"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor21AdditionalModelOutput4nameE", "tensorrt_llm::executor::AdditionalModelOutput::name"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor21AdditionalModelOutputeqERK21AdditionalModelOutput", "tensorrt_llm::executor::AdditionalModelOutput::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor21AdditionalModelOutputeqERK21AdditionalModelOutput", "tensorrt_llm::executor::AdditionalModelOutput::operator==::other"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutputE", "tensorrt_llm::executor::AdditionalOutput"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutput16AdditionalOutputENSt6stringE6Tensor", "tensorrt_llm::executor::AdditionalOutput::AdditionalOutput"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutput16AdditionalOutputERK16AdditionalOutput", "tensorrt_llm::executor::AdditionalOutput::AdditionalOutput"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutput16AdditionalOutputERR16AdditionalOutput", "tensorrt_llm::executor::AdditionalOutput::AdditionalOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutput16AdditionalOutputENSt6stringE6Tensor", "tensorrt_llm::executor::AdditionalOutput::AdditionalOutput::name"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutput16AdditionalOutputERK16AdditionalOutput", "tensorrt_llm::executor::AdditionalOutput::AdditionalOutput::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutput16AdditionalOutputERR16AdditionalOutput", "tensorrt_llm::executor::AdditionalOutput::AdditionalOutput::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutput16AdditionalOutputENSt6stringE6Tensor", "tensorrt_llm::executor::AdditionalOutput::AdditionalOutput::output"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutput4nameE", "tensorrt_llm::executor::AdditionalOutput::name"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutputaSERK16AdditionalOutput", "tensorrt_llm::executor::AdditionalOutput::operator="], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutputaSERR16AdditionalOutput", "tensorrt_llm::executor::AdditionalOutput::operator="], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutputaSERK16AdditionalOutput", "tensorrt_llm::executor::AdditionalOutput::operator=::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutputaSERR16AdditionalOutput", "tensorrt_llm::executor::AdditionalOutput::operator=::other"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutput6outputE", "tensorrt_llm::executor::AdditionalOutput::output"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor16AdditionalOutputD0Ev", "tensorrt_llm::executor::AdditionalOutput::~AdditionalOutput"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor12BatchingTypeE", "tensorrt_llm::executor::BatchingType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12BatchingType9kINFLIGHTE", "tensorrt_llm::executor::BatchingType::kINFLIGHT"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12BatchingType7kSTATICE", "tensorrt_llm::executor::BatchingType::kSTATIC"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor10BeamTokensE", "tensorrt_llm::executor::BeamTokens"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor10BufferViewE", "tensorrt_llm::executor::BufferView"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor15CacheSaltIDTypeE", "tensorrt_llm::executor::CacheSaltIDType"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfigE", "tensorrt_llm::executor::CacheTransceiverConfig"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig11BackendTypeE", "tensorrt_llm::executor::CacheTransceiverConfig::BackendType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig11BackendType7DEFAULTE", "tensorrt_llm::executor::CacheTransceiverConfig::BackendType::DEFAULT"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig11BackendType3MPIE", "tensorrt_llm::executor::CacheTransceiverConfig::BackendType::MPI"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig11BackendType4NIXLE", "tensorrt_llm::executor::CacheTransceiverConfig::BackendType::NIXL"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig11BackendType3UCXE", "tensorrt_llm::executor::CacheTransceiverConfig::BackendType::UCX"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig22CacheTransceiverConfigENSt8optionalI11BackendTypeEENSt8optionalI6size_tEE", "tensorrt_llm::executor::CacheTransceiverConfig::CacheTransceiverConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig22CacheTransceiverConfigENSt8optionalI11BackendTypeEENSt8optionalI6size_tEE", "tensorrt_llm::executor::CacheTransceiverConfig::CacheTransceiverConfig::backendType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig22CacheTransceiverConfigENSt8optionalI11BackendTypeEENSt8optionalI6size_tEE", "tensorrt_llm::executor::CacheTransceiverConfig::CacheTransceiverConfig::maxNumTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22CacheTransceiverConfig14getBackendTypeEv", "tensorrt_llm::executor::CacheTransceiverConfig::getBackendType"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22CacheTransceiverConfig20getMaxTokensInBufferEv", "tensorrt_llm::executor::CacheTransceiverConfig::getMaxTokensInBuffer"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig12mBackendTypeE", "tensorrt_llm::executor::CacheTransceiverConfig::mBackendType"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig18mMaxTokensInBufferE", "tensorrt_llm::executor::CacheTransceiverConfig::mMaxTokensInBuffer"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22CacheTransceiverConfigeqERK22CacheTransceiverConfig", "tensorrt_llm::executor::CacheTransceiverConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor22CacheTransceiverConfigeqERK22CacheTransceiverConfig", "tensorrt_llm::executor::CacheTransceiverConfig::operator==::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig14setBackendTypeENSt8optionalI11BackendTypeEE", "tensorrt_llm::executor::CacheTransceiverConfig::setBackendType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig14setBackendTypeENSt8optionalI11BackendTypeEE", "tensorrt_llm::executor::CacheTransceiverConfig::setBackendType::backendType"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig20setMaxTokensInBufferENSt8optionalI6size_tEE", "tensorrt_llm::executor::CacheTransceiverConfig::setMaxTokensInBuffer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22CacheTransceiverConfig20setMaxTokensInBufferENSt8optionalI6size_tEE", "tensorrt_llm::executor::CacheTransceiverConfig::setMaxTokensInBuffer::maxTokensInBuffer"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor23CapacitySchedulerPolicyE", "tensorrt_llm::executor::CapacitySchedulerPolicy"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor23CapacitySchedulerPolicy20kGUARANTEED_NO_EVICTE", "tensorrt_llm::executor::CapacitySchedulerPolicy::kGUARANTEED_NO_EVICT"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor23CapacitySchedulerPolicy16kMAX_UTILIZATIONE", "tensorrt_llm::executor::CapacitySchedulerPolicy::kMAX_UTILIZATION"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor23CapacitySchedulerPolicy13kSTATIC_BATCHE", "tensorrt_llm::executor::CapacitySchedulerPolicy::kSTATIC_BATCH"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor17CommunicationModeE", "tensorrt_llm::executor::CommunicationMode"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor17CommunicationMode7kLEADERE", "tensorrt_llm::executor::CommunicationMode::kLEADER"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor17CommunicationMode13kORCHESTRATORE", "tensorrt_llm::executor::CommunicationMode::kORCHESTRATOR"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor17CommunicationTypeE", "tensorrt_llm::executor::CommunicationType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor17CommunicationType4kMPIE", "tensorrt_llm::executor::CommunicationType::kMPI"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor21ContextChunkingPolicyE", "tensorrt_llm::executor::ContextChunkingPolicy"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor21ContextChunkingPolicy15kEQUAL_PROGRESSE", "tensorrt_llm::executor::ContextChunkingPolicy::kEQUAL_PROGRESS"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor21ContextChunkingPolicy24kFIRST_COME_FIRST_SERVEDE", "tensorrt_llm::executor::ContextChunkingPolicy::kFIRST_COME_FIRST_SERVED"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParamsE", "tensorrt_llm::executor::ContextPhaseParams"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypeNSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypePvNSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypeRKNSt6vectorIcEENSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsERK18ContextPhaseParams", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsERR18ContextPhaseParams", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypeNSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::draftTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypePvNSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::draftTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypeRKNSt6vectorIcEENSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::draftTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypeNSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::firstGenTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypePvNSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::firstGenTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypeRKNSt6vectorIcEENSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::firstGenTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypeNSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::reqId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypePvNSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::reqId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypeRKNSt6vectorIcEENSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::reqId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypeRKNSt6vectorIcEENSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::serializedState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams18ContextPhaseParamsE9VecTokens13RequestIdTypePvNSt8optionalI9VecTokensEE", "tensorrt_llm::executor::ContextPhaseParams::ContextPhaseParams::state"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams13RequestIdTypeE", "tensorrt_llm::executor::ContextPhaseParams::RequestIdType"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams8StatePtrE", "tensorrt_llm::executor::ContextPhaseParams::StatePtr"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams7deleterEPKv", "tensorrt_llm::executor::ContextPhaseParams::deleter"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams7deleterEPKv", "tensorrt_llm::executor::ContextPhaseParams::deleter::data"], [0, 3, 1, "_CPPv4NKR12tensorrt_llm8executor18ContextPhaseParams14getDraftTokensEv", "tensorrt_llm::executor::ContextPhaseParams::getDraftTokens"], [0, 3, 1, "_CPPv4NKR12tensorrt_llm8executor18ContextPhaseParams17getFirstGenTokensEv", "tensorrt_llm::executor::ContextPhaseParams::getFirstGenTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18ContextPhaseParams8getReqIdEv", "tensorrt_llm::executor::ContextPhaseParams::getReqId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18ContextPhaseParams18getSerializedStateEv", "tensorrt_llm::executor::ContextPhaseParams::getSerializedState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams8getStateEv", "tensorrt_llm::executor::ContextPhaseParams::getState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18ContextPhaseParams8getStateEv", "tensorrt_llm::executor::ContextPhaseParams::getState"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams12mDraftTokensE", "tensorrt_llm::executor::ContextPhaseParams::mDraftTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams15mFirstGenTokensE", "tensorrt_llm::executor::ContextPhaseParams::mFirstGenTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams6mReqIdE", "tensorrt_llm::executor::ContextPhaseParams::mReqId"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams6mStateE", "tensorrt_llm::executor::ContextPhaseParams::mState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParamsaSERK18ContextPhaseParams", "tensorrt_llm::executor::ContextPhaseParams::operator="], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParamsaSERR18ContextPhaseParams", "tensorrt_llm::executor::ContextPhaseParams::operator="], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18ContextPhaseParamseqERK18ContextPhaseParams", "tensorrt_llm::executor::ContextPhaseParams::operator=="], [0, 3, 1, "_CPPv4NO12tensorrt_llm8executor18ContextPhaseParams17popFirstGenTokensEv", "tensorrt_llm::executor::ContextPhaseParams::popFirstGenTokens"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParams12releaseStateEv", "tensorrt_llm::executor::ContextPhaseParams::releaseState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18ContextPhaseParamsD0Ev", "tensorrt_llm::executor::ContextPhaseParams::~ContextPhaseParams"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverStateE", "tensorrt_llm::executor::DataTransceiverState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState20DataTransceiverStateEN8kv_cache10CacheStateEN8kv_cache9CommStateE", "tensorrt_llm::executor::DataTransceiverState::DataTransceiverState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState20DataTransceiverStateEv", "tensorrt_llm::executor::DataTransceiverState::DataTransceiverState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState20DataTransceiverStateEN8kv_cache10CacheStateEN8kv_cache9CommStateE", "tensorrt_llm::executor::DataTransceiverState::DataTransceiverState::cacheState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState20DataTransceiverStateEN8kv_cache10CacheStateEN8kv_cache9CommStateE", "tensorrt_llm::executor::DataTransceiverState::DataTransceiverState::commState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20DataTransceiverState13getCacheStateEv", "tensorrt_llm::executor::DataTransceiverState::getCacheState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20DataTransceiverState12getCommStateEv", "tensorrt_llm::executor::DataTransceiverState::getCommState"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState11mCacheStateE", "tensorrt_llm::executor::DataTransceiverState::mCacheState"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState10mCommStateE", "tensorrt_llm::executor::DataTransceiverState::mCommState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20DataTransceiverStateeqERK20DataTransceiverState", "tensorrt_llm::executor::DataTransceiverState::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor20DataTransceiverStateeqERK20DataTransceiverState", "tensorrt_llm::executor::DataTransceiverState::operator==::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState13setCacheStateEN8kv_cache10CacheStateE", "tensorrt_llm::executor::DataTransceiverState::setCacheState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState13setCacheStateEN8kv_cache10CacheStateE", "tensorrt_llm::executor::DataTransceiverState::setCacheState::state"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState12setCommStateEN8kv_cache9CommStateE", "tensorrt_llm::executor::DataTransceiverState::setCommState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20DataTransceiverState12setCommStateEN8kv_cache9CommStateE", "tensorrt_llm::executor::DataTransceiverState::setCommState::state"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20DataTransceiverState8toStringEv", "tensorrt_llm::executor::DataTransceiverState::toString"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor8DataTypeE", "tensorrt_llm::executor::DataType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType5kBF16E", "tensorrt_llm::executor::DataType::kBF16"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType5kBOOLE", "tensorrt_llm::executor::DataType::kBOOL"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType5kFP16E", "tensorrt_llm::executor::DataType::kFP16"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType5kFP32E", "tensorrt_llm::executor::DataType::kFP32"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType4kFP8E", "tensorrt_llm::executor::DataType::kFP8"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType6kINT32E", "tensorrt_llm::executor::DataType::kINT32"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType6kINT64E", "tensorrt_llm::executor::DataType::kINT64"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType5kINT8E", "tensorrt_llm::executor::DataType::kINT8"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType6kUINT8E", "tensorrt_llm::executor::DataType::kUINT8"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8DataType8kUNKNOWNE", "tensorrt_llm::executor::DataType::kUNKNOWN"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfigE", "tensorrt_llm::executor::DebugConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig11DebugConfigEbb9StringVec10SizeType32", "tensorrt_llm::executor::DebugConfig::DebugConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig11DebugConfigEbb9StringVec10SizeType32", "tensorrt_llm::executor::DebugConfig::DebugConfig::debugInputTensors"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig11DebugConfigEbb9StringVec10SizeType32", "tensorrt_llm::executor::DebugConfig::DebugConfig::debugOutputTensors"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig11DebugConfigEbb9StringVec10SizeType32", "tensorrt_llm::executor::DebugConfig::DebugConfig::debugTensorNames"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig11DebugConfigEbb9StringVec10SizeType32", "tensorrt_llm::executor::DebugConfig::DebugConfig::debugTensorsMaxIterations"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig9StringVecE", "tensorrt_llm::executor::DebugConfig::StringVec"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11DebugConfig20getDebugInputTensorsEv", "tensorrt_llm::executor::DebugConfig::getDebugInputTensors"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11DebugConfig21getDebugOutputTensorsEv", "tensorrt_llm::executor::DebugConfig::getDebugOutputTensors"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11DebugConfig19getDebugTensorNamesEv", "tensorrt_llm::executor::DebugConfig::getDebugTensorNames"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11DebugConfig28getDebugTensorsMaxIterationsEv", "tensorrt_llm::executor::DebugConfig::getDebugTensorsMaxIterations"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig18mDebugInputTensorsE", "tensorrt_llm::executor::DebugConfig::mDebugInputTensors"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig19mDebugOutputTensorsE", "tensorrt_llm::executor::DebugConfig::mDebugOutputTensors"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig17mDebugTensorNamesE", "tensorrt_llm::executor::DebugConfig::mDebugTensorNames"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig26mDebugTensorsMaxIterationsE", "tensorrt_llm::executor::DebugConfig::mDebugTensorsMaxIterations"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11DebugConfigeqERK11DebugConfig", "tensorrt_llm::executor::DebugConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor11DebugConfigeqERK11DebugConfig", "tensorrt_llm::executor::DebugConfig::operator==::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig20setDebugInputTensorsEb", "tensorrt_llm::executor::DebugConfig::setDebugInputTensors"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig20setDebugInputTensorsEb", "tensorrt_llm::executor::DebugConfig::setDebugInputTensors::debugInputTensors"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig21setDebugOutputTensorsEb", "tensorrt_llm::executor::DebugConfig::setDebugOutputTensors"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig21setDebugOutputTensorsEb", "tensorrt_llm::executor::DebugConfig::setDebugOutputTensors::debugOutputTensors"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig19setDebugTensorNamesERK9StringVec", "tensorrt_llm::executor::DebugConfig::setDebugTensorNames"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig19setDebugTensorNamesERK9StringVec", "tensorrt_llm::executor::DebugConfig::setDebugTensorNames::debugTensorNames"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig28setDebugTensorsMaxIterationsE10SizeType32", "tensorrt_llm::executor::DebugConfig::setDebugTensorsMaxIterations"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11DebugConfig28setDebugTensorsMaxIterationsE10SizeType32", "tensorrt_llm::executor::DebugConfig::setDebugTensorsMaxIterations::debugTensorsMaxIterations"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor24DebugTensorsPerIterationE", "tensorrt_llm::executor::DebugTensorsPerIteration"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor24DebugTensorsPerIteration12debugTensorsE", "tensorrt_llm::executor::DebugTensorsPerIteration::debugTensors"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor24DebugTensorsPerIteration4iterE", "tensorrt_llm::executor::DebugTensorsPerIteration::iter"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfigE", "tensorrt_llm::executor::DecodingConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig14DecodingConfigENSt8optionalI12DecodingModeEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI13MedusaChoicesEENSt8optionalI11EagleConfigEE", "tensorrt_llm::executor::DecodingConfig::DecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig14DecodingConfigENSt8optionalI12DecodingModeEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI13MedusaChoicesEENSt8optionalI11EagleConfigEE", "tensorrt_llm::executor::DecodingConfig::DecodingConfig::decodingMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig14DecodingConfigENSt8optionalI12DecodingModeEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI13MedusaChoicesEENSt8optionalI11EagleConfigEE", "tensorrt_llm::executor::DecodingConfig::DecodingConfig::eagleConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig14DecodingConfigENSt8optionalI12DecodingModeEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI13MedusaChoicesEENSt8optionalI11EagleConfigEE", "tensorrt_llm::executor::DecodingConfig::DecodingConfig::lookaheadDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig14DecodingConfigENSt8optionalI12DecodingModeEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI13MedusaChoicesEENSt8optionalI11EagleConfigEE", "tensorrt_llm::executor::DecodingConfig::DecodingConfig::medusaChoices"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig31enableSeamlessLookaheadDecodingEv", "tensorrt_llm::executor::DecodingConfig::enableSeamlessLookaheadDecoding"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14DecodingConfig15getDecodingModeEv", "tensorrt_llm::executor::DecodingConfig::getDecodingMode"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14DecodingConfig14getEagleConfigEv", "tensorrt_llm::executor::DecodingConfig::getEagleConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14DecodingConfig26getLookaheadDecodingConfigEv", "tensorrt_llm::executor::DecodingConfig::getLookaheadDecodingConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14DecodingConfig33getLookaheadDecodingMaxNumRequestEv", "tensorrt_llm::executor::DecodingConfig::getLookaheadDecodingMaxNumRequest"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14DecodingConfig16getMedusaChoicesEv", "tensorrt_llm::executor::DecodingConfig::getMedusaChoices"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig13mDecodingModeE", "tensorrt_llm::executor::DecodingConfig::mDecodingMode"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig12mEagleConfigE", "tensorrt_llm::executor::DecodingConfig::mEagleConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig24mLookaheadDecodingConfigE", "tensorrt_llm::executor::DecodingConfig::mLookaheadDecodingConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig31mLookaheadDecodingMaxNumRequestE", "tensorrt_llm::executor::DecodingConfig::mLookaheadDecodingMaxNumRequest"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig14mMedusaChoicesE", "tensorrt_llm::executor::DecodingConfig::mMedusaChoices"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14DecodingConfigeqERK14DecodingConfig", "tensorrt_llm::executor::DecodingConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor14DecodingConfigeqERK14DecodingConfig", "tensorrt_llm::executor::DecodingConfig::operator==::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig15setDecodingModeERK12DecodingMode", "tensorrt_llm::executor::DecodingConfig::setDecodingMode"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig14setEagleConfigERK11EagleConfig", "tensorrt_llm::executor::DecodingConfig::setEagleConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig26setLookaheadDecodingConfigERK23LookaheadDecodingConfig", "tensorrt_llm::executor::DecodingConfig::setLookaheadDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig26setLookaheadDecodingConfigERK23LookaheadDecodingConfig", "tensorrt_llm::executor::DecodingConfig::setLookaheadDecodingConfig::lookaheadDecodingConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14DecodingConfig16setMedusaChoicesERK13MedusaChoices", "tensorrt_llm::executor::DecodingConfig::setMedusaChoices"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor12DecodingModeE", "tensorrt_llm::executor::DecodingMode"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode4AutoEv", "tensorrt_llm::executor::DecodingMode::Auto"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode10BeamSearchEv", "tensorrt_llm::executor::DecodingMode::BeamSearch"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode12DecodingModeE14UnderlyingType", "tensorrt_llm::executor::DecodingMode::DecodingMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode12DecodingModeE14UnderlyingType", "tensorrt_llm::executor::DecodingMode::DecodingMode::state"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode5EagleEv", "tensorrt_llm::executor::DecodingMode::Eagle"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode19ExplicitDraftTokensEv", "tensorrt_llm::executor::DecodingMode::ExplicitDraftTokens"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode19ExternalDraftTokensEv", "tensorrt_llm::executor::DecodingMode::ExternalDraftTokens"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode9LookaheadEv", "tensorrt_llm::executor::DecodingMode::Lookahead"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode6MedusaEv", "tensorrt_llm::executor::DecodingMode::Medusa"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode4TopKEv", "tensorrt_llm::executor::DecodingMode::TopK"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode8TopKTopPEv", "tensorrt_llm::executor::DecodingMode::TopKTopP"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode4TopPEv", "tensorrt_llm::executor::DecodingMode::TopP"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode14UnderlyingTypeE", "tensorrt_llm::executor::DecodingMode::UnderlyingType"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode9allBitSetE14UnderlyingType", "tensorrt_llm::executor::DecodingMode::allBitSet"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode9allBitSetE14UnderlyingType", "tensorrt_llm::executor::DecodingMode::allBitSet::bits"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode9anyBitSetE14UnderlyingType", "tensorrt_llm::executor::DecodingMode::anyBitSet"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode9anyBitSetE14UnderlyingType", "tensorrt_llm::executor::DecodingMode::anyBitSet::bits"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode7getNameEv", "tensorrt_llm::executor::DecodingMode::getName"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode8getStateEv", "tensorrt_llm::executor::DecodingMode::getState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode6isAutoEv", "tensorrt_llm::executor::DecodingMode::isAuto"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode12isBeamSearchEv", "tensorrt_llm::executor::DecodingMode::isBeamSearch"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode7isEagleEv", "tensorrt_llm::executor::DecodingMode::isEagle"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode21isExplicitDraftTokensEv", "tensorrt_llm::executor::DecodingMode::isExplicitDraftTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode21isExternalDraftTokensEv", "tensorrt_llm::executor::DecodingMode::isExternalDraftTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode11isLookaheadEv", "tensorrt_llm::executor::DecodingMode::isLookahead"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode8isMedusaEv", "tensorrt_llm::executor::DecodingMode::isMedusa"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode6isTopKEv", "tensorrt_llm::executor::DecodingMode::isTopK"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode13isTopKandTopPEv", "tensorrt_llm::executor::DecodingMode::isTopKandTopP"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode12isTopKorTopPEv", "tensorrt_llm::executor::DecodingMode::isTopKorTopP"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode6isTopPEv", "tensorrt_llm::executor::DecodingMode::isTopP"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode14isUseBanTokensEv", "tensorrt_llm::executor::DecodingMode::isUseBanTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode13isUseBanWordsEv", "tensorrt_llm::executor::DecodingMode::isUseBanWords"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode20isUseExplicitEosStopEv", "tensorrt_llm::executor::DecodingMode::isUseExplicitEosStop"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode21isUseFrequencyPenaltyEv", "tensorrt_llm::executor::DecodingMode::isUseFrequencyPenalty"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode18isUseMaxLengthStopEv", "tensorrt_llm::executor::DecodingMode::isUseMaxLengthStop"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode14isUseMinLengthEv", "tensorrt_llm::executor::DecodingMode::isUseMinLength"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode9isUseMinPEv", "tensorrt_llm::executor::DecodingMode::isUseMinP"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode22isUseNoRepeatNgramSizeEv", "tensorrt_llm::executor::DecodingMode::isUseNoRepeatNgramSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode22isUseOccurrencePenaltyEv", "tensorrt_llm::executor::DecodingMode::isUseOccurrencePenalty"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode12isUsePenaltyEv", "tensorrt_llm::executor::DecodingMode::isUsePenalty"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode20isUsePresencePenaltyEv", "tensorrt_llm::executor::DecodingMode::isUsePresencePenalty"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode22isUseRepetitionPenaltyEv", "tensorrt_llm::executor::DecodingMode::isUseRepetitionPenalty"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode17isUseStopCriteriaEv", "tensorrt_llm::executor::DecodingMode::isUseStopCriteria"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode14isUseStopWordsEv", "tensorrt_llm::executor::DecodingMode::isUseStopWords"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode16isUseTemperatureEv", "tensorrt_llm::executor::DecodingMode::isUseTemperature"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingMode28isUseVariableBeamWidthSearchEv", "tensorrt_llm::executor::DecodingMode::isUseVariableBeamWidthSearch"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode5kAutoE", "tensorrt_llm::executor::DecodingMode::kAuto"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode11kBeamSearchE", "tensorrt_llm::executor::DecodingMode::kBeamSearch"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode6kEagleE", "tensorrt_llm::executor::DecodingMode::kEagle"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode20kExplicitDraftTokensE", "tensorrt_llm::executor::DecodingMode::kExplicitDraftTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode20kExternalDraftTokensE", "tensorrt_llm::executor::DecodingMode::kExternalDraftTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode10kLookaheadE", "tensorrt_llm::executor::DecodingMode::kLookahead"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode7kMedusaE", "tensorrt_llm::executor::DecodingMode::kMedusa"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode9kNumFlagsE", "tensorrt_llm::executor::DecodingMode::kNumFlags"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode5kTopKE", "tensorrt_llm::executor::DecodingMode::kTopK"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode9kTopKTopPE", "tensorrt_llm::executor::DecodingMode::kTopKTopP"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode5kTopPE", "tensorrt_llm::executor::DecodingMode::kTopP"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode13kUseBanTokensE", "tensorrt_llm::executor::DecodingMode::kUseBanTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode12kUseBanWordsE", "tensorrt_llm::executor::DecodingMode::kUseBanWords"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode19kUseExplicitEosStopE", "tensorrt_llm::executor::DecodingMode::kUseExplicitEosStop"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode22kUseFrequencyPenaltiesE", "tensorrt_llm::executor::DecodingMode::kUseFrequencyPenalties"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode17kUseMaxLengthStopE", "tensorrt_llm::executor::DecodingMode::kUseMaxLengthStop"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode13kUseMinLengthE", "tensorrt_llm::executor::DecodingMode::kUseMinLength"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode8kUseMinPE", "tensorrt_llm::executor::DecodingMode::kUseMinP"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode21kUseNoRepeatNgramSizeE", "tensorrt_llm::executor::DecodingMode::kUseNoRepeatNgramSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode23kUseOccurrencePenaltiesE", "tensorrt_llm::executor::DecodingMode::kUseOccurrencePenalties"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode13kUsePenaltiesE", "tensorrt_llm::executor::DecodingMode::kUsePenalties"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode21kUsePresencePenaltiesE", "tensorrt_llm::executor::DecodingMode::kUsePresencePenalties"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode23kUseRepetitionPenaltiesE", "tensorrt_llm::executor::DecodingMode::kUseRepetitionPenalties"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode24kUseStandardStopCriteriaE", "tensorrt_llm::executor::DecodingMode::kUseStandardStopCriteria"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode13kUseStopWordsE", "tensorrt_llm::executor::DecodingMode::kUseStopWords"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode15kUseTemperatureE", "tensorrt_llm::executor::DecodingMode::kUseTemperature"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode27kUseVariableBeamWidthSearchE", "tensorrt_llm::executor::DecodingMode::kUseVariableBeamWidthSearch"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode6mStateE", "tensorrt_llm::executor::DecodingMode::mState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingModeeqERK12DecodingMode", "tensorrt_llm::executor::DecodingMode::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor12DecodingModeeqERK12DecodingMode", "tensorrt_llm::executor::DecodingMode::operator==::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode8setBitToE14UnderlyingTypeb", "tensorrt_llm::executor::DecodingMode::setBitTo"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode8setBitToE14UnderlyingTypeb", "tensorrt_llm::executor::DecodingMode::setBitTo::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode8setBitToE14UnderlyingTypeb", "tensorrt_llm::executor::DecodingMode::setBitTo::x"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode12useBanTokensEb", "tensorrt_llm::executor::DecodingMode::useBanTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode12useBanTokensEb", "tensorrt_llm::executor::DecodingMode::useBanTokens::banTokens"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode11useBanWordsEb", "tensorrt_llm::executor::DecodingMode::useBanWords"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode11useBanWordsEb", "tensorrt_llm::executor::DecodingMode::useBanWords::banWords"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode18useExplicitEosStopEb", "tensorrt_llm::executor::DecodingMode::useExplicitEosStop"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode18useExplicitEosStopEb", "tensorrt_llm::executor::DecodingMode::useExplicitEosStop::explicitEosStop"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode19useFrequencyPenaltyEb", "tensorrt_llm::executor::DecodingMode::useFrequencyPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode19useFrequencyPenaltyEb", "tensorrt_llm::executor::DecodingMode::useFrequencyPenalty::usePenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode16useMaxLengthStopEb", "tensorrt_llm::executor::DecodingMode::useMaxLengthStop"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode16useMaxLengthStopEb", "tensorrt_llm::executor::DecodingMode::useMaxLengthStop::maxLengthStop"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode12useMinLengthEb", "tensorrt_llm::executor::DecodingMode::useMinLength"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode12useMinLengthEb", "tensorrt_llm::executor::DecodingMode::useMinLength::useMinLen"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode7useMinPEb", "tensorrt_llm::executor::DecodingMode::useMinP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode7useMinPEb", "tensorrt_llm::executor::DecodingMode::useMinP::useMinP"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode20useNoRepeatNgramSizeEb", "tensorrt_llm::executor::DecodingMode::useNoRepeatNgramSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode20useNoRepeatNgramSizeEb", "tensorrt_llm::executor::DecodingMode::useNoRepeatNgramSize::noRepeatNgramSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode22useOccurrencePenaltiesEb", "tensorrt_llm::executor::DecodingMode::useOccurrencePenalties"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode22useOccurrencePenaltiesEb", "tensorrt_llm::executor::DecodingMode::useOccurrencePenalties::usePenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode18usePresencePenaltyEb", "tensorrt_llm::executor::DecodingMode::usePresencePenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode18usePresencePenaltyEb", "tensorrt_llm::executor::DecodingMode::usePresencePenalty::usePenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode20useRepetitionPenaltyEb", "tensorrt_llm::executor::DecodingMode::useRepetitionPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode20useRepetitionPenaltyEb", "tensorrt_llm::executor::DecodingMode::useRepetitionPenalty::usePenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode12useStopWordsEb", "tensorrt_llm::executor::DecodingMode::useStopWords"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode12useStopWordsEb", "tensorrt_llm::executor::DecodingMode::useStopWords::stopWords"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode14useTemperatureEb", "tensorrt_llm::executor::DecodingMode::useTemperature"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode14useTemperatureEb", "tensorrt_llm::executor::DecodingMode::useTemperature::useTemp"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode26useVariableBeamWidthSearchEb", "tensorrt_llm::executor::DecodingMode::useVariableBeamWidthSearch"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12DecodingMode26useVariableBeamWidthSearchEb", "tensorrt_llm::executor::DecodingMode::useVariableBeamWidthSearch::useVariableBeamWidthSearch"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor22DisServingRequestStatsE", "tensorrt_llm::executor::DisServingRequestStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22DisServingRequestStats11kvCacheSizeE", "tensorrt_llm::executor::DisServingRequestStats::kvCacheSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22DisServingRequestStats17kvCacheTransferMSE", "tensorrt_llm::executor::DisServingRequestStats::kvCacheTransferMS"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfigE", "tensorrt_llm::executor::DynamicBatchConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig18DynamicBatchConfigEbb10SizeType32NSt6vectorINSt4pairI10SizeType3210SizeType32EEEE", "tensorrt_llm::executor::DynamicBatchConfig::DynamicBatchConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig18DynamicBatchConfigEbb10SizeType32NSt6vectorINSt4pairI10SizeType3210SizeType32EEEE", "tensorrt_llm::executor::DynamicBatchConfig::DynamicBatchConfig::batchSizeTable"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig18DynamicBatchConfigEbb10SizeType32NSt6vectorINSt4pairI10SizeType3210SizeType32EEEE", "tensorrt_llm::executor::DynamicBatchConfig::DynamicBatchConfig::dynamicBatchMovingAverageWindow"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig18DynamicBatchConfigEbb10SizeType32NSt6vectorINSt4pairI10SizeType3210SizeType32EEEE", "tensorrt_llm::executor::DynamicBatchConfig::DynamicBatchConfig::enableBatchSizeTuning"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig18DynamicBatchConfigEbb10SizeType32NSt6vectorINSt4pairI10SizeType3210SizeType32EEEE", "tensorrt_llm::executor::DynamicBatchConfig::DynamicBatchConfig::enableMaxNumTokensTuning"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18DynamicBatchConfig17getBatchSizeTableEv", "tensorrt_llm::executor::DynamicBatchConfig::getBatchSizeTable"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18DynamicBatchConfig34getDynamicBatchMovingAverageWindowEv", "tensorrt_llm::executor::DynamicBatchConfig::getDynamicBatchMovingAverageWindow"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18DynamicBatchConfig24getEnableBatchSizeTuningEv", "tensorrt_llm::executor::DynamicBatchConfig::getEnableBatchSizeTuning"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18DynamicBatchConfig27getEnableMaxNumTokensTuningEv", "tensorrt_llm::executor::DynamicBatchConfig::getEnableMaxNumTokensTuning"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig22kDefaultBatchSizeTableE", "tensorrt_llm::executor::DynamicBatchConfig::kDefaultBatchSizeTable"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig39kDefaultDynamicBatchMovingAverageWindowE", "tensorrt_llm::executor::DynamicBatchConfig::kDefaultDynamicBatchMovingAverageWindow"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig15mBatchSizeTableE", "tensorrt_llm::executor::DynamicBatchConfig::mBatchSizeTable"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig32mDynamicBatchMovingAverageWindowE", "tensorrt_llm::executor::DynamicBatchConfig::mDynamicBatchMovingAverageWindow"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig22mEnableBatchSizeTuningE", "tensorrt_llm::executor::DynamicBatchConfig::mEnableBatchSizeTuning"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18DynamicBatchConfig25mEnableMaxNumTokensTuningE", "tensorrt_llm::executor::DynamicBatchConfig::mEnableMaxNumTokensTuning"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor12EagleChoicesE", "tensorrt_llm::executor::EagleChoices"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfigE", "tensorrt_llm::executor::EagleConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig11EagleConfigENSt8optionalI12EagleChoicesEEbNSt8optionalIfEEbNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::EagleConfig::EagleConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig11EagleConfigENSt8optionalI12EagleChoicesEEbNSt8optionalIfEEbNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::EagleConfig::EagleConfig::dynamicTreeMaxTopK"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig11EagleConfigENSt8optionalI12EagleChoicesEEbNSt8optionalIfEEbNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::EagleConfig::EagleConfig::eagleChoices"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig11EagleConfigENSt8optionalI12EagleChoicesEEbNSt8optionalIfEEbNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::EagleConfig::EagleConfig::greedySampling"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig11EagleConfigENSt8optionalI12EagleChoicesEEbNSt8optionalIfEEbNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::EagleConfig::EagleConfig::posteriorThreshold"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig11EagleConfigENSt8optionalI12EagleChoicesEEbNSt8optionalIfEEbNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::EagleConfig::EagleConfig::useDynamicTree"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig19checkPosteriorValueERKNSt8optionalIfEE", "tensorrt_llm::executor::EagleConfig::checkPosteriorValue"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig19checkPosteriorValueERKNSt8optionalIfEE", "tensorrt_llm::executor::EagleConfig::checkPosteriorValue::value"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11EagleConfig21getDynamicTreeMaxTopKEv", "tensorrt_llm::executor::EagleConfig::getDynamicTreeMaxTopK"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11EagleConfig15getEagleChoicesEv", "tensorrt_llm::executor::EagleConfig::getEagleChoices"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11EagleConfig21getPosteriorThresholdEv", "tensorrt_llm::executor::EagleConfig::getPosteriorThreshold"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11EagleConfig16isGreedySamplingEv", "tensorrt_llm::executor::EagleConfig::isGreedySampling"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig19mDynamicTreeMaxTopKE", "tensorrt_llm::executor::EagleConfig::mDynamicTreeMaxTopK"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig13mEagleChoicesE", "tensorrt_llm::executor::EagleConfig::mEagleChoices"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig15mGreedySamplingE", "tensorrt_llm::executor::EagleConfig::mGreedySampling"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig19mPosteriorThresholdE", "tensorrt_llm::executor::EagleConfig::mPosteriorThreshold"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11EagleConfig15mUseDynamicTreeE", "tensorrt_llm::executor::EagleConfig::mUseDynamicTree"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11EagleConfigeqERK11EagleConfig", "tensorrt_llm::executor::EagleConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor11EagleConfigeqERK11EagleConfig", "tensorrt_llm::executor::EagleConfig::operator==::other"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11EagleConfig14useDynamicTreeEv", "tensorrt_llm::executor::EagleConfig::useDynamicTree"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8ExecutorE", "tensorrt_llm::executor::Executor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorENSt10shared_ptrI5ModelEENSt10shared_ptrI5ModelEERK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorENSt10shared_ptrI5ModelEERK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfigRKNSt8optionalINSt3mapINSt6stringE6TensorEEEE", "tensorrt_llm::executor::Executor::Executor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK8Executor", "tensorrt_llm::executor::Executor::Executor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERKNSt10filesystem4pathE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERKNSt10filesystem4pathERKNSt10filesystem4pathE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERR8Executor", "tensorrt_llm::executor::Executor::Executor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::decoderEngineBuffer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::decoderJsonConfigStr"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorENSt10shared_ptrI5ModelEENSt10shared_ptrI5ModelEERK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::decoderModel"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERKNSt10filesystem4pathERKNSt10filesystem4pathE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::decoderModelPath"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::encoderEngineBuffer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::encoderJsonConfigStr"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorENSt10shared_ptrI5ModelEENSt10shared_ptrI5ModelEERK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::encoderModel"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERKNSt10filesystem4pathERKNSt10filesystem4pathE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::encoderModelPath"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfigRKNSt8optionalINSt3mapINSt6stringE6TensorEEEE", "tensorrt_llm::executor::Executor::Executor::engineBuffer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK8Executor", "tensorrt_llm::executor::Executor::Executor::executor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorENSt10shared_ptrI5ModelEENSt10shared_ptrI5ModelEERK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::executorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorENSt10shared_ptrI5ModelEERK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::executorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfigRKNSt8optionalINSt3mapINSt6stringE6TensorEEEE", "tensorrt_llm::executor::Executor::Executor::executorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::executorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERKNSt10filesystem4pathE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::executorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERKNSt10filesystem4pathERKNSt10filesystem4pathE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::executorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfigRKNSt8optionalINSt3mapINSt6stringE6TensorEEEE", "tensorrt_llm::executor::Executor::Executor::jsonConfigStr"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfigRKNSt8optionalINSt3mapINSt6stringE6TensorEEEE", "tensorrt_llm::executor::Executor::Executor::managedWeights"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorENSt10shared_ptrI5ModelEERK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::model"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERKNSt10filesystem4pathE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::modelPath"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfigRKNSt8optionalINSt3mapINSt6stringE6TensorEEEE", "tensorrt_llm::executor::Executor::Executor::modelType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERK10BufferViewRKNSt6stringERK10BufferViewRKNSt6stringE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::modelType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERKNSt10filesystem4pathE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::modelType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor8ExecutorERKNSt10filesystem4pathERKNSt10filesystem4pathE9ModelTypeRK14ExecutorConfig", "tensorrt_llm::executor::Executor::Executor::modelType"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor14awaitResponsesERK6IdTypeRKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::Executor::awaitResponses"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor14awaitResponsesERKNSt6vectorI6IdTypeEERKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::Executor::awaitResponses"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor14awaitResponsesERKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::Executor::awaitResponses"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor14awaitResponsesERK6IdTypeRKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::Executor::awaitResponses::requestId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor14awaitResponsesERKNSt6vectorI6IdTypeEERKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::Executor::awaitResponses::requestIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor14awaitResponsesERK6IdTypeRKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::Executor::awaitResponses::timeout"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor14awaitResponsesERKNSt6vectorI6IdTypeEERKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::Executor::awaitResponses::timeout"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor14awaitResponsesERKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::Executor::awaitResponses::timeout"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8Executor18canEnqueueRequestsEv", "tensorrt_llm::executor::Executor::canEnqueueRequests"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor13cancelRequestE6IdType", "tensorrt_llm::executor::Executor::cancelRequest"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor13cancelRequestE6IdType", "tensorrt_llm::executor::Executor::cancelRequest::requestId"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor14enqueueRequestERK7Request", "tensorrt_llm::executor::Executor::enqueueRequest"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor14enqueueRequestERK7Request", "tensorrt_llm::executor::Executor::enqueueRequest::request"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor15enqueueRequestsERKNSt6vectorI7RequestEE", "tensorrt_llm::executor::Executor::enqueueRequests"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Executor15enqueueRequestsERKNSt6vectorI7RequestEE", "tensorrt_llm::executor::Executor::enqueueRequests::requests"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8Executor22getKVCacheEventManagerEv", "tensorrt_llm::executor::Executor::getKVCacheEventManager"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor21getLatestDebugTensorsEv", "tensorrt_llm::executor::Executor::getLatestDebugTensors"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor23getLatestIterationStatsEv", "tensorrt_llm::executor::Executor::getLatestIterationStats"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor21getLatestRequestStatsEv", "tensorrt_llm::executor::Executor::getLatestRequestStats"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8Executor20getNumResponsesReadyERKNSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Executor::getNumResponsesReady"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8Executor20getNumResponsesReadyERKNSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Executor::getNumResponsesReady::requestId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8Executor13isParticipantEv", "tensorrt_llm::executor::Executor::isParticipant"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8Executor5mImplE", "tensorrt_llm::executor::Executor::mImpl"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8ExecutoraSERK8Executor", "tensorrt_llm::executor::Executor::operator="], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8ExecutoraSERR8Executor", "tensorrt_llm::executor::Executor::operator="], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8ExecutoraSERK8Executor", "tensorrt_llm::executor::Executor::operator=::executor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Executor8shutdownEv", "tensorrt_llm::executor::Executor::shutdown"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8ExecutorD0Ev", "tensorrt_llm::executor::Executor::~Executor"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfigE", "tensorrt_llm::executor::ExecutorConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::additionalModelOutputs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::batchingType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::cacheTransceiverConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::debugConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::decodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::enableChunkedContext"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::enableTrtOverlap"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::extendedRuntimePerfKnobConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::failFastOnAttentionWindowTooLarge"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::gatherGenerationLogits"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::gpuWeightsPercent"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::guidedDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::iterStatsMaxIterations"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::kvCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::logitsPostProcessorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::maxBatchSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::maxBeamWidth"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::maxNumTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::maxQueueSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::maxSeqIdleMicroseconds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::normalizeLogProbs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::parallelConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::peftCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::promptTableOffloading"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::recvPollPeriodMs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::requestStatsMaxIterations"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::schedulerConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::specDecConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14ExecutorConfigE10SizeType3215SchedulerConfig13KvCacheConfigbb10SizeType3210SizeType3212BatchingTypeNSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI14ParallelConfigEERKNSt8optionalI15PeftCacheConfigEENSt8optionalI25LogitsPostProcessorConfigEENSt8optionalI14DecodingConfigEEbfNSt8optionalI10SizeType32EERK29ExtendedRuntimePerfKnobConfigNSt8optionalI11DebugConfigEE10SizeType328uint64_tNSt8optionalI25SpeculativeDecodingConfigEENSt8optionalI20GuidedDecodingConfigEENSt8optionalINSt6vectorI21AdditionalModelOutputEEEENSt8optionalI22CacheTransceiverConfigEEbbbb", "tensorrt_llm::executor::ExecutorConfig::ExecutorConfig::useGpuDirectStorage"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig25getAdditionalModelOutputsEv", "tensorrt_llm::executor::ExecutorConfig::getAdditionalModelOutputs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig15getBatchingTypeEv", "tensorrt_llm::executor::ExecutorConfig::getBatchingType"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig25getCacheTransceiverConfigEv", "tensorrt_llm::executor::ExecutorConfig::getCacheTransceiverConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig14getDebugConfigEv", "tensorrt_llm::executor::ExecutorConfig::getDebugConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig17getDecodingConfigEv", "tensorrt_llm::executor::ExecutorConfig::getDecodingConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig23getEnableChunkedContextEv", "tensorrt_llm::executor::ExecutorConfig::getEnableChunkedContext"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig19getEnableTrtOverlapEv", "tensorrt_llm::executor::ExecutorConfig::getEnableTrtOverlap"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig32getExtendedRuntimePerfKnobConfigEv", "tensorrt_llm::executor::ExecutorConfig::getExtendedRuntimePerfKnobConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig36getFailFastOnAttentionWindowTooLargeEv", "tensorrt_llm::executor::ExecutorConfig::getFailFastOnAttentionWindowTooLarge"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig25getGatherGenerationLogitsEv", "tensorrt_llm::executor::ExecutorConfig::getGatherGenerationLogits"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig20getGpuWeightsPercentEv", "tensorrt_llm::executor::ExecutorConfig::getGpuWeightsPercent"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig23getGuidedDecodingConfigEv", "tensorrt_llm::executor::ExecutorConfig::getGuidedDecodingConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig25getIterStatsMaxIterationsEv", "tensorrt_llm::executor::ExecutorConfig::getIterStatsMaxIterations"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig16getKvCacheConfigEv", "tensorrt_llm::executor::ExecutorConfig::getKvCacheConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig19getKvCacheConfigRefEv", "tensorrt_llm::executor::ExecutorConfig::getKvCacheConfigRef"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig28getLogitsPostProcessorConfigEv", "tensorrt_llm::executor::ExecutorConfig::getLogitsPostProcessorConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig15getMaxBatchSizeEv", "tensorrt_llm::executor::ExecutorConfig::getMaxBatchSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig15getMaxBeamWidthEv", "tensorrt_llm::executor::ExecutorConfig::getMaxBeamWidth"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig15getMaxNumTokensEv", "tensorrt_llm::executor::ExecutorConfig::getMaxNumTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig15getMaxQueueSizeEv", "tensorrt_llm::executor::ExecutorConfig::getMaxQueueSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig25getMaxSeqIdleMicrosecondsEv", "tensorrt_llm::executor::ExecutorConfig::getMaxSeqIdleMicroseconds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig20getNormalizeLogProbsEv", "tensorrt_llm::executor::ExecutorConfig::getNormalizeLogProbs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig17getParallelConfigEv", "tensorrt_llm::executor::ExecutorConfig::getParallelConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig18getPeftCacheConfigEv", "tensorrt_llm::executor::ExecutorConfig::getPeftCacheConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig24getPromptTableOffloadingEv", "tensorrt_llm::executor::ExecutorConfig::getPromptTableOffloading"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig19getRecvPollPeriodMsEv", "tensorrt_llm::executor::ExecutorConfig::getRecvPollPeriodMs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig28getRequestStatsMaxIterationsEv", "tensorrt_llm::executor::ExecutorConfig::getRequestStatsMaxIterations"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig18getSchedulerConfigEv", "tensorrt_llm::executor::ExecutorConfig::getSchedulerConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig21getSchedulerConfigRefEv", "tensorrt_llm::executor::ExecutorConfig::getSchedulerConfigRef"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig16getSpecDecConfigEv", "tensorrt_llm::executor::ExecutorConfig::getSpecDecConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ExecutorConfig22getUseGpuDirectStorageEv", "tensorrt_llm::executor::ExecutorConfig::getUseGpuDirectStorage"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig30kDefaultIterStatsMaxIterationsE", "tensorrt_llm::executor::ExecutorConfig::kDefaultIterStatsMaxIterations"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig30kDefaultMaxSeqIdleMicrosecondsE", "tensorrt_llm::executor::ExecutorConfig::kDefaultMaxSeqIdleMicroseconds"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig33kDefaultRequestStatsMaxIterationsE", "tensorrt_llm::executor::ExecutorConfig::kDefaultRequestStatsMaxIterations"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig23mAdditionalModelOutputsE", "tensorrt_llm::executor::ExecutorConfig::mAdditionalModelOutputs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig13mBatchingTypeE", "tensorrt_llm::executor::ExecutorConfig::mBatchingType"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig23mCacheTransceiverConfigE", "tensorrt_llm::executor::ExecutorConfig::mCacheTransceiverConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig12mDebugConfigE", "tensorrt_llm::executor::ExecutorConfig::mDebugConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15mDecodingConfigE", "tensorrt_llm::executor::ExecutorConfig::mDecodingConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig21mEnableChunkedContextE", "tensorrt_llm::executor::ExecutorConfig::mEnableChunkedContext"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig17mEnableTrtOverlapE", "tensorrt_llm::executor::ExecutorConfig::mEnableTrtOverlap"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig30mExtendedRuntimePerfKnobConfigE", "tensorrt_llm::executor::ExecutorConfig::mExtendedRuntimePerfKnobConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig34mFailFastOnAttentionWindowTooLargeE", "tensorrt_llm::executor::ExecutorConfig::mFailFastOnAttentionWindowTooLarge"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig23mGatherGenerationLogitsE", "tensorrt_llm::executor::ExecutorConfig::mGatherGenerationLogits"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig18mGpuWeightsPercentE", "tensorrt_llm::executor::ExecutorConfig::mGpuWeightsPercent"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig21mGuidedDecodingConfigE", "tensorrt_llm::executor::ExecutorConfig::mGuidedDecodingConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig23mIterStatsMaxIterationsE", "tensorrt_llm::executor::ExecutorConfig::mIterStatsMaxIterations"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14mKvCacheConfigE", "tensorrt_llm::executor::ExecutorConfig::mKvCacheConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig26mLogitsPostProcessorConfigE", "tensorrt_llm::executor::ExecutorConfig::mLogitsPostProcessorConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig13mMaxBatchSizeE", "tensorrt_llm::executor::ExecutorConfig::mMaxBatchSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig13mMaxBeamWidthE", "tensorrt_llm::executor::ExecutorConfig::mMaxBeamWidth"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig13mMaxNumTokensE", "tensorrt_llm::executor::ExecutorConfig::mMaxNumTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig13mMaxQueueSizeE", "tensorrt_llm::executor::ExecutorConfig::mMaxQueueSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig23mMaxSeqIdleMicrosecondsE", "tensorrt_llm::executor::ExecutorConfig::mMaxSeqIdleMicroseconds"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig18mNormalizeLogProbsE", "tensorrt_llm::executor::ExecutorConfig::mNormalizeLogProbs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15mParallelConfigE", "tensorrt_llm::executor::ExecutorConfig::mParallelConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig16mPeftCacheConfigE", "tensorrt_llm::executor::ExecutorConfig::mPeftCacheConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig22mPromptTableOffloadingE", "tensorrt_llm::executor::ExecutorConfig::mPromptTableOffloading"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig17mRecvPollPeriodMsE", "tensorrt_llm::executor::ExecutorConfig::mRecvPollPeriodMs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig26mRequestStatsMaxIterationsE", "tensorrt_llm::executor::ExecutorConfig::mRequestStatsMaxIterations"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig16mSchedulerConfigE", "tensorrt_llm::executor::ExecutorConfig::mSchedulerConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig26mSpeculativeDecodingConfigE", "tensorrt_llm::executor::ExecutorConfig::mSpeculativeDecodingConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig20mUseGpuDirectStorageE", "tensorrt_llm::executor::ExecutorConfig::mUseGpuDirectStorage"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setAdditionalModelOutputsERKNSt6vectorI21AdditionalModelOutputEE", "tensorrt_llm::executor::ExecutorConfig::setAdditionalModelOutputs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setAdditionalModelOutputsERKNSt6vectorI21AdditionalModelOutputEE", "tensorrt_llm::executor::ExecutorConfig::setAdditionalModelOutputs::additionalModelOutputs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setBatchingTypeE12BatchingType", "tensorrt_llm::executor::ExecutorConfig::setBatchingType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setBatchingTypeE12BatchingType", "tensorrt_llm::executor::ExecutorConfig::setBatchingType::batchingType"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setCacheTransceiverConfigERK22CacheTransceiverConfig", "tensorrt_llm::executor::ExecutorConfig::setCacheTransceiverConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setCacheTransceiverConfigERK22CacheTransceiverConfig", "tensorrt_llm::executor::ExecutorConfig::setCacheTransceiverConfig::cacheTransceiverConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14setDebugConfigERK11DebugConfig", "tensorrt_llm::executor::ExecutorConfig::setDebugConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig14setDebugConfigERK11DebugConfig", "tensorrt_llm::executor::ExecutorConfig::setDebugConfig::debugConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig17setDecodingConfigERK14DecodingConfig", "tensorrt_llm::executor::ExecutorConfig::setDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig17setDecodingConfigERK14DecodingConfig", "tensorrt_llm::executor::ExecutorConfig::setDecodingConfig::decodingConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig23setEnableChunkedContextEb", "tensorrt_llm::executor::ExecutorConfig::setEnableChunkedContext"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig23setEnableChunkedContextEb", "tensorrt_llm::executor::ExecutorConfig::setEnableChunkedContext::enableChunkedContext"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig19setEnableTrtOverlapEb", "tensorrt_llm::executor::ExecutorConfig::setEnableTrtOverlap"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig19setEnableTrtOverlapEb", "tensorrt_llm::executor::ExecutorConfig::setEnableTrtOverlap::enableTrtOverlap"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig32setExtendedRuntimePerfKnobConfigERK29ExtendedRuntimePerfKnobConfig", "tensorrt_llm::executor::ExecutorConfig::setExtendedRuntimePerfKnobConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig32setExtendedRuntimePerfKnobConfigERK29ExtendedRuntimePerfKnobConfig", "tensorrt_llm::executor::ExecutorConfig::setExtendedRuntimePerfKnobConfig::extendedRuntimePerfKnobConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig36setFailFastOnAttentionWindowTooLargeEb", "tensorrt_llm::executor::ExecutorConfig::setFailFastOnAttentionWindowTooLarge"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig36setFailFastOnAttentionWindowTooLargeEb", "tensorrt_llm::executor::ExecutorConfig::setFailFastOnAttentionWindowTooLarge::failFastOnAttentionWindowTooLarge"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setGatherGenerationLogitsEb", "tensorrt_llm::executor::ExecutorConfig::setGatherGenerationLogits"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setGatherGenerationLogitsEb", "tensorrt_llm::executor::ExecutorConfig::setGatherGenerationLogits::gatherGenerationLogits"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig20setGpuWeightsPercentERKf", "tensorrt_llm::executor::ExecutorConfig::setGpuWeightsPercent"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig20setGpuWeightsPercentERKf", "tensorrt_llm::executor::ExecutorConfig::setGpuWeightsPercent::gpuWeightsPercent"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig23setGuidedDecodingConfigERK20GuidedDecodingConfig", "tensorrt_llm::executor::ExecutorConfig::setGuidedDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig23setGuidedDecodingConfigERK20GuidedDecodingConfig", "tensorrt_llm::executor::ExecutorConfig::setGuidedDecodingConfig::guidedDecodingConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setIterStatsMaxIterationsE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setIterStatsMaxIterations"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setIterStatsMaxIterationsE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setIterStatsMaxIterations::iterStatsMaxIterations"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig16setKvCacheConfigERK13KvCacheConfig", "tensorrt_llm::executor::ExecutorConfig::setKvCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig16setKvCacheConfigERK13KvCacheConfig", "tensorrt_llm::executor::ExecutorConfig::setKvCacheConfig::kvCacheConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig28setLogitsPostProcessorConfigERK25LogitsPostProcessorConfig", "tensorrt_llm::executor::ExecutorConfig::setLogitsPostProcessorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig28setLogitsPostProcessorConfigERK25LogitsPostProcessorConfig", "tensorrt_llm::executor::ExecutorConfig::setLogitsPostProcessorConfig::logitsPostProcessorConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setMaxBatchSizeE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setMaxBatchSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setMaxBatchSizeE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setMaxBatchSize::maxBatchSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setMaxBeamWidthE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setMaxBeamWidth"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setMaxBeamWidthE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setMaxBeamWidth::maxBeamWidth"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setMaxNumTokensE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setMaxNumTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setMaxNumTokensE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setMaxNumTokens::maxNumTokens"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setMaxQueueSizeERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::ExecutorConfig::setMaxQueueSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig15setMaxQueueSizeERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::ExecutorConfig::setMaxQueueSize::maxQueueSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setMaxSeqIdleMicrosecondsE8uint64_t", "tensorrt_llm::executor::ExecutorConfig::setMaxSeqIdleMicroseconds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig25setMaxSeqIdleMicrosecondsE8uint64_t", "tensorrt_llm::executor::ExecutorConfig::setMaxSeqIdleMicroseconds::maxSeqIdleMicroseconds"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig20setNormalizeLogProbsEb", "tensorrt_llm::executor::ExecutorConfig::setNormalizeLogProbs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig20setNormalizeLogProbsEb", "tensorrt_llm::executor::ExecutorConfig::setNormalizeLogProbs::normalizeLogProbs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig17setParallelConfigERK14ParallelConfig", "tensorrt_llm::executor::ExecutorConfig::setParallelConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig17setParallelConfigERK14ParallelConfig", "tensorrt_llm::executor::ExecutorConfig::setParallelConfig::parallelConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig18setPeftCacheConfigERK15PeftCacheConfig", "tensorrt_llm::executor::ExecutorConfig::setPeftCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig18setPeftCacheConfigERK15PeftCacheConfig", "tensorrt_llm::executor::ExecutorConfig::setPeftCacheConfig::peftCacheConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig24setPromptTableOffloadingEb", "tensorrt_llm::executor::ExecutorConfig::setPromptTableOffloading"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig24setPromptTableOffloadingEb", "tensorrt_llm::executor::ExecutorConfig::setPromptTableOffloading::promptTableOffloading"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig19setRecvPollPeriodMsERK10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setRecvPollPeriodMs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig19setRecvPollPeriodMsERK10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setRecvPollPeriodMs::recvPollPeriodMs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig28setRequestStatsMaxIterationsE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setRequestStatsMaxIterations"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig28setRequestStatsMaxIterationsE10SizeType32", "tensorrt_llm::executor::ExecutorConfig::setRequestStatsMaxIterations::requestStatsMaxIterations"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig18setSchedulerConfigERK15SchedulerConfig", "tensorrt_llm::executor::ExecutorConfig::setSchedulerConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig18setSchedulerConfigERK15SchedulerConfig", "tensorrt_llm::executor::ExecutorConfig::setSchedulerConfig::schedulerConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig16setSpecDecConfigERK25SpeculativeDecodingConfig", "tensorrt_llm::executor::ExecutorConfig::setSpecDecConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig16setSpecDecConfigERK25SpeculativeDecodingConfig", "tensorrt_llm::executor::ExecutorConfig::setSpecDecConfig::specDecConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig22setUseGpuDirectStorageERKb", "tensorrt_llm::executor::ExecutorConfig::setUseGpuDirectStorage"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ExecutorConfig22setUseGpuDirectStorageERKb", "tensorrt_llm::executor::ExecutorConfig::setUseGpuDirectStorage::useGpuDirectStorage"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfigE", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig29ExtendedRuntimePerfKnobConfigEbbb10SizeType32", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::ExtendedRuntimePerfKnobConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig29ExtendedRuntimePerfKnobConfigEbbb10SizeType32", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::ExtendedRuntimePerfKnobConfig::cudaGraphCacheSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig29ExtendedRuntimePerfKnobConfigEbbb10SizeType32", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::ExtendedRuntimePerfKnobConfig::cudaGraphMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig29ExtendedRuntimePerfKnobConfigEbbb10SizeType32", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::ExtendedRuntimePerfKnobConfig::enableContextFMHAFP32Acc"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig29ExtendedRuntimePerfKnobConfigEbbb10SizeType32", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::ExtendedRuntimePerfKnobConfig::multiBlockMode"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig21getCudaGraphCacheSizeEv", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::getCudaGraphCacheSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig16getCudaGraphModeEv", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::getCudaGraphMode"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig27getEnableContextFMHAFP32AccEv", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::getEnableContextFMHAFP32Acc"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig17getMultiBlockModeEv", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::getMultiBlockMode"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig19mCudaGraphCacheSizeE", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::mCudaGraphCacheSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig14mCudaGraphModeE", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::mCudaGraphMode"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig25mEnableContextFMHAFP32AccE", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::mEnableContextFMHAFP32Acc"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig15mMultiBlockModeE", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::mMultiBlockMode"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfigeqERK29ExtendedRuntimePerfKnobConfig", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfigeqERK29ExtendedRuntimePerfKnobConfig", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::operator==::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig21setCudaGraphCacheSizeE10SizeType32", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::setCudaGraphCacheSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig21setCudaGraphCacheSizeE10SizeType32", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::setCudaGraphCacheSize::cacheSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig16setCudaGraphModeEb", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::setCudaGraphMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig16setCudaGraphModeEb", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::setCudaGraphMode::cudaGraphMode"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig27setEnableContextFMHAFP32AccEb", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::setEnableContextFMHAFP32Acc"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig27setEnableContextFMHAFP32AccEb", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::setEnableContextFMHAFP32Acc::enableContextFMHAFP32Acc"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig17setMultiBlockModeEb", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::setMultiBlockMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor29ExtendedRuntimePerfKnobConfig17setMultiBlockModeEb", "tensorrt_llm::executor::ExtendedRuntimePerfKnobConfig::setMultiBlockMode::multiBlockMode"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfigE", "tensorrt_llm::executor::ExternalDraftTokensConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfig25ExternalDraftTokensConfigE9VecTokensNSt8optionalI6TensorEERKNSt8optionalI9FloatTypeEERKNSt8optionalIbEE", "tensorrt_llm::executor::ExternalDraftTokensConfig::ExternalDraftTokensConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfig25ExternalDraftTokensConfigE9VecTokensNSt8optionalI6TensorEERKNSt8optionalI9FloatTypeEERKNSt8optionalIbEE", "tensorrt_llm::executor::ExternalDraftTokensConfig::ExternalDraftTokensConfig::acceptanceThreshold"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfig25ExternalDraftTokensConfigE9VecTokensNSt8optionalI6TensorEERKNSt8optionalI9FloatTypeEERKNSt8optionalIbEE", "tensorrt_llm::executor::ExternalDraftTokensConfig::ExternalDraftTokensConfig::fastLogits"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfig25ExternalDraftTokensConfigE9VecTokensNSt8optionalI6TensorEERKNSt8optionalI9FloatTypeEERKNSt8optionalIbEE", "tensorrt_llm::executor::ExternalDraftTokensConfig::ExternalDraftTokensConfig::logits"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfig25ExternalDraftTokensConfigE9VecTokensNSt8optionalI6TensorEERKNSt8optionalI9FloatTypeEERKNSt8optionalIbEE", "tensorrt_llm::executor::ExternalDraftTokensConfig::ExternalDraftTokensConfig::tokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor25ExternalDraftTokensConfig22getAcceptanceThresholdEv", "tensorrt_llm::executor::ExternalDraftTokensConfig::getAcceptanceThreshold"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor25ExternalDraftTokensConfig13getFastLogitsEv", "tensorrt_llm::executor::ExternalDraftTokensConfig::getFastLogits"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor25ExternalDraftTokensConfig9getLogitsEv", "tensorrt_llm::executor::ExternalDraftTokensConfig::getLogits"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor25ExternalDraftTokensConfig9getTokensEv", "tensorrt_llm::executor::ExternalDraftTokensConfig::getTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfig20mAcceptanceThresholdE", "tensorrt_llm::executor::ExternalDraftTokensConfig::mAcceptanceThreshold"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfig11mFastLogitsE", "tensorrt_llm::executor::ExternalDraftTokensConfig::mFastLogits"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfig7mLogitsE", "tensorrt_llm::executor::ExternalDraftTokensConfig::mLogits"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor25ExternalDraftTokensConfig7mTokensE", "tensorrt_llm::executor::ExternalDraftTokensConfig::mTokens"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor12FinishReasonE", "tensorrt_llm::executor::FinishReason"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12FinishReason10kCANCELLEDE", "tensorrt_llm::executor::FinishReason::kCANCELLED"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12FinishReason7kEND_IDE", "tensorrt_llm::executor::FinishReason::kEND_ID"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12FinishReason7kLENGTHE", "tensorrt_llm::executor::FinishReason::kLENGTH"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12FinishReason13kNOT_FINISHEDE", "tensorrt_llm::executor::FinishReason::kNOT_FINISHED"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12FinishReason11kSTOP_WORDSE", "tensorrt_llm::executor::FinishReason::kSTOP_WORDS"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12FinishReason10kTIMED_OUTE", "tensorrt_llm::executor::FinishReason::kTIMED_OUT"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor9FloatTypeE", "tensorrt_llm::executor::FloatType"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfigE", "tensorrt_llm::executor::GuidedDecodingConfig"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig21GuidedDecodingBackendE", "tensorrt_llm::executor::GuidedDecodingConfig::GuidedDecodingBackend"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig21GuidedDecodingBackend11kLLGUIDANCEE", "tensorrt_llm::executor::GuidedDecodingConfig::GuidedDecodingBackend::kLLGUIDANCE"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig21GuidedDecodingBackend9kXGRAMMARE", "tensorrt_llm::executor::GuidedDecodingConfig::GuidedDecodingBackend::kXGRAMMAR"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig20GuidedDecodingConfigE21GuidedDecodingBackendNSt8optionalINSt6vectorINSt6stringEEEEENSt8optionalINSt6stringEEENSt8optionalINSt6vectorI11TokenIdTypeEEEE", "tensorrt_llm::executor::GuidedDecodingConfig::GuidedDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig20GuidedDecodingConfigE21GuidedDecodingBackendNSt8optionalINSt6vectorINSt6stringEEEEENSt8optionalINSt6stringEEENSt8optionalINSt6vectorI11TokenIdTypeEEEE", "tensorrt_llm::executor::GuidedDecodingConfig::GuidedDecodingConfig::backend"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig20GuidedDecodingConfigE21GuidedDecodingBackendNSt8optionalINSt6vectorINSt6stringEEEEENSt8optionalINSt6stringEEENSt8optionalINSt6vectorI11TokenIdTypeEEEE", "tensorrt_llm::executor::GuidedDecodingConfig::GuidedDecodingConfig::encodedVocab"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig20GuidedDecodingConfigE21GuidedDecodingBackendNSt8optionalINSt6vectorINSt6stringEEEEENSt8optionalINSt6stringEEENSt8optionalINSt6vectorI11TokenIdTypeEEEE", "tensorrt_llm::executor::GuidedDecodingConfig::GuidedDecodingConfig::stopTokenIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig20GuidedDecodingConfigE21GuidedDecodingBackendNSt8optionalINSt6vectorINSt6stringEEEEENSt8optionalINSt6stringEEENSt8optionalINSt6vectorI11TokenIdTypeEEEE", "tensorrt_llm::executor::GuidedDecodingConfig::GuidedDecodingConfig::tokenizerStr"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingConfig10getBackendEv", "tensorrt_llm::executor::GuidedDecodingConfig::getBackend"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingConfig15getEncodedVocabEv", "tensorrt_llm::executor::GuidedDecodingConfig::getEncodedVocab"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingConfig15getStopTokenIdsEv", "tensorrt_llm::executor::GuidedDecodingConfig::getStopTokenIds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingConfig15getTokenizerStrEv", "tensorrt_llm::executor::GuidedDecodingConfig::getTokenizerStr"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig8mBackendE", "tensorrt_llm::executor::GuidedDecodingConfig::mBackend"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig13mEncodedVocabE", "tensorrt_llm::executor::GuidedDecodingConfig::mEncodedVocab"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig13mStopTokenIdsE", "tensorrt_llm::executor::GuidedDecodingConfig::mStopTokenIds"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig13mTokenizerStrE", "tensorrt_llm::executor::GuidedDecodingConfig::mTokenizerStr"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingConfigeqERK20GuidedDecodingConfig", "tensorrt_llm::executor::GuidedDecodingConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingConfigeqERK20GuidedDecodingConfig", "tensorrt_llm::executor::GuidedDecodingConfig::operator==::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig10setBackendERK21GuidedDecodingBackend", "tensorrt_llm::executor::GuidedDecodingConfig::setBackend"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig10setBackendERK21GuidedDecodingBackend", "tensorrt_llm::executor::GuidedDecodingConfig::setBackend::backend"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig15setEncodedVocabERKNSt6vectorINSt6stringEEE", "tensorrt_llm::executor::GuidedDecodingConfig::setEncodedVocab"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig15setEncodedVocabERKNSt6vectorINSt6stringEEE", "tensorrt_llm::executor::GuidedDecodingConfig::setEncodedVocab::encodedVocab"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig15setStopTokenIdsERKNSt6vectorI11TokenIdTypeEE", "tensorrt_llm::executor::GuidedDecodingConfig::setStopTokenIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig15setStopTokenIdsERKNSt6vectorI11TokenIdTypeEE", "tensorrt_llm::executor::GuidedDecodingConfig::setStopTokenIds::stopTokenIds"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig15setTokenizerStrERKNSt6stringE", "tensorrt_llm::executor::GuidedDecodingConfig::setTokenizerStr"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingConfig15setTokenizerStrERKNSt6stringE", "tensorrt_llm::executor::GuidedDecodingConfig::setTokenizerStr::tokenizerStr"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingConfig8validateEv", "tensorrt_llm::executor::GuidedDecodingConfig::validate"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParamsE", "tensorrt_llm::executor::GuidedDecodingParams"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams9GuideTypeE", "tensorrt_llm::executor::GuidedDecodingParams::GuideType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams9GuideType13kEBNF_GRAMMARE", "tensorrt_llm::executor::GuidedDecodingParams::GuideType::kEBNF_GRAMMAR"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams9GuideType5kJSONE", "tensorrt_llm::executor::GuidedDecodingParams::GuideType::kJSON"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams9GuideType12kJSON_SCHEMAE", "tensorrt_llm::executor::GuidedDecodingParams::GuideType::kJSON_SCHEMA"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams9GuideType6kREGEXE", "tensorrt_llm::executor::GuidedDecodingParams::GuideType::kREGEX"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams9GuideType15kSTRUCTURAL_TAGE", "tensorrt_llm::executor::GuidedDecodingParams::GuideType::kSTRUCTURAL_TAG"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams20GuidedDecodingParamsE9GuideTypeNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::GuidedDecodingParams::GuidedDecodingParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams20GuidedDecodingParamsE9GuideTypeNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::GuidedDecodingParams::GuidedDecodingParams::guide"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams20GuidedDecodingParamsE9GuideTypeNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::GuidedDecodingParams::GuidedDecodingParams::guideType"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingParams8getGuideEv", "tensorrt_llm::executor::GuidedDecodingParams::getGuide"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingParams12getGuideTypeEv", "tensorrt_llm::executor::GuidedDecodingParams::getGuideType"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams6mGuideE", "tensorrt_llm::executor::GuidedDecodingParams::mGuide"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor20GuidedDecodingParams10mGuideTypeE", "tensorrt_llm::executor::GuidedDecodingParams::mGuideType"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingParamseqERK20GuidedDecodingParams", "tensorrt_llm::executor::GuidedDecodingParams::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor20GuidedDecodingParamseqERK20GuidedDecodingParams", "tensorrt_llm::executor::GuidedDecodingParams::operator==::other"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor6IdTypeE", "tensorrt_llm::executor::IdType"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor21InflightBatchingStatsE", "tensorrt_llm::executor::InflightBatchingStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor21InflightBatchingStats26avgNumDecodedTokensPerIterE", "tensorrt_llm::executor::InflightBatchingStats::avgNumDecodedTokensPerIter"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor21InflightBatchingStats12microBatchIdE", "tensorrt_llm::executor::InflightBatchingStats::microBatchId"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor21InflightBatchingStats18numContextRequestsE", "tensorrt_llm::executor::InflightBatchingStats::numContextRequests"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor21InflightBatchingStats12numCtxTokensE", "tensorrt_llm::executor::InflightBatchingStats::numCtxTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor21InflightBatchingStats14numGenRequestsE", "tensorrt_llm::executor::InflightBatchingStats::numGenRequests"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor21InflightBatchingStats17numPausedRequestsE", "tensorrt_llm::executor::InflightBatchingStats::numPausedRequests"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor21InflightBatchingStats20numScheduledRequestsE", "tensorrt_llm::executor::InflightBatchingStats::numScheduledRequests"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor14IterationStatsE", "tensorrt_llm::executor::IterationStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats11cpuMemUsageE", "tensorrt_llm::executor::IterationStats::cpuMemUsage"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats17crossKvCacheStatsE", "tensorrt_llm::executor::IterationStats::crossKvCacheStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats11gpuMemUsageE", "tensorrt_llm::executor::IterationStats::gpuMemUsage"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats21inflightBatchingStatsE", "tensorrt_llm::executor::IterationStats::inflightBatchingStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats4iterE", "tensorrt_llm::executor::IterationStats::iter"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats13iterLatencyMSE", "tensorrt_llm::executor::IterationStats::iterLatencyMS"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats12kvCacheStatsE", "tensorrt_llm::executor::IterationStats::kvCacheStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats19maxBatchSizeRuntimeE", "tensorrt_llm::executor::IterationStats::maxBatchSizeRuntime"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats18maxBatchSizeStaticE", "tensorrt_llm::executor::IterationStats::maxBatchSizeStatic"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats28maxBatchSizeTunerRecommendedE", "tensorrt_llm::executor::IterationStats::maxBatchSizeTunerRecommended"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats20maxNumActiveRequestsE", "tensorrt_llm::executor::IterationStats::maxNumActiveRequests"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats19maxNumTokensRuntimeE", "tensorrt_llm::executor::IterationStats::maxNumTokensRuntime"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats18maxNumTokensStaticE", "tensorrt_llm::executor::IterationStats::maxNumTokensStatic"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats28maxNumTokensTunerRecommendedE", "tensorrt_llm::executor::IterationStats::maxNumTokensTunerRecommended"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats31newActiveRequestsQueueLatencyMSE", "tensorrt_llm::executor::IterationStats::newActiveRequestsQueueLatencyMS"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats17numActiveRequestsE", "tensorrt_llm::executor::IterationStats::numActiveRequests"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats20numCompletedRequestsE", "tensorrt_llm::executor::IterationStats::numCompletedRequests"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats20numNewActiveRequestsE", "tensorrt_llm::executor::IterationStats::numNewActiveRequests"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats17numQueuedRequestsE", "tensorrt_llm::executor::IterationStats::numQueuedRequests"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats14pinnedMemUsageE", "tensorrt_llm::executor::IterationStats::pinnedMemUsage"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats17specDecodingStatsE", "tensorrt_llm::executor::IterationStats::specDecodingStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats19staticBatchingStatsE", "tensorrt_llm::executor::IterationStats::staticBatchingStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14IterationStats9timestampE", "tensorrt_llm::executor::IterationStats::timestamp"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor13IterationTypeE", "tensorrt_llm::executor::IterationType"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor17JsonSerializationE", "tensorrt_llm::executor::JsonSerialization"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor17JsonSerialization9toJsonStrERK12RequestStats", "tensorrt_llm::executor::JsonSerialization::toJsonStr"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor17JsonSerialization9toJsonStrERK14IterationStats", "tensorrt_llm::executor::JsonSerialization::toJsonStr"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor17JsonSerialization9toJsonStrERK24RequestStatsPerIteration", "tensorrt_llm::executor::JsonSerialization::toJsonStr"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor17JsonSerialization9toJsonStrERK14IterationStats", "tensorrt_llm::executor::JsonSerialization::toJsonStr::iterationStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor17JsonSerialization9toJsonStrERK12RequestStats", "tensorrt_llm::executor::JsonSerialization::toJsonStr::requestStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor17JsonSerialization9toJsonStrERK24RequestStatsPerIteration", "tensorrt_llm::executor::JsonSerialization::toJsonStr::requestStatsPerIter"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheCreatedDataE", "tensorrt_llm::executor::KVCacheCreatedData"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheCreatedData22numBlocksPerCacheLevelE", "tensorrt_llm::executor::KVCacheCreatedData::numBlocksPerCacheLevel"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEventE", "tensorrt_llm::executor::KVCacheEvent"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEvent12KVCacheEventE6IdType16KVCacheEventData10SizeType32NSt8optionalI10SizeType32EE", "tensorrt_llm::executor::KVCacheEvent::KVCacheEvent"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEvent12KVCacheEventE6IdType16KVCacheEventData10SizeType32NSt8optionalI10SizeType32EE", "tensorrt_llm::executor::KVCacheEvent::KVCacheEvent::attentionDpRank"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEvent12KVCacheEventE6IdType16KVCacheEventData10SizeType32NSt8optionalI10SizeType32EE", "tensorrt_llm::executor::KVCacheEvent::KVCacheEvent::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEvent12KVCacheEventE6IdType16KVCacheEventData10SizeType32NSt8optionalI10SizeType32EE", "tensorrt_llm::executor::KVCacheEvent::KVCacheEvent::eventId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEvent12KVCacheEventE6IdType16KVCacheEventData10SizeType32NSt8optionalI10SizeType32EE", "tensorrt_llm::executor::KVCacheEvent::KVCacheEvent::windowSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEvent15attentionDpRankE", "tensorrt_llm::executor::KVCacheEvent::attentionDpRank"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEvent4dataE", "tensorrt_llm::executor::KVCacheEvent::data"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEvent7eventIdE", "tensorrt_llm::executor::KVCacheEvent::eventId"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KVCacheEvent10windowSizeE", "tensorrt_llm::executor::KVCacheEvent::windowSize"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor16KVCacheEventDataE", "tensorrt_llm::executor::KVCacheEventData"], [0, 2, 1, "_CPPv4I0EN12tensorrt_llm8executor16KVCacheEventDiffE", "tensorrt_llm::executor::KVCacheEventDiff"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor16KVCacheEventDiffE", "tensorrt_llm::executor::KVCacheEventDiff::T"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor16KVCacheEventDiff8newValueE", "tensorrt_llm::executor::KVCacheEventDiff::newValue"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor16KVCacheEventDiff8oldValueE", "tensorrt_llm::executor::KVCacheEventDiff::oldValue"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor19KVCacheEventManagerE", "tensorrt_llm::executor::KVCacheEventManager"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor19KVCacheEventManager19KVCacheEventManagerENSt10shared_ptrIN12tensorrt_llm13batch_manager16kv_cache_manager18BaseKVCacheManagerEEE", "tensorrt_llm::executor::KVCacheEventManager::KVCacheEventManager"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor19KVCacheEventManager19KVCacheEventManagerENSt10shared_ptrIN12tensorrt_llm13batch_manager16kv_cache_manager18BaseKVCacheManagerEEE", "tensorrt_llm::executor::KVCacheEventManager::KVCacheEventManager::kvCacheManager"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor19KVCacheEventManager15getLatestEventsENSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::KVCacheEventManager::getLatestEvents"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor19KVCacheEventManager15getLatestEventsENSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::KVCacheEventManager::getLatestEvents::timeout"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor19KVCacheEventManager14kvCacheManagerE", "tensorrt_llm::executor::KVCacheEventManager::kvCacheManager"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheRemovedDataE", "tensorrt_llm::executor::KVCacheRemovedData"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheRemovedData11blockHashesE", "tensorrt_llm::executor::KVCacheRemovedData::blockHashes"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockDataE", "tensorrt_llm::executor::KVCacheStoredBlockData"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData22KVCacheStoredBlockDataE6IdTypeN12tensorrt_llm7runtime15VecUniqueTokensENSt8optionalIN12tensorrt_llm7runtime14LoraTaskIdTypeEEE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheStoredBlockData::KVCacheStoredBlockData"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData22KVCacheStoredBlockDataE6IdTypeN12tensorrt_llm7runtime15VecUniqueTokensENSt8optionalIN12tensorrt_llm7runtime14LoraTaskIdTypeEEE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheStoredBlockData::KVCacheStoredBlockData::blockHash"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData22KVCacheStoredBlockDataE6IdTypeN12tensorrt_llm7runtime15VecUniqueTokensENSt8optionalIN12tensorrt_llm7runtime14LoraTaskIdTypeEEE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheStoredBlockData::KVCacheStoredBlockData::cacheLevel"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData22KVCacheStoredBlockDataE6IdTypeN12tensorrt_llm7runtime15VecUniqueTokensENSt8optionalIN12tensorrt_llm7runtime14LoraTaskIdTypeEEE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheStoredBlockData::KVCacheStoredBlockData::loraId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData22KVCacheStoredBlockDataE6IdTypeN12tensorrt_llm7runtime15VecUniqueTokensENSt8optionalIN12tensorrt_llm7runtime14LoraTaskIdTypeEEE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheStoredBlockData::KVCacheStoredBlockData::priority"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData22KVCacheStoredBlockDataE6IdTypeN12tensorrt_llm7runtime15VecUniqueTokensENSt8optionalIN12tensorrt_llm7runtime14LoraTaskIdTypeEEE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheStoredBlockData::KVCacheStoredBlockData::tokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData9blockHashE", "tensorrt_llm::executor::KVCacheStoredBlockData::blockHash"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData10cacheLevelE", "tensorrt_llm::executor::KVCacheStoredBlockData::cacheLevel"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData6loraIdE", "tensorrt_llm::executor::KVCacheStoredBlockData::loraId"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData8priorityE", "tensorrt_llm::executor::KVCacheStoredBlockData::priority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KVCacheStoredBlockData6tokensE", "tensorrt_llm::executor::KVCacheStoredBlockData::tokens"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor17KVCacheStoredDataE", "tensorrt_llm::executor::KVCacheStoredData"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor17KVCacheStoredData6blocksE", "tensorrt_llm::executor::KVCacheStoredData::blocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor17KVCacheStoredData10parentHashE", "tensorrt_llm::executor::KVCacheStoredData::parentHash"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedDataE", "tensorrt_llm::executor::KVCacheUpdatedData"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData18KVCacheUpdatedDataE6IdType", "tensorrt_llm::executor::KVCacheUpdatedData::KVCacheUpdatedData"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData18KVCacheUpdatedDataE6IdTypeNSt8optionalI16KVCacheEventDiffI10SizeType32EEENSt8optionalI16KVCacheEventDiffI10SizeType32EEE", "tensorrt_llm::executor::KVCacheUpdatedData::KVCacheUpdatedData"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData18KVCacheUpdatedDataE6IdType", "tensorrt_llm::executor::KVCacheUpdatedData::KVCacheUpdatedData::blockHash"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData18KVCacheUpdatedDataE6IdTypeNSt8optionalI16KVCacheEventDiffI10SizeType32EEENSt8optionalI16KVCacheEventDiffI10SizeType32EEE", "tensorrt_llm::executor::KVCacheUpdatedData::KVCacheUpdatedData::blockHash"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData18KVCacheUpdatedDataE6IdTypeNSt8optionalI16KVCacheEventDiffI10SizeType32EEENSt8optionalI16KVCacheEventDiffI10SizeType32EEE", "tensorrt_llm::executor::KVCacheUpdatedData::KVCacheUpdatedData::cacheLevel"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData18KVCacheUpdatedDataE6IdTypeNSt8optionalI16KVCacheEventDiffI10SizeType32EEENSt8optionalI16KVCacheEventDiffI10SizeType32EEE", "tensorrt_llm::executor::KVCacheUpdatedData::KVCacheUpdatedData::priority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData9blockHashE", "tensorrt_llm::executor::KVCacheUpdatedData::blockHash"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData10cacheLevelE", "tensorrt_llm::executor::KVCacheUpdatedData::cacheLevel"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData17cacheLevelUpdatedE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheUpdatedData::cacheLevelUpdated"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData17cacheLevelUpdatedE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheUpdatedData::cacheLevelUpdated::newValue"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData17cacheLevelUpdatedE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheUpdatedData::cacheLevelUpdated::oldValue"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData8priorityE", "tensorrt_llm::executor::KVCacheUpdatedData::priority"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData15priorityUpdatedE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheUpdatedData::priorityUpdated"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData15priorityUpdatedE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheUpdatedData::priorityUpdated::newValue"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18KVCacheUpdatedData15priorityUpdatedE10SizeType3210SizeType32", "tensorrt_llm::executor::KVCacheUpdatedData::priorityUpdated::oldValue"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfigE", "tensorrt_llm::executor::KvCacheConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::attentionDpEventsGatherPeriodMs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::copyOnPartialReuse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::crossKvCacheFraction"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::enableBlockReuse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::enablePartialReuse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::eventBufferMaxSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::freeGpuMemoryFraction"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::hostCacheSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::maxAttentionWindowVec"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::maxGpuTotalBytes"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::maxTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::onboardBlocks"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::runtimeDefaults"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::secondaryOffloadMinPriority"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::sinkTokenLength"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig13KvCacheConfigEbRKNSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI6size_tEEbRKNSt8optionalI9FloatTypeEENSt8optionalI17RetentionPriorityEE6size_tbbb10SizeType32RKNSt8optionalIN12tensorrt_llm7runtime15RuntimeDefaultsEEERK8uint64_t", "tensorrt_llm::executor::KvCacheConfig::KvCacheConfig::useUvm"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig34fillEmptyFieldsFromRuntimeDefaultsERKN12tensorrt_llm7runtime15RuntimeDefaultsE", "tensorrt_llm::executor::KvCacheConfig::fillEmptyFieldsFromRuntimeDefaults"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig34fillEmptyFieldsFromRuntimeDefaultsERKN12tensorrt_llm7runtime15RuntimeDefaultsE", "tensorrt_llm::executor::KvCacheConfig::fillEmptyFieldsFromRuntimeDefaults::runtimeDefaults"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig34getAttentionDpEventsGatherPeriodMsEv", "tensorrt_llm::executor::KvCacheConfig::getAttentionDpEventsGatherPeriodMs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig21getCopyOnPartialReuseEv", "tensorrt_llm::executor::KvCacheConfig::getCopyOnPartialReuse"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig23getCrossKvCacheFractionEv", "tensorrt_llm::executor::KvCacheConfig::getCrossKvCacheFraction"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig19getEnableBlockReuseEv", "tensorrt_llm::executor::KvCacheConfig::getEnableBlockReuse"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig21getEnablePartialReuseEv", "tensorrt_llm::executor::KvCacheConfig::getEnablePartialReuse"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig21getEventBufferMaxSizeEv", "tensorrt_llm::executor::KvCacheConfig::getEventBufferMaxSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig24getFreeGpuMemoryFractionEv", "tensorrt_llm::executor::KvCacheConfig::getFreeGpuMemoryFraction"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig16getHostCacheSizeEv", "tensorrt_llm::executor::KvCacheConfig::getHostCacheSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig24getMaxAttentionWindowVecEv", "tensorrt_llm::executor::KvCacheConfig::getMaxAttentionWindowVec"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig19getMaxGpuTotalBytesEv", "tensorrt_llm::executor::KvCacheConfig::getMaxGpuTotalBytes"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig12getMaxTokensEv", "tensorrt_llm::executor::KvCacheConfig::getMaxTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig16getOnboardBlocksEv", "tensorrt_llm::executor::KvCacheConfig::getOnboardBlocks"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig30getSecondaryOffloadMinPriorityEv", "tensorrt_llm::executor::KvCacheConfig::getSecondaryOffloadMinPriority"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig18getSinkTokenLengthEv", "tensorrt_llm::executor::KvCacheConfig::getSinkTokenLength"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor13KvCacheConfig9getUseUvmEv", "tensorrt_llm::executor::KvCacheConfig::getUseUvm"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig22kDefaultGpuMemFractionE", "tensorrt_llm::executor::KvCacheConfig::kDefaultGpuMemFraction"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig32mAttentionDpEventsGatherPeriodMsE", "tensorrt_llm::executor::KvCacheConfig::mAttentionDpEventsGatherPeriodMs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig19mCopyOnPartialReuseE", "tensorrt_llm::executor::KvCacheConfig::mCopyOnPartialReuse"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig21mCrossKvCacheFractionE", "tensorrt_llm::executor::KvCacheConfig::mCrossKvCacheFraction"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig17mEnableBlockReuseE", "tensorrt_llm::executor::KvCacheConfig::mEnableBlockReuse"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig19mEnablePartialReuseE", "tensorrt_llm::executor::KvCacheConfig::mEnablePartialReuse"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig19mEventBufferMaxSizeE", "tensorrt_llm::executor::KvCacheConfig::mEventBufferMaxSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig22mFreeGpuMemoryFractionE", "tensorrt_llm::executor::KvCacheConfig::mFreeGpuMemoryFraction"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig14mHostCacheSizeE", "tensorrt_llm::executor::KvCacheConfig::mHostCacheSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig22mMaxAttentionWindowVecE", "tensorrt_llm::executor::KvCacheConfig::mMaxAttentionWindowVec"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig17mMaxGpuTotalBytesE", "tensorrt_llm::executor::KvCacheConfig::mMaxGpuTotalBytes"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig10mMaxTokensE", "tensorrt_llm::executor::KvCacheConfig::mMaxTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig14mOnboardBlocksE", "tensorrt_llm::executor::KvCacheConfig::mOnboardBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig28mSecondaryOffloadMinPriorityE", "tensorrt_llm::executor::KvCacheConfig::mSecondaryOffloadMinPriority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig16mSinkTokenLengthE", "tensorrt_llm::executor::KvCacheConfig::mSinkTokenLength"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig7mUseUvmE", "tensorrt_llm::executor::KvCacheConfig::mUseUvm"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig34setAttentionDpEventsGatherPeriodMsE10SizeType32", "tensorrt_llm::executor::KvCacheConfig::setAttentionDpEventsGatherPeriodMs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig34setAttentionDpEventsGatherPeriodMsE10SizeType32", "tensorrt_llm::executor::KvCacheConfig::setAttentionDpEventsGatherPeriodMs::attentionDpEventsGatherPeriodMs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig21setCopyOnPartialReuseEb", "tensorrt_llm::executor::KvCacheConfig::setCopyOnPartialReuse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig21setCopyOnPartialReuseEb", "tensorrt_llm::executor::KvCacheConfig::setCopyOnPartialReuse::copyOnPartialReuse"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig23setCrossKvCacheFractionE9FloatType", "tensorrt_llm::executor::KvCacheConfig::setCrossKvCacheFraction"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig23setCrossKvCacheFractionE9FloatType", "tensorrt_llm::executor::KvCacheConfig::setCrossKvCacheFraction::crossKvCacheFraction"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig19setEnableBlockReuseEb", "tensorrt_llm::executor::KvCacheConfig::setEnableBlockReuse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig19setEnableBlockReuseEb", "tensorrt_llm::executor::KvCacheConfig::setEnableBlockReuse::enableBlockReuse"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig21setEnablePartialReuseEb", "tensorrt_llm::executor::KvCacheConfig::setEnablePartialReuse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig21setEnablePartialReuseEb", "tensorrt_llm::executor::KvCacheConfig::setEnablePartialReuse::enablePartialReuse"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig21setEventBufferMaxSizeE6size_t", "tensorrt_llm::executor::KvCacheConfig::setEventBufferMaxSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig21setEventBufferMaxSizeE6size_t", "tensorrt_llm::executor::KvCacheConfig::setEventBufferMaxSize::eventBufferMaxSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig24setFreeGpuMemoryFractionE9FloatType", "tensorrt_llm::executor::KvCacheConfig::setFreeGpuMemoryFraction"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig24setFreeGpuMemoryFractionE9FloatType", "tensorrt_llm::executor::KvCacheConfig::setFreeGpuMemoryFraction::freeGpuMemoryFraction"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig16setHostCacheSizeE6size_t", "tensorrt_llm::executor::KvCacheConfig::setHostCacheSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig16setHostCacheSizeE6size_t", "tensorrt_llm::executor::KvCacheConfig::setHostCacheSize::hostCacheSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig24setMaxAttentionWindowVecENSt6vectorI10SizeType32EE", "tensorrt_llm::executor::KvCacheConfig::setMaxAttentionWindowVec"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig24setMaxAttentionWindowVecENSt6vectorI10SizeType32EE", "tensorrt_llm::executor::KvCacheConfig::setMaxAttentionWindowVec::maxAttentionWindowVec"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig19setMaxGpuTotalBytesE8uint64_t", "tensorrt_llm::executor::KvCacheConfig::setMaxGpuTotalBytes"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig19setMaxGpuTotalBytesE8uint64_t", "tensorrt_llm::executor::KvCacheConfig::setMaxGpuTotalBytes::maxGpuTotalBytes"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig12setMaxTokensENSt8optionalI10SizeType32EE", "tensorrt_llm::executor::KvCacheConfig::setMaxTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig12setMaxTokensENSt8optionalI10SizeType32EE", "tensorrt_llm::executor::KvCacheConfig::setMaxTokens::maxTokens"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig16setOnboardBlocksEb", "tensorrt_llm::executor::KvCacheConfig::setOnboardBlocks"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig16setOnboardBlocksEb", "tensorrt_llm::executor::KvCacheConfig::setOnboardBlocks::onboardBlocks"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig30setSecondaryOffloadMinPriorityENSt8optionalI17RetentionPriorityEE", "tensorrt_llm::executor::KvCacheConfig::setSecondaryOffloadMinPriority"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig30setSecondaryOffloadMinPriorityENSt8optionalI17RetentionPriorityEE", "tensorrt_llm::executor::KvCacheConfig::setSecondaryOffloadMinPriority::secondaryOffloadMinPriority"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig18setSinkTokenLengthE10SizeType32", "tensorrt_llm::executor::KvCacheConfig::setSinkTokenLength"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig18setSinkTokenLengthE10SizeType32", "tensorrt_llm::executor::KvCacheConfig::setSinkTokenLength::sinkTokenLength"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig9setUseUvmEb", "tensorrt_llm::executor::KvCacheConfig::setUseUvm"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13KvCacheConfig9setUseUvmEb", "tensorrt_llm::executor::KvCacheConfig::setUseUvm::useUvm"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfigE", "tensorrt_llm::executor::KvCacheRetentionConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig22KvCacheRetentionConfigERKNSt6vectorI25TokenRangeRetentionConfigEE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE19KvCacheTransferModeRKNSt6stringE", "tensorrt_llm::executor::KvCacheRetentionConfig::KvCacheRetentionConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig22KvCacheRetentionConfigEv", "tensorrt_llm::executor::KvCacheRetentionConfig::KvCacheRetentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig22KvCacheRetentionConfigERKNSt6vectorI25TokenRangeRetentionConfigEE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE19KvCacheTransferModeRKNSt6stringE", "tensorrt_llm::executor::KvCacheRetentionConfig::KvCacheRetentionConfig::decodeDurationMs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig22KvCacheRetentionConfigERKNSt6vectorI25TokenRangeRetentionConfigEE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE19KvCacheTransferModeRKNSt6stringE", "tensorrt_llm::executor::KvCacheRetentionConfig::KvCacheRetentionConfig::decodeRetentionPriority"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig22KvCacheRetentionConfigERKNSt6vectorI25TokenRangeRetentionConfigEE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE19KvCacheTransferModeRKNSt6stringE", "tensorrt_llm::executor::KvCacheRetentionConfig::KvCacheRetentionConfig::directory"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig22KvCacheRetentionConfigERKNSt6vectorI25TokenRangeRetentionConfigEE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE19KvCacheTransferModeRKNSt6stringE", "tensorrt_llm::executor::KvCacheRetentionConfig::KvCacheRetentionConfig::tokenRangeRetentionPriorities"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig22KvCacheRetentionConfigERKNSt6vectorI25TokenRangeRetentionConfigEE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE19KvCacheTransferModeRKNSt6stringE", "tensorrt_llm::executor::KvCacheRetentionConfig::KvCacheRetentionConfig::transferMode"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfigE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfig25TokenRangeRetentionConfigE10SizeType32NSt8optionalI10SizeType32EE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::TokenRangeRetentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfig25TokenRangeRetentionConfigE10SizeType32NSt8optionalI10SizeType32EE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::TokenRangeRetentionConfig::durationMs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfig25TokenRangeRetentionConfigE10SizeType32NSt8optionalI10SizeType32EE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::TokenRangeRetentionConfig::priority"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfig25TokenRangeRetentionConfigE10SizeType32NSt8optionalI10SizeType32EE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::TokenRangeRetentionConfig::tokenEnd"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfig25TokenRangeRetentionConfigE10SizeType32NSt8optionalI10SizeType32EE17RetentionPriorityNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::TokenRangeRetentionConfig::tokenStart"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfig10durationMsE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::durationMs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfigeqERK25TokenRangeRetentionConfig", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfigeqERK25TokenRangeRetentionConfig", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::operator==::other"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfig8priorityE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::priority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfig8tokenEndE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::tokenEnd"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25TokenRangeRetentionConfig10tokenStartE", "tensorrt_llm::executor::KvCacheRetentionConfig::TokenRangeRetentionConfig::tokenStart"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig19getDecodeDurationMsEv", "tensorrt_llm::executor::KvCacheRetentionConfig::getDecodeDurationMs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig26getDecodeRetentionPriorityEv", "tensorrt_llm::executor::KvCacheRetentionConfig::getDecodeRetentionPriority"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig12getDirectoryEv", "tensorrt_llm::executor::KvCacheRetentionConfig::getDirectory"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig36getPerBlockRetentionPriorityDurationE10SizeType3210SizeType32", "tensorrt_llm::executor::KvCacheRetentionConfig::getPerBlockRetentionPriorityDuration"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig36getPerBlockRetentionPriorityDurationE10SizeType3210SizeType32", "tensorrt_llm::executor::KvCacheRetentionConfig::getPerBlockRetentionPriorityDuration::blockSize"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig36getPerBlockRetentionPriorityDurationE10SizeType3210SizeType32", "tensorrt_llm::executor::KvCacheRetentionConfig::getPerBlockRetentionPriorityDuration::seqLen"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig29getTokenRangeRetentionConfigsEv", "tensorrt_llm::executor::KvCacheRetentionConfig::getTokenRangeRetentionConfigs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfig15getTransferModeEv", "tensorrt_llm::executor::KvCacheRetentionConfig::getTransferMode"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig25kDefaultRetentionPriorityE", "tensorrt_llm::executor::KvCacheRetentionConfig::kDefaultRetentionPriority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig21kMaxRetentionPriorityE", "tensorrt_llm::executor::KvCacheRetentionConfig::kMaxRetentionPriority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig21kMinRetentionPriorityE", "tensorrt_llm::executor::KvCacheRetentionConfig::kMinRetentionPriority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig17mDecodeDurationMsE", "tensorrt_llm::executor::KvCacheRetentionConfig::mDecodeDurationMs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig24mDecodeRetentionPriorityE", "tensorrt_llm::executor::KvCacheRetentionConfig::mDecodeRetentionPriority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig10mDirectoryE", "tensorrt_llm::executor::KvCacheRetentionConfig::mDirectory"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig27mTokenRangeRetentionConfigsE", "tensorrt_llm::executor::KvCacheRetentionConfig::mTokenRangeRetentionConfigs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor22KvCacheRetentionConfig13mTransferModeE", "tensorrt_llm::executor::KvCacheRetentionConfig::mTransferMode"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfigeqERK22KvCacheRetentionConfig", "tensorrt_llm::executor::KvCacheRetentionConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor22KvCacheRetentionConfigeqERK22KvCacheRetentionConfig", "tensorrt_llm::executor::KvCacheRetentionConfig::operator==::other"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStatsE", "tensorrt_llm::executor::KvCacheStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStats14allocNewBlocksE", "tensorrt_llm::executor::KvCacheStats::allocNewBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStats16allocTotalBlocksE", "tensorrt_llm::executor::KvCacheStats::allocTotalBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStats12cacheHitRateE", "tensorrt_llm::executor::KvCacheStats::cacheHitRate"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStats13freeNumBlocksE", "tensorrt_llm::executor::KvCacheStats::freeNumBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStats12maxNumBlocksE", "tensorrt_llm::executor::KvCacheStats::maxNumBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStats12missedBlocksE", "tensorrt_llm::executor::KvCacheStats::missedBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStats12reusedBlocksE", "tensorrt_llm::executor::KvCacheStats::reusedBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStats14tokensPerBlockE", "tensorrt_llm::executor::KvCacheStats::tokensPerBlock"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12KvCacheStats13usedNumBlocksE", "tensorrt_llm::executor::KvCacheStats::usedNumBlocks"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor19KvCacheTransferModeE", "tensorrt_llm::executor::KvCacheTransferMode"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor19KvCacheTransferMode4DRAME", "tensorrt_llm::executor::KvCacheTransferMode::DRAM"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor19KvCacheTransferMode3GDSE", "tensorrt_llm::executor::KvCacheTransferMode::GDS"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor19KvCacheTransferMode20POSIX_DEBUG_FALLBACKE", "tensorrt_llm::executor::KvCacheTransferMode::POSIX_DEBUG_FALLBACK"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor19LogitsPostProcessorE", "tensorrt_llm::executor::LogitsPostProcessor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor26LogitsPostProcessorBatchedE", "tensorrt_llm::executor::LogitsPostProcessorBatched"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfigE", "tensorrt_llm::executor::LogitsPostProcessorConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig25LogitsPostProcessorConfigENSt8optionalI22LogitsPostProcessorMapEENSt8optionalI26LogitsPostProcessorBatchedEEb", "tensorrt_llm::executor::LogitsPostProcessorConfig::LogitsPostProcessorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig25LogitsPostProcessorConfigENSt8optionalI22LogitsPostProcessorMapEENSt8optionalI26LogitsPostProcessorBatchedEEb", "tensorrt_llm::executor::LogitsPostProcessorConfig::LogitsPostProcessorConfig::processorBatched"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig25LogitsPostProcessorConfigENSt8optionalI22LogitsPostProcessorMapEENSt8optionalI26LogitsPostProcessorBatchedEEb", "tensorrt_llm::executor::LogitsPostProcessorConfig::LogitsPostProcessorConfig::processorMap"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig25LogitsPostProcessorConfigENSt8optionalI22LogitsPostProcessorMapEENSt8optionalI26LogitsPostProcessorBatchedEEb", "tensorrt_llm::executor::LogitsPostProcessorConfig::LogitsPostProcessorConfig::replicate"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor25LogitsPostProcessorConfig19getProcessorBatchedEv", "tensorrt_llm::executor::LogitsPostProcessorConfig::getProcessorBatched"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor25LogitsPostProcessorConfig15getProcessorMapEv", "tensorrt_llm::executor::LogitsPostProcessorConfig::getProcessorMap"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor25LogitsPostProcessorConfig12getReplicateEv", "tensorrt_llm::executor::LogitsPostProcessorConfig::getReplicate"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig17mProcessorBatchedE", "tensorrt_llm::executor::LogitsPostProcessorConfig::mProcessorBatched"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig13mProcessorMapE", "tensorrt_llm::executor::LogitsPostProcessorConfig::mProcessorMap"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig10mReplicateE", "tensorrt_llm::executor::LogitsPostProcessorConfig::mReplicate"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig19setProcessorBatchedERK26LogitsPostProcessorBatched", "tensorrt_llm::executor::LogitsPostProcessorConfig::setProcessorBatched"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig19setProcessorBatchedERK26LogitsPostProcessorBatched", "tensorrt_llm::executor::LogitsPostProcessorConfig::setProcessorBatched::processorBatched"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig15setProcessorMapERK22LogitsPostProcessorMap", "tensorrt_llm::executor::LogitsPostProcessorConfig::setProcessorMap"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig15setProcessorMapERK22LogitsPostProcessorMap", "tensorrt_llm::executor::LogitsPostProcessorConfig::setProcessorMap::processorMap"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig12setReplicateEb", "tensorrt_llm::executor::LogitsPostProcessorConfig::setReplicate"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25LogitsPostProcessorConfig12setReplicateEb", "tensorrt_llm::executor::LogitsPostProcessorConfig::setReplicate::replicate"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor22LogitsPostProcessorMapE", "tensorrt_llm::executor::LogitsPostProcessorMap"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfigE", "tensorrt_llm::executor::LookaheadDecodingConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig23LookaheadDecodingConfigE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::LookaheadDecodingConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig23LookaheadDecodingConfigEv", "tensorrt_llm::executor::LookaheadDecodingConfig::LookaheadDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig23LookaheadDecodingConfigE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::LookaheadDecodingConfig::ngramSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig23LookaheadDecodingConfigE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::LookaheadDecodingConfig::verificationSetSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig23LookaheadDecodingConfigE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::LookaheadDecodingConfig::windowSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor23LookaheadDecodingConfig28calculateSpeculativeResourceEv", "tensorrt_llm::executor::LookaheadDecodingConfig::calculateSpeculativeResource"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig33calculateSpeculativeResourceTupleE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::calculateSpeculativeResourceTuple"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig33calculateSpeculativeResourceTupleE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::calculateSpeculativeResourceTuple::ngramSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig33calculateSpeculativeResourceTupleE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::calculateSpeculativeResourceTuple::verificationSetSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig33calculateSpeculativeResourceTupleE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::calculateSpeculativeResourceTuple::windowSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor23LookaheadDecodingConfig3getEv", "tensorrt_llm::executor::LookaheadDecodingConfig::get"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor23LookaheadDecodingConfig12getNgramSizeEv", "tensorrt_llm::executor::LookaheadDecodingConfig::getNgramSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor23LookaheadDecodingConfig22getVerificationSetSizeEv", "tensorrt_llm::executor::LookaheadDecodingConfig::getVerificationSetSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor23LookaheadDecodingConfig13getWindowSizeEv", "tensorrt_llm::executor::LookaheadDecodingConfig::getWindowSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor23LookaheadDecodingConfig4isLEERK23LookaheadDecodingConfig", "tensorrt_llm::executor::LookaheadDecodingConfig::isLE"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor23LookaheadDecodingConfig4isLEERK23LookaheadDecodingConfig", "tensorrt_llm::executor::LookaheadDecodingConfig::isLE::that"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig7isLegalE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::isLegal"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig7isLegalE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::isLegal::ngramSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig7isLegalE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::isLegal::verificationSetSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig7isLegalE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::executor::LookaheadDecodingConfig::isLegal::windowSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig30kDefaultLookaheadDecodingNgramE", "tensorrt_llm::executor::LookaheadDecodingConfig::kDefaultLookaheadDecodingNgram"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig40kDefaultLookaheadDecodingVerificationSetE", "tensorrt_llm::executor::LookaheadDecodingConfig::kDefaultLookaheadDecodingVerificationSet"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig31kDefaultLookaheadDecodingWindowE", "tensorrt_llm::executor::LookaheadDecodingConfig::kDefaultLookaheadDecodingWindow"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig10mNgramSizeE", "tensorrt_llm::executor::LookaheadDecodingConfig::mNgramSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig20mVerificationSetSizeE", "tensorrt_llm::executor::LookaheadDecodingConfig::mVerificationSetSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor23LookaheadDecodingConfig11mWindowSizeE", "tensorrt_llm::executor::LookaheadDecodingConfig::mWindowSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor23LookaheadDecodingConfigeqERK23LookaheadDecodingConfig", "tensorrt_llm::executor::LookaheadDecodingConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor23LookaheadDecodingConfigeqERK23LookaheadDecodingConfig", "tensorrt_llm::executor::LookaheadDecodingConfig::operator==::other"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor10LoraConfigE", "tensorrt_llm::executor::LoraConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor10LoraConfig10LoraConfigE6IdTypeNSt8optionalI6TensorEENSt8optionalI6TensorEE", "tensorrt_llm::executor::LoraConfig::LoraConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor10LoraConfig10LoraConfigE6IdTypeNSt8optionalI6TensorEENSt8optionalI6TensorEE", "tensorrt_llm::executor::LoraConfig::LoraConfig::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor10LoraConfig10LoraConfigE6IdTypeNSt8optionalI6TensorEENSt8optionalI6TensorEE", "tensorrt_llm::executor::LoraConfig::LoraConfig::taskId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor10LoraConfig10LoraConfigE6IdTypeNSt8optionalI6TensorEENSt8optionalI6TensorEE", "tensorrt_llm::executor::LoraConfig::LoraConfig::weights"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor10LoraConfig9getConfigEv", "tensorrt_llm::executor::LoraConfig::getConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor10LoraConfig9getTaskIdEv", "tensorrt_llm::executor::LoraConfig::getTaskId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor10LoraConfig10getWeightsEv", "tensorrt_llm::executor::LoraConfig::getWeights"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10LoraConfig7mConfigE", "tensorrt_llm::executor::LoraConfig::mConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10LoraConfig7mTaskIdE", "tensorrt_llm::executor::LoraConfig::mTaskId"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10LoraConfig8mWeightsE", "tensorrt_llm::executor::LoraConfig::mWeights"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor13MedusaChoicesE", "tensorrt_llm::executor::MedusaChoices"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor10MemoryTypeE", "tensorrt_llm::executor::MemoryType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor10MemoryType4kCPUE", "tensorrt_llm::executor::MemoryType::kCPU"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor10MemoryType11kCPU_PINNEDE", "tensorrt_llm::executor::MemoryType::kCPU_PINNED"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor10MemoryType15kCPU_PINNEDPOOLE", "tensorrt_llm::executor::MemoryType::kCPU_PINNEDPOOL"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor10MemoryType4kGPUE", "tensorrt_llm::executor::MemoryType::kGPU"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor10MemoryType8kUNKNOWNE", "tensorrt_llm::executor::MemoryType::kUNKNOWN"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor10MemoryType4kUVME", "tensorrt_llm::executor::MemoryType::kUVM"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor16MillisecondsTypeE", "tensorrt_llm::executor::MillisecondsType"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor9ModelTypeE", "tensorrt_llm::executor::ModelType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor9ModelType13kDECODER_ONLYE", "tensorrt_llm::executor::ModelType::kDECODER_ONLY"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor9ModelType16kENCODER_DECODERE", "tensorrt_llm::executor::ModelType::kENCODER_DECODER"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor9ModelType13kENCODER_ONLYE", "tensorrt_llm::executor::ModelType::kENCODER_ONLY"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor11MropeConfigE", "tensorrt_llm::executor::MropeConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor11MropeConfig11MropeConfigE6Tensor10SizeType32", "tensorrt_llm::executor::MropeConfig::MropeConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11MropeConfig11MropeConfigE6Tensor10SizeType32", "tensorrt_llm::executor::MropeConfig::MropeConfig::mropePositionDeltas"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor11MropeConfig11MropeConfigE6Tensor10SizeType32", "tensorrt_llm::executor::MropeConfig::MropeConfig::mropeRoratySinCos"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11MropeConfig22getMRopePositionDeltasEv", "tensorrt_llm::executor::MropeConfig::getMRopePositionDeltas"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor11MropeConfig20getMRopeRotaryCosSinEv", "tensorrt_llm::executor::MropeConfig::getMRopeRotaryCosSin"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11MropeConfig20mMRopePositionDeltasE", "tensorrt_llm::executor::MropeConfig::mMRopePositionDeltas"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor11MropeConfig18mMRopeRotaryCosSinE", "tensorrt_llm::executor::MropeConfig::mMRopeRotaryCosSin"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor15MultimodalInputE", "tensorrt_llm::executor::MultimodalInput"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15MultimodalInput15MultimodalInputENSt6vectorINSt6vectorI10SizeType32EEEENSt6vectorI10SizeType32EENSt6vectorI10SizeType32EE", "tensorrt_llm::executor::MultimodalInput::MultimodalInput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15MultimodalInput15MultimodalInputENSt6vectorINSt6vectorI10SizeType32EEEENSt6vectorI10SizeType32EENSt6vectorI10SizeType32EE", "tensorrt_llm::executor::MultimodalInput::MultimodalInput::multimodalHashes"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15MultimodalInput15MultimodalInputENSt6vectorINSt6vectorI10SizeType32EEEENSt6vectorI10SizeType32EENSt6vectorI10SizeType32EE", "tensorrt_llm::executor::MultimodalInput::MultimodalInput::multimodalLengths"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15MultimodalInput15MultimodalInputENSt6vectorINSt6vectorI10SizeType32EEEENSt6vectorI10SizeType32EENSt6vectorI10SizeType32EE", "tensorrt_llm::executor::MultimodalInput::MultimodalInput::multimodalPositions"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15MultimodalInput19getMultimodalHashesEv", "tensorrt_llm::executor::MultimodalInput::getMultimodalHashes"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15MultimodalInput20getMultimodalLengthsEv", "tensorrt_llm::executor::MultimodalInput::getMultimodalLengths"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15MultimodalInput22getMultimodalPositionsEv", "tensorrt_llm::executor::MultimodalInput::getMultimodalPositions"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15MultimodalInput17mMultimodalHashesE", "tensorrt_llm::executor::MultimodalInput::mMultimodalHashes"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15MultimodalInput18mMultimodalLengthsE", "tensorrt_llm::executor::MultimodalInput::mMultimodalLengths"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15MultimodalInput20mMultimodalPositionsE", "tensorrt_llm::executor::MultimodalInput::mMultimodalPositions"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfigE", "tensorrt_llm::executor::OrchestratorConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig18OrchestratorConfigEbNSt6stringENSt10shared_ptrIN3mpi7MpiCommEEEb", "tensorrt_llm::executor::OrchestratorConfig::OrchestratorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig18OrchestratorConfigEbNSt6stringENSt10shared_ptrIN3mpi7MpiCommEEEb", "tensorrt_llm::executor::OrchestratorConfig::OrchestratorConfig::isOrchestrator"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig18OrchestratorConfigEbNSt6stringENSt10shared_ptrIN3mpi7MpiCommEEEb", "tensorrt_llm::executor::OrchestratorConfig::OrchestratorConfig::orchLeaderComm"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig18OrchestratorConfigEbNSt6stringENSt10shared_ptrIN3mpi7MpiCommEEEb", "tensorrt_llm::executor::OrchestratorConfig::OrchestratorConfig::spawnProcesses"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig18OrchestratorConfigEbNSt6stringENSt10shared_ptrIN3mpi7MpiCommEEEb", "tensorrt_llm::executor::OrchestratorConfig::OrchestratorConfig::workerExecutablePath"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18OrchestratorConfig17getIsOrchestratorEv", "tensorrt_llm::executor::OrchestratorConfig::getIsOrchestrator"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18OrchestratorConfig17getOrchLeaderCommEv", "tensorrt_llm::executor::OrchestratorConfig::getOrchLeaderComm"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18OrchestratorConfig17getSpawnProcessesEv", "tensorrt_llm::executor::OrchestratorConfig::getSpawnProcesses"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18OrchestratorConfig23getWorkerExecutablePathEv", "tensorrt_llm::executor::OrchestratorConfig::getWorkerExecutablePath"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig15mIsOrchestratorE", "tensorrt_llm::executor::OrchestratorConfig::mIsOrchestrator"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig15mOrchLeaderCommE", "tensorrt_llm::executor::OrchestratorConfig::mOrchLeaderComm"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig15mSpawnProcessesE", "tensorrt_llm::executor::OrchestratorConfig::mSpawnProcesses"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig21mWorkerExecutablePathE", "tensorrt_llm::executor::OrchestratorConfig::mWorkerExecutablePath"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig17setIsOrchestratorEb", "tensorrt_llm::executor::OrchestratorConfig::setIsOrchestrator"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig17setIsOrchestratorEb", "tensorrt_llm::executor::OrchestratorConfig::setIsOrchestrator::isOrchestrator"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig17setOrchLeaderCommERKNSt10shared_ptrIN3mpi7MpiCommEEE", "tensorrt_llm::executor::OrchestratorConfig::setOrchLeaderComm"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig17setOrchLeaderCommERKNSt10shared_ptrIN3mpi7MpiCommEEE", "tensorrt_llm::executor::OrchestratorConfig::setOrchLeaderComm::orchLeaderComm"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig17setSpawnProcessesEb", "tensorrt_llm::executor::OrchestratorConfig::setSpawnProcesses"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig17setSpawnProcessesEb", "tensorrt_llm::executor::OrchestratorConfig::setSpawnProcesses::spawnProcesses"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig23setWorkerExecutablePathERKNSt6stringE", "tensorrt_llm::executor::OrchestratorConfig::setWorkerExecutablePath"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18OrchestratorConfig23setWorkerExecutablePathERKNSt6stringE", "tensorrt_llm::executor::OrchestratorConfig::setWorkerExecutablePath::workerExecutablePath"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfigE", "tensorrt_llm::executor::OutputConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig12OutputConfigEbbbbbbNSt8optionalINSt6vectorI21AdditionalModelOutputEEEE", "tensorrt_llm::executor::OutputConfig::OutputConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig12OutputConfigEbbbbbbNSt8optionalINSt6vectorI21AdditionalModelOutputEEEE", "tensorrt_llm::executor::OutputConfig::OutputConfig::additionalModelOutputs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig12OutputConfigEbbbbbbNSt8optionalINSt6vectorI21AdditionalModelOutputEEEE", "tensorrt_llm::executor::OutputConfig::OutputConfig::excludeInputFromOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig12OutputConfigEbbbbbbNSt8optionalINSt6vectorI21AdditionalModelOutputEEEE", "tensorrt_llm::executor::OutputConfig::OutputConfig::returnContextLogits"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig12OutputConfigEbbbbbbNSt8optionalINSt6vectorI21AdditionalModelOutputEEEE", "tensorrt_llm::executor::OutputConfig::OutputConfig::returnEncoderOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig12OutputConfigEbbbbbbNSt8optionalINSt6vectorI21AdditionalModelOutputEEEE", "tensorrt_llm::executor::OutputConfig::OutputConfig::returnGenerationLogits"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig12OutputConfigEbbbbbbNSt8optionalINSt6vectorI21AdditionalModelOutputEEEE", "tensorrt_llm::executor::OutputConfig::OutputConfig::returnLogProbs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig12OutputConfigEbbbbbbNSt8optionalINSt6vectorI21AdditionalModelOutputEEEE", "tensorrt_llm::executor::OutputConfig::OutputConfig::returnPerfMetrics"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig22additionalModelOutputsE", "tensorrt_llm::executor::OutputConfig::additionalModelOutputs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig22excludeInputFromOutputE", "tensorrt_llm::executor::OutputConfig::excludeInputFromOutput"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig19returnContextLogitsE", "tensorrt_llm::executor::OutputConfig::returnContextLogits"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig19returnEncoderOutputE", "tensorrt_llm::executor::OutputConfig::returnEncoderOutput"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig22returnGenerationLogitsE", "tensorrt_llm::executor::OutputConfig::returnGenerationLogits"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig14returnLogProbsE", "tensorrt_llm::executor::OutputConfig::returnLogProbs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12OutputConfig17returnPerfMetricsE", "tensorrt_llm::executor::OutputConfig::returnPerfMetrics"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfigE", "tensorrt_llm::executor::ParallelConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig14ParallelConfigE17CommunicationType17CommunicationModeNSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI18OrchestratorConfigEENSt8optionalI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::ParallelConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig14ParallelConfigE17CommunicationType17CommunicationModeNSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI18OrchestratorConfigEENSt8optionalI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::ParallelConfig::commMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig14ParallelConfigE17CommunicationType17CommunicationModeNSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI18OrchestratorConfigEENSt8optionalI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::ParallelConfig::commType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig14ParallelConfigE17CommunicationType17CommunicationModeNSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI18OrchestratorConfigEENSt8optionalI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::ParallelConfig::deviceIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig14ParallelConfigE17CommunicationType17CommunicationModeNSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI18OrchestratorConfigEENSt8optionalI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::ParallelConfig::numNodes"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig14ParallelConfigE17CommunicationType17CommunicationModeNSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI18OrchestratorConfigEENSt8optionalI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::ParallelConfig::orchestratorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig14ParallelConfigE17CommunicationType17CommunicationModeNSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt6vectorI10SizeType32EEEERKNSt8optionalI18OrchestratorConfigEENSt8optionalI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::ParallelConfig::participantIds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ParallelConfig20getCommunicationModeEv", "tensorrt_llm::executor::ParallelConfig::getCommunicationMode"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ParallelConfig20getCommunicationTypeEv", "tensorrt_llm::executor::ParallelConfig::getCommunicationType"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ParallelConfig12getDeviceIdsEv", "tensorrt_llm::executor::ParallelConfig::getDeviceIds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ParallelConfig11getNumNodesEv", "tensorrt_llm::executor::ParallelConfig::getNumNodes"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ParallelConfig21getOrchestratorConfigEv", "tensorrt_llm::executor::ParallelConfig::getOrchestratorConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14ParallelConfig17getParticipantIdsEv", "tensorrt_llm::executor::ParallelConfig::getParticipantIds"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig9mCommModeE", "tensorrt_llm::executor::ParallelConfig::mCommMode"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig9mCommTypeE", "tensorrt_llm::executor::ParallelConfig::mCommType"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig10mDeviceIdsE", "tensorrt_llm::executor::ParallelConfig::mDeviceIds"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig9mNumNodesE", "tensorrt_llm::executor::ParallelConfig::mNumNodes"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig19mOrchestratorConfigE", "tensorrt_llm::executor::ParallelConfig::mOrchestratorConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig15mParticipantIdsE", "tensorrt_llm::executor::ParallelConfig::mParticipantIds"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig20setCommunicationModeE17CommunicationMode", "tensorrt_llm::executor::ParallelConfig::setCommunicationMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig20setCommunicationModeE17CommunicationMode", "tensorrt_llm::executor::ParallelConfig::setCommunicationMode::mode"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig20setCommunicationTypeE17CommunicationType", "tensorrt_llm::executor::ParallelConfig::setCommunicationType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig20setCommunicationTypeE17CommunicationType", "tensorrt_llm::executor::ParallelConfig::setCommunicationType::type"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig12setDeviceIdsERKNSt6vectorI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::setDeviceIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig12setDeviceIdsERKNSt6vectorI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::setDeviceIds::deviceIds"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig11setNumNodesE10SizeType32", "tensorrt_llm::executor::ParallelConfig::setNumNodes"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig11setNumNodesE10SizeType32", "tensorrt_llm::executor::ParallelConfig::setNumNodes::numNodes"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig21setOrchestratorConfigERK18OrchestratorConfig", "tensorrt_llm::executor::ParallelConfig::setOrchestratorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig21setOrchestratorConfigERK18OrchestratorConfig", "tensorrt_llm::executor::ParallelConfig::setOrchestratorConfig::orchestratorConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig17setParticipantIdsERKNSt6vectorI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::setParticipantIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14ParallelConfig17setParticipantIdsERKNSt6vectorI10SizeType32EE", "tensorrt_llm::executor::ParallelConfig::setParticipantIds::participantIds"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfigE", "tensorrt_llm::executor::PeftCacheConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::deviceCachePercent"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::hostCacheSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::loraPrefetchDir"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::maxAdapterSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::maxPagesPerBlockDevice"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::maxPagesPerBlockHost"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::numCopyStreams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::numDeviceModuleLayer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::numEnsureWorkers"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::numHostModuleLayer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::numPutWorkers"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15PeftCacheConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalIfEERKNSt8optionalI6size_tEERKNSt8optionalINSt6stringEEE", "tensorrt_llm::executor::PeftCacheConfig::PeftCacheConfig::optimalAdapterSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig21getDeviceCachePercentEv", "tensorrt_llm::executor::PeftCacheConfig::getDeviceCachePercent"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig16getHostCacheSizeEv", "tensorrt_llm::executor::PeftCacheConfig::getHostCacheSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig18getLoraPrefetchDirEv", "tensorrt_llm::executor::PeftCacheConfig::getLoraPrefetchDir"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig17getMaxAdapterSizeEv", "tensorrt_llm::executor::PeftCacheConfig::getMaxAdapterSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig25getMaxPagesPerBlockDeviceEv", "tensorrt_llm::executor::PeftCacheConfig::getMaxPagesPerBlockDevice"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig23getMaxPagesPerBlockHostEv", "tensorrt_llm::executor::PeftCacheConfig::getMaxPagesPerBlockHost"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig17getNumCopyStreamsEv", "tensorrt_llm::executor::PeftCacheConfig::getNumCopyStreams"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig23getNumDeviceModuleLayerEv", "tensorrt_llm::executor::PeftCacheConfig::getNumDeviceModuleLayer"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig19getNumEnsureWorkersEv", "tensorrt_llm::executor::PeftCacheConfig::getNumEnsureWorkers"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig21getNumHostModuleLayerEv", "tensorrt_llm::executor::PeftCacheConfig::getNumHostModuleLayer"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig16getNumPutWorkersEv", "tensorrt_llm::executor::PeftCacheConfig::getNumPutWorkers"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfig21getOptimalAdapterSizeEv", "tensorrt_llm::executor::PeftCacheConfig::getOptimalAdapterSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig22kDefaultMaxAdapterSizeE", "tensorrt_llm::executor::PeftCacheConfig::kDefaultMaxAdapterSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig30kDefaultMaxPagesPerBlockDeviceE", "tensorrt_llm::executor::PeftCacheConfig::kDefaultMaxPagesPerBlockDevice"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig28kDefaultMaxPagesPerBlockHostE", "tensorrt_llm::executor::PeftCacheConfig::kDefaultMaxPagesPerBlockHost"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig26kDefaultOptimalAdapterSizeE", "tensorrt_llm::executor::PeftCacheConfig::kDefaultOptimalAdapterSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig19mDeviceCachePercentE", "tensorrt_llm::executor::PeftCacheConfig::mDeviceCachePercent"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig14mHostCacheSizeE", "tensorrt_llm::executor::PeftCacheConfig::mHostCacheSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig16mLoraPrefetchDirE", "tensorrt_llm::executor::PeftCacheConfig::mLoraPrefetchDir"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15mMaxAdapterSizeE", "tensorrt_llm::executor::PeftCacheConfig::mMaxAdapterSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig23mMaxPagesPerBlockDeviceE", "tensorrt_llm::executor::PeftCacheConfig::mMaxPagesPerBlockDevice"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig21mMaxPagesPerBlockHostE", "tensorrt_llm::executor::PeftCacheConfig::mMaxPagesPerBlockHost"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig15mNumCopyStreamsE", "tensorrt_llm::executor::PeftCacheConfig::mNumCopyStreams"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig21mNumDeviceModuleLayerE", "tensorrt_llm::executor::PeftCacheConfig::mNumDeviceModuleLayer"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig17mNumEnsureWorkersE", "tensorrt_llm::executor::PeftCacheConfig::mNumEnsureWorkers"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig19mNumHostModuleLayerE", "tensorrt_llm::executor::PeftCacheConfig::mNumHostModuleLayer"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig14mNumPutWorkersE", "tensorrt_llm::executor::PeftCacheConfig::mNumPutWorkers"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15PeftCacheConfig19mOptimalAdapterSizeE", "tensorrt_llm::executor::PeftCacheConfig::mOptimalAdapterSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfigeqERK15PeftCacheConfig", "tensorrt_llm::executor::PeftCacheConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor15PeftCacheConfigeqERK15PeftCacheConfig", "tensorrt_llm::executor::PeftCacheConfig::operator==::other"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor12PriorityTypeE", "tensorrt_llm::executor::PriorityType"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18PromptTuningConfigE", "tensorrt_llm::executor::PromptTuningConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor18PromptTuningConfig18PromptTuningConfigE6TensorNSt8optionalI16VecTokenExtraIdsEE", "tensorrt_llm::executor::PromptTuningConfig::PromptTuningConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18PromptTuningConfig18PromptTuningConfigE6TensorNSt8optionalI16VecTokenExtraIdsEE", "tensorrt_llm::executor::PromptTuningConfig::PromptTuningConfig::embeddingTable"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor18PromptTuningConfig18PromptTuningConfigE6TensorNSt8optionalI16VecTokenExtraIdsEE", "tensorrt_llm::executor::PromptTuningConfig::PromptTuningConfig::inputTokenExtraIds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18PromptTuningConfig17getEmbeddingTableEv", "tensorrt_llm::executor::PromptTuningConfig::getEmbeddingTable"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor18PromptTuningConfig21getInputTokenExtraIdsEv", "tensorrt_llm::executor::PromptTuningConfig::getInputTokenExtraIds"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18PromptTuningConfig15mEmbeddingTableE", "tensorrt_llm::executor::PromptTuningConfig::mEmbeddingTable"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18PromptTuningConfig19mInputTokenExtraIdsE", "tensorrt_llm::executor::PromptTuningConfig::mInputTokenExtraIds"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor14RandomSeedTypeE", "tensorrt_llm::executor::RandomSeedType"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor7RequestE", "tensorrt_llm::executor::Request"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestERK7Request", "tensorrt_llm::executor::Request::Request"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestERR7Request", "tensorrt_llm::executor::Request::Request"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::allottedTimeMs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::badWords"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::cacheSaltID"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::clientId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::contextPhaseParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::crossAttentionMask"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::eagleConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::embeddingBias"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::encoderInputFeatures"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::encoderInputTokenIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::encoderOutputLength"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::endId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::externalDraftTokensConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::guidedDecodingParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::inputTokenIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::kvCacheRetentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::languageAdapterUid"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::logitsPostProcessor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::logitsPostProcessorName"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::lookaheadConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::loraConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::mRopeConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::maxTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::multimodalEmbedding"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::multimodalInput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::numReturnSequences"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestERK7Request", "tensorrt_llm::executor::Request::Request::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestERR7Request", "tensorrt_llm::executor::Request::Request::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::outputConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::pTuningConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::padId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::positionIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::priority"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::returnAllGeneratedTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::samplingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::skipCrossAttnBlocks"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::stopWords"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::streaming"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request7RequestE9VecTokens10SizeType32bRK14SamplingConfigRK12OutputConfigRKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalINSt4listI9VecTokensEEEENSt8optionalI6TensorEENSt8optionalI25ExternalDraftTokensConfigEENSt8optionalI18PromptTuningConfigEENSt8optionalI15MultimodalInputEENSt8optionalI6TensorEENSt8optionalI11MropeConfigEENSt8optionalI10LoraConfigEENSt8optionalI23LookaheadDecodingConfigEENSt8optionalI22KvCacheRetentionConfigEENSt8optionalINSt6stringEEENSt8optionalI19LogitsPostProcessorEENSt8optionalI9VecTokensEENSt8optionalI6IdTypeEEb12PriorityType11RequestTypeNSt8optionalI18ContextPhaseParamsEENSt8optionalI6TensorEENSt8optionalI10SizeType32EENSt8optionalI6TensorEE10SizeType32NSt8optionalI11EagleConfigEENSt8optionalI6TensorEENSt8optionalI20GuidedDecodingParamsEENSt8optionalI10SizeType32EENSt8optionalI16MillisecondsTypeEENSt8optionalI15CacheSaltIDTypeEE", "tensorrt_llm::executor::Request::Request::type"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request24getAdditionalOutputNamesEv", "tensorrt_llm::executor::Request::getAdditionalOutputNames"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request17getAllottedTimeMsEv", "tensorrt_llm::executor::Request::getAllottedTimeMs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request11getBadWordsEv", "tensorrt_llm::executor::Request::getBadWords"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request14getCacheSaltIDEv", "tensorrt_llm::executor::Request::getCacheSaltID"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request11getClientIdEv", "tensorrt_llm::executor::Request::getClientId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request21getContextPhaseParamsEv", "tensorrt_llm::executor::Request::getContextPhaseParams"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request21getCrossAttentionMaskEv", "tensorrt_llm::executor::Request::getCrossAttentionMask"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request14getEagleConfigEv", "tensorrt_llm::executor::Request::getEagleConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request16getEmbeddingBiasEv", "tensorrt_llm::executor::Request::getEmbeddingBias"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request23getEncoderInputFeaturesEv", "tensorrt_llm::executor::Request::getEncoderInputFeatures"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request23getEncoderInputTokenIdsEv", "tensorrt_llm::executor::Request::getEncoderInputTokenIds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request22getEncoderOutputLengthEv", "tensorrt_llm::executor::Request::getEncoderOutputLength"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request8getEndIdEv", "tensorrt_llm::executor::Request::getEndId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request28getExternalDraftTokensConfigEv", "tensorrt_llm::executor::Request::getExternalDraftTokensConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request23getGuidedDecodingParamsEv", "tensorrt_llm::executor::Request::getGuidedDecodingParams"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request16getInputTokenIdsEv", "tensorrt_llm::executor::Request::getInputTokenIds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request25getKvCacheRetentionConfigEv", "tensorrt_llm::executor::Request::getKvCacheRetentionConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request21getLanguageAdapterUidEv", "tensorrt_llm::executor::Request::getLanguageAdapterUid"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request22getLogitsPostProcessorEv", "tensorrt_llm::executor::Request::getLogitsPostProcessor"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request26getLogitsPostProcessorNameEv", "tensorrt_llm::executor::Request::getLogitsPostProcessorName"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request18getLookaheadConfigEv", "tensorrt_llm::executor::Request::getLookaheadConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request13getLoraConfigEv", "tensorrt_llm::executor::Request::getLoraConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request12getMaxTokensEv", "tensorrt_llm::executor::Request::getMaxTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request14getMropeConfigEv", "tensorrt_llm::executor::Request::getMropeConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request22getMultimodalEmbeddingEv", "tensorrt_llm::executor::Request::getMultimodalEmbedding"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request18getMultimodalInputEv", "tensorrt_llm::executor::Request::getMultimodalInput"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request15getOutputConfigEv", "tensorrt_llm::executor::Request::getOutputConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request8getPadIdEv", "tensorrt_llm::executor::Request::getPadId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request14getPositionIdsEv", "tensorrt_llm::executor::Request::getPositionIds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request11getPriorityEv", "tensorrt_llm::executor::Request::getPriority"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request21getPromptTuningConfigEv", "tensorrt_llm::executor::Request::getPromptTuningConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request14getRequestTypeEv", "tensorrt_llm::executor::Request::getRequestType"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request27getReturnAllGeneratedTokensEv", "tensorrt_llm::executor::Request::getReturnAllGeneratedTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request17getSamplingConfigEv", "tensorrt_llm::executor::Request::getSamplingConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request22getSkipCrossAttnBlocksEv", "tensorrt_llm::executor::Request::getSkipCrossAttnBlocks"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request12getStopWordsEv", "tensorrt_llm::executor::Request::getStopWords"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor7Request12getStreamingEv", "tensorrt_llm::executor::Request::getStreaming"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor7Request25kBatchedPostProcessorNameE", "tensorrt_llm::executor::Request::kBatchedPostProcessorName"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor7Request16kDefaultPriorityE", "tensorrt_llm::executor::Request::kDefaultPriority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor7Request31kDynamicPostProcessorNamePrefixE", "tensorrt_llm::executor::Request::kDynamicPostProcessorNamePrefix"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor7Request5mImplE", "tensorrt_llm::executor::Request::mImpl"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7RequestaSERK7Request", "tensorrt_llm::executor::Request::operator="], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7RequestaSERR7Request", "tensorrt_llm::executor::Request::operator="], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7RequestaSERK7Request", "tensorrt_llm::executor::Request::operator=::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7RequestaSERR7Request", "tensorrt_llm::executor::Request::operator=::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request17setAllottedTimeMsE16MillisecondsType", "tensorrt_llm::executor::Request::setAllottedTimeMs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request17setAllottedTimeMsE16MillisecondsType", "tensorrt_llm::executor::Request::setAllottedTimeMs::allottedTimeMs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request11setBadWordsERKNSt4listI9VecTokensEE", "tensorrt_llm::executor::Request::setBadWords"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request11setBadWordsERKNSt4listI9VecTokensEE", "tensorrt_llm::executor::Request::setBadWords::badWords"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request14setCacheSaltIDE15CacheSaltIDType", "tensorrt_llm::executor::Request::setCacheSaltID"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request14setCacheSaltIDE15CacheSaltIDType", "tensorrt_llm::executor::Request::setCacheSaltID::cacheSaltID"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request11setClientIdE6IdType", "tensorrt_llm::executor::Request::setClientId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request11setClientIdE6IdType", "tensorrt_llm::executor::Request::setClientId::clientId"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request21setContextPhaseParamsE18ContextPhaseParams", "tensorrt_llm::executor::Request::setContextPhaseParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request21setContextPhaseParamsE18ContextPhaseParams", "tensorrt_llm::executor::Request::setContextPhaseParams::contextPhaseParams"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request21setCrossAttentionMaskE6Tensor", "tensorrt_llm::executor::Request::setCrossAttentionMask"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request21setCrossAttentionMaskE6Tensor", "tensorrt_llm::executor::Request::setCrossAttentionMask::crossAttentionMask"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request14setEagleConfigERKNSt8optionalI11EagleConfigEE", "tensorrt_llm::executor::Request::setEagleConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request14setEagleConfigERKNSt8optionalI11EagleConfigEE", "tensorrt_llm::executor::Request::setEagleConfig::eagleConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request16setEmbeddingBiasERK6Tensor", "tensorrt_llm::executor::Request::setEmbeddingBias"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request16setEmbeddingBiasERK6Tensor", "tensorrt_llm::executor::Request::setEmbeddingBias::embeddingBias"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request23setEncoderInputFeaturesE6Tensor", "tensorrt_llm::executor::Request::setEncoderInputFeatures"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request23setEncoderInputFeaturesE6Tensor", "tensorrt_llm::executor::Request::setEncoderInputFeatures::encoderInputFeatures"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request23setEncoderInputTokenIdsERK9VecTokens", "tensorrt_llm::executor::Request::setEncoderInputTokenIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request23setEncoderInputTokenIdsERK9VecTokens", "tensorrt_llm::executor::Request::setEncoderInputTokenIds::encoderInputTokenIds"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request22setEncoderOutputLengthE10SizeType32", "tensorrt_llm::executor::Request::setEncoderOutputLength"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request22setEncoderOutputLengthE10SizeType32", "tensorrt_llm::executor::Request::setEncoderOutputLength::encoderOutputLength"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request8setEndIdE10SizeType32", "tensorrt_llm::executor::Request::setEndId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request8setEndIdE10SizeType32", "tensorrt_llm::executor::Request::setEndId::endId"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request28setExternalDraftTokensConfigERK25ExternalDraftTokensConfig", "tensorrt_llm::executor::Request::setExternalDraftTokensConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request28setExternalDraftTokensConfigERK25ExternalDraftTokensConfig", "tensorrt_llm::executor::Request::setExternalDraftTokensConfig::externalDraftTokensConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request23setGuidedDecodingParamsERK20GuidedDecodingParams", "tensorrt_llm::executor::Request::setGuidedDecodingParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request23setGuidedDecodingParamsERK20GuidedDecodingParams", "tensorrt_llm::executor::Request::setGuidedDecodingParams::guidedDecodingParams"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request25setKvCacheRetentionConfigERK22KvCacheRetentionConfig", "tensorrt_llm::executor::Request::setKvCacheRetentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request25setKvCacheRetentionConfigERK22KvCacheRetentionConfig", "tensorrt_llm::executor::Request::setKvCacheRetentionConfig::kvCacheRetentionConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request21setLanguageAdapterUidE10SizeType32", "tensorrt_llm::executor::Request::setLanguageAdapterUid"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request21setLanguageAdapterUidE10SizeType32", "tensorrt_llm::executor::Request::setLanguageAdapterUid::languageAdapterUid"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request22setLogitsPostProcessorERKNSt8optionalI19LogitsPostProcessorEE", "tensorrt_llm::executor::Request::setLogitsPostProcessor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request22setLogitsPostProcessorERKNSt8optionalI19LogitsPostProcessorEE", "tensorrt_llm::executor::Request::setLogitsPostProcessor::logitsPostProcessor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request26setLogitsPostProcessorNameERKNSt6stringE", "tensorrt_llm::executor::Request::setLogitsPostProcessorName"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request26setLogitsPostProcessorNameERKNSt6stringE", "tensorrt_llm::executor::Request::setLogitsPostProcessorName::logitsPostProcessorName"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request18setLookaheadConfigERK23LookaheadDecodingConfig", "tensorrt_llm::executor::Request::setLookaheadConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request18setLookaheadConfigERK23LookaheadDecodingConfig", "tensorrt_llm::executor::Request::setLookaheadConfig::lookaheadConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request13setLoraConfigERK10LoraConfig", "tensorrt_llm::executor::Request::setLoraConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request13setLoraConfigERK10LoraConfig", "tensorrt_llm::executor::Request::setLoraConfig::loraConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request14setMropeConfigERK11MropeConfig", "tensorrt_llm::executor::Request::setMropeConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request14setMropeConfigERK11MropeConfig", "tensorrt_llm::executor::Request::setMropeConfig::mRopeConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request22setMultimodalEmbeddingERK6Tensor", "tensorrt_llm::executor::Request::setMultimodalEmbedding"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request22setMultimodalEmbeddingERK6Tensor", "tensorrt_llm::executor::Request::setMultimodalEmbedding::multimodalEmbedding"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request18setMultimodalInputERK15MultimodalInput", "tensorrt_llm::executor::Request::setMultimodalInput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request18setMultimodalInputERK15MultimodalInput", "tensorrt_llm::executor::Request::setMultimodalInput::multimodalInput"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request15setOutputConfigERK12OutputConfig", "tensorrt_llm::executor::Request::setOutputConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request15setOutputConfigERK12OutputConfig", "tensorrt_llm::executor::Request::setOutputConfig::outputConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request8setPadIdE10SizeType32", "tensorrt_llm::executor::Request::setPadId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request8setPadIdE10SizeType32", "tensorrt_llm::executor::Request::setPadId::padId"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request14setPositionIdsERKNSt6vectorI10SizeType32EE", "tensorrt_llm::executor::Request::setPositionIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request14setPositionIdsERKNSt6vectorI10SizeType32EE", "tensorrt_llm::executor::Request::setPositionIds::positionIds"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request11setPriorityE12PriorityType", "tensorrt_llm::executor::Request::setPriority"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request11setPriorityE12PriorityType", "tensorrt_llm::executor::Request::setPriority::priority"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request21setPromptTuningConfigERK18PromptTuningConfig", "tensorrt_llm::executor::Request::setPromptTuningConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request21setPromptTuningConfigERK18PromptTuningConfig", "tensorrt_llm::executor::Request::setPromptTuningConfig::pTuningConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request14setRequestTypeERK11RequestType", "tensorrt_llm::executor::Request::setRequestType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request14setRequestTypeERK11RequestType", "tensorrt_llm::executor::Request::setRequestType::requestType"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request27setReturnAllGeneratedTokensEb", "tensorrt_llm::executor::Request::setReturnAllGeneratedTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request27setReturnAllGeneratedTokensEb", "tensorrt_llm::executor::Request::setReturnAllGeneratedTokens::returnAllGeneratedTokens"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request17setSamplingConfigERK14SamplingConfig", "tensorrt_llm::executor::Request::setSamplingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request17setSamplingConfigERK14SamplingConfig", "tensorrt_llm::executor::Request::setSamplingConfig::config"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request22setSkipCrossAttnBlocksE6Tensor", "tensorrt_llm::executor::Request::setSkipCrossAttnBlocks"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request22setSkipCrossAttnBlocksE6Tensor", "tensorrt_llm::executor::Request::setSkipCrossAttnBlocks::skipCrossAttnBlocks"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request12setStopWordsERKNSt4listI9VecTokensEE", "tensorrt_llm::executor::Request::setStopWords"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request12setStopWordsERKNSt4listI9VecTokensEE", "tensorrt_llm::executor::Request::setStopWords::stopWords"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7Request12setStreamingEb", "tensorrt_llm::executor::Request::setStreaming"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor7Request12setStreamingEb", "tensorrt_llm::executor::Request::setStreaming::streaming"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7RequestD0Ev", "tensorrt_llm::executor::Request::~Request"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetricsE", "tensorrt_llm::executor::RequestPerfMetrics"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics14KvCacheMetricsE", "tensorrt_llm::executor::RequestPerfMetrics::KvCacheMetrics"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics14KvCacheMetrics14kvCacheHitRateE", "tensorrt_llm::executor::RequestPerfMetrics::KvCacheMetrics::kvCacheHitRate"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics14KvCacheMetrics15numMissedBlocksE", "tensorrt_llm::executor::RequestPerfMetrics::KvCacheMetrics::numMissedBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics14KvCacheMetrics21numNewAllocatedBlocksE", "tensorrt_llm::executor::RequestPerfMetrics::KvCacheMetrics::numNewAllocatedBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics14KvCacheMetrics15numReusedBlocksE", "tensorrt_llm::executor::RequestPerfMetrics::KvCacheMetrics::numReusedBlocks"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics14KvCacheMetrics23numTotalAllocatedBlocksE", "tensorrt_llm::executor::RequestPerfMetrics::KvCacheMetrics::numTotalAllocatedBlocks"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics26SpeculativeDecodingMetricsE", "tensorrt_llm::executor::RequestPerfMetrics::SpeculativeDecodingMetrics"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics26SpeculativeDecodingMetrics14acceptanceRateE", "tensorrt_llm::executor::RequestPerfMetrics::SpeculativeDecodingMetrics::acceptanceRate"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics26SpeculativeDecodingMetrics24totalAcceptedDraftTokensE", "tensorrt_llm::executor::RequestPerfMetrics::SpeculativeDecodingMetrics::totalAcceptedDraftTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics26SpeculativeDecodingMetrics16totalDraftTokensE", "tensorrt_llm::executor::RequestPerfMetrics::SpeculativeDecodingMetrics::totalDraftTokens"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics9TimePointE", "tensorrt_llm::executor::RequestPerfMetrics::TimePoint"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics13TimingMetricsE", "tensorrt_llm::executor::RequestPerfMetrics::TimingMetrics"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics13TimingMetrics11arrivalTimeE", "tensorrt_llm::executor::RequestPerfMetrics::TimingMetrics::arrivalTime"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics13TimingMetrics18firstScheduledTimeE", "tensorrt_llm::executor::RequestPerfMetrics::TimingMetrics::firstScheduledTime"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics13TimingMetrics14firstTokenTimeE", "tensorrt_llm::executor::RequestPerfMetrics::TimingMetrics::firstTokenTime"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics13TimingMetrics11kvCacheSizeE", "tensorrt_llm::executor::RequestPerfMetrics::TimingMetrics::kvCacheSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics13TimingMetrics18kvCacheTransferEndE", "tensorrt_llm::executor::RequestPerfMetrics::TimingMetrics::kvCacheTransferEnd"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics13TimingMetrics20kvCacheTransferStartE", "tensorrt_llm::executor::RequestPerfMetrics::TimingMetrics::kvCacheTransferStart"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics13TimingMetrics13lastTokenTimeE", "tensorrt_llm::executor::RequestPerfMetrics::TimingMetrics::lastTokenTime"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics9firstIterE", "tensorrt_llm::executor::RequestPerfMetrics::firstIter"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics4iterE", "tensorrt_llm::executor::RequestPerfMetrics::iter"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics14kvCacheMetricsE", "tensorrt_llm::executor::RequestPerfMetrics::kvCacheMetrics"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics8lastIterE", "tensorrt_llm::executor::RequestPerfMetrics::lastIter"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics19speculativeDecodingE", "tensorrt_llm::executor::RequestPerfMetrics::speculativeDecoding"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor18RequestPerfMetrics13timingMetricsE", "tensorrt_llm::executor::RequestPerfMetrics::timingMetrics"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor12RequestStageE", "tensorrt_llm::executor::RequestStage"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12RequestStage20kCONTEXT_IN_PROGRESSE", "tensorrt_llm::executor::RequestStage::kCONTEXT_IN_PROGRESS"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12RequestStage20kENCODER_IN_PROGRESSE", "tensorrt_llm::executor::RequestStage::kENCODER_IN_PROGRESS"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12RequestStage20kGENERATION_COMPLETEE", "tensorrt_llm::executor::RequestStage::kGENERATION_COMPLETE"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12RequestStage23kGENERATION_IN_PROGRESSE", "tensorrt_llm::executor::RequestStage::kGENERATION_IN_PROGRESS"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor12RequestStage7kQUEUEDE", "tensorrt_llm::executor::RequestStage::kQUEUED"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor12RequestStatsE", "tensorrt_llm::executor::RequestStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats24allocNewBlocksPerRequestE", "tensorrt_llm::executor::RequestStats::allocNewBlocksPerRequest"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats26allocTotalBlocksPerRequestE", "tensorrt_llm::executor::RequestStats::allocTotalBlocksPerRequest"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats26avgNumDecodedTokensPerIterE", "tensorrt_llm::executor::RequestStats::avgNumDecodedTokensPerIter"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats22contextPrefillPositionE", "tensorrt_llm::executor::RequestStats::contextPrefillPosition"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats15disServingStatsE", "tensorrt_llm::executor::RequestStats::disServingStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats2idE", "tensorrt_llm::executor::RequestStats::id"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats24kvCacheHitRatePerRequestE", "tensorrt_llm::executor::RequestStats::kvCacheHitRatePerRequest"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats22missedBlocksPerRequestE", "tensorrt_llm::executor::RequestStats::missedBlocksPerRequest"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats18numGeneratedTokensE", "tensorrt_llm::executor::RequestStats::numGeneratedTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats6pausedE", "tensorrt_llm::executor::RequestStats::paused"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats22reusedBlocksPerRequestE", "tensorrt_llm::executor::RequestStats::reusedBlocksPerRequest"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats9scheduledE", "tensorrt_llm::executor::RequestStats::scheduled"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor12RequestStats5stageE", "tensorrt_llm::executor::RequestStats::stage"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor24RequestStatsPerIterationE", "tensorrt_llm::executor::RequestStatsPerIteration"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor24RequestStatsPerIteration4iterE", "tensorrt_llm::executor::RequestStatsPerIteration::iter"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor24RequestStatsPerIteration12requestStatsE", "tensorrt_llm::executor::RequestStatsPerIteration::requestStats"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor11RequestTypeE", "tensorrt_llm::executor::RequestType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor11RequestType35REQUEST_TYPE_CONTEXT_AND_GENERATIONE", "tensorrt_llm::executor::RequestType::REQUEST_TYPE_CONTEXT_AND_GENERATION"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor11RequestType25REQUEST_TYPE_CONTEXT_ONLYE", "tensorrt_llm::executor::RequestType::REQUEST_TYPE_CONTEXT_ONLY"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor11RequestType28REQUEST_TYPE_GENERATION_ONLYE", "tensorrt_llm::executor::RequestType::REQUEST_TYPE_GENERATION_ONLY"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8ResponseE", "tensorrt_llm::executor::Response"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseE6IdType6ResultNSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Response::Response"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseE6IdTypeNSt6stringENSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Response::Response"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseERK8Response", "tensorrt_llm::executor::Response::Response"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseERR8Response", "tensorrt_llm::executor::Response::Response"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseE6IdType6ResultNSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Response::Response::Result"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseE6IdType6ResultNSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Response::Response::clientId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseE6IdTypeNSt6stringENSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Response::Response::clientId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseE6IdTypeNSt6stringENSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Response::Response::errorMsg"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseERK8Response", "tensorrt_llm::executor::Response::Response::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseERR8Response", "tensorrt_llm::executor::Response::Response::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseE6IdType6ResultNSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Response::Response::requestId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8Response8ResponseE6IdTypeNSt6stringENSt8optionalI6IdTypeEE", "tensorrt_llm::executor::Response::Response::requestId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8Response11getClientIdEv", "tensorrt_llm::executor::Response::getClientId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8Response11getErrorMsgEv", "tensorrt_llm::executor::Response::getErrorMsg"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8Response12getRequestIdEv", "tensorrt_llm::executor::Response::getRequestId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8Response9getResultEv", "tensorrt_llm::executor::Response::getResult"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8Response8hasErrorEv", "tensorrt_llm::executor::Response::hasError"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8Response5mImplE", "tensorrt_llm::executor::Response::mImpl"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8ResponseaSERK8Response", "tensorrt_llm::executor::Response::operator="], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8ResponseaSERR8Response", "tensorrt_llm::executor::Response::operator="], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8ResponseaSERK8Response", "tensorrt_llm::executor::Response::operator=::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8ResponseaSERR8Response", "tensorrt_llm::executor::Response::operator=::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8ResponseD0Ev", "tensorrt_llm::executor::Response::~Response"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor6ResultE", "tensorrt_llm::executor::Result"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result17additionalOutputsE", "tensorrt_llm::executor::Result::additionalOutputs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result23avgDecodedTokensPerIterE", "tensorrt_llm::executor::Result::avgDecodedTokensPerIter"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result13contextLogitsE", "tensorrt_llm::executor::Result::contextLogits"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result18contextPhaseParamsE", "tensorrt_llm::executor::Result::contextPhaseParams"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result11cumLogProbsE", "tensorrt_llm::executor::Result::cumLogProbs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result12decodingIterE", "tensorrt_llm::executor::Result::decodingIter"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result13encoderOutputE", "tensorrt_llm::executor::Result::encoderOutput"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result13finishReasonsE", "tensorrt_llm::executor::Result::finishReasons"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result16generationLogitsE", "tensorrt_llm::executor::Result::generationLogits"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result7isFinalE", "tensorrt_llm::executor::Result::isFinal"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result15isSequenceFinalE", "tensorrt_llm::executor::Result::isSequenceFinal"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result8logProbsE", "tensorrt_llm::executor::Result::logProbs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result14outputTokenIdsE", "tensorrt_llm::executor::Result::outputTokenIds"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result18requestPerfMetricsE", "tensorrt_llm::executor::Result::requestPerfMetrics"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result13sequenceIndexE", "tensorrt_llm::executor::Result::sequenceIndex"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Result21specDecFastLogitsInfoE", "tensorrt_llm::executor::Result::specDecFastLogitsInfo"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor17RetentionPriorityE", "tensorrt_llm::executor::RetentionPriority"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor28RetentionPriorityAndDurationE", "tensorrt_llm::executor::RetentionPriorityAndDuration"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor28RetentionPriorityAndDuration28RetentionPriorityAndDurationERKNSt8optionalI17RetentionPriorityEERKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::RetentionPriorityAndDuration::RetentionPriorityAndDuration"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor28RetentionPriorityAndDuration28RetentionPriorityAndDurationERKNSt8optionalI17RetentionPriorityEERKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::RetentionPriorityAndDuration::RetentionPriorityAndDuration::durationMs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor28RetentionPriorityAndDuration28RetentionPriorityAndDurationERKNSt8optionalI17RetentionPriorityEERKNSt8optionalINSt6chrono12millisecondsEEE", "tensorrt_llm::executor::RetentionPriorityAndDuration::RetentionPriorityAndDuration::retentionPriority"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor28RetentionPriorityAndDuration10durationMsE", "tensorrt_llm::executor::RetentionPriorityAndDuration::durationMs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor28RetentionPriorityAndDuration17retentionPriorityE", "tensorrt_llm::executor::RetentionPriorityAndDuration::retentionPriority"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfigE", "tensorrt_llm::executor::SamplingConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::beamSearchDiversityRate"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::beamWidth"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::beamWidthArray"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::earlyStopping"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::frequencyPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::lengthPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::minP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::minTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::noRepeatNgramSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::numReturnSequences"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::presencePenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::repetitionPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::seed"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::temperature"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::topK"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::topP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::topPDecay"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::topPMin"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14SamplingConfigE10SizeType32RKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI11TokenIdTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI14RandomSeedTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI9FloatTypeEERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI10SizeType32EERKNSt8optionalI9FloatTypeEERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::SamplingConfig::topPResetIds"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig28checkBeamSearchDiversityRateERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkBeamSearchDiversityRate"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig28checkBeamSearchDiversityRateERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkBeamSearchDiversityRate::beamSearchDiversityRate"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14checkBeamWidthE10SizeType32", "tensorrt_llm::executor::SamplingConfig::checkBeamWidth"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14checkBeamWidthE10SizeType32", "tensorrt_llm::executor::SamplingConfig::checkBeamWidth::beamWidth"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig19checkBeamWidthArrayERKNSt8optionalINSt6vectorI10SizeType32EEEEK10SizeType32", "tensorrt_llm::executor::SamplingConfig::checkBeamWidthArray"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig19checkBeamWidthArrayERKNSt8optionalINSt6vectorI10SizeType32EEEEK10SizeType32", "tensorrt_llm::executor::SamplingConfig::checkBeamWidthArray::beamWidth"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig19checkBeamWidthArrayERKNSt8optionalINSt6vectorI10SizeType32EEEEK10SizeType32", "tensorrt_llm::executor::SamplingConfig::checkBeamWidthArray::beamWidthArray"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig18checkEarlyStoppingERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::checkEarlyStopping"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig18checkEarlyStoppingERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::checkEarlyStopping::earlyStopping"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig18checkLengthPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkLengthPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig18checkLengthPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkLengthPenalty::lengthPenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig9checkMinPERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkMinP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig9checkMinPERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkMinP::minP"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14checkMinTokensERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::checkMinTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14checkMinTokensERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::checkMinTokens::minTokens"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig22checkNoRepeatNgramSizeERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::checkNoRepeatNgramSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig22checkNoRepeatNgramSizeERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::checkNoRepeatNgramSize::noRepeatNgramSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig23checkNumReturnSequencesERKNSt8optionalI10SizeType32EE10SizeType32", "tensorrt_llm::executor::SamplingConfig::checkNumReturnSequences"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig23checkNumReturnSequencesERKNSt8optionalI10SizeType32EE10SizeType32", "tensorrt_llm::executor::SamplingConfig::checkNumReturnSequences::beamWidth"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig23checkNumReturnSequencesERKNSt8optionalI10SizeType32EE10SizeType32", "tensorrt_llm::executor::SamplingConfig::checkNumReturnSequences::numReturnSequences"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig22checkRepetitionPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkRepetitionPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig22checkRepetitionPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkRepetitionPenalty::repetitionpenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig16checkTemperatureERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTemperature"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig16checkTemperatureERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTemperature::temperature"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig9checkTopKERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopK"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig9checkTopKERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopK::topK"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig9checkTopPERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig9checkTopPERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopP::topP"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14checkTopPDecayERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopPDecay"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14checkTopPDecayERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopPDecay::topPDecay"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig12checkTopPMinERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopPMin"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig12checkTopPMinERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopPMin::topPMin"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig17checkTopPResetIdsERKNSt8optionalI11TokenIdTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopPResetIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig17checkTopPResetIdsERKNSt8optionalI11TokenIdTypeEE", "tensorrt_llm::executor::SamplingConfig::checkTopPResetIds::topPResetIds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig26getBeamSearchDiversityRateEv", "tensorrt_llm::executor::SamplingConfig::getBeamSearchDiversityRate"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig12getBeamWidthEv", "tensorrt_llm::executor::SamplingConfig::getBeamWidth"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig17getBeamWidthArrayEv", "tensorrt_llm::executor::SamplingConfig::getBeamWidthArray"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig16getEarlyStoppingEv", "tensorrt_llm::executor::SamplingConfig::getEarlyStopping"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig19getFrequencyPenaltyEv", "tensorrt_llm::executor::SamplingConfig::getFrequencyPenalty"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig16getLengthPenaltyEv", "tensorrt_llm::executor::SamplingConfig::getLengthPenalty"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig7getMinPEv", "tensorrt_llm::executor::SamplingConfig::getMinP"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig12getMinTokensEv", "tensorrt_llm::executor::SamplingConfig::getMinTokens"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig20getNoRepeatNgramSizeEv", "tensorrt_llm::executor::SamplingConfig::getNoRepeatNgramSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig17getNumReturnBeamsEv", "tensorrt_llm::executor::SamplingConfig::getNumReturnBeams"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig21getNumReturnSequencesEv", "tensorrt_llm::executor::SamplingConfig::getNumReturnSequences"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig18getPresencePenaltyEv", "tensorrt_llm::executor::SamplingConfig::getPresencePenalty"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig20getRepetitionPenaltyEv", "tensorrt_llm::executor::SamplingConfig::getRepetitionPenalty"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig7getSeedEv", "tensorrt_llm::executor::SamplingConfig::getSeed"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig14getTemperatureEv", "tensorrt_llm::executor::SamplingConfig::getTemperature"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig7getTopKEv", "tensorrt_llm::executor::SamplingConfig::getTopK"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig7getTopPEv", "tensorrt_llm::executor::SamplingConfig::getTopP"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig12getTopPDecayEv", "tensorrt_llm::executor::SamplingConfig::getTopPDecay"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig10getTopPMinEv", "tensorrt_llm::executor::SamplingConfig::getTopPMin"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfig15getTopPResetIdsEv", "tensorrt_llm::executor::SamplingConfig::getTopPResetIds"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig24mBeamSearchDiversityRateE", "tensorrt_llm::executor::SamplingConfig::mBeamSearchDiversityRate"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig10mBeamWidthE", "tensorrt_llm::executor::SamplingConfig::mBeamWidth"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig15mBeamWidthArrayE", "tensorrt_llm::executor::SamplingConfig::mBeamWidthArray"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14mEarlyStoppingE", "tensorrt_llm::executor::SamplingConfig::mEarlyStopping"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig17mFrequencyPenaltyE", "tensorrt_llm::executor::SamplingConfig::mFrequencyPenalty"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14mLengthPenaltyE", "tensorrt_llm::executor::SamplingConfig::mLengthPenalty"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig5mMinPE", "tensorrt_llm::executor::SamplingConfig::mMinP"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig10mMinTokensE", "tensorrt_llm::executor::SamplingConfig::mMinTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig18mNoRepeatNgramSizeE", "tensorrt_llm::executor::SamplingConfig::mNoRepeatNgramSize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig15mNumReturnBeamsE", "tensorrt_llm::executor::SamplingConfig::mNumReturnBeams"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig19mNumReturnSequencesE", "tensorrt_llm::executor::SamplingConfig::mNumReturnSequences"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig16mPresencePenaltyE", "tensorrt_llm::executor::SamplingConfig::mPresencePenalty"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig18mRepetitionPenaltyE", "tensorrt_llm::executor::SamplingConfig::mRepetitionPenalty"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig5mSeedE", "tensorrt_llm::executor::SamplingConfig::mSeed"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig12mTemperatureE", "tensorrt_llm::executor::SamplingConfig::mTemperature"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig5mTopKE", "tensorrt_llm::executor::SamplingConfig::mTopK"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig5mTopPE", "tensorrt_llm::executor::SamplingConfig::mTopP"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig10mTopPDecayE", "tensorrt_llm::executor::SamplingConfig::mTopPDecay"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig8mTopPMinE", "tensorrt_llm::executor::SamplingConfig::mTopPMin"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig13mTopPResetIdsE", "tensorrt_llm::executor::SamplingConfig::mTopPResetIds"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfigeqERK14SamplingConfig", "tensorrt_llm::executor::SamplingConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor14SamplingConfigeqERK14SamplingConfig", "tensorrt_llm::executor::SamplingConfig::operator==::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig26setBeamSearchDiversityRateERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setBeamSearchDiversityRate"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig26setBeamSearchDiversityRateERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setBeamSearchDiversityRate::beamSearchDiversityRate"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig12setBeamWidthE10SizeType32", "tensorrt_llm::executor::SamplingConfig::setBeamWidth"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig12setBeamWidthE10SizeType32", "tensorrt_llm::executor::SamplingConfig::setBeamWidth::beamWidth"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig17setBeamWidthArrayERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::setBeamWidthArray"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig17setBeamWidthArrayERKNSt8optionalINSt6vectorI10SizeType32EEEE", "tensorrt_llm::executor::SamplingConfig::setBeamWidthArray::beamWidthArray"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig16setEarlyStoppingERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setEarlyStopping"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig16setEarlyStoppingERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setEarlyStopping::earlyStopping"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig19setFrequencyPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setFrequencyPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig19setFrequencyPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setFrequencyPenalty::frequencyPenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig16setLengthPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setLengthPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig16setLengthPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setLengthPenalty::lengthPenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig7setMinPERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setMinP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig7setMinPERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setMinP::minP"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig12setMinTokensERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setMinTokens"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig12setMinTokensERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setMinTokens::minTokens"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig20setNoRepeatNgramSizeERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setNoRepeatNgramSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig20setNoRepeatNgramSizeERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setNoRepeatNgramSize::noRepeatNgramSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig21setNumReturnSequencesERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setNumReturnSequences"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig21setNumReturnSequencesERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setNumReturnSequences::numReturnSequences"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig18setPresencePenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setPresencePenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig18setPresencePenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setPresencePenalty::presencePenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig20setRepetitionPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setRepetitionPenalty"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig20setRepetitionPenaltyERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setRepetitionPenalty::repetitionPenalty"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig7setSeedERKNSt8optionalI14RandomSeedTypeEE", "tensorrt_llm::executor::SamplingConfig::setSeed"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig7setSeedERKNSt8optionalI14RandomSeedTypeEE", "tensorrt_llm::executor::SamplingConfig::setSeed::seed"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14setTemperatureERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setTemperature"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig14setTemperatureERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setTemperature::temperature"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig7setTopKERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setTopK"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig7setTopKERKNSt8optionalI10SizeType32EE", "tensorrt_llm::executor::SamplingConfig::setTopK::topK"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig7setTopPERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setTopP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig7setTopPERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setTopP::topP"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig12setTopPDecayERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setTopPDecay"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig12setTopPDecayERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setTopPDecay::topPDecay"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig10setTopPMinERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setTopPMin"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig10setTopPMinERKNSt8optionalI9FloatTypeEE", "tensorrt_llm::executor::SamplingConfig::setTopPMin::topPMin"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig15setTopPResetIdsERKNSt8optionalI11TokenIdTypeEE", "tensorrt_llm::executor::SamplingConfig::setTopPResetIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig15setTopPResetIdsERKNSt8optionalI11TokenIdTypeEE", "tensorrt_llm::executor::SamplingConfig::setTopPResetIds::topPResetIds"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor14SamplingConfig20updateNumReturnBeamsEv", "tensorrt_llm::executor::SamplingConfig::updateNumReturnBeams"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor15SchedulerConfigE", "tensorrt_llm::executor::SchedulerConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15SchedulerConfig15SchedulerConfigE23CapacitySchedulerPolicyNSt8optionalI21ContextChunkingPolicyEENSt8optionalI18DynamicBatchConfigEE", "tensorrt_llm::executor::SchedulerConfig::SchedulerConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15SchedulerConfig15SchedulerConfigE23CapacitySchedulerPolicyNSt8optionalI21ContextChunkingPolicyEENSt8optionalI18DynamicBatchConfigEE", "tensorrt_llm::executor::SchedulerConfig::SchedulerConfig::capacitySchedulerPolicy"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15SchedulerConfig15SchedulerConfigE23CapacitySchedulerPolicyNSt8optionalI21ContextChunkingPolicyEENSt8optionalI18DynamicBatchConfigEE", "tensorrt_llm::executor::SchedulerConfig::SchedulerConfig::contextChunkingPolicy"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15SchedulerConfig15SchedulerConfigE23CapacitySchedulerPolicyNSt8optionalI21ContextChunkingPolicyEENSt8optionalI18DynamicBatchConfigEE", "tensorrt_llm::executor::SchedulerConfig::SchedulerConfig::dynamicBatchConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15SchedulerConfig26getCapacitySchedulerPolicyEv", "tensorrt_llm::executor::SchedulerConfig::getCapacitySchedulerPolicy"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15SchedulerConfig24getContextChunkingPolicyEv", "tensorrt_llm::executor::SchedulerConfig::getContextChunkingPolicy"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15SchedulerConfig21getDynamicBatchConfigEv", "tensorrt_llm::executor::SchedulerConfig::getDynamicBatchConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15SchedulerConfig24mCapacitySchedulerPolicyE", "tensorrt_llm::executor::SchedulerConfig::mCapacitySchedulerPolicy"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15SchedulerConfig22mContextChunkingPolicyE", "tensorrt_llm::executor::SchedulerConfig::mContextChunkingPolicy"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15SchedulerConfig19mDynamicBatchConfigE", "tensorrt_llm::executor::SchedulerConfig::mDynamicBatchConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15SchedulerConfigeqERK15SchedulerConfig", "tensorrt_llm::executor::SchedulerConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor15SchedulerConfigeqERK15SchedulerConfig", "tensorrt_llm::executor::SchedulerConfig::operator==::other"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor13SerializationE", "tensorrt_llm::executor::Serialization"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization32deserializeAdditionalModelOutputERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeAdditionalModelOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization32deserializeAdditionalModelOutputERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeAdditionalModelOutput::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization27deserializeAdditionalOutputERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeAdditionalOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization27deserializeAdditionalOutputERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeAdditionalOutput::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization21deserializeAgentStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeAgentState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization21deserializeAgentStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeAgentState::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization19deserializeBlockKeyERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeBlockKey"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization19deserializeBlockKeyERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeBlockKey::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization15deserializeBoolERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeBool"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization15deserializeBoolERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeBool::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization21deserializeCacheStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeCacheState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization21deserializeCacheStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeCacheState::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization33deserializeCacheTransceiverConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeCacheTransceiverConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization33deserializeCacheTransceiverConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeCacheTransceiverConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization20deserializeCommStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeCommState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization20deserializeCommStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeCommState::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeContextPhaseParamsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeContextPhaseParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeContextPhaseParamsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeContextPhaseParams::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization31deserializeDataTransceiverStateERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeDataTransceiverState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization31deserializeDataTransceiverStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDataTransceiverState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization31deserializeDataTransceiverStateERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeDataTransceiverState::buffer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization31deserializeDataTransceiverStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDataTransceiverState::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeDebugConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDebugConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeDebugConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDebugConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeDecodingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeDecodingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDecodingConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeDecodingModeERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDecodingMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeDecodingModeERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDecodingMode::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization33deserializeDisServingRequestStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDisServingRequestStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization33deserializeDisServingRequestStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDisServingRequestStats::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeDynamicBatchConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDynamicBatchConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeDynamicBatchConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeDynamicBatchConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeEagleConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeEagleConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeEagleConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeEagleConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeExecutorConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeExecutorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeExecutorConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeExecutorConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization40deserializeExtendedRuntimePerfKnobConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeExtendedRuntimePerfKnobConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization40deserializeExtendedRuntimePerfKnobConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeExtendedRuntimePerfKnobConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization36deserializeExternalDraftTokensConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeExternalDraftTokensConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization36deserializeExternalDraftTokensConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeExternalDraftTokensConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization31deserializeGuidedDecodingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeGuidedDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization31deserializeGuidedDecodingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeGuidedDecodingConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization31deserializeGuidedDecodingParamsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeGuidedDecodingParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization31deserializeGuidedDecodingParamsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeGuidedDecodingParams::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization32deserializeInflightBatchingStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeInflightBatchingStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization32deserializeInflightBatchingStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeInflightBatchingStats::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeIterationStatsERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeIterationStats"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeIterationStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeIterationStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeIterationStatsERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeIterationStats::buffer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeIterationStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeIterationStats::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization28deserializeIterationStatsVecERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeIterationStatsVec"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization28deserializeIterationStatsVecERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeIterationStatsVec::buffer"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeKVCacheCreatedDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheCreatedData"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeKVCacheCreatedDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheCreatedData::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeKVCacheEventERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheEvent"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeKVCacheEventERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheEvent::is"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization27deserializeKVCacheEventDiffE16KVCacheEventDiffI1TERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheEventDiff"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization27deserializeKVCacheEventDiffE16KVCacheEventDiffI1TERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheEventDiff::T"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization27deserializeKVCacheEventDiffE16KVCacheEventDiffI1TERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheEventDiff::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization24deserializeKVCacheEventsERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeKVCacheEvents"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization24deserializeKVCacheEventsERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeKVCacheEvents::buffer"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeKVCacheRemovedDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheRemovedData"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeKVCacheRemovedDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheRemovedData::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization33deserializeKVCacheStoredBlockDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheStoredBlockData"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization33deserializeKVCacheStoredBlockDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheStoredBlockData::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization28deserializeKVCacheStoredDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheStoredData"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization28deserializeKVCacheStoredDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheStoredData::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeKVCacheUpdatedDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheUpdatedData"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeKVCacheUpdatedDataERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKVCacheUpdatedData::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization24deserializeKvCacheConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKvCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization24deserializeKvCacheConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKvCacheConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization33deserializeKvCacheRetentionConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKvCacheRetentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization33deserializeKvCacheRetentionConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKvCacheRetentionConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeKvCacheStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKvCacheStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeKvCacheStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeKvCacheStats::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization34deserializeLookaheadDecodingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeLookaheadDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization34deserializeLookaheadDecodingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeLookaheadDecodingConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization21deserializeLoraConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeLoraConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization21deserializeLoraConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeLoraConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization20deserializeModelTypeERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeModelType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization20deserializeModelTypeERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeModelType::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeMropeConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeMropeConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeMropeConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeMropeConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization26deserializeMultimodalInputERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeMultimodalInput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization26deserializeMultimodalInputERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeMultimodalInput::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeOrchestratorConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeOrchestratorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeOrchestratorConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeOrchestratorConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeOutputConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeOutputConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeOutputConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeOutputConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeParallelConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeParallelConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeParallelConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeParallelConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization26deserializePeftCacheConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializePeftCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization26deserializePeftCacheConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializePeftCacheConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializePromptTuningConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializePromptTuningConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializePromptTuningConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializePromptTuningConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization18deserializeRequestERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequest"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization18deserializeRequestERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequest::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeRequestPerfMetricsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequestPerfMetrics"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization29deserializeRequestPerfMetricsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequestPerfMetrics::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeRequestStageERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequestStage"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeRequestStageERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequestStage::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeRequestStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequestStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization23deserializeRequestStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequestStats::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization35deserializeRequestStatsPerIterationERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeRequestStatsPerIteration"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization35deserializeRequestStatsPerIterationERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequestStatsPerIteration"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization35deserializeRequestStatsPerIterationERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeRequestStatsPerIteration::buffer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization35deserializeRequestStatsPerIterationERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeRequestStatsPerIteration::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization38deserializeRequestStatsPerIterationVecERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeRequestStatsPerIterationVec"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization38deserializeRequestStatsPerIterationVecERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeRequestStatsPerIterationVec::buffer"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization19deserializeResponseERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeResponse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization19deserializeResponseERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeResponse::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization20deserializeResponsesERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeResponses"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization20deserializeResponsesERNSt6vectorIcEE", "tensorrt_llm::executor::Serialization::deserializeResponses::buffer"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization17deserializeResultERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeResult"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization17deserializeResultERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeResult::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeSamplingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSamplingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization25deserializeSamplingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSamplingConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization26deserializeSchedulerConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSchedulerConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization26deserializeSchedulerConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSchedulerConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeSocketStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSocketState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeSocketStateERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSocketState::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization32deserializeSpecDecFastLogitsInfoERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSpecDecFastLogitsInfo"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization32deserializeSpecDecFastLogitsInfoERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSpecDecFastLogitsInfo::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization28deserializeSpecDecodingStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSpecDecodingStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization28deserializeSpecDecodingStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSpecDecodingStats::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization36deserializeSpeculativeDecodingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSpeculativeDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization36deserializeSpeculativeDecodingConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeSpeculativeDecodingConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization30deserializeStaticBatchingStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeStaticBatchingStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization30deserializeStaticBatchingStatsERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeStaticBatchingStats::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization17deserializeStringERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeString"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization17deserializeStringERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeString::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization17deserializeTensorERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeTensor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization17deserializeTensorERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeTensor::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization20deserializeTimePointERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeTimePoint"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization20deserializeTimePointERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeTimePoint::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization36deserializeTokenRangeRetentionConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeTokenRangeRetentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization36deserializeTokenRangeRetentionConfigERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeTokenRangeRetentionConfig::is"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeUniqueTokenERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeUniqueToken"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization22deserializeUniqueTokenERNSt7istreamE", "tensorrt_llm::executor::Serialization::deserializeUniqueToken::is"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization9serializeEvRK16KVCacheEventDiffI1TERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK10LoraConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK11DebugConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK11EagleConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK11MropeConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12DecodingModeRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12KVCacheEventRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12KvCacheStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12OutputConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12RequestStageRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12RequestStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK13KvCacheConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14DecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14ExecutorConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14IterationStats", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14IterationStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14ParallelConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14SamplingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK15MultimodalInputRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK15PeftCacheConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK15SchedulerConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK16AdditionalOutputRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK17KVCacheStoredDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK17SpecDecodingStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18ContextPhaseParamsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18DynamicBatchConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18KVCacheCreatedDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18KVCacheRemovedDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18KVCacheUpdatedDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18OrchestratorConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18PromptTuningConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18RequestPerfMetricsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK19StaticBatchingStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20DataTransceiverState", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20DataTransceiverStateRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20GuidedDecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20GuidedDecodingParamsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK21AdditionalModelOutputRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK21InflightBatchingStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22CacheTransceiverConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22DisServingRequestStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22KVCacheStoredBlockDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22KvCacheRetentionConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK23LookaheadDecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK24RequestStatsPerIteration", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK24RequestStatsPerIterationRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK25ExternalDraftTokensConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK25SpeculativeDecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK29ExtendedRuntimePerfKnobConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK33SpeculativeDecodingFastLogitsInfoRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK6ResultRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK6TensorRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK7RequestRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK8ResponseRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN12tensorrt_llm13batch_manager16kv_cache_manager8BlockKeyERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN12tensorrt_llm7runtime11UniqueTokenERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN18RequestPerfMetrics9TimePointERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN22KvCacheRetentionConfig25TokenRangeRetentionConfigERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache10AgentStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache10CacheStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache11SocketStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache9CommStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKNSt5dequeI12KVCacheEventEE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKNSt6vectorI14IterationStatsEE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKNSt6vectorI24RequestStatsPerIterationEE", "tensorrt_llm::executor::Serialization::serialize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKNSt6vectorI8ResponseEE", "tensorrt_llm::executor::Serialization::serialize"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization9serializeEvRK16KVCacheEventDiffI1TERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::T"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK21AdditionalModelOutputRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::additionalModelOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK16AdditionalOutputRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::additionalOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22CacheTransceiverConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::cacheTransceiverConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK10LoraConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK11MropeConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12OutputConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14SamplingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18PromptTuningConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK25ExternalDraftTokensConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18ContextPhaseParamsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::contextPhaseParams"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization9serializeEvRK16KVCacheEventDiffI1TERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK17KVCacheStoredDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18KVCacheCreatedDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18KVCacheRemovedDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18KVCacheUpdatedDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22KVCacheStoredBlockDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20DataTransceiverState", "tensorrt_llm::executor::Serialization::serialize::dataTransceiverState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20DataTransceiverStateRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::dataTransceiverState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK11DebugConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::debugConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14DecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::decodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12DecodingModeRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::decodingMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18DynamicBatchConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::dynamicBatchConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK11EagleConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::eagleConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12KVCacheEventRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::event"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14ExecutorConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::executorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK29ExtendedRuntimePerfKnobConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::extendedRuntimePerfKnobConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20GuidedDecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::guidedDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20GuidedDecodingParamsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::guidedDecodingParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK21InflightBatchingStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::inflightBatchingStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK33SpeculativeDecodingFastLogitsInfoRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::info"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14IterationStats", "tensorrt_llm::executor::Serialization::serialize::iterStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14IterationStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::iterStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKNSt6vectorI14IterationStatsEE", "tensorrt_llm::executor::Serialization::serialize::iterStatsVec"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN12tensorrt_llm13batch_manager16kv_cache_manager8BlockKeyERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::key"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK13KvCacheConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::kvCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKNSt5dequeI12KVCacheEventEE", "tensorrt_llm::executor::Serialization::serialize::kvCacheEvents"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22KvCacheRetentionConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::kvCacheRetentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12KvCacheStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::kvCacheStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK23LookaheadDecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::lookaheadDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18RequestPerfMetricsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::metrics"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK15MultimodalInputRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::multimodalInput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18OrchestratorConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::orchestratorConfig"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization9serializeEvRK16KVCacheEventDiffI1TERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK10LoraConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK11DebugConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK11EagleConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK11MropeConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12DecodingModeRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12KVCacheEventRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12KvCacheStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12OutputConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12RequestStageRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12RequestStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK13KvCacheConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14DecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14ExecutorConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14IterationStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14ParallelConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14SamplingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK15MultimodalInputRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK15PeftCacheConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK15SchedulerConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK16AdditionalOutputRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK17KVCacheStoredDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK17SpecDecodingStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18ContextPhaseParamsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18DynamicBatchConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18KVCacheCreatedDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18KVCacheRemovedDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18KVCacheUpdatedDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18OrchestratorConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18PromptTuningConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK18RequestPerfMetricsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK19StaticBatchingStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20DataTransceiverStateRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20GuidedDecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK20GuidedDecodingParamsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK21AdditionalModelOutputRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK21InflightBatchingStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22CacheTransceiverConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22DisServingRequestStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22KVCacheStoredBlockDataRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22KvCacheRetentionConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK23LookaheadDecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK24RequestStatsPerIterationRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK25ExternalDraftTokensConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK25SpeculativeDecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK29ExtendedRuntimePerfKnobConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK33SpeculativeDecodingFastLogitsInfoRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK6ResultRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK6TensorRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK7RequestRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK8ResponseRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN12tensorrt_llm13batch_manager16kv_cache_manager8BlockKeyERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN12tensorrt_llm7runtime11UniqueTokenERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN18RequestPerfMetrics9TimePointERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN22KvCacheRetentionConfig25TokenRangeRetentionConfigERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache10AgentStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache10CacheStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache11SocketStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache9CommStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK14ParallelConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::parallelConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK15PeftCacheConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::peftCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK7RequestRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::request"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12RequestStageRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::requestStage"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKNSt6vectorI24RequestStatsPerIterationEE", "tensorrt_llm::executor::Serialization::serialize::requestStatsVec"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK8ResponseRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::response"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKNSt6vectorI8ResponseEE", "tensorrt_llm::executor::Serialization::serialize::responses"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK6ResultRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::result"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK15SchedulerConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::schedulerConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK25SpeculativeDecodingConfigRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::specDecConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK17SpecDecodingStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::specDecodingStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK12RequestStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK24RequestStatsPerIteration", "tensorrt_llm::executor::Serialization::serialize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK24RequestStatsPerIterationRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache10AgentStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache10CacheStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache11SocketStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN8kv_cache9CommStateERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK19StaticBatchingStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::staticBatchingStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK22DisServingRequestStatsRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::stats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERK6TensorRNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::tensor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN12tensorrt_llm7runtime11UniqueTokenERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::token"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN22KvCacheRetentionConfig25TokenRangeRetentionConfigERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::tokenRangeRetentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization9serializeERKN18RequestPerfMetrics9TimePointERNSt7ostreamE", "tensorrt_llm::executor::Serialization::serialize::tp"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization14serializedSizeE6size_tRK16KVCacheEventDiffI1TE", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK10LoraConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK11DebugConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK11EagleConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK11MropeConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12DecodingMode", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12KVCacheEvent", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12KvCacheStats", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12OutputConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12RequestStage", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12RequestStats", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK13KvCacheConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14DecodingConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14ExecutorConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14IterationStats", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14ParallelConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14SamplingConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK15MultimodalInput", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK15PeftCacheConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK15SchedulerConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK16AdditionalOutput", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK17KVCacheStoredData", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK17SpecDecodingStats", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18ContextPhaseParams", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18DynamicBatchConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18KVCacheCreatedData", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18KVCacheRemovedData", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18KVCacheUpdatedData", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18OrchestratorConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18PromptTuningConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18RequestPerfMetrics", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK19StaticBatchingStats", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK20DataTransceiverState", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK20GuidedDecodingConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK20GuidedDecodingParams", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK21AdditionalModelOutput", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK21InflightBatchingStats", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK22CacheTransceiverConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK22DisServingRequestStats", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK22KVCacheStoredBlockData", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK22KvCacheRetentionConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK23LookaheadDecodingConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK24RequestStatsPerIteration", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK25ExternalDraftTokensConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK25SpeculativeDecodingConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK29ExtendedRuntimePerfKnobConfig", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK33SpeculativeDecodingFastLogitsInfo", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK6Result", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK6Tensor", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK7Request", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK8Response", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN12tensorrt_llm13batch_manager16kv_cache_manager8BlockKeyE", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN12tensorrt_llm7runtime11UniqueTokenE", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN18RequestPerfMetrics9TimePointE", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN22KvCacheRetentionConfig25TokenRangeRetentionConfigE", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN8kv_cache10AgentStateE", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN8kv_cache10CacheStateE", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN8kv_cache11SocketStateE", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN8kv_cache9CommStateE", "tensorrt_llm::executor::Serialization::serializedSize"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization14serializedSizeE6size_tRK16KVCacheEventDiffI1TE", "tensorrt_llm::executor::Serialization::serializedSize::T"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK21AdditionalModelOutput", "tensorrt_llm::executor::Serialization::serializedSize::additionalModelOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK16AdditionalOutput", "tensorrt_llm::executor::Serialization::serializedSize::additionalOutput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK22CacheTransceiverConfig", "tensorrt_llm::executor::Serialization::serializedSize::cacheTransceiverConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK10LoraConfig", "tensorrt_llm::executor::Serialization::serializedSize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK11MropeConfig", "tensorrt_llm::executor::Serialization::serializedSize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12OutputConfig", "tensorrt_llm::executor::Serialization::serializedSize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14SamplingConfig", "tensorrt_llm::executor::Serialization::serializedSize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18PromptTuningConfig", "tensorrt_llm::executor::Serialization::serializedSize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK25ExternalDraftTokensConfig", "tensorrt_llm::executor::Serialization::serializedSize::config"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18ContextPhaseParams", "tensorrt_llm::executor::Serialization::serializedSize::contextPhaseParams"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor13Serialization14serializedSizeE6size_tRK16KVCacheEventDiffI1TE", "tensorrt_llm::executor::Serialization::serializedSize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK17KVCacheStoredData", "tensorrt_llm::executor::Serialization::serializedSize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18KVCacheCreatedData", "tensorrt_llm::executor::Serialization::serializedSize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18KVCacheRemovedData", "tensorrt_llm::executor::Serialization::serializedSize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18KVCacheUpdatedData", "tensorrt_llm::executor::Serialization::serializedSize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK22KVCacheStoredBlockData", "tensorrt_llm::executor::Serialization::serializedSize::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK20DataTransceiverState", "tensorrt_llm::executor::Serialization::serializedSize::dataTransceiverState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK11DebugConfig", "tensorrt_llm::executor::Serialization::serializedSize::debugConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14DecodingConfig", "tensorrt_llm::executor::Serialization::serializedSize::decodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12DecodingMode", "tensorrt_llm::executor::Serialization::serializedSize::decodingMode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK22DisServingRequestStats", "tensorrt_llm::executor::Serialization::serializedSize::disServingRequestStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18DynamicBatchConfig", "tensorrt_llm::executor::Serialization::serializedSize::dynamicBatchConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK11EagleConfig", "tensorrt_llm::executor::Serialization::serializedSize::eagleConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12KVCacheEvent", "tensorrt_llm::executor::Serialization::serializedSize::event"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14ExecutorConfig", "tensorrt_llm::executor::Serialization::serializedSize::executorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK29ExtendedRuntimePerfKnobConfig", "tensorrt_llm::executor::Serialization::serializedSize::extendedRuntimePerfKnobConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK20GuidedDecodingConfig", "tensorrt_llm::executor::Serialization::serializedSize::guidedDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK20GuidedDecodingParams", "tensorrt_llm::executor::Serialization::serializedSize::guidedDecodingParams"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK21InflightBatchingStats", "tensorrt_llm::executor::Serialization::serializedSize::inflightBatchingStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK33SpeculativeDecodingFastLogitsInfo", "tensorrt_llm::executor::Serialization::serializedSize::info"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14IterationStats", "tensorrt_llm::executor::Serialization::serializedSize::iterStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN12tensorrt_llm13batch_manager16kv_cache_manager8BlockKeyE", "tensorrt_llm::executor::Serialization::serializedSize::key"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK13KvCacheConfig", "tensorrt_llm::executor::Serialization::serializedSize::kvCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK22KvCacheRetentionConfig", "tensorrt_llm::executor::Serialization::serializedSize::kvCacheRetentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12KvCacheStats", "tensorrt_llm::executor::Serialization::serializedSize::kvCacheStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK23LookaheadDecodingConfig", "tensorrt_llm::executor::Serialization::serializedSize::lookaheadDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18RequestPerfMetrics", "tensorrt_llm::executor::Serialization::serializedSize::metrics"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK15MultimodalInput", "tensorrt_llm::executor::Serialization::serializedSize::multimodalInput"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK18OrchestratorConfig", "tensorrt_llm::executor::Serialization::serializedSize::orchestratorConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK14ParallelConfig", "tensorrt_llm::executor::Serialization::serializedSize::parallelConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK15PeftCacheConfig", "tensorrt_llm::executor::Serialization::serializedSize::peftCacheConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK7Request", "tensorrt_llm::executor::Serialization::serializedSize::request"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12RequestStage", "tensorrt_llm::executor::Serialization::serializedSize::requestStage"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK8Response", "tensorrt_llm::executor::Serialization::serializedSize::response"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK6Result", "tensorrt_llm::executor::Serialization::serializedSize::result"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK15SchedulerConfig", "tensorrt_llm::executor::Serialization::serializedSize::schedulerConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK25SpeculativeDecodingConfig", "tensorrt_llm::executor::Serialization::serializedSize::specDecConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK17SpecDecodingStats", "tensorrt_llm::executor::Serialization::serializedSize::specDecodingStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK12RequestStats", "tensorrt_llm::executor::Serialization::serializedSize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK24RequestStatsPerIteration", "tensorrt_llm::executor::Serialization::serializedSize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN8kv_cache10AgentStateE", "tensorrt_llm::executor::Serialization::serializedSize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN8kv_cache10CacheStateE", "tensorrt_llm::executor::Serialization::serializedSize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN8kv_cache11SocketStateE", "tensorrt_llm::executor::Serialization::serializedSize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN8kv_cache9CommStateE", "tensorrt_llm::executor::Serialization::serializedSize::state"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK19StaticBatchingStats", "tensorrt_llm::executor::Serialization::serializedSize::staticBatchingStats"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERK6Tensor", "tensorrt_llm::executor::Serialization::serializedSize::tensor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN12tensorrt_llm7runtime11UniqueTokenE", "tensorrt_llm::executor::Serialization::serializedSize::token"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor13Serialization14serializedSizeERKN22KvCacheRetentionConfig25TokenRangeRetentionConfigE", "tensorrt_llm::executor::Serialization::serializedSize::tokenRangeRetentionConfig"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor5ShapeE", "tensorrt_llm::executor::Shape"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor5Shape4BaseE", "tensorrt_llm::executor::Shape::Base"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor5Shape9DimType64E", "tensorrt_llm::executor::Shape::DimType64"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor5Shape5ShapeENSt16initializer_listI9DimType64EE", "tensorrt_llm::executor::Shape::Shape"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor5Shape5ShapeEPK9DimType64N4Base9size_typeE", "tensorrt_llm::executor::Shape::Shape"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor5Shape5ShapeEv", "tensorrt_llm::executor::Shape::Shape"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor5Shape5ShapeEPK9DimType64N4Base9size_typeE", "tensorrt_llm::executor::Shape::Shape::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor5Shape5ShapeENSt16initializer_listI9DimType64EE", "tensorrt_llm::executor::Shape::Shape::dims"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor5Shape5ShapeEPK9DimType64N4Base9size_typeE", "tensorrt_llm::executor::Shape::Shape::size"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor10SizeType32E", "tensorrt_llm::executor::SizeType32"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor10SizeType64E", "tensorrt_llm::executor::SizeType64"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor17SpecDecodingStatsE", "tensorrt_llm::executor::SpecDecodingStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor17SpecDecodingStats16acceptanceLengthE", "tensorrt_llm::executor::SpecDecodingStats::acceptanceLength"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor17SpecDecodingStats13draftOverheadE", "tensorrt_llm::executor::SpecDecodingStats::draftOverhead"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor17SpecDecodingStats13iterLatencyMSE", "tensorrt_llm::executor::SpecDecodingStats::iterLatencyMS"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor17SpecDecodingStats17numAcceptedTokensE", "tensorrt_llm::executor::SpecDecodingStats::numAcceptedTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor17SpecDecodingStats14numDraftTokensE", "tensorrt_llm::executor::SpecDecodingStats::numDraftTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor17SpecDecodingStats26numRequestsWithDraftTokensE", "tensorrt_llm::executor::SpecDecodingStats::numRequestsWithDraftTokens"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor25SpeculativeDecodingConfigE", "tensorrt_llm::executor::SpeculativeDecodingConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor25SpeculativeDecodingConfig25SpeculativeDecodingConfigEb", "tensorrt_llm::executor::SpeculativeDecodingConfig::SpeculativeDecodingConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor25SpeculativeDecodingConfig25SpeculativeDecodingConfigEb", "tensorrt_llm::executor::SpeculativeDecodingConfig::SpeculativeDecodingConfig::fastLogits"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor25SpeculativeDecodingConfig10fastLogitsE", "tensorrt_llm::executor::SpeculativeDecodingConfig::fastLogits"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor25SpeculativeDecodingConfigeqERK25SpeculativeDecodingConfig", "tensorrt_llm::executor::SpeculativeDecodingConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor25SpeculativeDecodingConfigeqERK25SpeculativeDecodingConfig", "tensorrt_llm::executor::SpeculativeDecodingConfig::operator==::other"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor33SpeculativeDecodingFastLogitsInfoE", "tensorrt_llm::executor::SpeculativeDecodingFastLogitsInfo"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor33SpeculativeDecodingFastLogitsInfo18draftParticipantIdE", "tensorrt_llm::executor::SpeculativeDecodingFastLogitsInfo::draftParticipantId"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor33SpeculativeDecodingFastLogitsInfo14draftRequestIdE", "tensorrt_llm::executor::SpeculativeDecodingFastLogitsInfo::draftRequestId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor33SpeculativeDecodingFastLogitsInfo8toTensorEv", "tensorrt_llm::executor::SpeculativeDecodingFastLogitsInfo::toTensor"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor19StaticBatchingStatsE", "tensorrt_llm::executor::StaticBatchingStats"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor19StaticBatchingStats13emptyGenSlotsE", "tensorrt_llm::executor::StaticBatchingStats::emptyGenSlots"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor19StaticBatchingStats18numContextRequestsE", "tensorrt_llm::executor::StaticBatchingStats::numContextRequests"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor19StaticBatchingStats12numCtxTokensE", "tensorrt_llm::executor::StaticBatchingStats::numCtxTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor19StaticBatchingStats12numGenTokensE", "tensorrt_llm::executor::StaticBatchingStats::numGenTokens"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor19StaticBatchingStats20numScheduledRequestsE", "tensorrt_llm::executor::StaticBatchingStats::numScheduledRequests"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor9StreamPtrE", "tensorrt_llm::executor::StreamPtr"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor6TensorE", "tensorrt_llm::executor::Tensor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::CudaStreamPtr"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor6Tensor4ImplE", "tensorrt_llm::executor::Tensor::Impl"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6TensorENSt10shared_ptrIN7runtime7ITensorEEE", "tensorrt_llm::executor::Tensor::Tensor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6TensorERK6Tensor", "tensorrt_llm::executor::Tensor::Tensor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6TensorERR6Tensor", "tensorrt_llm::executor::Tensor::Tensor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6TensorEv", "tensorrt_llm::executor::Tensor::Tensor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6TensorERK6Tensor", "tensorrt_llm::executor::Tensor::Tensor::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6TensorERR6Tensor", "tensorrt_llm::executor::Tensor::Tensor::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6TensorENSt10shared_ptrIN7runtime7ITensorEEE", "tensorrt_llm::executor::Tensor::Tensor::tensor"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor6copyToENSt10shared_ptrI4ImplEE13CudaStreamPtr", "tensorrt_llm::executor::Tensor::copyTo"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor6copyToENSt10shared_ptrI4ImplEE13CudaStreamPtr", "tensorrt_llm::executor::Tensor::copyTo::stream"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor6copyToENSt10shared_ptrI4ImplEE13CudaStreamPtr", "tensorrt_llm::executor::Tensor::copyTo::tensor"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor9copyToCpuEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToCpu"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor9copyToCpuEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToCpu::stream"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor9copyToGpuEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToGpu"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor9copyToGpuEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToGpu::stream"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor13copyToManagedEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToManaged"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor13copyToManagedEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToManaged::stream"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor12copyToPinnedEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToPinned"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor12copyToPinnedEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToPinned::stream"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor18copyToPooledPinnedEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToPooledPinned"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor18copyToPooledPinnedEN6Tensor13CudaStreamPtrE", "tensorrt_llm::executor::Tensor::copyToPooledPinned::stream"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor3cpuE6Tensor5Shape", "tensorrt_llm::executor::Tensor::cpu"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor3cpuE8DataType5Shape", "tensorrt_llm::executor::Tensor::cpu"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor3cpuE6Tensor5Shape", "tensorrt_llm::executor::Tensor::cpu::T"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor3cpuE8DataType5Shape", "tensorrt_llm::executor::Tensor::cpu::dataType"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor3cpuE6Tensor5Shape", "tensorrt_llm::executor::Tensor::cpu::shape"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor3cpuE8DataType5Shape", "tensorrt_llm::executor::Tensor::cpu::shape"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6detail9ofITensorENSt10shared_ptrIN7runtime7ITensorEEE", "tensorrt_llm::executor::Tensor::detail::ofITensor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6detail9ofITensorENSt10shared_ptrIN7runtime7ITensorEEE", "tensorrt_llm::executor::Tensor::detail::ofITensor::tensor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6detail9toITensorERK6Tensor", "tensorrt_llm::executor::Tensor::detail::toITensor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6detail9toITensorERK6Tensor", "tensorrt_llm::executor::Tensor::detail::toITensor::tensor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7getDataEv", "tensorrt_llm::executor::Tensor::getData"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor7getDataEv", "tensorrt_llm::executor::Tensor::getData"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor11getDataTypeEv", "tensorrt_llm::executor::Tensor::getDataType"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor13getMemoryTypeEv", "tensorrt_llm::executor::Tensor::getMemoryType"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor14getRuntimeTypeE8DataTypev", "tensorrt_llm::executor::Tensor::getRuntimeType"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor14getRuntimeTypeE8DataTypev", "tensorrt_llm::executor::Tensor::getRuntimeType::T"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor8getShapeEv", "tensorrt_llm::executor::Tensor::getShape"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor7getSizeEv", "tensorrt_llm::executor::Tensor::getSize"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6Tensor14getSizeInBytesEv", "tensorrt_llm::executor::Tensor::getSizeInBytes"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor3gpuE6Tensor13CudaStreamPtr5Shape", "tensorrt_llm::executor::Tensor::gpu"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor3gpuE8DataType13CudaStreamPtr5Shape", "tensorrt_llm::executor::Tensor::gpu"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor3gpuE6Tensor13CudaStreamPtr5Shape", "tensorrt_llm::executor::Tensor::gpu::T"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor3gpuE8DataType13CudaStreamPtr5Shape", "tensorrt_llm::executor::Tensor::gpu::dataType"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor3gpuE6Tensor13CudaStreamPtr5Shape", "tensorrt_llm::executor::Tensor::gpu::shape"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor3gpuE8DataType13CudaStreamPtr5Shape", "tensorrt_llm::executor::Tensor::gpu::shape"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor3gpuE6Tensor13CudaStreamPtr5Shape", "tensorrt_llm::executor::Tensor::gpu::stream"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor3gpuE8DataType13CudaStreamPtr5Shape", "tensorrt_llm::executor::Tensor::gpu::stream"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7mTensorE", "tensorrt_llm::executor::Tensor::mTensor"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor7managedE6Tensor5Shape", "tensorrt_llm::executor::Tensor::managed"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7managedE8DataType5Shape", "tensorrt_llm::executor::Tensor::managed"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor7managedE6Tensor5Shape", "tensorrt_llm::executor::Tensor::managed::T"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7managedE8DataType5Shape", "tensorrt_llm::executor::Tensor::managed::dataType"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor7managedE6Tensor5Shape", "tensorrt_llm::executor::Tensor::managed::shape"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7managedE8DataType5Shape", "tensorrt_llm::executor::Tensor::managed::shape"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor2ofE6TensorP1T5Shape", "tensorrt_llm::executor::Tensor::of"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor2ofE6TensorR1T", "tensorrt_llm::executor::Tensor::of"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor2ofE8DataTypePv5Shape", "tensorrt_llm::executor::Tensor::of"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor2ofE6TensorP1T5Shape", "tensorrt_llm::executor::Tensor::of::T"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor2ofE6TensorR1T", "tensorrt_llm::executor::Tensor::of::T"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor2ofE6TensorP1T5Shape", "tensorrt_llm::executor::Tensor::of::data"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor2ofE6TensorR1T", "tensorrt_llm::executor::Tensor::of::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor2ofE8DataTypePv5Shape", "tensorrt_llm::executor::Tensor::of::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor2ofE8DataTypePv5Shape", "tensorrt_llm::executor::Tensor::of::dataType"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor2ofE6TensorP1T5Shape", "tensorrt_llm::executor::Tensor::of::shape"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor2ofE8DataTypePv5Shape", "tensorrt_llm::executor::Tensor::of::shape"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6TensorcvbEv", "tensorrt_llm::executor::Tensor::operator bool"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6TensorneERK6Tensor", "tensorrt_llm::executor::Tensor::operator!="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor6TensorneERK6Tensor", "tensorrt_llm::executor::Tensor::operator!=::rhs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6TensoraSERK6Tensor", "tensorrt_llm::executor::Tensor::operator="], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6TensoraSERR6Tensor", "tensorrt_llm::executor::Tensor::operator="], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6TensoraSERK6Tensor", "tensorrt_llm::executor::Tensor::operator=::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6TensoraSERR6Tensor", "tensorrt_llm::executor::Tensor::operator=::other"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor6TensoreqERK6Tensor", "tensorrt_llm::executor::Tensor::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor6TensoreqERK6Tensor", "tensorrt_llm::executor::Tensor::operator==::rhs"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor6pinnedE6Tensor5Shape", "tensorrt_llm::executor::Tensor::pinned"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6pinnedE8DataType5Shape", "tensorrt_llm::executor::Tensor::pinned"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor6pinnedE6Tensor5Shape", "tensorrt_llm::executor::Tensor::pinned::T"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6pinnedE8DataType5Shape", "tensorrt_llm::executor::Tensor::pinned::dataType"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor6pinnedE6Tensor5Shape", "tensorrt_llm::executor::Tensor::pinned::shape"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor6pinnedE8DataType5Shape", "tensorrt_llm::executor::Tensor::pinned::shape"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor12pooledPinnedE6Tensor5Shape", "tensorrt_llm::executor::Tensor::pooledPinned"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor12pooledPinnedE8DataType5Shape", "tensorrt_llm::executor::Tensor::pooledPinned"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor12pooledPinnedE6Tensor5Shape", "tensorrt_llm::executor::Tensor::pooledPinned::T"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor12pooledPinnedE8DataType5Shape", "tensorrt_llm::executor::Tensor::pooledPinned::dataType"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor6Tensor12pooledPinnedE6Tensor5Shape", "tensorrt_llm::executor::Tensor::pooledPinned::shape"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor12pooledPinnedE8DataType5Shape", "tensorrt_llm::executor::Tensor::pooledPinned::shape"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7setFromERK6Tensor13CudaStreamPtr", "tensorrt_llm::executor::Tensor::setFrom"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7setFromERK6Tensor13CudaStreamPtr", "tensorrt_llm::executor::Tensor::setFrom::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7setFromERK6Tensor13CudaStreamPtr", "tensorrt_llm::executor::Tensor::setFrom::stream"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7setZeroE13CudaStreamPtr", "tensorrt_llm::executor::Tensor::setZero"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6Tensor7setZeroE13CudaStreamPtr", "tensorrt_llm::executor::Tensor::setZero::stream"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6TensorD0Ev", "tensorrt_llm::executor::Tensor::~Tensor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor9TensorPtrE", "tensorrt_llm::executor::TensorPtr"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor11TokenIdTypeE", "tensorrt_llm::executor::TokenIdType"], [0, 2, 1, "_CPPv4I0_bEN12tensorrt_llm8executor10TypeTraitsE", "tensorrt_llm::executor::TypeTraits"], [0, 8, 1, "_CPPv4I0_bEN12tensorrt_llm8executor10TypeTraitsE", "tensorrt_llm::executor::TypeTraits::T"], [0, 2, 1, "_CPPv4I0EN12tensorrt_llm8executor10TypeTraitsIP1TEE", "tensorrt_llm::executor::TypeTraits<T*>"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor10TypeTraitsIP1TEE", "tensorrt_llm::executor::TypeTraits<T*>::T"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10TypeTraitsIP1TE5valueE", "tensorrt_llm::executor::TypeTraits<T*>::value"], [0, 2, 1, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsIbEE", "tensorrt_llm::executor::TypeTraits<bool>"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10TypeTraitsIbE5valueE", "tensorrt_llm::executor::TypeTraits<bool>::value"], [0, 2, 1, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsIfEE", "tensorrt_llm::executor::TypeTraits<float>"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10TypeTraitsIfE5valueE", "tensorrt_llm::executor::TypeTraits<float>::value"], [0, 2, 1, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsI4halfEE", "tensorrt_llm::executor::TypeTraits<half>"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10TypeTraitsI4halfE5valueE", "tensorrt_llm::executor::TypeTraits<half>::value"], [0, 2, 1, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsINSt7int32_tEEE", "tensorrt_llm::executor::TypeTraits<std::int32_t>"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10TypeTraitsINSt7int32_tEE5valueE", "tensorrt_llm::executor::TypeTraits<std::int32_t>::value"], [0, 2, 1, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsINSt7int64_tEEE", "tensorrt_llm::executor::TypeTraits<std::int64_t>"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10TypeTraitsINSt7int64_tEE5valueE", "tensorrt_llm::executor::TypeTraits<std::int64_t>::value"], [0, 2, 1, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsINSt6int8_tEEE", "tensorrt_llm::executor::TypeTraits<std::int8_t>"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10TypeTraitsINSt6int8_tEE5valueE", "tensorrt_llm::executor::TypeTraits<std::int8_t>::value"], [0, 2, 1, "_CPPv4IEN12tensorrt_llm8executor10TypeTraitsINSt7uint8_tEEE", "tensorrt_llm::executor::TypeTraits<std::uint8_t>"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor10TypeTraitsINSt7uint8_tEE5valueE", "tensorrt_llm::executor::TypeTraits<std::uint8_t>::value"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor11VecLogProbsE", "tensorrt_llm::executor::VecLogProbs"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor16VecTokenExtraIdsE", "tensorrt_llm::executor::VecTokenExtraIds"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor9VecTokensE", "tensorrt_llm::executor::VecTokens"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor6detailE", "tensorrt_llm::executor::detail"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor6detail9DimType64E", "tensorrt_llm::executor::detail::DimType64"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6detail9ofITensorENSt10shared_ptrIN7runtime7ITensorEEE", "tensorrt_llm::executor::detail::ofITensor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6detail9ofITensorENSt10shared_ptrIN7runtime7ITensorEEE", "tensorrt_llm::executor::detail::ofITensor::tensor"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor6detail9toITensorERK6Tensor", "tensorrt_llm::executor::detail::toITensor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor6detail9toITensorERK6Tensor", "tensorrt_llm::executor::detail::toITensor::tensor"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executorE", "tensorrt_llm::executor::disagg_executor"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestratorE", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator26DisaggExecutorOrchestratorERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorIN8executor14ExecutorConfigEEERKNSt6vectorIN8executor14ExecutorConfigEEEbb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::DisaggExecutorOrchestrator"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator26DisaggExecutorOrchestratorERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorIN8executor14ExecutorConfigEEERKNSt6vectorIN8executor14ExecutorConfigEEEbb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::DisaggExecutorOrchestrator::ctxEnginePaths"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator26DisaggExecutorOrchestratorERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorIN8executor14ExecutorConfigEEERKNSt6vectorIN8executor14ExecutorConfigEEEbb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::DisaggExecutorOrchestrator::ctxExecutorConfigs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator26DisaggExecutorOrchestratorERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorIN8executor14ExecutorConfigEEERKNSt6vectorIN8executor14ExecutorConfigEEEbb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::DisaggExecutorOrchestrator::genEnginePaths"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator26DisaggExecutorOrchestratorERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorIN8executor14ExecutorConfigEEERKNSt6vectorIN8executor14ExecutorConfigEEEbb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::DisaggExecutorOrchestrator::genExecutorConfigs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator26DisaggExecutorOrchestratorERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorIN8executor14ExecutorConfigEEERKNSt6vectorIN8executor14ExecutorConfigEEEbb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::DisaggExecutorOrchestrator::hasContextAwaitThreads"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator26DisaggExecutorOrchestratorERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorINSt10filesystem4pathEEERKNSt6vectorIN8executor14ExecutorConfigEEERKNSt6vectorIN8executor14ExecutorConfigEEEbb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::DisaggExecutorOrchestrator::hasGenAwaitThreads"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator21awaitContextResponsesERKNSt8optionalINSt6chrono12millisecondsEEENSt8optionalIiEE", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::awaitContextResponses"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator21awaitContextResponsesERKNSt8optionalINSt6chrono12millisecondsEEENSt8optionalIiEE", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::awaitContextResponses::contextIdx"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator21awaitContextResponsesERKNSt8optionalINSt6chrono12millisecondsEEENSt8optionalIiEE", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::awaitContextResponses::timeout"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator24awaitGenerationResponsesERKNSt8optionalINSt6chrono12millisecondsEEENSt8optionalIiEE", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::awaitGenerationResponses"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator24awaitGenerationResponsesERKNSt8optionalINSt6chrono12millisecondsEEENSt8optionalIiEE", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::awaitGenerationResponses::genIdx"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator24awaitGenerationResponsesERKNSt8optionalINSt6chrono12millisecondsEEENSt8optionalIiEE", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::awaitGenerationResponses::timeout"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator10canEnqueueEv", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::canEnqueue"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator14enqueueContextERKNSt6vectorIN5texec7RequestEEENSt8optionalIiEEb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::enqueueContext"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator14enqueueContextERKNSt6vectorIN5texec7RequestEEENSt8optionalIiEEb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::enqueueContext::batch"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator14enqueueContextERKNSt6vectorIN5texec7RequestEEENSt8optionalIiEEb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::enqueueContext::requests"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator14enqueueContextERKNSt6vectorIN5texec7RequestEEENSt8optionalIiEEb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::enqueueContext::selectContextId"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator17enqueueGenerationERKNSt6vectorIN5texec7RequestEEERKNSt6vectorI6IdTypeEENSt8optionalIiEEb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::enqueueGeneration"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator17enqueueGenerationERKNSt6vectorIN5texec7RequestEEERKNSt6vectorI6IdTypeEENSt8optionalIiEEb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::enqueueGeneration::batch"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator17enqueueGenerationERKNSt6vectorIN5texec7RequestEEERKNSt6vectorI6IdTypeEENSt8optionalIiEEb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::enqueueGeneration::globalRequestIds"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator17enqueueGenerationERKNSt6vectorIN5texec7RequestEEERKNSt6vectorI6IdTypeEENSt8optionalIiEEb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::enqueueGeneration::requests"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator17enqueueGenerationERKNSt6vectorIN5texec7RequestEEERKNSt6vectorI6IdTypeEENSt8optionalIiEEb", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::enqueueGeneration::selectGenIdx"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator19getContextExecutorsEv", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::getContextExecutors"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator15getGenExecutorsEv", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::getGenExecutors"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestrator5mImplE", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::mImpl"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor26DisaggExecutorOrchestratorD0Ev", "tensorrt_llm::executor::disagg_executor::DisaggExecutorOrchestrator::~DisaggExecutorOrchestrator"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithIdE", "tensorrt_llm::executor::disagg_executor::ResponseWithId"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERK14ResponseWithId", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERKN12tensorrt_llm8executor8ResponseE6IdType", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERR14ResponseWithId", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERRN12tensorrt_llm8executor8ResponseE6IdType", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERKN12tensorrt_llm8executor8ResponseE6IdType", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId::gid"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERRN12tensorrt_llm8executor8ResponseE6IdType", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId::gid"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERK14ResponseWithId", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERR14ResponseWithId", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERKN12tensorrt_llm8executor8ResponseE6IdType", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId::response"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId14ResponseWithIdERRN12tensorrt_llm8executor8ResponseE6IdType", "tensorrt_llm::executor::disagg_executor::ResponseWithId::ResponseWithId::response"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId3gidE", "tensorrt_llm::executor::disagg_executor::ResponseWithId::gid"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithIdaSERK14ResponseWithId", "tensorrt_llm::executor::disagg_executor::ResponseWithId::operator="], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithIdaSERR14ResponseWithId", "tensorrt_llm::executor::disagg_executor::ResponseWithId::operator="], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithIdaSERK14ResponseWithId", "tensorrt_llm::executor::disagg_executor::ResponseWithId::operator=::other"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithIdaSERR14ResponseWithId", "tensorrt_llm::executor::disagg_executor::ResponseWithId::operator=::other"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithId8responseE", "tensorrt_llm::executor::disagg_executor::ResponseWithId::response"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor15disagg_executor14ResponseWithIdD0Ev", "tensorrt_llm::executor::disagg_executor::ResponseWithId::~ResponseWithId"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor8kv_cacheE", "tensorrt_llm::executor::kv_cache"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor8kv_cacheE", "tensorrt_llm::executor::kv_cache"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor8kv_cacheE", "tensorrt_llm::executor::kv_cache"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor8kv_cacheE", "tensorrt_llm::executor::kv_cache"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9AgentDescE", "tensorrt_llm::executor::kv_cache::AgentDesc"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9AgentDesc9AgentDescENSt6stringE", "tensorrt_llm::executor::kv_cache::AgentDesc::AgentDesc"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9AgentDesc9AgentDescENSt6stringE", "tensorrt_llm::executor::kv_cache::AgentDesc::AgentDesc::backendAgentDesc"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9AgentDesc19getBackendAgentDescEv", "tensorrt_llm::executor::kv_cache::AgentDesc::getBackendAgentDesc"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9AgentDesc17mBackendAgentDescE", "tensorrt_llm::executor::kv_cache::AgentDesc::mBackendAgentDesc"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10AgentStateE", "tensorrt_llm::executor::kv_cache::AgentState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10AgentState10AgentStateENSt6stringENSt6stringE", "tensorrt_llm::executor::kv_cache::AgentState::AgentState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10AgentState10AgentStateEv", "tensorrt_llm::executor::kv_cache::AgentState::AgentState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10AgentState10AgentStateENSt6stringENSt6stringE", "tensorrt_llm::executor::kv_cache::AgentState::AgentState::agentName"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10AgentState10AgentStateENSt6stringENSt6stringE", "tensorrt_llm::executor::kv_cache::AgentState::AgentState::connectionInfo"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10AgentState10mAgentNameE", "tensorrt_llm::executor::kv_cache::AgentState::mAgentName"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10AgentState15mConnectionInfoE", "tensorrt_llm::executor::kv_cache::AgentState::mConnectionInfo"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10AgentStateeqERK10AgentState", "tensorrt_llm::executor::kv_cache::AgentState::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10AgentStateeqERK10AgentState", "tensorrt_llm::executor::kv_cache::AgentState::operator==::other"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10AgentState8toStringEv", "tensorrt_llm::executor::kv_cache::AgentState::toString"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15BaseAgentConfigE", "tensorrt_llm::executor::kv_cache::BaseAgentConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15BaseAgentConfig5mNameE", "tensorrt_llm::executor::kv_cache::BaseAgentConfig::mName"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15BaseAgentConfig11multiThreadE", "tensorrt_llm::executor::kv_cache::BaseAgentConfig::multiThread"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15BaseAgentConfig13useProgThreadE", "tensorrt_llm::executor::kv_cache::BaseAgentConfig::useProgThread"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseLoopbackAgentE", "tensorrt_llm::executor::kv_cache::BaseLoopbackAgent"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseLoopbackAgent22executeLoopbackRequestERK11MemoryDescsRK9FileDescsb", "tensorrt_llm::executor::kv_cache::BaseLoopbackAgent::executeLoopbackRequest"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseLoopbackAgent22executeLoopbackRequestERK11MemoryDescsRK9FileDescsb", "tensorrt_llm::executor::kv_cache::BaseLoopbackAgent::executeLoopbackRequest::fileDescs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseLoopbackAgent22executeLoopbackRequestERK11MemoryDescsRK9FileDescsb", "tensorrt_llm::executor::kv_cache::BaseLoopbackAgent::executeLoopbackRequest::isOffload"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseLoopbackAgent22executeLoopbackRequestERK11MemoryDescsRK9FileDescsb", "tensorrt_llm::executor::kv_cache::BaseLoopbackAgent::executeLoopbackRequest::memoryDescs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseLoopbackAgentD0Ev", "tensorrt_llm::executor::kv_cache::BaseLoopbackAgent::~BaseLoopbackAgent"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgentE", "tensorrt_llm::executor::kv_cache::BaseTransferAgent"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent16checkRemoteDescsERKNSt6stringERK11MemoryDescs", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::checkRemoteDescs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent16checkRemoteDescsERKNSt6stringERK11MemoryDescs", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::checkRemoteDescs::memoryDescs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent16checkRemoteDescsERKNSt6stringERK11MemoryDescs", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::checkRemoteDescs::name"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent18connectRemoteAgentERKNSt6stringERK18ConnectionInfoType", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::connectRemoteAgent"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent18connectRemoteAgentERKNSt6stringERK18ConnectionInfoType", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::connectRemoteAgent::connectionInfo"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent18connectRemoteAgentERKNSt6stringERK18ConnectionInfoType", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::connectRemoteAgent::name"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent16deregisterMemoryERK13RegisterDescs", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::deregisterMemory"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent16deregisterMemoryERK13RegisterDescs", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::deregisterMemory::descs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent17getConnectionInfoEv", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::getConnectionInfo"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent17getLocalAgentDescEv", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::getLocalAgentDesc"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent23getNotifiedSyncMessagesEv", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::getNotifiedSyncMessages"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent21invalidateRemoteAgentERKNSt6stringE", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::invalidateRemoteAgent"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent21invalidateRemoteAgentERKNSt6stringE", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::invalidateRemoteAgent::name"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent15loadRemoteAgentERKNSt6stringERK9AgentDesc", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::loadRemoteAgent"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent15loadRemoteAgentERKNSt6stringERK9AgentDesc", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::loadRemoteAgent::agentDesc"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent15loadRemoteAgentERKNSt6stringERK9AgentDesc", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::loadRemoteAgent::name"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent17notifySyncMessageERKNSt6stringERK11SyncMessage", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::notifySyncMessage"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent17notifySyncMessageERKNSt6stringERK11SyncMessage", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::notifySyncMessage::name"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent17notifySyncMessageERKNSt6stringERK11SyncMessage", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::notifySyncMessage::syncMessage"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent14registerMemoryERK13RegisterDescs", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::registerMemory"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent14registerMemoryERK13RegisterDescs", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::registerMemory::descs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent22submitTransferRequestsERK15TransferRequest", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::submitTransferRequests"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgent22submitTransferRequestsERK15TransferRequest", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::submitTransferRequests::request"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17BaseTransferAgentD0Ev", "tensorrt_llm::executor::kv_cache::BaseTransferAgent::~BaseTransferAgent"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheStateE", "tensorrt_llm::executor::kv_cache::CacheState"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState15AttentionConfigE", "tensorrt_llm::executor::kv_cache::CacheState::AttentionConfig"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState15AttentionConfig15AttentionConfigE13AttentionTypei", "tensorrt_llm::executor::kv_cache::CacheState::AttentionConfig::AttentionConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState15AttentionConfig15AttentionConfigE13AttentionTypei", "tensorrt_llm::executor::kv_cache::CacheState::AttentionConfig::AttentionConfig::attentionType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState15AttentionConfig15AttentionConfigE13AttentionTypei", "tensorrt_llm::executor::kv_cache::CacheState::AttentionConfig::AttentionConfig::kvFactor"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState15AttentionConfig14mAttentionTypeE", "tensorrt_llm::executor::kv_cache::CacheState::AttentionConfig::mAttentionType"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState15AttentionConfig9mKvFactorE", "tensorrt_llm::executor::kv_cache::CacheState::AttentionConfig::mKvFactor"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState15AttentionConfigeqERK15AttentionConfig", "tensorrt_llm::executor::kv_cache::CacheState::AttentionConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState15AttentionConfigeqERK15AttentionConfig", "tensorrt_llm::executor::kv_cache::CacheState::AttentionConfig::operator==::other"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState13AttentionTypeE", "tensorrt_llm::executor::kv_cache::CacheState::AttentionType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState13AttentionType8kDEFAULTE", "tensorrt_llm::executor::kv_cache::CacheState::AttentionType::kDEFAULT"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState13AttentionType4kMLAE", "tensorrt_llm::executor::kv_cache::CacheState::AttentionType::kMLA"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE11ModelConfigRKN7runtime11WorldConfigERKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::DPrank"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::DPrank"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::DPsize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::DPsize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::attentionLayerNumPerPP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE11ModelConfigRKN7runtime11WorldConfigERKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::attentionLayerNumPerPP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::attentionLayerNumPerPP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::attentionType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE11ModelConfigRKN7runtime11WorldConfigERKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::attentionType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::attentionType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::contextParallelism"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::contextParallelism"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::dataType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE11ModelConfigRKN7runtime11WorldConfigERKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::dataType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::dataType"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::enableAttentionDP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::enableAttentionDP"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::enableBlockReuse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE11ModelConfigRKN7runtime11WorldConfigERKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::enableBlockReuse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::enableBlockReuse"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::kvFactor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE11ModelConfigRKN7runtime11WorldConfigERKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::kvFactor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::kvFactor"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE11ModelConfigRKN7runtime11WorldConfigERKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::modelConfig"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::nbAttentionLayers"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::nbKvHeadPerLayer"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::nbKvHeads"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::pipelineParallelism"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::pipelineParallelism"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::sizePerHead"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::sizePerHead"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::tensorParallelism"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::tensorParallelism"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::tokensPerBlock"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateENSt6vectorI10SizeType32EE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeibiib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::tokensPerBlock"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState10CacheStateE11ModelConfigRKN7runtime11WorldConfigERKNSt6vectorI10SizeType32EEN8nvinfer18DataTypeE13AttentionTypeib", "tensorrt_llm::executor::kv_cache::CacheState::CacheState::worldConfig"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState11ModelConfigE", "tensorrt_llm::executor::kv_cache::CacheState::ModelConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState11ModelConfig18mNbKvHeadsPerLayerE", "tensorrt_llm::executor::kv_cache::CacheState::ModelConfig::mNbKvHeadsPerLayer"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState11ModelConfig12mSizePerHeadE", "tensorrt_llm::executor::kv_cache::CacheState::ModelConfig::mSizePerHead"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState11ModelConfig15mTokensPerBlockE", "tensorrt_llm::executor::kv_cache::CacheState::ModelConfig::mTokensPerBlock"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState11ModelConfigeqERK11ModelConfig", "tensorrt_llm::executor::kv_cache::CacheState::ModelConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState11ModelConfigeqERK11ModelConfig", "tensorrt_llm::executor::kv_cache::CacheState::ModelConfig::operator==::other"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfigE", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfig23mAttentionLayerNumPerPPE", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig::mAttentionLayerNumPerPP"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfig19mContextParallelismE", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig::mContextParallelism"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfig7mDPrankE", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig::mDPrank"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfig7mDPsizeE", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig::mDPsize"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfig18mEnableAttentionDPE", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig::mEnableAttentionDP"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfig20mPipelineParallelismE", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig::mPipelineParallelism"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfig18mTensorParallelismE", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig::mTensorParallelism"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfigeqERK14ParallelConfig", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState14ParallelConfigeqERK14ParallelConfig", "tensorrt_llm::executor::kv_cache::CacheState::ParallelConfig::operator==::other"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState18getAttentionConfigEv", "tensorrt_llm::executor::kv_cache::CacheState::getAttentionConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState11getDataTypeEv", "tensorrt_llm::executor::kv_cache::CacheState::getDataType"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState19getEnableBlockReuseEv", "tensorrt_llm::executor::kv_cache::CacheState::getEnableBlockReuse"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState14getModelConfigEv", "tensorrt_llm::executor::kv_cache::CacheState::getModelConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState17getParallelConfigEv", "tensorrt_llm::executor::kv_cache::CacheState::getParallelConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState16mAttentionConfigE", "tensorrt_llm::executor::kv_cache::CacheState::mAttentionConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState9mDataTypeE", "tensorrt_llm::executor::kv_cache::CacheState::mDataType"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState17mEnableBlockReuseE", "tensorrt_llm::executor::kv_cache::CacheState::mEnableBlockReuse"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState12mModelConfigE", "tensorrt_llm::executor::kv_cache::CacheState::mModelConfig"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10CacheState15mParallelConfigE", "tensorrt_llm::executor::kv_cache::CacheState::mParallelConfig"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheStateeqERKN8kv_cache10CacheStateE", "tensorrt_llm::executor::kv_cache::CacheState::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheStateeqERKN8kv_cache10CacheStateE", "tensorrt_llm::executor::kv_cache::CacheState::operator==::other"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10CacheState8toStringEv", "tensorrt_llm::executor::kv_cache::CacheState::toString"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommStateE", "tensorrt_llm::executor::kv_cache::CommState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt6vectorI10AgentStateEEi", "tensorrt_llm::executor::kv_cache::CommState::CommState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt6vectorI10SizeType32EEi", "tensorrt_llm::executor::kv_cache::CommState::CommState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt6vectorI11SocketStateEEi", "tensorrt_llm::executor::kv_cache::CommState::CommState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt8uint16_tENSt6stringE", "tensorrt_llm::executor::kv_cache::CommState::CommState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateEv", "tensorrt_llm::executor::kv_cache::CommState::CommState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt6vectorI10AgentStateEEi", "tensorrt_llm::executor::kv_cache::CommState::CommState::agentState"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt8uint16_tENSt6stringE", "tensorrt_llm::executor::kv_cache::CommState::CommState::ip"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt8uint16_tENSt6stringE", "tensorrt_llm::executor::kv_cache::CommState::CommState::port"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt6vectorI10SizeType32EEi", "tensorrt_llm::executor::kv_cache::CommState::CommState::ranks"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt6vectorI10AgentStateEEi", "tensorrt_llm::executor::kv_cache::CommState::CommState::selfIdx"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt6vectorI10SizeType32EEi", "tensorrt_llm::executor::kv_cache::CommState::CommState::selfIdx"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt6vectorI11SocketStateEEi", "tensorrt_llm::executor::kv_cache::CommState::CommState::selfIdx"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState9CommStateENSt6vectorI11SocketStateEEi", "tensorrt_llm::executor::kv_cache::CommState::CommState::socketState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommState13getAgentStateEv", "tensorrt_llm::executor::kv_cache::CommState::getAgentState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommState11getMpiStateEv", "tensorrt_llm::executor::kv_cache::CommState::getMpiState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommState10getSelfIdxEv", "tensorrt_llm::executor::kv_cache::CommState::getSelfIdx"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommState14getSocketStateEv", "tensorrt_llm::executor::kv_cache::CommState::getSocketState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommState12isAgentStateEv", "tensorrt_llm::executor::kv_cache::CommState::isAgentState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommState10isMpiStateEv", "tensorrt_llm::executor::kv_cache::CommState::isMpiState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommState13isSocketStateEv", "tensorrt_llm::executor::kv_cache::CommState::isSocketState"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState8mSelfIdxE", "tensorrt_llm::executor::kv_cache::CommState::mSelfIdx"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9CommState6mStateE", "tensorrt_llm::executor::kv_cache::CommState::mState"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommStateeqERK9CommState", "tensorrt_llm::executor::kv_cache::CommState::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommStateeqERK9CommState", "tensorrt_llm::executor::kv_cache::CommState::operator==::other"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9CommState8toStringEv", "tensorrt_llm::executor::kv_cache::CommState::toString"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10ConnectionE", "tensorrt_llm::executor::kv_cache::Connection"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10Connection12isThreadSafeEv", "tensorrt_llm::executor::kv_cache::Connection::isThreadSafe"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10Connection4recvERK11DataContextPv6size_t", "tensorrt_llm::executor::kv_cache::Connection::recv"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10Connection4recvERK11DataContextPv6size_t", "tensorrt_llm::executor::kv_cache::Connection::recv::ctx"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10Connection4recvERK11DataContextPv6size_t", "tensorrt_llm::executor::kv_cache::Connection::recv::data"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10Connection4recvERK11DataContextPv6size_t", "tensorrt_llm::executor::kv_cache::Connection::recv::size"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10Connection4sendERK11DataContextPKv6size_t", "tensorrt_llm::executor::kv_cache::Connection::send"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10Connection4sendERK11DataContextPKv6size_t", "tensorrt_llm::executor::kv_cache::Connection::send::ctx"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10Connection4sendERK11DataContextPKv6size_t", "tensorrt_llm::executor::kv_cache::Connection::send::data"], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10Connection4sendERK11DataContextPKv6size_t", "tensorrt_llm::executor::kv_cache::Connection::send::size"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10ConnectionD0Ev", "tensorrt_llm::executor::kv_cache::Connection::~Connection"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache18ConnectionInfoTypeE", "tensorrt_llm::executor::kv_cache::ConnectionInfoType"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17ConnectionManagerE", "tensorrt_llm::executor::kv_cache::ConnectionManager"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache17ConnectionManager12getCommStateEv", "tensorrt_llm::executor::kv_cache::ConnectionManager::getCommState"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17ConnectionManager14getConnectionsERK9CommState", "tensorrt_llm::executor::kv_cache::ConnectionManager::getConnections"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17ConnectionManager14getConnectionsERK9CommState", "tensorrt_llm::executor::kv_cache::ConnectionManager::getConnections::state"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17ConnectionManager11recvConnectERK11DataContextPv6size_t", "tensorrt_llm::executor::kv_cache::ConnectionManager::recvConnect"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17ConnectionManager11recvConnectERK11DataContextPv6size_t", "tensorrt_llm::executor::kv_cache::ConnectionManager::recvConnect::ctx"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17ConnectionManager11recvConnectERK11DataContextPv6size_t", "tensorrt_llm::executor::kv_cache::ConnectionManager::recvConnect::data"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17ConnectionManager11recvConnectERK11DataContextPv6size_t", "tensorrt_llm::executor::kv_cache::ConnectionManager::recvConnect::size"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache17ConnectionManagerD0Ev", "tensorrt_llm::executor::kv_cache::ConnectionManager::~ConnectionManager"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11DataContextE", "tensorrt_llm::executor::kv_cache::DataContext"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11DataContext11DataContextEi", "tensorrt_llm::executor::kv_cache::DataContext::DataContext"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11DataContext11DataContextEi", "tensorrt_llm::executor::kv_cache::DataContext::DataContext::tag"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache11DataContext6getTagEv", "tensorrt_llm::executor::kv_cache::DataContext::getTag"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11DataContext4mTagE", "tensorrt_llm::executor::kv_cache::DataContext::mTag"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoaderE", "tensorrt_llm::executor::kv_cache::DynLibLoader"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader12DynLibLoaderERK12DynLibLoader", "tensorrt_llm::executor::kv_cache::DynLibLoader::DynLibLoader"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader12DynLibLoaderEv", "tensorrt_llm::executor::kv_cache::DynLibLoader::DynLibLoader"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader5dlSymEPvPKc", "tensorrt_llm::executor::kv_cache::DynLibLoader::dlSym"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader5dlSymEPvPKc", "tensorrt_llm::executor::kv_cache::DynLibLoader::dlSym::handle"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader5dlSymEPvPKc", "tensorrt_llm::executor::kv_cache::DynLibLoader::dlSym::symbol"], [0, 3, 1, "_CPPv4I0EN12tensorrt_llm8executor8kv_cache12DynLibLoader18getFunctionPointerE9FunctionTRKNSt6stringERKNSt6stringE", "tensorrt_llm::executor::kv_cache::DynLibLoader::getFunctionPointer"], [0, 8, 1, "_CPPv4I0EN12tensorrt_llm8executor8kv_cache12DynLibLoader18getFunctionPointerE9FunctionTRKNSt6stringERKNSt6stringE", "tensorrt_llm::executor::kv_cache::DynLibLoader::getFunctionPointer::FunctionT"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor8kv_cache12DynLibLoader18getFunctionPointerE9FunctionTRKNSt6stringERKNSt6stringE", "tensorrt_llm::executor::kv_cache::DynLibLoader::getFunctionPointer::funcName"], [0, 4, 1, "_CPPv4I0EN12tensorrt_llm8executor8kv_cache12DynLibLoader18getFunctionPointerE9FunctionTRKNSt6stringERKNSt6stringE", "tensorrt_llm::executor::kv_cache::DynLibLoader::getFunctionPointer::libName"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader9getHandleERKNSt6stringE", "tensorrt_llm::executor::kv_cache::DynLibLoader::getHandle"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader9getHandleERKNSt6stringE", "tensorrt_llm::executor::kv_cache::DynLibLoader::getHandle::name"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader11getInstanceEv", "tensorrt_llm::executor::kv_cache::DynLibLoader::getInstance"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader9mDllMutexE", "tensorrt_llm::executor::kv_cache::DynLibLoader::mDllMutex"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoader9mHandlersE", "tensorrt_llm::executor::kv_cache::DynLibLoader::mHandlers"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoaderaSERK12DynLibLoader", "tensorrt_llm::executor::kv_cache::DynLibLoader::operator="], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache12DynLibLoaderD0Ev", "tensorrt_llm::executor::kv_cache::DynLibLoader::~DynLibLoader"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDescE", "tensorrt_llm::executor::kv_cache::FileDesc"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc8FileDescERK8FileDesc", "tensorrt_llm::executor::kv_cache::FileDesc::FileDesc"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc8FileDescERKNSt6stringEi6mode_t6size_t", "tensorrt_llm::executor::kv_cache::FileDesc::FileDesc"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc8FileDescERR8FileDesc", "tensorrt_llm::executor::kv_cache::FileDesc::FileDesc"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc8FileDescERKNSt6stringEi6mode_t6size_t", "tensorrt_llm::executor::kv_cache::FileDesc::FileDesc::filename"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc8FileDescERKNSt6stringEi6mode_t6size_t", "tensorrt_llm::executor::kv_cache::FileDesc::FileDesc::flags"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc8FileDescERKNSt6stringEi6mode_t6size_t", "tensorrt_llm::executor::kv_cache::FileDesc::FileDesc::len"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc8FileDescERKNSt6stringEi6mode_t6size_t", "tensorrt_llm::executor::kv_cache::FileDesc::FileDesc::mode"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc8FileDescERR8FileDesc", "tensorrt_llm::executor::kv_cache::FileDesc::FileDesc::other"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc2fdE", "tensorrt_llm::executor::kv_cache::FileDesc::fd"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache8FileDesc5getFdEv", "tensorrt_llm::executor::kv_cache::FileDesc::getFd"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache8FileDesc6getLenEv", "tensorrt_llm::executor::kv_cache::FileDesc::getLen"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDesc4mLenE", "tensorrt_llm::executor::kv_cache::FileDesc::mLen"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDescaSERK8FileDesc", "tensorrt_llm::executor::kv_cache::FileDesc::operator="], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDescaSERR8FileDesc", "tensorrt_llm::executor::kv_cache::FileDesc::operator="], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDescaSERR8FileDesc", "tensorrt_llm::executor::kv_cache::FileDesc::operator=::other"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8FileDescD0Ev", "tensorrt_llm::executor::kv_cache::FileDesc::~FileDesc"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9FileDescsE", "tensorrt_llm::executor::kv_cache::FileDescs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9FileDescs9FileDescsERRNSt6vectorI8FileDescEE", "tensorrt_llm::executor::kv_cache::FileDescs::FileDescs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9FileDescs9FileDescsERRNSt6vectorI8FileDescEE", "tensorrt_llm::executor::kv_cache::FileDescs::FileDescs::descs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache9FileDescs8getDescsEv", "tensorrt_llm::executor::kv_cache::FileDescs::getDescs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache9FileDescs6mDescsE", "tensorrt_llm::executor::kv_cache::FileDescs::mDescs"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDescE", "tensorrt_llm::executor::kv_cache::MemoryDesc"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescE9uintptr_t6size_t8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescEPv6size_t8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescERKNSt6vectorIcEE8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescE9uintptr_t6size_t8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc::addr"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescEPv6size_t8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc::addr"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescE9uintptr_t6size_t8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc::deviceId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescEPv6size_t8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc::deviceId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescERKNSt6vectorIcEE8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc::deviceId"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescE9uintptr_t6size_t8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc::len"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescEPv6size_t8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc::len"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc10MemoryDescERKNSt6vectorIcEE8uint32_t", "tensorrt_llm::executor::kv_cache::MemoryDesc::MemoryDesc::vec"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc11deserializeERNSt7istreamE", "tensorrt_llm::executor::kv_cache::MemoryDesc::deserialize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc11deserializeERNSt7istreamE", "tensorrt_llm::executor::kv_cache::MemoryDesc::deserialize::is"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10MemoryDesc7getAddrEv", "tensorrt_llm::executor::kv_cache::MemoryDesc::getAddr"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10MemoryDesc11getDeviceIdEv", "tensorrt_llm::executor::kv_cache::MemoryDesc::getDeviceId"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache10MemoryDesc6getLenEv", "tensorrt_llm::executor::kv_cache::MemoryDesc::getLen"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc5mAddrE", "tensorrt_llm::executor::kv_cache::MemoryDesc::mAddr"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc9mDeviceIdE", "tensorrt_llm::executor::kv_cache::MemoryDesc::mDeviceId"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc4mLenE", "tensorrt_llm::executor::kv_cache::MemoryDesc::mLen"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc9serializeERK10MemoryDescRNSt7ostreamE", "tensorrt_llm::executor::kv_cache::MemoryDesc::serialize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc9serializeERK10MemoryDescRNSt7ostreamE", "tensorrt_llm::executor::kv_cache::MemoryDesc::serialize::memoryDesc"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc9serializeERK10MemoryDescRNSt7ostreamE", "tensorrt_llm::executor::kv_cache::MemoryDesc::serialize::os"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc14serializedSizeERK10MemoryDesc", "tensorrt_llm::executor::kv_cache::MemoryDesc::serializedSize"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryDesc14serializedSizeERK10MemoryDesc", "tensorrt_llm::executor::kv_cache::MemoryDesc::serializedSize::memoryDesc"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11MemoryDescsE", "tensorrt_llm::executor::kv_cache::MemoryDescs"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11MemoryDescs11MemoryDescsE10MemoryTypeNSt6vectorI10MemoryDescEE", "tensorrt_llm::executor::kv_cache::MemoryDescs::MemoryDescs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11MemoryDescs11MemoryDescsE10MemoryTypeNSt6vectorI10MemoryDescEE", "tensorrt_llm::executor::kv_cache::MemoryDescs::MemoryDescs::descs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11MemoryDescs11MemoryDescsE10MemoryTypeNSt6vectorI10MemoryDescEE", "tensorrt_llm::executor::kv_cache::MemoryDescs::MemoryDescs::type"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache11MemoryDescs8getDescsEv", "tensorrt_llm::executor::kv_cache::MemoryDescs::getDescs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache11MemoryDescs7getTypeEv", "tensorrt_llm::executor::kv_cache::MemoryDescs::getType"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11MemoryDescs6mDescsE", "tensorrt_llm::executor::kv_cache::MemoryDescs::mDescs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11MemoryDescs5mTypeE", "tensorrt_llm::executor::kv_cache::MemoryDescs::mType"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryTypeE", "tensorrt_llm::executor::kv_cache::MemoryType"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryType4kBLKE", "tensorrt_llm::executor::kv_cache::MemoryType::kBLK"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryType5kDRAME", "tensorrt_llm::executor::kv_cache::MemoryType::kDRAM"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryType5kFILEE", "tensorrt_llm::executor::kv_cache::MemoryType::kFILE"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryType4kOBJE", "tensorrt_llm::executor::kv_cache::MemoryType::kOBJ"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10MemoryType5kVRAME", "tensorrt_llm::executor::kv_cache::MemoryType::kVRAM"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8MpiStateE", "tensorrt_llm::executor::kv_cache::MpiState"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache8MpiState6mRanksE", "tensorrt_llm::executor::kv_cache::MpiState::mRanks"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache8MpiStateeqERK8MpiState", "tensorrt_llm::executor::kv_cache::MpiState::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache8MpiStateeqERK8MpiState", "tensorrt_llm::executor::kv_cache::MpiState::operator==::other"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache8MpiState8toStringEv", "tensorrt_llm::executor::kv_cache::MpiState::toString"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache13RegisterDescsE", "tensorrt_llm::executor::kv_cache::RegisterDescs"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11SocketStateE", "tensorrt_llm::executor::kv_cache::SocketState"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11SocketState3mIpE", "tensorrt_llm::executor::kv_cache::SocketState::mIp"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11SocketState5mPortE", "tensorrt_llm::executor::kv_cache::SocketState::mPort"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache11SocketStateeqERK11SocketState", "tensorrt_llm::executor::kv_cache::SocketState::operator=="], [0, 4, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache11SocketStateeqERK11SocketState", "tensorrt_llm::executor::kv_cache::SocketState::operator==::other"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache11SocketState8toStringEv", "tensorrt_llm::executor::kv_cache::SocketState::toString"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache11SyncMessageE", "tensorrt_llm::executor::kv_cache::SyncMessage"], [0, 1, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache13TransferDescsE", "tensorrt_llm::executor::kv_cache::TransferDescs"], [0, 6, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10TransferOpE", "tensorrt_llm::executor::kv_cache::TransferOp"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10TransferOp5kREADE", "tensorrt_llm::executor::kv_cache::TransferOp::kREAD"], [0, 7, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache10TransferOp6kWRITEE", "tensorrt_llm::executor::kv_cache::TransferOp::kWRITE"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequestE", "tensorrt_llm::executor::kv_cache::TransferRequest"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest15TransferRequestE10TransferOp13TransferDescs13TransferDescsRKNSt6stringENSt8optionalI11SyncMessageEE", "tensorrt_llm::executor::kv_cache::TransferRequest::TransferRequest"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest15TransferRequestE10TransferOp13TransferDescs13TransferDescsRKNSt6stringENSt8optionalI11SyncMessageEE", "tensorrt_llm::executor::kv_cache::TransferRequest::TransferRequest::dstDescs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest15TransferRequestE10TransferOp13TransferDescs13TransferDescsRKNSt6stringENSt8optionalI11SyncMessageEE", "tensorrt_llm::executor::kv_cache::TransferRequest::TransferRequest::op"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest15TransferRequestE10TransferOp13TransferDescs13TransferDescsRKNSt6stringENSt8optionalI11SyncMessageEE", "tensorrt_llm::executor::kv_cache::TransferRequest::TransferRequest::remoteName"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest15TransferRequestE10TransferOp13TransferDescs13TransferDescsRKNSt6stringENSt8optionalI11SyncMessageEE", "tensorrt_llm::executor::kv_cache::TransferRequest::TransferRequest::srcDescs"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest15TransferRequestE10TransferOp13TransferDescs13TransferDescsRKNSt6stringENSt8optionalI11SyncMessageEE", "tensorrt_llm::executor::kv_cache::TransferRequest::TransferRequest::syncMessage"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache15TransferRequest11getDstDescsEv", "tensorrt_llm::executor::kv_cache::TransferRequest::getDstDescs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache15TransferRequest5getOpEv", "tensorrt_llm::executor::kv_cache::TransferRequest::getOp"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache15TransferRequest13getRemoteNameEv", "tensorrt_llm::executor::kv_cache::TransferRequest::getRemoteName"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache15TransferRequest11getSrcDescsEv", "tensorrt_llm::executor::kv_cache::TransferRequest::getSrcDescs"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache15TransferRequest14getSyncMessageEv", "tensorrt_llm::executor::kv_cache::TransferRequest::getSyncMessage"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest9mDstDescsE", "tensorrt_llm::executor::kv_cache::TransferRequest::mDstDescs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest3mOpE", "tensorrt_llm::executor::kv_cache::TransferRequest::mOp"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest11mRemoteNameE", "tensorrt_llm::executor::kv_cache::TransferRequest::mRemoteName"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest9mSrcDescsE", "tensorrt_llm::executor::kv_cache::TransferRequest::mSrcDescs"], [0, 5, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache15TransferRequest12mSyncMessageE", "tensorrt_llm::executor::kv_cache::TransferRequest::mSyncMessage"], [0, 2, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache14TransferStatusE", "tensorrt_llm::executor::kv_cache::TransferStatus"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache14TransferStatus11isCompletedEv", "tensorrt_llm::executor::kv_cache::TransferStatus::isCompleted"], [0, 3, 1, "_CPPv4NK12tensorrt_llm8executor8kv_cache14TransferStatus4waitEv", "tensorrt_llm::executor::kv_cache::TransferStatus::wait"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor8kv_cache14TransferStatusD0Ev", "tensorrt_llm::executor::kv_cache::TransferStatus::~TransferStatus"], [0, 3, 1, "_CPPv4IDpEN12tensorrt_llm8executor8kv_cache17makeLoopbackAgentENSt10shared_ptrI17BaseLoopbackAgentEERKNSt6stringEDpRR4Args", "tensorrt_llm::executor::kv_cache::makeLoopbackAgent"], [0, 8, 1, "_CPPv4IDpEN12tensorrt_llm8executor8kv_cache17makeLoopbackAgentENSt10shared_ptrI17BaseLoopbackAgentEERKNSt6stringEDpRR4Args", "tensorrt_llm::executor::kv_cache::makeLoopbackAgent::Args"], [0, 4, 1, "_CPPv4IDpEN12tensorrt_llm8executor8kv_cache17makeLoopbackAgentENSt10shared_ptrI17BaseLoopbackAgentEERKNSt6stringEDpRR4Args", "tensorrt_llm::executor::kv_cache::makeLoopbackAgent::args"], [0, 4, 1, "_CPPv4IDpEN12tensorrt_llm8executor8kv_cache17makeLoopbackAgentENSt10shared_ptrI17BaseLoopbackAgentEERKNSt6stringEDpRR4Args", "tensorrt_llm::executor::kv_cache::makeLoopbackAgent::backend"], [0, 3, 1, "_CPPv4IDpEN12tensorrt_llm8executor8kv_cache17makeTransferAgentENSt10unique_ptrI17BaseTransferAgentEERKNSt6stringEDpRR4Args", "tensorrt_llm::executor::kv_cache::makeTransferAgent"], [0, 8, 1, "_CPPv4IDpEN12tensorrt_llm8executor8kv_cache17makeTransferAgentENSt10unique_ptrI17BaseTransferAgentEERKNSt6stringEDpRR4Args", "tensorrt_llm::executor::kv_cache::makeTransferAgent::Args"], [0, 4, 1, "_CPPv4IDpEN12tensorrt_llm8executor8kv_cache17makeTransferAgentENSt10unique_ptrI17BaseTransferAgentEERKNSt6stringEDpRR4Args", "tensorrt_llm::executor::kv_cache::makeTransferAgent::args"], [0, 4, 1, "_CPPv4IDpEN12tensorrt_llm8executor8kv_cache17makeTransferAgentENSt10unique_ptrI17BaseTransferAgentEERKNSt6stringEDpRR4Args", "tensorrt_llm::executor::kv_cache::makeTransferAgent::backend"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executorlsERNSt7ostreamE21ContextChunkingPolicy", "tensorrt_llm::executor::operator<<"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executorlsERNSt7ostreamE23CapacitySchedulerPolicy", "tensorrt_llm::executor::operator<<"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executorlsERNSt7ostreamE21ContextChunkingPolicy", "tensorrt_llm::executor::operator<<::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executorlsERNSt7ostreamE23CapacitySchedulerPolicy", "tensorrt_llm::executor::operator<<::os"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executorlsERNSt7ostreamE21ContextChunkingPolicy", "tensorrt_llm::executor::operator<<::policy"], [0, 4, 1, "_CPPv4N12tensorrt_llm8executorlsERNSt7ostreamE23CapacitySchedulerPolicy", "tensorrt_llm::executor::operator<<::policy"], [0, 3, 1, "_CPPv4N12tensorrt_llm8executor7versionEv", "tensorrt_llm::executor::version"], [1, 1, 1, "_CPPv4N12tensorrt_llm6layersE", "tensorrt_llm::layers"], [0, 1, 1, "_CPPv4N12tensorrt_llm3mpiE", "tensorrt_llm::mpi"], [0, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [0, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtimeE", "tensorrt_llm::runtime"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffersE", "tensorrt_llm::runtime::AllReduceBuffers"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers16AllReduceBuffersE10SizeType3210SizeType3210SizeType3210SizeType32RK13BufferManagerRK11WorldConfigKb", "tensorrt_llm::runtime::AllReduceBuffers::AllReduceBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers16AllReduceBuffersE10SizeType3210SizeType3210SizeType3210SizeType32RK13BufferManagerRK11WorldConfigKb", "tensorrt_llm::runtime::AllReduceBuffers::AllReduceBuffers::fakeBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers16AllReduceBuffersE10SizeType3210SizeType3210SizeType3210SizeType32RK13BufferManagerRK11WorldConfigKb", "tensorrt_llm::runtime::AllReduceBuffers::AllReduceBuffers::hiddenSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers16AllReduceBuffersE10SizeType3210SizeType3210SizeType3210SizeType32RK13BufferManagerRK11WorldConfigKb", "tensorrt_llm::runtime::AllReduceBuffers::AllReduceBuffers::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers16AllReduceBuffersE10SizeType3210SizeType3210SizeType3210SizeType32RK13BufferManagerRK11WorldConfigKb", "tensorrt_llm::runtime::AllReduceBuffers::AllReduceBuffers::maxBatchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers16AllReduceBuffersE10SizeType3210SizeType3210SizeType3210SizeType32RK13BufferManagerRK11WorldConfigKb", "tensorrt_llm::runtime::AllReduceBuffers::AllReduceBuffers::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers16AllReduceBuffersE10SizeType3210SizeType3210SizeType3210SizeType32RK13BufferManagerRK11WorldConfigKb", "tensorrt_llm::runtime::AllReduceBuffers::AllReduceBuffers::maxSequenceLength"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers16AllReduceBuffersE10SizeType3210SizeType3210SizeType3210SizeType32RK13BufferManagerRK11WorldConfigKb", "tensorrt_llm::runtime::AllReduceBuffers::AllReduceBuffers::worldConfig"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers9TensorPtrE", "tensorrt_llm::runtime::AllReduceBuffers::TensorPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers18mAllReduceCommPtrsE", "tensorrt_llm::runtime::AllReduceBuffers::mAllReduceCommPtrs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers9mFlagPtrsE", "tensorrt_llm::runtime::AllReduceBuffers::mFlagPtrs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime16AllReduceBuffers17mIpcMemoryHandlesE", "tensorrt_llm::runtime::AllReduceBuffers::mIpcMemoryHandles"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime14BufferDataTypeE", "tensorrt_llm::runtime::BufferDataType"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType14BufferDataTypeEN8nvinfer18DataTypeEbb", "tensorrt_llm::runtime::BufferDataType::BufferDataType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType14BufferDataTypeEN8nvinfer18DataTypeEbb", "tensorrt_llm::runtime::BufferDataType::BufferDataType::_unsigned"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType14BufferDataTypeEN8nvinfer18DataTypeEbb", "tensorrt_llm::runtime::BufferDataType::BufferDataType::dataType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType14BufferDataTypeEN8nvinfer18DataTypeEbb", "tensorrt_llm::runtime::BufferDataType::BufferDataType::pointer"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType11getDataTypeEv", "tensorrt_llm::runtime::BufferDataType::getDataType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType7getSizeEv", "tensorrt_llm::runtime::BufferDataType::getSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType13getSizeInBitsEv", "tensorrt_llm::runtime::BufferDataType::getSizeInBits"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType9isPointerEv", "tensorrt_llm::runtime::BufferDataType::isPointer"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataType10isUnsignedEv", "tensorrt_llm::runtime::BufferDataType::isUnsigned"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType15kTrtPointerTypeE", "tensorrt_llm::runtime::BufferDataType::kTrtPointerType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType9mDataTypeE", "tensorrt_llm::runtime::BufferDataType::mDataType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType8mPointerE", "tensorrt_llm::runtime::BufferDataType::mPointer"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14BufferDataType9mUnsignedE", "tensorrt_llm::runtime::BufferDataType::mUnsigned"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14BufferDataTypecvN8nvinfer18DataTypeEEv", "tensorrt_llm::runtime::BufferDataType::operator nvinfer1::DataType"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManagerE", "tensorrt_llm::runtime::BufferManager"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager13BufferManagerE13CudaStreamPtrb", "tensorrt_llm::runtime::BufferManager::BufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager13BufferManagerE13CudaStreamPtrb", "tensorrt_llm::runtime::BufferManager::BufferManager::stream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager13BufferManagerE13CudaStreamPtrb", "tensorrt_llm::runtime::BufferManager::BufferManager::trimPool"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager14CudaMemPoolPtrE", "tensorrt_llm::runtime::BufferManager::CudaMemPoolPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager13CudaStreamPtrE", "tensorrt_llm::runtime::BufferManager::CudaStreamPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10IBufferPtrE", "tensorrt_llm::runtime::BufferManager::IBufferPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10ITensorPtrE", "tensorrt_llm::runtime::BufferManager::ITensorPtr"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::allocate"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeNSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::allocate"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::allocate::dims"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::allocate::memoryType"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeNSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::allocate::memoryType"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeNSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::allocate::size"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::allocate::type"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8allocateE10MemoryTypeNSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::allocate::type"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyEPKvR7IBuffer", "tensorrt_llm::runtime::BufferManager::copy"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyEPKvR7IBuffer10MemoryType", "tensorrt_llm::runtime::BufferManager::copy"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferPv", "tensorrt_llm::runtime::BufferManager::copy"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferPv10MemoryType", "tensorrt_llm::runtime::BufferManager::copy"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferR7IBuffer", "tensorrt_llm::runtime::BufferManager::copy"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyEPKvR7IBuffer", "tensorrt_llm::runtime::BufferManager::copy::dst"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyEPKvR7IBuffer10MemoryType", "tensorrt_llm::runtime::BufferManager::copy::dst"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferPv", "tensorrt_llm::runtime::BufferManager::copy::dst"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferPv10MemoryType", "tensorrt_llm::runtime::BufferManager::copy::dst"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferR7IBuffer", "tensorrt_llm::runtime::BufferManager::copy::dst"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferPv10MemoryType", "tensorrt_llm::runtime::BufferManager::copy::dstType"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyEPKvR7IBuffer", "tensorrt_llm::runtime::BufferManager::copy::src"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyEPKvR7IBuffer10MemoryType", "tensorrt_llm::runtime::BufferManager::copy::src"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferPv", "tensorrt_llm::runtime::BufferManager::copy::src"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferPv10MemoryType", "tensorrt_llm::runtime::BufferManager::copy::src"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyERK7IBufferR7IBuffer", "tensorrt_llm::runtime::BufferManager::copy::src"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager4copyEPKvR7IBuffer10MemoryType", "tensorrt_llm::runtime::BufferManager::copy::srcType"], [1, 3, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10IBufferPtrRKNSt6vectorI1TEE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom"], [1, 3, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrP1TN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom"], [1, 3, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrRKNSt6vectorI1TEEN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8copyFromERK7IBuffer10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8copyFromERK7ITensor10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom"], [1, 8, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10IBufferPtrRKNSt6vectorI1TEE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::T"], [1, 8, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrP1TN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::T"], [1, 8, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrRKNSt6vectorI1TEEN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::T"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrP1TN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::dims"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrRKNSt6vectorI1TEEN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::dims"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10IBufferPtrRKNSt6vectorI1TEE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::memoryType"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrP1TN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::memoryType"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrRKNSt6vectorI1TEEN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::memoryType"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8copyFromERK7IBuffer10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::memoryType"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8copyFromERK7ITensor10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::memoryType"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10IBufferPtrRKNSt6vectorI1TEE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::src"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrP1TN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::src"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime13BufferManager8copyFromE10ITensorPtrRKNSt6vectorI1TEEN8nvinfer14DimsE10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::src"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8copyFromERK7IBuffer10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::src"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager8copyFromERK7ITensor10MemoryType", "tensorrt_llm::runtime::BufferManager::copyFrom::src"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager3cpuEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::cpu"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager3cpuENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::cpu"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager3cpuEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::cpu::dims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager3cpuENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::cpu::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager3cpuEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::cpu::type"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager3cpuENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::cpu::type"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager11emptyBufferE10MemoryTypeN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::emptyBuffer"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager11emptyBufferE10MemoryTypeN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::emptyBuffer::memoryType"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager11emptyBufferE10MemoryTypeN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::emptyBuffer::type"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager11emptyTensorE10MemoryTypeN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::emptyTensor"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager11emptyTensorE10MemoryTypeN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::emptyTensor::memoryType"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager11emptyTensorE10MemoryTypeN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::emptyTensor::type"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager9getStreamEv", "tensorrt_llm::runtime::BufferManager::getStream"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager3gpuEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpu"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager3gpuENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpu"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager3gpuEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpu::dims"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager3gpuENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpu::size"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager3gpuEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpu::type"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager3gpuENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpu::type"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7gpuSyncEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpuSync"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7gpuSyncENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpuSync"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7gpuSyncEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpuSync::dims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7gpuSyncENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpuSync::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7gpuSyncEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpuSync::type"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7gpuSyncENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::gpuSync::type"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7ipcNvlsENSt3setIiEEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::ipcNvls"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7ipcNvlsENSt3setIiEEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::ipcNvls::dims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7ipcNvlsENSt3setIiEEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::ipcNvls::ranks"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7ipcNvlsENSt3setIiEEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::ipcNvls::type"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10kBYTE_TYPEE", "tensorrt_llm::runtime::BufferManager::kBYTE_TYPE"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager5mPoolE", "tensorrt_llm::runtime::BufferManager::mPool"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7mStreamE", "tensorrt_llm::runtime::BufferManager::mStream"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager9mTrimPoolE", "tensorrt_llm::runtime::BufferManager::mTrimPool"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7managedEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::managed"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7managedENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::managed"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7managedEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::managed::dims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7managedENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::managed::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7managedEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::managed::type"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager7managedENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::managed::type"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager14memoryPoolFreeEv", "tensorrt_llm::runtime::BufferManager::memoryPoolFree"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager18memoryPoolReservedEv", "tensorrt_llm::runtime::BufferManager::memoryPoolReserved"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager16memoryPoolTrimToENSt6size_tE", "tensorrt_llm::runtime::BufferManager::memoryPoolTrimTo"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager16memoryPoolTrimToENSt6size_tE", "tensorrt_llm::runtime::BufferManager::memoryPoolTrimTo::size"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager14memoryPoolUsedEv", "tensorrt_llm::runtime::BufferManager::memoryPoolUsed"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager6pinnedEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinned"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager6pinnedENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinned"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager6pinnedEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinned::dims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager6pinnedENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinned::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager6pinnedEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinned::type"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager6pinnedENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinned::type"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10pinnedPoolEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinnedPool"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10pinnedPoolENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinnedPool"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10pinnedPoolEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinnedPool::dims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10pinnedPoolENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinnedPool::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10pinnedPoolEN8nvinfer14DimsEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinnedPool::type"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManager10pinnedPoolENSt6size_tEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::BufferManager::pinnedPool::type"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager6setMemER7IBuffer7int32_t", "tensorrt_llm::runtime::BufferManager::setMem"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager6setMemER7IBuffer7int32_t", "tensorrt_llm::runtime::BufferManager::setMem::buffer"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager6setMemER7IBuffer7int32_t", "tensorrt_llm::runtime::BufferManager::setMem::value"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager7setZeroER7IBuffer", "tensorrt_llm::runtime::BufferManager::setZero"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13BufferManager7setZeroER7IBuffer", "tensorrt_llm::runtime::BufferManager::setZero::buffer"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13BufferManagerD0Ev", "tensorrt_llm::runtime::BufferManager::~BufferManager"], [1, 2, 1, "_CPPv4I0EN12tensorrt_llm7runtime11BufferRangeE", "tensorrt_llm::runtime::BufferRange"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime11BufferRange4BaseE", "tensorrt_llm::runtime::BufferRange::Base"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI1UEEbEEEN12tensorrt_llm7runtime11BufferRange11BufferRangeERK7IBuffer", "tensorrt_llm::runtime::BufferRange::BufferRange"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tIXntNSt10is_const_vI1UEEEbEEEN12tensorrt_llm7runtime11BufferRange11BufferRangeER7IBuffer", "tensorrt_llm::runtime::BufferRange::BufferRange"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11BufferRange11BufferRangeEP1T9size_type", "tensorrt_llm::runtime::BufferRange::BufferRange"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI1UEEbEEEN12tensorrt_llm7runtime11BufferRange11BufferRangeERK7IBuffer", "tensorrt_llm::runtime::BufferRange::BufferRange::U"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tIXntNSt10is_const_vI1UEEEbEEEN12tensorrt_llm7runtime11BufferRange11BufferRangeER7IBuffer", "tensorrt_llm::runtime::BufferRange::BufferRange::U"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI1UEEbEEEN12tensorrt_llm7runtime11BufferRange11BufferRangeERK7IBuffer", "tensorrt_llm::runtime::BufferRange::BufferRange::buffer"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tIXntNSt10is_const_vI1UEEEbEEEN12tensorrt_llm7runtime11BufferRange11BufferRangeER7IBuffer", "tensorrt_llm::runtime::BufferRange::BufferRange::buffer"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11BufferRange11BufferRangeEP1T9size_type", "tensorrt_llm::runtime::BufferRange::BufferRange::data"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11BufferRange11BufferRangeEP1T9size_type", "tensorrt_llm::runtime::BufferRange::BufferRange::size"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime11BufferRangeE", "tensorrt_llm::runtime::BufferRange::T"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunkE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkERK22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::CUDAVirtualMemoryChunk"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkERR10CreatorPtrRR13Configurators", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::CUDAVirtualMemoryChunk"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkERR22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::CUDAVirtualMemoryChunk"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkEv", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::CUDAVirtualMemoryChunk"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkERR10CreatorPtrRR13Configurators", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::CUDAVirtualMemoryChunk::configurators"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkERR10CreatorPtrRR13Configurators", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::CUDAVirtualMemoryChunk::creator"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk22CUDAVirtualMemoryChunkERR22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::CUDAVirtualMemoryChunk::other"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12ConfiguratorE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator12ConfiguratorERK12Configurator", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::Configurator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator12ConfiguratorERR12Configurator", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::Configurator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator12ConfiguratorEv", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::Configurator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12ConfiguratoraSERK12Configurator", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::operator="], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12ConfiguratoraSERR12Configurator", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::operator="], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator5setupE28CUmemGenericAllocationHandle", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::setup"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator5setupE28CUmemGenericAllocationHandle", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::setup::handle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator8teardownE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::teardown"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator8teardownE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::teardown::destructing"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12Configurator8teardownE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::teardown::handle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk12ConfiguratorD0Ev", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurator::~Configurator"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk15ConfiguratorPtrE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::ConfiguratorPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk13ConfiguratorsE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Configurators"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7CreatorE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7CreatorERK7Creator", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::Creator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7CreatorERR7Creator", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::Creator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7CreatorEv", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::Creator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator6createEv", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::create"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7CreatoraSERK7Creator", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::operator="], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7CreatoraSERR7Creator", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::operator="], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7releaseE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::release"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7releaseE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::release::destructing"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7Creator7releaseE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::release::handle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7CreatorD0Ev", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Creator::~Creator"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk10CreatorPtrE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::CreatorPtr"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status7ERROREDE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::ERRORED"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status7INVALIDE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::INVALID"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk13INVALID_STATEE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::INVALID_STATE"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status12MATERIALIZEDE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::MATERIALIZED"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status8RELEASEDE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::RELEASED"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6StatusE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Status"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status7ERROREDE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Status::ERRORED"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status7INVALIDE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Status::INVALID"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status12MATERIALIZEDE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Status::MATERIALIZED"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6Status8RELEASEDE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::Status::RELEASED"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk8_releaseEb", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::_release"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk8_releaseEb", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::_release::destructing"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk14mConfiguratorsE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::mConfigurators"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk8mCreatorE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::mCreator"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7mHandleE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::mHandle"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6mStateE", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::mState"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk11materializeEv", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::materialize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime22CUDAVirtualMemoryChunkcvbEv", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::operator bool"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunkaSERK22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::operator="], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunkaSERR22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::operator="], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunkaSERR22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::operator=::other"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunk7releaseEv", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::release"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime22CUDAVirtualMemoryChunk6statusEv", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::status"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22CUDAVirtualMemoryChunkD0Ev", "tensorrt_llm::runtime::CUDAVirtualMemoryChunk::~CUDAVirtualMemoryChunk"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime15CacheSaltIDTypeE", "tensorrt_llm::runtime::CacheSaltIDType"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEventE", "tensorrt_llm::runtime::CudaEvent"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent9CudaEventE7pointerb", "tensorrt_llm::runtime::CudaEvent::CudaEvent"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent9CudaEventEj", "tensorrt_llm::runtime::CudaEvent::CudaEvent"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent9CudaEventE7pointerb", "tensorrt_llm::runtime::CudaEvent::CudaEvent::event"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent9CudaEventEj", "tensorrt_llm::runtime::CudaEvent::CudaEvent::flags"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent9CudaEventE7pointerb", "tensorrt_llm::runtime::CudaEvent::CudaEvent::ownsEvent"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7DeleterE", "tensorrt_llm::runtime::CudaEvent::Deleter"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7Deleter7DeleterEb", "tensorrt_llm::runtime::CudaEvent::Deleter::Deleter"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7Deleter7DeleterEv", "tensorrt_llm::runtime::CudaEvent::Deleter::Deleter"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7Deleter7DeleterEb", "tensorrt_llm::runtime::CudaEvent::Deleter::Deleter::ownsEvent"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7Deleter10mOwnsEventE", "tensorrt_llm::runtime::CudaEvent::Deleter::mOwnsEvent"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9CudaEvent7DeleterclE7pointer", "tensorrt_llm::runtime::CudaEvent::Deleter::operator()"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9CudaEvent7DeleterclE7pointer", "tensorrt_llm::runtime::CudaEvent::Deleter::operator()::event"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent8EventPtrE", "tensorrt_llm::runtime::CudaEvent::EventPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent12element_typeE", "tensorrt_llm::runtime::CudaEvent::element_type"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9CudaEvent3getEv", "tensorrt_llm::runtime::CudaEvent::get"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent6mEventE", "tensorrt_llm::runtime::CudaEvent::mEvent"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime9CudaEvent7pointerE", "tensorrt_llm::runtime::CudaEvent::pointer"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9CudaEvent11synchronizeEv", "tensorrt_llm::runtime::CudaEvent::synchronize"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStreamE", "tensorrt_llm::runtime::CudaStream"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamE12cudaStream_t", "tensorrt_llm::runtime::CudaStream::CudaStream"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamE12cudaStream_tib", "tensorrt_llm::runtime::CudaStream::CudaStream"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamEji", "tensorrt_llm::runtime::CudaStream::CudaStream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamE12cudaStream_tib", "tensorrt_llm::runtime::CudaStream::CudaStream::device"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamEji", "tensorrt_llm::runtime::CudaStream::CudaStream::flags"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamE12cudaStream_tib", "tensorrt_llm::runtime::CudaStream::CudaStream::ownsStream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamEji", "tensorrt_llm::runtime::CudaStream::CudaStream::priority"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamE12cudaStream_t", "tensorrt_llm::runtime::CudaStream::CudaStream::stream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream10CudaStreamE12cudaStream_tib", "tensorrt_llm::runtime::CudaStream::CudaStream::stream"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7DeleterE", "tensorrt_llm::runtime::CudaStream::Deleter"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7Deleter7DeleterEb", "tensorrt_llm::runtime::CudaStream::Deleter::Deleter"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7Deleter7DeleterEv", "tensorrt_llm::runtime::CudaStream::Deleter::Deleter"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7Deleter7DeleterEb", "tensorrt_llm::runtime::CudaStream::Deleter::Deleter::ownsStream"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7Deleter11mOwnsStreamE", "tensorrt_llm::runtime::CudaStream::Deleter::mOwnsStream"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream7DeleterclE12cudaStream_t", "tensorrt_llm::runtime::CudaStream::Deleter::operator()"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream7DeleterclE12cudaStream_t", "tensorrt_llm::runtime::CudaStream::Deleter::operator()::stream"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream9StreamPtrE", "tensorrt_llm::runtime::CudaStream::StreamPtr"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream3getEv", "tensorrt_llm::runtime::CudaStream::get"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream9getDeviceEv", "tensorrt_llm::runtime::CudaStream::getDevice"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7mDeviceE", "tensorrt_llm::runtime::CudaStream::mDevice"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10CudaStream7mStreamE", "tensorrt_llm::runtime::CudaStream::mStream"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream6recordEN9CudaEvent7pointerE", "tensorrt_llm::runtime::CudaStream::record"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream6recordERK9CudaEvent", "tensorrt_llm::runtime::CudaStream::record"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream6recordEN9CudaEvent7pointerE", "tensorrt_llm::runtime::CudaStream::record::event"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream6recordERK9CudaEvent", "tensorrt_llm::runtime::CudaStream::record::event"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream11synchronizeEv", "tensorrt_llm::runtime::CudaStream::synchronize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream4waitEN9CudaEvent7pointerE", "tensorrt_llm::runtime::CudaStream::wait"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream4waitERK9CudaEvent", "tensorrt_llm::runtime::CudaStream::wait"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream4waitEN9CudaEvent7pointerE", "tensorrt_llm::runtime::CudaStream::wait::event"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10CudaStream4waitERK9CudaEvent", "tensorrt_llm::runtime::CudaStream::wait::event"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocatorE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode3CPUE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::CPU"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13ConfigurationE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtr", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtrb", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtr", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration::backStream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtrb", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration::backStream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtrb", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration::background"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtr", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtrb", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtr", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration::mode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtrb", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration::mode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtr", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration::tag"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration13ConfigurationER24CudaVirtualMemoryManagerNSt6stringE11RestoreMode13CudaStreamPtrb", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::Configuration::tag"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration23backgroundConfigurationE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::backgroundConfiguration"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration11mBackStreamE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::mBackStream"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration11mBackgroundE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::mBackground"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration8mManagerE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::mManager"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration5mModeE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::mMode"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration9mPageSizeE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::mPageSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration4mTagE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::mTag"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration11pageAlignedENSt6size_tE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::pageAligned"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration11pageAlignedENSt6size_tE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::pageAligned::n"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration25setVirtualMemoryAllocatorERKNSt6stringE11RestoreModeNSt10shared_ptrI10CudaStreamEE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::setVirtualMemoryAllocator"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration25setVirtualMemoryAllocatorERKNSt6stringE11RestoreModeNSt10shared_ptrI10CudaStreamEE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::setVirtualMemoryAllocator::backStream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration25setVirtualMemoryAllocatorERKNSt6stringE11RestoreModeNSt10shared_ptrI10CudaStreamEE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::setVirtualMemoryAllocator::mode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13Configuration25setVirtualMemoryAllocatorERKNSt6stringE11RestoreModeNSt10shared_ptrI10CudaStreamEE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Configuration::setVirtualMemoryAllocator::tag"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator13CudaStreamPtrE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::CudaStreamPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator26CudaVirtualMemoryAllocatorENSt10shared_ptrI13ConfigurationEE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::CudaVirtualMemoryAllocator"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator26CudaVirtualMemoryAllocatorENSt10shared_ptrI13ConfigurationEE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::CudaVirtualMemoryAllocator::config"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode6MEMSETE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::MEMSET"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode4NONEE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::NONE"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode6PINNEDE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::PINNED"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator7PointerE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::Pointer"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreModeE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::RestoreMode"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode3CPUE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::RestoreMode::CPU"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode6MEMSETE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::RestoreMode::MEMSET"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode4NONEE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::RestoreMode::NONE"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator11RestoreMode6PINNEDE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::RestoreMode::PINNED"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator8allocateEP7PointerNSt6size_tEi", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::allocate"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator8allocateEP7PointerNSt6size_tEi", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::allocate::device"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator8allocateEP7PointerNSt6size_tEi", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::allocate::n"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator8allocateEP7PointerNSt6size_tEi", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::allocate::ptr"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator10deallocateE7PointerNSt6size_tE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::deallocate"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator10deallocateE7PointerNSt6size_tE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::deallocate::n"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocator10deallocateE7PointerNSt6size_tE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::deallocate::ptr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26CudaVirtualMemoryAllocator7mConfigE", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::mConfig"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26CudaVirtualMemoryAllocatorcvbEv", "tensorrt_llm::runtime::CudaVirtualMemoryAllocator::operator bool"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManagerE", "tensorrt_llm::runtime::CudaVirtualMemoryManager"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager5EntryE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::Entry"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager5Entry8mEntryItE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::Entry::mEntryIt"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager5Entry7mMemoryE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::Entry::mMemory"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager16PointerMemoryMapE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::PointerMemoryMap"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager11TagEntryMapE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::TagEntryMap"], [1, 3, 1, "_CPPv4IDpEN12tensorrt_llm7runtime24CudaVirtualMemoryManager3addEv9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrEDpRR13Configurators", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERR22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrERRN22CUDAVirtualMemoryChunk13ConfiguratorsE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add"], [1, 8, 1, "_CPPv4IDpEN12tensorrt_llm7runtime24CudaVirtualMemoryManager3addEv9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrEDpRR13Configurators", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::Configurators"], [1, 4, 1, "_CPPv4IDpEN12tensorrt_llm7runtime24CudaVirtualMemoryManager3addEv9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrEDpRR13Configurators", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::configurators"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrERRN22CUDAVirtualMemoryChunk13ConfiguratorsE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::configurators"], [1, 4, 1, "_CPPv4IDpEN12tensorrt_llm7runtime24CudaVirtualMemoryManager3addEv9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrEDpRR13Configurators", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::creator"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrERRN22CUDAVirtualMemoryChunk13ConfiguratorsE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::creator"], [1, 4, 1, "_CPPv4IDpEN12tensorrt_llm7runtime24CudaVirtualMemoryManager3addEv9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrEDpRR13Configurators", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::handle"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERR22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::handle"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrERRN22CUDAVirtualMemoryChunk13ConfiguratorsE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::handle"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERR22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::memory"], [1, 4, 1, "_CPPv4IDpEN12tensorrt_llm7runtime24CudaVirtualMemoryManager3addEv9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrEDpRR13Configurators", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::tag"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERR22CUDAVirtualMemoryChunk", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::tag"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager3addE9uintptr_tNSt6stringERRN22CUDAVirtualMemoryChunk10CreatorPtrERRN22CUDAVirtualMemoryChunk13ConfiguratorsE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::add::tag"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager12addBadHandleE9uintptr_t", "tensorrt_llm::runtime::CudaVirtualMemoryManager::addBadHandle"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager12addBadHandleE9uintptr_t", "tensorrt_llm::runtime::CudaVirtualMemoryManager::addBadHandle::handle"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager11mBadHandlesE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::mBadHandles"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager8mEntriesE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::mEntries"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager9mMemoriesE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::mMemories"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager6mMutexE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::mMutex"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager18materializeWithTagERKNSt6stringE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::materializeWithTag"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager18materializeWithTagERKNSt6stringE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::materializeWithTag::tag"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager14releaseWithTagERKNSt6stringE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::releaseWithTag"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager14releaseWithTagERKNSt6stringE", "tensorrt_llm::runtime::CudaVirtualMemoryManager::releaseWithTag::tag"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager6removeE9uintptr_t", "tensorrt_llm::runtime::CudaVirtualMemoryManager::remove"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager6removeE9uintptr_t", "tensorrt_llm::runtime::CudaVirtualMemoryManager::remove::handle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager18retrieveBadHandlesEv", "tensorrt_llm::runtime::CudaVirtualMemoryManager::retrieveBadHandles"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager12unsafeRemoveE9uintptr_t", "tensorrt_llm::runtime::CudaVirtualMemoryManager::unsafeRemove"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24CudaVirtualMemoryManager12unsafeRemoveE9uintptr_t", "tensorrt_llm::runtime::CudaVirtualMemoryManager::unsafeRemove::handle"], [1, 2, 1, "_CPPv4I_N8nvinfer18DataTypeE_b_bEN12tensorrt_llm7runtime14DataTypeTraitsE", "tensorrt_llm::runtime::DataTypeTraits"], [1, 8, 1, "_CPPv4I_N8nvinfer18DataTypeE_b_bEN12tensorrt_llm7runtime14DataTypeTraitsE", "tensorrt_llm::runtime::DataTypeTraits::kDataType"], [1, 8, 1, "_CPPv4I_N8nvinfer18DataTypeE_b_bEN12tensorrt_llm7runtime14DataTypeTraitsE", "tensorrt_llm::runtime::DataTypeTraits::kIsPointer"], [1, 8, 1, "_CPPv4I_N8nvinfer18DataTypeE_b_bEN12tensorrt_llm7runtime14DataTypeTraitsE", "tensorrt_llm::runtime::DataTypeTraits::kIsUnsigned"], [1, 2, 1, "_CPPv4I_N8nvinfer18DataTypeE_bEN12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEEE", "tensorrt_llm::runtime::DataTypeTraits<kDataType, kUnsigned, true>"], [1, 8, 1, "_CPPv4I_N8nvinfer18DataTypeE_bEN12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEEE", "tensorrt_llm::runtime::DataTypeTraits<kDataType, kUnsigned, true>::kDataType"], [1, 8, 1, "_CPPv4I_N8nvinfer18DataTypeE_bEN12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEEE", "tensorrt_llm::runtime::DataTypeTraits<kDataType, kUnsigned, true>::kUnsigned"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEE4nameE", "tensorrt_llm::runtime::DataTypeTraits<kDataType, kUnsigned, true>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<kDataType, kUnsigned, true>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsI9kDataType9kUnsignedXL1EEE4typeE", "tensorrt_llm::runtime::DataTypeTraits<kDataType, kUnsigned, true>::type"], [1, 2, 1, "_CPPv4I_bEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kBOOLE9kUnsignedEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kBOOL, kUnsigned>"], [1, 8, 1, "_CPPv4I_bEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kBOOLE9kUnsignedEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kBOOL, kUnsigned>::kUnsigned"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kBOOLE9kUnsignedE4nameE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kBOOL, kUnsigned>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kBOOLE9kUnsignedE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kBOOL, kUnsigned>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kBOOLE9kUnsignedE4typeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kBOOL, kUnsigned>::type"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kFLOATEEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kFLOAT>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kFLOATEE4nameE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kFLOAT>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kFLOATEE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kFLOAT>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kFLOATEE4typeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kFLOAT>::type"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kHALFEEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kHALF>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kHALFEE4nameE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kHALF>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kHALFEE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kHALF>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kHALFEE4typeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kHALF>::type"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EXL1EEEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT32, true>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EXL1EEE4nameE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT32, true>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EXL1EEE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT32, true>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EXL1EEE4typeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT32, true>::type"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT32>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EE4nameE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT32>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT32>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT32EE4typeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT32>::type"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EXL1EEEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT64, true>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EXL1EEE4nameE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT64, true>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EXL1EEE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT64, true>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EXL1EEE4typeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT64, true>::type"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT64>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EE4nameE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT64>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT64>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kINT64EE4typeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT64>::type"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kINT8EEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT8>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kINT8EE4nameE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT8>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kINT8EE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT8>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType5kINT8EE4typeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kINT8>::type"], [1, 2, 1, "_CPPv4I_bEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kUINT8E9kUnsignedEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kUINT8, kUnsigned>"], [1, 8, 1, "_CPPv4I_bEN12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kUINT8E9kUnsignedEE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kUINT8, kUnsigned>::kUnsigned"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kUINT8E9kUnsignedE4nameE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kUINT8, kUnsigned>::name"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kUINT8E9kUnsignedE4sizeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kUINT8, kUnsigned>::size"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DataTypeTraitsIN8nvinfer18DataType6kUINT8E9kUnsignedE4typeE", "tensorrt_llm::runtime::DataTypeTraits<nvinfer1::DataType::kUINT8, kUnsigned>::type"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInputE", "tensorrt_llm::runtime::DecodingInput"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13DecodingInputEv", "tensorrt_llm::runtime::DecodingInput::DecodingInput"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputsE", "tensorrt_llm::runtime::DecodingInput::EagleInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs12acceptedLensE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::acceptedLens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs15acceptedPathIdsE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::acceptedPathIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs14acceptedTokensE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::acceptedTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs24chunkedContextNextTokensE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::chunkedContextNextTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs13lastDraftLensE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::lastDraftLens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs14lastDraftPathsE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::lastDraftPaths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs15lastDraftTokensE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::lastDraftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs13nextDraftLensE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::nextDraftLens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs14nextDraftPathsE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::nextDraftPaths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs15nextDraftTokensE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::nextDraftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11EagleInputs8seqSlotsE", "tensorrt_llm::runtime::DecodingInput::EagleInputs::seqSlots"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputsE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs15bestPathIndicesE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::bestPathIndices"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs15bestPathLengthsE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::bestPathLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs16lastDraftIndicesE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::lastDraftIndices"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs15lastDraftTokensE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::lastDraftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs21lastGenerationLengthsE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::lastGenerationLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs19lastPositionIdsBaseE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::lastPositionIdsBase"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs5masksE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::masks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs18maxGenLengthDeviceE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::maxGenLengthDevice"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs16nextDraftIndicesE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::nextDraftIndices"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs14nextDraftProbsE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::nextDraftProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs15nextDraftTokensE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::nextDraftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs14nextFlatTokensE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::nextFlatTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs21nextGenerationLengthsE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::nextGenerationLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs17packedPositionIdsE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::packedPositionIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExplicitDraftTokensInputs8seqSlotsE", "tensorrt_llm::runtime::DecodingInput::ExplicitDraftTokensInputs::seqSlots"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputsE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs17constantThresholdE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::constantThreshold"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs11draftLogitsE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::draftLogits"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs15draftLogitsHostE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::draftLogitsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs10draftProbsE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::draftProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs13draftTokenIdsE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::draftTokenIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs17draftTokenIdsHostE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::draftTokenIdsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs14numDraftTokensE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::numDraftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs18numDraftTokensHostE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::numDraftTokensHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs4stepE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::step"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs11targetProbsE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::targetProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs14useDraftLogitsE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::useDraftLogits"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs18useDraftLogitsHostE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::useDraftLogitsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25ExternalDraftTokensInputs28useRandomAcceptanceThresholdE", "tensorrt_llm::runtime::DecodingInput::ExternalDraftTokensInputs::useRandomAcceptanceThreshold"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15LookaheadInputsE", "tensorrt_llm::runtime::DecodingInput::LookaheadInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15LookaheadInputs13tokensPerStepE", "tensorrt_llm::runtime::DecodingInput::LookaheadInputs::tokensPerStep"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputsE", "tensorrt_llm::runtime::DecodingInput::MedusaInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs22medusaCurTokensPerStepE", "tensorrt_llm::runtime::DecodingInput::MedusaInputs::medusaCurTokensPerStep"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs12medusaLogitsE", "tensorrt_llm::runtime::DecodingInput::MedusaInputs::medusaLogits"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs11medusaPathsE", "tensorrt_llm::runtime::DecodingInput::MedusaInputs::medusaPaths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs25medusaTargetTokensPerStepE", "tensorrt_llm::runtime::DecodingInput::MedusaInputs::medusaTargetTokensPerStep"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12MedusaInputs13medusaTreeIdsE", "tensorrt_llm::runtime::DecodingInput::MedusaInputs::medusaTreeIds"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput14TensorConstPtrE", "tensorrt_llm::runtime::DecodingInput::TensorConstPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput9TensorPtrE", "tensorrt_llm::runtime::DecodingInput::TensorPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12badWordsLensE", "tensorrt_llm::runtime::DecodingInput::badWordsLens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13badWordsListsE", "tensorrt_llm::runtime::DecodingInput::badWordsLists"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12badWordsPtrsE", "tensorrt_llm::runtime::DecodingInput::badWordsPtrs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput9batchSizeE", "tensorrt_llm::runtime::DecodingInput::batchSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput10batchSlotsE", "tensorrt_llm::runtime::DecodingInput::batchSlots"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput10beamWidthsE", "tensorrt_llm::runtime::DecodingInput::beamWidths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput16cacheIndirectionE", "tensorrt_llm::runtime::DecodingInput::cacheIndirection"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput11eagleInputsE", "tensorrt_llm::runtime::DecodingInput::eagleInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13embeddingBiasE", "tensorrt_llm::runtime::DecodingInput::embeddingBias"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput6endIdsE", "tensorrt_llm::runtime::DecodingInput::endIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25explicitDraftTokensInputsE", "tensorrt_llm::runtime::DecodingInput::explicitDraftTokensInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput25externalDraftTokensInputsE", "tensorrt_llm::runtime::DecodingInput::externalDraftTokensInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13finishReasonsE", "tensorrt_llm::runtime::DecodingInput::finishReasons"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15generationStepsE", "tensorrt_llm::runtime::DecodingInput::generationSteps"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput7lengthsE", "tensorrt_llm::runtime::DecodingInput::lengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput9logitsVecE", "tensorrt_llm::runtime::DecodingInput::logitsVec"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15lookaheadInputsE", "tensorrt_llm::runtime::DecodingInput::lookaheadInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput18maxAttentionWindowE", "tensorrt_llm::runtime::DecodingInput::maxAttentionWindow"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput14maxBadWordsLenE", "tensorrt_llm::runtime::DecodingInput::maxBadWordsLen"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput9maxLengthE", "tensorrt_llm::runtime::DecodingInput::maxLength"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15maxStopWordsLenE", "tensorrt_llm::runtime::DecodingInput::maxStopWordsLen"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput12medusaInputsE", "tensorrt_llm::runtime::DecodingInput::medusaInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput17noRepeatNgramSizeE", "tensorrt_llm::runtime::DecodingInput::noRepeatNgramSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput19sequenceLimitLengthE", "tensorrt_llm::runtime::DecodingInput::sequenceLimitLength"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput15sinkTokenLengthE", "tensorrt_llm::runtime::DecodingInput::sinkTokenLength"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput4stepE", "tensorrt_llm::runtime::DecodingInput::step"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13stopWordsLensE", "tensorrt_llm::runtime::DecodingInput::stopWordsLens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput14stopWordsListsE", "tensorrt_llm::runtime::DecodingInput::stopWordsLists"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13DecodingInput13stopWordsPtrsE", "tensorrt_llm::runtime::DecodingInput::stopWordsPtrs"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutputE", "tensorrt_llm::runtime::DecodingOutput"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypothesesE", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses10batchDonesE", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::batchDones"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses14cumLogProbsCBAE", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::cumLogProbsCBA"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses5emptyERK13BufferManager", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::empty"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses5emptyERK13BufferManager", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::empty::manager"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses4initERK13BufferManager11TokenIdType", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::init"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses4initERK13BufferManager11TokenIdType", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::init::endId"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses4initERK13BufferManager11TokenIdType", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::init::manager"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses11logProbsCBAE", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::logProbsCBA"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses18minNormedScoresCBAE", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::minNormedScoresCBA"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses15normedScoresCBAE", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::normedScoresCBA"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses11numBeamsCBAE", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::numBeamsCBA"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses12outputIdsCBAE", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::outputIdsCBA"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses7releaseEv", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::release"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses7reshapeE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::reshape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses7reshapeE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::reshape::batchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses7reshapeE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::reshape::beamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses7reshapeE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::reshape::maxSequenceLength"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses18sequenceLengthsCBAE", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::sequenceLengthsCBA"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses5sliceE10SizeType3210SizeType32", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::slice"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses5sliceE10SizeType3210SizeType32", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::slice::batchIndex"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime14DecodingOutput14BeamHypotheses5sliceE10SizeType3210SizeType32", "tensorrt_llm::runtime::DecodingOutput::BeamHypotheses::slice::size"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14DecodingOutputEv", "tensorrt_llm::runtime::DecodingOutput::DecodingOutput"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputsE", "tensorrt_llm::runtime::DecodingOutput::SpeculativeDecodingOutputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs21acceptedLengthsCumSumE", "tensorrt_llm::runtime::DecodingOutput::SpeculativeDecodingOutputs::acceptedLengthsCumSum"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs17acceptedTokensLenE", "tensorrt_llm::runtime::DecodingOutput::SpeculativeDecodingOutputs::acceptedTokensLen"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs15nextDraftTokensE", "tensorrt_llm::runtime::DecodingOutput::SpeculativeDecodingOutputs::nextDraftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs18nextDraftTokensLenE", "tensorrt_llm::runtime::DecodingOutput::SpeculativeDecodingOutputs::nextDraftTokensLen"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs12pathsOffsetsE", "tensorrt_llm::runtime::DecodingOutput::SpeculativeDecodingOutputs::pathsOffsets"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26SpeculativeDecodingOutputs18prevDraftTokensLenE", "tensorrt_llm::runtime::DecodingOutput::SpeculativeDecodingOutputs::prevDraftTokensLen"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput9TensorPtrE", "tensorrt_llm::runtime::DecodingOutput::TensorPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14beamHypothesesE", "tensorrt_llm::runtime::DecodingOutput::beamHypotheses"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput16cacheIndirectionE", "tensorrt_llm::runtime::DecodingOutput::cacheIndirection"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput11cumLogProbsE", "tensorrt_llm::runtime::DecodingOutput::cumLogProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput12eagleBuffersE", "tensorrt_llm::runtime::DecodingOutput::eagleBuffers"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26explicitDraftTokensBuffersE", "tensorrt_llm::runtime::DecodingOutput::explicitDraftTokensBuffers"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput13finishReasonsE", "tensorrt_llm::runtime::DecodingOutput::finishReasons"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput11finishedSumE", "tensorrt_llm::runtime::DecodingOutput::finishedSum"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput11gatheredIdsE", "tensorrt_llm::runtime::DecodingOutput::gatheredIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput3idsE", "tensorrt_llm::runtime::DecodingOutput::ids"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput17kNegativeInfinityE", "tensorrt_llm::runtime::DecodingOutput::kNegativeInfinity"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput7lengthsE", "tensorrt_llm::runtime::DecodingOutput::lengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput8logProbsE", "tensorrt_llm::runtime::DecodingOutput::logProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput13logProbsTiledE", "tensorrt_llm::runtime::DecodingOutput::logProbsTiled"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput16lookaheadOutputsE", "tensorrt_llm::runtime::DecodingOutput::lookaheadOutputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput9newTokensE", "tensorrt_llm::runtime::DecodingOutput::newTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput14newTokensStepsE", "tensorrt_llm::runtime::DecodingOutput::newTokensSteps"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput12newTokensVecE", "tensorrt_llm::runtime::DecodingOutput::newTokensVec"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput9parentIdsE", "tensorrt_llm::runtime::DecodingOutput::parentIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14DecodingOutput26speculativeDecodingOutputsE", "tensorrt_llm::runtime::DecodingOutput::speculativeDecodingOutputs"], [1, 2, 1, "_CPPv4I0EN12tensorrt_llm7runtime20DeviceAllocationNvlsE", "tensorrt_llm::runtime::DeviceAllocationNvls"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls20DeviceAllocationNvlsEv", "tensorrt_llm::runtime::DeviceAllocationNvls::DeviceAllocationNvls"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime20DeviceAllocationNvlsE", "tensorrt_llm::runtime::DeviceAllocationNvls::T"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls9_capacityE", "tensorrt_llm::runtime::DeviceAllocationNvls::_capacity"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls7_handleE", "tensorrt_llm::runtime::DeviceAllocationNvls::_handle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls4freeEv", "tensorrt_llm::runtime::DeviceAllocationNvls::free"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime20DeviceAllocationNvls11getCapacityEv", "tensorrt_llm::runtime::DeviceAllocationNvls::getCapacity"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls21getIpcUnicastPointersEv", "tensorrt_llm::runtime::DeviceAllocationNvls::getIpcUnicastPointers"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime20DeviceAllocationNvls19getMulticastPointerEv", "tensorrt_llm::runtime::DeviceAllocationNvls::getMulticastPointer"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime20DeviceAllocationNvls17getUnicastPointerEv", "tensorrt_llm::runtime::DeviceAllocationNvls::getUnicastPointer"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls5resetE6size_tNSt3setIiEE", "tensorrt_llm::runtime::DeviceAllocationNvls::reset"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls5resetE6size_tNSt3setIiEE", "tensorrt_llm::runtime::DeviceAllocationNvls::reset::ranks"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvls5resetE6size_tNSt3setIiEE", "tensorrt_llm::runtime::DeviceAllocationNvls::reset::size"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20DeviceAllocationNvlsD0Ev", "tensorrt_llm::runtime::DeviceAllocationNvls::~DeviceAllocationNvls"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffersE", "tensorrt_llm::runtime::EagleBuffers"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers9BufferPtrE", "tensorrt_llm::runtime::EagleBuffers::BufferPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12EagleBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN8executor14DecodingConfigE", "tensorrt_llm::runtime::EagleBuffers::EagleBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12EagleBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN8executor14DecodingConfigE", "tensorrt_llm::runtime::EagleBuffers::EagleBuffers::decodingConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12EagleBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN8executor14DecodingConfigE", "tensorrt_llm::runtime::EagleBuffers::EagleBuffers::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12EagleBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN8executor14DecodingConfigE", "tensorrt_llm::runtime::EagleBuffers::EagleBuffers::maxBatchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12EagleBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN8executor14DecodingConfigE", "tensorrt_llm::runtime::EagleBuffers::EagleBuffers::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12EagleBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN8executor14DecodingConfigE", "tensorrt_llm::runtime::EagleBuffers::EagleBuffers::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12EagleBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN8executor14DecodingConfigE", "tensorrt_llm::runtime::EagleBuffers::EagleBuffers::worldConfig"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputsE", "tensorrt_llm::runtime::EagleBuffers::EngineOutputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs12acceptedLensE", "tensorrt_llm::runtime::EagleBuffers::EngineOutputs::acceptedLens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs13acceptedPathsE", "tensorrt_llm::runtime::EagleBuffers::EngineOutputs::acceptedPaths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs14acceptedTokensE", "tensorrt_llm::runtime::EagleBuffers::EngineOutputs::acceptedTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs24chunkedContextNextTokensE", "tensorrt_llm::runtime::EagleBuffers::EngineOutputs::chunkedContextNextTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs13nextDraftLensE", "tensorrt_llm::runtime::EagleBuffers::EngineOutputs::nextDraftLens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs14nextDraftPathsE", "tensorrt_llm::runtime::EagleBuffers::EngineOutputs::nextDraftPaths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13EngineOutputs15nextDraftTokensE", "tensorrt_llm::runtime::EagleBuffers::EngineOutputs::nextDraftTokens"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers7ITensorE", "tensorrt_llm::runtime::EagleBuffers::ITensor"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6InputsE", "tensorrt_llm::runtime::EagleBuffers::Inputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs22allLayersDraftTokenIdsE", "tensorrt_llm::runtime::EagleBuffers::Inputs::allLayersDraftTokenIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs33allLayersDraftTokenIdsPredecessorE", "tensorrt_llm::runtime::EagleBuffers::Inputs::allLayersDraftTokenIdsPredecessor"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs15allLayersScoresE", "tensorrt_llm::runtime::EagleBuffers::Inputs::allLayersScores"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs24chunkedContextNextTokensE", "tensorrt_llm::runtime::EagleBuffers::Inputs::chunkedContextNextTokens"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs6createE10SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::EagleBuffers::Inputs::create"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs6createE10SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::EagleBuffers::Inputs::create::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs6createE10SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::EagleBuffers::Inputs::create::maxNumSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs6createE10SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::EagleBuffers::Inputs::create::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs6createE10SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::EagleBuffers::Inputs::create::worldConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs20currentExpandIndicesE", "tensorrt_llm::runtime::EagleBuffers::Inputs::currentExpandIndices"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs9draftLensE", "tensorrt_llm::runtime::EagleBuffers::Inputs::draftLens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs10draftPathsE", "tensorrt_llm::runtime::EagleBuffers::Inputs::draftPaths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs14draftPathsHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::draftPathsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs11draftTokensE", "tensorrt_llm::runtime::EagleBuffers::Inputs::draftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs22dynamicTreeMaxTopKHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::dynamicTreeMaxTopKHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs29eagleNetCtxContextLengthsHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::eagleNetCtxContextLengthsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs34eagleNetCtxPastKeyValueLengthsHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::eagleNetCtxPastKeyValueLengthsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs27eagleNetCtxRequestTypesHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::eagleNetCtxRequestTypesHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs29eagleNetGenContextLengthsHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::eagleNetGenContextLengthsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs34eagleNetGenPastKeyValueLengthsHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::eagleNetGenPastKeyValueLengthsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs27eagleNetGenRequestTypesHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::eagleNetGenRequestTypesHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs18inputGenTokensHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::inputGenTokensHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs14posteriorAlphaE", "tensorrt_llm::runtime::EagleBuffers::Inputs::posteriorAlpha"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs18posteriorThresholdE", "tensorrt_llm::runtime::EagleBuffers::Inputs::posteriorThreshold"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs10prevScoresE", "tensorrt_llm::runtime::EagleBuffers::Inputs::prevScores"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs16randomDataSampleE", "tensorrt_llm::runtime::EagleBuffers::Inputs::randomDataSample"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs20randomDataValidationE", "tensorrt_llm::runtime::EagleBuffers::Inputs::randomDataValidation"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs29specDecodingGenerationLengthsE", "tensorrt_llm::runtime::EagleBuffers::Inputs::specDecodingGenerationLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs33specDecodingGenerationLengthsHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::specDecodingGenerationLengthsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs23specDecodingPackedMasksE", "tensorrt_llm::runtime::EagleBuffers::Inputs::specDecodingPackedMasks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs27specDecodingPositionOffsetsE", "tensorrt_llm::runtime::EagleBuffers::Inputs::specDecodingPositionOffsets"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs12temperaturesE", "tensorrt_llm::runtime::EagleBuffers::Inputs::temperatures"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs18useDynamicTreeHostE", "tensorrt_llm::runtime::EagleBuffers::Inputs::useDynamicTreeHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers6Inputs15useSpecDecodingE", "tensorrt_llm::runtime::EagleBuffers::Inputs::useSpecDecoding"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13LlmRequestPtrE", "tensorrt_llm::runtime::EagleBuffers::LlmRequestPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13RequestVectorE", "tensorrt_llm::runtime::EagleBuffers::RequestVector"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers10SizeType32E", "tensorrt_llm::runtime::EagleBuffers::SizeType32"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers9TensorMapE", "tensorrt_llm::runtime::EagleBuffers::TensorMap"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers9TensorPtrE", "tensorrt_llm::runtime::EagleBuffers::TensorPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers28chunkedContextNextTokensHostE", "tensorrt_llm::runtime::EagleBuffers::chunkedContextNextTokensHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers23cumSumGenerationLengthsE", "tensorrt_llm::runtime::EagleBuffers::cumSumGenerationLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers12engineInputsE", "tensorrt_llm::runtime::EagleBuffers::engineInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers13engineOutputsE", "tensorrt_llm::runtime::EagleBuffers::engineOutputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers18greedySamplingHostE", "tensorrt_llm::runtime::EagleBuffers::greedySamplingHost"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::insertInputTensors"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::insertInputTensors::inputBuffers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::insertInputTensors::outputBuffers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::insertInputTensors::worldConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers26mDefaultPosteriorThresholdE", "tensorrt_llm::runtime::EagleBuffers::mDefaultPosteriorThreshold"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers17mDoGreedySamplingE", "tensorrt_llm::runtime::EagleBuffers::mDoGreedySampling"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers19maxGenerationLengthE", "tensorrt_llm::runtime::EagleBuffers::maxGenerationLength"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers18posteriorAlphaHostE", "tensorrt_llm::runtime::EagleBuffers::posteriorAlphaHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers22posteriorThresholdHostE", "tensorrt_llm::runtime::EagleBuffers::posteriorThresholdHost"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", "tensorrt_llm::runtime::EagleBuffers::reshape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", "tensorrt_llm::runtime::EagleBuffers::reshape::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", "tensorrt_llm::runtime::EagleBuffers::reshape::numCtxSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", "tensorrt_llm::runtime::EagleBuffers::reshape::numGenSequences"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers21scanReduceTempStorageE", "tensorrt_llm::runtime::EagleBuffers::scanReduceTempStorage"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12EagleBuffers26scanReduceTempStorageBytesE", "tensorrt_llm::runtime::EagleBuffers::scanReduceTempStorageBytes"], [1, 3, 1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs"], [1, 8, 1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::T"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::contextRequests"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::contextRequests"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::decoderBuffers"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::draftBuffers"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::eagleModule"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::genRequests"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::genRequests"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::manager"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::manager"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::modelConfig"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::requestTypes"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::seqSlots"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::seqSlots"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime12EagleBuffers13setFromInputsEvRK13RequestVectorRK13RequestVector10SizeType32RK7ITensorRKN12EagleBuffers6InputsERKN7runtime11EagleModuleERKN7runtime13BufferManagerE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::vocabSizePadded"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime12EagleBuffers13setFromInputsERK13RequestVectorRK13RequestVectorRKN7runtime7ITensorERK7ITensorRKN12EagleBuffers6InputsERKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::EagleBuffers::setFromInputs::worldConfig"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModuleE", "tensorrt_llm::runtime::EagleModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModule11EagleModuleE10SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::EagleModule::EagleModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModule11EagleModuleEv", "tensorrt_llm::runtime::EagleModule::EagleModule"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModule11EagleModuleE10SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::EagleModule::EagleModule::maxDecodingDraftTokens"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModule11EagleModuleE10SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::EagleModule::EagleModule::maxDraftPathLen"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModule11EagleModuleE10SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::EagleModule::EagleModule::maxNonLeafNodesPerLayer"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModule11EagleModuleE10SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::EagleModule::EagleModule::numTransformersLayer"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11EagleModule22getDefaultEagleChoicesEv", "tensorrt_llm::runtime::EagleModule::getDefaultEagleChoices"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11EagleModule26getMaxNonLeafNodesPerLayerEv", "tensorrt_llm::runtime::EagleModule::getMaxNonLeafNodesPerLayer"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11EagleModule23getNumTransformerLayersEv", "tensorrt_llm::runtime::EagleModule::getNumTransformerLayers"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModule20mDefaultEagleChoicesE", "tensorrt_llm::runtime::EagleModule::mDefaultEagleChoices"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModule24mMaxNonLeafNodesPerLayerE", "tensorrt_llm::runtime::EagleModule::mMaxNonLeafNodesPerLayer"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11EagleModule21mNumTransformersLayerE", "tensorrt_llm::runtime::EagleModule::mNumTransformersLayer"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffersE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers9BufferPtrE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::BufferPtr"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers12EngineInputsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers12EngineInputs15positionOffsetsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineInputs::positionOffsets"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers12EngineInputs18requestTypesDeviceE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineInputs::requestTypesDevice"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs15bestPathIndicesE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::bestPathIndices"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs15bestPathLengthsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::bestPathLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs5masksE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::masks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs11maxGenTokenE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::maxGenToken"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs16nextDraftIndicesE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::nextDraftIndices"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs14nextDraftProbsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::nextDraftProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs15nextDraftTokensE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::nextDraftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs14nextFlatTokensE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::nextFlatTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs21nextGenerationLengthsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::nextGenerationLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs19nextPositionOffsetsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::nextPositionOffsets"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs17packedPositionIdsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::packedPositionIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13EngineOutputs13totalGenTokenE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::EngineOutputs::totalGenToken"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers26ExplicitDraftTokensBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::ExplicitDraftTokensBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers26ExplicitDraftTokensBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::ExplicitDraftTokensBuffers::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers26ExplicitDraftTokensBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::ExplicitDraftTokensBuffers::maxBatchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers26ExplicitDraftTokensBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::ExplicitDraftTokensBuffers::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers26ExplicitDraftTokensBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::ExplicitDraftTokensBuffers::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers26ExplicitDraftTokensBuffersE10SizeType3210SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::ExplicitDraftTokensBuffers::worldConfig"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers7ITensorE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::ITensor"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6InputsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs6createE10SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::create"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs6createE10SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::create::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs6createE10SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::create::maxNumSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs6createE10SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::create::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs6createE10SizeType32RKN7runtime13BufferManagerERKN7runtime11ModelConfigERKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::create::worldConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs12draftIndicesE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::draftIndices"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs10draftProbsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::draftProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs11draftTokensE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::draftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs17generationLengthsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::generationLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs21generationLengthsHostE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::generationLengthsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs16maxGenLengthHostE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::maxGenLengthHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs11packedMasksE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::packedMasks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs11positionIdsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::positionIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs15positionIdsBaseE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::positionIdsBase"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs16randomDataSampleE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::randomDataSample"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs20randomDataValidationE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::randomDataValidation"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs12temperaturesE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::temperatures"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers6Inputs15useSpecDecodingE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::Inputs::useSpecDecoding"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers10SizeType32E", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::SizeType32"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers9TensorMapE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::TensorMap"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers9TensorPtrE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::TensorPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers23cumSumGenerationLengthsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::cumSumGenerationLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers12engineInputsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::engineInputs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13engineOutputsE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::engineOutputs"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::insertInputTensors"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::insertInputTensors::inputBuffers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::insertInputTensors::outputBuffers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers18insertInputTensorsER9TensorMapR9TensorMapRKN7runtime11WorldConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::insertInputTensors::worldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::reshape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::reshape::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::reshape::numCtxSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers7reshapeE10SizeType3210SizeType32RKN7runtime11ModelConfigE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::reshape::numGenSequences"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers15scanTempStorageE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::scanTempStorage"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26ExplicitDraftTokensBuffers20scanTempStorageBytesE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::scanTempStorageBytes"], [1, 3, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs"], [1, 8, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::T"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::contextPositionIds"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::contextPositionIds"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::decoderBuffers"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::draftBuffers"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::explicitDraftTokensModule"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::manager"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::modelConfig"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::numCtxSequences"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::numCtxSequences"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::numGenSequences"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::numGenSequences"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::requestTypes"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::seqSlots"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::seqSlots"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::stream"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::stream"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsEv10SizeType3210SizeType3210SizeType32RK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime25ExplicitDraftTokensModuleERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::vocabSizePadded"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime26ExplicitDraftTokensBuffers13setFromInputsE10SizeType3210SizeType32RKN7runtime7ITensorERK7ITensorRKN26ExplicitDraftTokensBuffers6InputsERK7ITensorRKN7runtime11ModelConfigERKN7runtime11WorldConfigERKN7runtime13BufferManagerERKN7runtime10CudaStreamE", "tensorrt_llm::runtime::ExplicitDraftTokensBuffers::setFromInputs::worldConfig"], [1, 2, 1, "_CPPv4I0EN12tensorrt_llm7runtime25GenericPromptTuningParamsE", "tensorrt_llm::runtime::GenericPromptTuningParams"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams25GenericPromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", "tensorrt_llm::runtime::GenericPromptTuningParams::GenericPromptTuningParams"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams25GenericPromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", "tensorrt_llm::runtime::GenericPromptTuningParams::GenericPromptTuningParams::embeddingTable"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams25GenericPromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", "tensorrt_llm::runtime::GenericPromptTuningParams::GenericPromptTuningParams::tasks"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams25GenericPromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", "tensorrt_llm::runtime::GenericPromptTuningParams::GenericPromptTuningParams::vocabSize"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams10SizeType32E", "tensorrt_llm::runtime::GenericPromptTuningParams::SizeType32"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime25GenericPromptTuningParamsE", "tensorrt_llm::runtime::GenericPromptTuningParams::TTensor"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams9TensorPtrE", "tensorrt_llm::runtime::GenericPromptTuningParams::TensorPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams14embeddingTableE", "tensorrt_llm::runtime::GenericPromptTuningParams::embeddingTable"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams19promptTuningEnabledE", "tensorrt_llm::runtime::GenericPromptTuningParams::promptTuningEnabled"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams5tasksE", "tensorrt_llm::runtime::GenericPromptTuningParams::tasks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime25GenericPromptTuningParams9vocabSizeE", "tensorrt_llm::runtime::GenericPromptTuningParams::vocabSize"], [1, 2, 1, "_CPPv4I0EN12tensorrt_llm7runtime10GptDecoderE", "tensorrt_llm::runtime::GptDecoder"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder13CudaStreamPtrE", "tensorrt_llm::runtime::GptDecoder::CudaStreamPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10GptDecoderERKN8executor12DecodingModeE6size_t6size_t6size_t6size_tRK13CudaStreamPtrNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::GptDecoder::GptDecoder"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10GptDecoderERKN8executor12DecodingModeE6size_t6size_t6size_t6size_tRK13CudaStreamPtrNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::GptDecoder::GptDecoder::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10GptDecoderERKN8executor12DecodingModeE6size_t6size_t6size_t6size_tRK13CudaStreamPtrNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::GptDecoder::GptDecoder::maxNumSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10GptDecoderERKN8executor12DecodingModeE6size_t6size_t6size_t6size_tRK13CudaStreamPtrNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::GptDecoder::GptDecoder::mode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10GptDecoderERKN8executor12DecodingModeE6size_t6size_t6size_t6size_tRK13CudaStreamPtrNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::GptDecoder::GptDecoder::speculativeDecodingModule"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10GptDecoderERKN8executor12DecodingModeE6size_t6size_t6size_t6size_tRK13CudaStreamPtrNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::GptDecoder::GptDecoder::stream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10GptDecoderERKN8executor12DecodingModeE6size_t6size_t6size_t6size_tRK13CudaStreamPtrNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::GptDecoder::GptDecoder::vocabSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10GptDecoderERKN8executor12DecodingModeE6size_t6size_t6size_t6size_tRK13CudaStreamPtrNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::GptDecoder::GptDecoder::vocabSizePadded"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime10GptDecoderE", "tensorrt_llm::runtime::GptDecoder::T"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder9TensorPtrE", "tensorrt_llm::runtime::GptDecoder::TensorPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", "tensorrt_llm::runtime::GptDecoder::disableLookahead"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", "tensorrt_llm::runtime::GptDecoder::disableLookahead::batchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", "tensorrt_llm::runtime::GptDecoder::disableLookahead::batchSlots"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", "tensorrt_llm::runtime::GptDecoder::disableLookahead::samplingConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder12forwardAsyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::GptDecoder::forwardAsync"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder12forwardAsyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::GptDecoder::forwardAsync::input"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder12forwardAsyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::GptDecoder::forwardAsync::output"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder11forwardSyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::GptDecoder::forwardSync"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder11forwardSyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::GptDecoder::forwardSync::input"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder11forwardSyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::GptDecoder::forwardSync::output"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder17getSamplingConfigEv", "tensorrt_llm::runtime::GptDecoder::getSamplingConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder23mDecodingLayerWorkspaceE", "tensorrt_llm::runtime::GptDecoder::mDecodingLayerWorkspace"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder13mDecodingModeE", "tensorrt_llm::runtime::GptDecoder::mDecodingMode"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder19mDynamicDecodeLayerE", "tensorrt_llm::runtime::GptDecoder::mDynamicDecodeLayer"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder8mManagerE", "tensorrt_llm::runtime::GptDecoder::mManager"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder16mMaxNumSequencesE", "tensorrt_llm::runtime::GptDecoder::mMaxNumSequences"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder15mSamplingConfigE", "tensorrt_llm::runtime::GptDecoder::mSamplingConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder10mVocabSizeE", "tensorrt_llm::runtime::GptDecoder::mVocabSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder16mVocabSizePaddedE", "tensorrt_llm::runtime::GptDecoder::mVocabSizePadded"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::GptDecoder::setup"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::GptDecoder::setup::batchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::GptDecoder::setup::batchSlots"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::GptDecoder::setup::explicitDraftTokensDType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::GptDecoder::setup::lookaheadAlgoConfigs"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::GptDecoder::setup::lookaheadPrompt"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::GptDecoder::setup::output"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10GptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::GptDecoder::setup::samplingConfig"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatchedE", "tensorrt_llm::runtime::GptDecoderBatched"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched13CudaStreamPtrE", "tensorrt_llm::runtime::GptDecoderBatched::CudaStreamPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched17GptDecoderBatchedE13CudaStreamPtr", "tensorrt_llm::runtime::GptDecoderBatched::GptDecoderBatched"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched17GptDecoderBatchedE13CudaStreamPtr", "tensorrt_llm::runtime::GptDecoderBatched::GptDecoderBatched::stream"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched13GptDecoderPtrE", "tensorrt_llm::runtime::GptDecoderBatched::GptDecoderPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched13LlmRequestPtrE", "tensorrt_llm::runtime::GptDecoderBatched::LlmRequestPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched13RequestVectorE", "tensorrt_llm::runtime::GptDecoderBatched::RequestVector"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched9TensorPtrE", "tensorrt_llm::runtime::GptDecoderBatched::TensorPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched16disableLookaheadERK13RequestVectorRK9TensorPtr", "tensorrt_llm::runtime::GptDecoderBatched::disableLookahead"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched16disableLookaheadERK13RequestVectorRK9TensorPtr", "tensorrt_llm::runtime::GptDecoderBatched::disableLookahead::batchSlots"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched16disableLookaheadERK13RequestVectorRK9TensorPtr", "tensorrt_llm::runtime::GptDecoderBatched::disableLookahead::genRequests"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::GptDecoderBatched::finalize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::GptDecoderBatched::finalize::batchSlot"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::GptDecoderBatched::finalize::decoderState"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::GptDecoderBatched::finalize::samplingConfig"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::GptDecoderBatched::finalize::streaming"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched7forwardERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::GptDecoderBatched::forward"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched7forwardERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::GptDecoderBatched::forward::decoderState"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched7forwardERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::GptDecoderBatched::forward::input"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched12forwardAsyncERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::GptDecoderBatched::forwardAsync"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched12forwardAsyncERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::GptDecoderBatched::forwardAsync::decoderState"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched12forwardAsyncERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::GptDecoderBatched::forwardAsync::input"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched15forwardDispatchERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::GptDecoderBatched::forwardDispatch"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched15forwardDispatchERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::GptDecoderBatched::forwardDispatch::decoderState"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched15forwardDispatchERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::GptDecoderBatched::forwardDispatch::input"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched16getBufferManagerEv", "tensorrt_llm::runtime::GptDecoderBatched::getBufferManager"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched16getDecoderStreamEv", "tensorrt_llm::runtime::GptDecoderBatched::getDecoderStream"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime17GptDecoderBatched20getUnderlyingDecoderEv", "tensorrt_llm::runtime::GptDecoderBatched::getUnderlyingDecoder"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched14mBufferManagerE", "tensorrt_llm::runtime::GptDecoderBatched::mBufferManager"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched8mDecoderE", "tensorrt_llm::runtime::GptDecoderBatched::mDecoder"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched14mDecoderStreamE", "tensorrt_llm::runtime::GptDecoderBatched::mDecoderStream"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched14mRuntimeStreamE", "tensorrt_llm::runtime::GptDecoderBatched::mRuntimeStream"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::GptDecoderBatched::setup"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::GptDecoderBatched::setup::dtype"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::GptDecoderBatched::setup::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::GptDecoderBatched::setup::maxNumSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::GptDecoderBatched::setup::mode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::GptDecoderBatched::setup::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17GptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::GptDecoderBatched::setup::worldConfig"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfigE", "tensorrt_llm::runtime::GptJsonConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig::contextParallelism"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig::gpusPerNode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig::name"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig::pipelineParallelism"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig::precision"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig::runtimeDefaults"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig::tensorParallelism"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig13GptJsonConfigENSt6stringENSt6stringENSt6stringE10SizeType3210SizeType3210SizeType3210SizeType3211ModelConfigNSt8optionalI15RuntimeDefaultsEE", "tensorrt_llm::runtime::GptJsonConfig::GptJsonConfig::version"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14engineFilenameERK11WorldConfig", "tensorrt_llm::runtime::GptJsonConfig::engineFilename"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14engineFilenameERK11WorldConfigRKNSt6stringE", "tensorrt_llm::runtime::GptJsonConfig::engineFilename"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14engineFilenameERK11WorldConfigRKNSt6stringE", "tensorrt_llm::runtime::GptJsonConfig::engineFilename::model"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14engineFilenameERK11WorldConfig", "tensorrt_llm::runtime::GptJsonConfig::engineFilename::worldConfig"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14engineFilenameERK11WorldConfigRKNSt6stringE", "tensorrt_llm::runtime::GptJsonConfig::engineFilename::worldConfig"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig21getContextParallelismEv", "tensorrt_llm::runtime::GptJsonConfig::getContextParallelism"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14getGpusPerNodeEv", "tensorrt_llm::runtime::GptJsonConfig::getGpusPerNode"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig14getModelConfigEv", "tensorrt_llm::runtime::GptJsonConfig::getModelConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig21getModelConfigMutableEv", "tensorrt_llm::runtime::GptJsonConfig::getModelConfigMutable"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig7getNameEv", "tensorrt_llm::runtime::GptJsonConfig::getName"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig22getPipelineParallelismEv", "tensorrt_llm::runtime::GptJsonConfig::getPipelineParallelism"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig12getPrecisionEv", "tensorrt_llm::runtime::GptJsonConfig::getPrecision"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig18getRuntimeDefaultsEv", "tensorrt_llm::runtime::GptJsonConfig::getRuntimeDefaults"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig20getTensorParallelismEv", "tensorrt_llm::runtime::GptJsonConfig::getTensorParallelism"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig10getVersionEv", "tensorrt_llm::runtime::GptJsonConfig::getVersion"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime13GptJsonConfig12getWorldSizeEv", "tensorrt_llm::runtime::GptJsonConfig::getWorldSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig19mContextParallelismE", "tensorrt_llm::runtime::GptJsonConfig::mContextParallelism"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig12mGpusPerNodeE", "tensorrt_llm::runtime::GptJsonConfig::mGpusPerNode"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig12mModelConfigE", "tensorrt_llm::runtime::GptJsonConfig::mModelConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5mNameE", "tensorrt_llm::runtime::GptJsonConfig::mName"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig20mPipelineParallelismE", "tensorrt_llm::runtime::GptJsonConfig::mPipelineParallelism"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig10mPrecisionE", "tensorrt_llm::runtime::GptJsonConfig::mPrecision"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig16mRuntimeDefaultsE", "tensorrt_llm::runtime::GptJsonConfig::mRuntimeDefaults"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig18mTensorParallelismE", "tensorrt_llm::runtime::GptJsonConfig::mTensorParallelism"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig8mVersionE", "tensorrt_llm::runtime::GptJsonConfig::mVersion"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5parseERKNSt10filesystem4pathE", "tensorrt_llm::runtime::GptJsonConfig::parse"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5parseERKNSt6stringE", "tensorrt_llm::runtime::GptJsonConfig::parse"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5parseERNSt7istreamE", "tensorrt_llm::runtime::GptJsonConfig::parse"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5parseERKNSt6stringE", "tensorrt_llm::runtime::GptJsonConfig::parse::json"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5parseERNSt7istreamE", "tensorrt_llm::runtime::GptJsonConfig::parse::json"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13GptJsonConfig5parseERKNSt10filesystem4pathE", "tensorrt_llm::runtime::GptJsonConfig::parse::path"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime7IBufferE", "tensorrt_llm::runtime::IBuffer"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer8DataTypeE", "tensorrt_llm::runtime::IBuffer::DataType"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer7IBufferERK7IBuffer", "tensorrt_llm::runtime::IBuffer::IBuffer"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer7IBufferEv", "tensorrt_llm::runtime::IBuffer::IBuffer"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer14SharedConstPtrE", "tensorrt_llm::runtime::IBuffer::SharedConstPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer9SharedPtrE", "tensorrt_llm::runtime::IBuffer::SharedPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer14UniqueConstPtrE", "tensorrt_llm::runtime::IBuffer::UniqueConstPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer9UniquePtrE", "tensorrt_llm::runtime::IBuffer::UniquePtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4dataENSt6size_tE", "tensorrt_llm::runtime::IBuffer::data"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4dataEv", "tensorrt_llm::runtime::IBuffer::data"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer4dataENSt6size_tE", "tensorrt_llm::runtime::IBuffer::data"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer4dataEv", "tensorrt_llm::runtime::IBuffer::data"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4dataENSt6size_tE", "tensorrt_llm::runtime::IBuffer::data::index"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer4dataENSt6size_tE", "tensorrt_llm::runtime::IBuffer::data::index"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer11getCapacityEv", "tensorrt_llm::runtime::IBuffer::getCapacity"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer11getDataTypeEv", "tensorrt_llm::runtime::IBuffer::getDataType"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer15getDataTypeNameE8DataType", "tensorrt_llm::runtime::IBuffer::getDataTypeName"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer15getDataTypeNameEv", "tensorrt_llm::runtime::IBuffer::getDataTypeName"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer15getDataTypeNameE8DataType", "tensorrt_llm::runtime::IBuffer::getDataTypeName::dataType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer13getMemoryTypeEv", "tensorrt_llm::runtime::IBuffer::getMemoryType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer17getMemoryTypeNameEv", "tensorrt_llm::runtime::IBuffer::getMemoryTypeName"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer7getSizeEv", "tensorrt_llm::runtime::IBuffer::getSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer14getSizeInBytesEv", "tensorrt_llm::runtime::IBuffer::getSizeInBytes"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer10memoryTypeEPKv", "tensorrt_llm::runtime::IBuffer::memoryType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer10memoryTypeEPKv", "tensorrt_llm::runtime::IBuffer::memoryType::data"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBufferaSERK7IBuffer", "tensorrt_llm::runtime::IBuffer::operator="], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer7releaseEv", "tensorrt_llm::runtime::IBuffer::release"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer6resizeENSt6size_tE", "tensorrt_llm::runtime::IBuffer::resize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer6resizeENSt6size_tE", "tensorrt_llm::runtime::IBuffer::resize::newSize"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer5sliceE9SharedPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer5sliceE9SharedPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::TConstPtr"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::TConstPtr"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer5sliceE9SharedPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::buffer"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer5sliceE9SharedPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::buffer"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::offset"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::offset"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer5sliceE9SharedPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::offset"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer5sliceE9SharedPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::offset"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer5sliceE9SharedPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::size"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::tensor"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::slice::tensor"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer7toBytesENSt6size_tE", "tensorrt_llm::runtime::IBuffer::toBytes"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7IBuffer7toBytesENSt6size_tE", "tensorrt_llm::runtime::IBuffer::toBytes::size"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer4viewE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::view"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4viewE9SharedPtr", "tensorrt_llm::runtime::IBuffer::view"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4viewE9SharedPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::view"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer4viewE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::view::TConstPtr"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer4viewE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::view::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4viewE9SharedPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::view::size"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7IBuffer4viewE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::view::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4viewE9SharedPtr", "tensorrt_llm::runtime::IBuffer::view::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4viewE9SharedPtrNSt6size_tE", "tensorrt_llm::runtime::IBuffer::view::tensor"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrRNSt6vectorI1TEE", "tensorrt_llm::runtime::IBuffer::wrap"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrRNSt6vectorI1TEE", "tensorrt_llm::runtime::IBuffer::wrap::T"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::capacity"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::capacity"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::data"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::data"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::data"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::data"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::size"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrP1TNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::type"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7IBuffer4wrapEPv8DataTypeNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::IBuffer::wrap::type"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7IBuffer4wrapE9UniquePtrRNSt6vectorI1TEE", "tensorrt_llm::runtime::IBuffer::wrap::v"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7IBufferD0Ev", "tensorrt_llm::runtime::IBuffer::~IBuffer"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoderE", "tensorrt_llm::runtime::IGptDecoder"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder14TensorConstPtrE", "tensorrt_llm::runtime::IGptDecoder::TensorConstPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder9TensorPtrE", "tensorrt_llm::runtime::IGptDecoder::TensorPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::IGptDecoder::create"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::IGptDecoder::create::dtype"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::IGptDecoder::create::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::IGptDecoder::create::maxNumSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::IGptDecoder::create::mode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::IGptDecoder::create::speculativeDecodingModule"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::IGptDecoder::create::stream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::IGptDecoder::create::vocabSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder6createERKN8executor12DecodingModeEN8nvinfer18DataTypeE6size_t6size_t6size_t6size_tRKN13BufferManager13CudaStreamPtrERKNSt10shared_ptrIK25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::IGptDecoder::create::vocabSizePadded"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", "tensorrt_llm::runtime::IGptDecoder::disableLookahead"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", "tensorrt_llm::runtime::IGptDecoder::disableLookahead::batchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", "tensorrt_llm::runtime::IGptDecoder::disableLookahead::batchSlots"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder16disableLookaheadERKNSt8optionalI14SamplingConfigEE10SizeType3214TensorConstPtr", "tensorrt_llm::runtime::IGptDecoder::disableLookahead::samplingConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder12forwardAsyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::IGptDecoder::forwardAsync"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder12forwardAsyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::IGptDecoder::forwardAsync::input"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder12forwardAsyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::IGptDecoder::forwardAsync::output"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder11forwardSyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::IGptDecoder::forwardSync"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder11forwardSyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::IGptDecoder::forwardSync::input"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder11forwardSyncER14DecodingOutputRK13DecodingInput", "tensorrt_llm::runtime::IGptDecoder::forwardSync::output"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder17getSamplingConfigEv", "tensorrt_llm::runtime::IGptDecoder::getSamplingConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::IGptDecoder::setup"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::IGptDecoder::setup::batchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::IGptDecoder::setup::batchSlots"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::IGptDecoder::setup::explicitDraftTokensDType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::IGptDecoder::setup::lookaheadAlgoConfigs"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::IGptDecoder::setup::lookaheadPrompt"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::IGptDecoder::setup::output"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoder5setupERK14SamplingConfig6size_tRK14TensorConstPtrRKNSt8optionalI14DecodingOutputEENSt8optionalIN8nvinfer18DataTypeEEERKNSt8optionalINSt6vectorI14TensorConstPtrEEEERKNSt8optionalINSt6vectorIN8executor23LookaheadDecodingConfigEEEEE", "tensorrt_llm::runtime::IGptDecoder::setup::samplingConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11IGptDecoderD0Ev", "tensorrt_llm::runtime::IGptDecoder::~IGptDecoder"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatchedE", "tensorrt_llm::runtime::IGptDecoderBatched"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched13CudaStreamPtrE", "tensorrt_llm::runtime::IGptDecoderBatched::CudaStreamPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched18IGptDecoderBatchedEv", "tensorrt_llm::runtime::IGptDecoderBatched::IGptDecoderBatched"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched13LlmRequestPtrE", "tensorrt_llm::runtime::IGptDecoderBatched::LlmRequestPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched13RequestVectorE", "tensorrt_llm::runtime::IGptDecoderBatched::RequestVector"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched9TensorPtrE", "tensorrt_llm::runtime::IGptDecoderBatched::TensorPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched16disableLookaheadERK13RequestVectorRK9TensorPtr", "tensorrt_llm::runtime::IGptDecoderBatched::disableLookahead"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched16disableLookaheadERK13RequestVectorRK9TensorPtr", "tensorrt_llm::runtime::IGptDecoderBatched::disableLookahead::batchSlots"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched16disableLookaheadERK13RequestVectorRK9TensorPtr", "tensorrt_llm::runtime::IGptDecoderBatched::disableLookahead::genRequests"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime18IGptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::IGptDecoderBatched::finalize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime18IGptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::IGptDecoderBatched::finalize::batchSlot"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime18IGptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::IGptDecoderBatched::finalize::decoderState"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime18IGptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::IGptDecoderBatched::finalize::samplingConfig"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime18IGptDecoderBatched8finalizeERKN7decoder12DecoderStateE10SizeType32RK14SamplingConfigb", "tensorrt_llm::runtime::IGptDecoderBatched::finalize::streaming"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched7forwardERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::IGptDecoderBatched::forward"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched7forwardERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::IGptDecoderBatched::forward::decoderState"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched7forwardERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::IGptDecoderBatched::forward::input"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched12forwardAsyncERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::IGptDecoderBatched::forwardAsync"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched12forwardAsyncERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::IGptDecoderBatched::forwardAsync::decoderState"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched12forwardAsyncERKN7decoder12DecoderStateERKN13decoder_batch5InputE", "tensorrt_llm::runtime::IGptDecoderBatched::forwardAsync::input"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::IGptDecoderBatched::setup"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::IGptDecoderBatched::setup::dtype"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::IGptDecoderBatched::setup::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::IGptDecoderBatched::setup::maxNumSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::IGptDecoderBatched::setup::mode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::IGptDecoderBatched::setup::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatched5setupERKN8executor12DecodingModeE10SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::IGptDecoderBatched::setup::worldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18IGptDecoderBatchedD0Ev", "tensorrt_llm::runtime::IGptDecoderBatched::~IGptDecoderBatched"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime7ITensorE", "tensorrt_llm::runtime::ITensor"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9DimType64E", "tensorrt_llm::runtime::ITensor::DimType64"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7ITensorERK7ITensor", "tensorrt_llm::runtime::ITensor::ITensor"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7ITensorEv", "tensorrt_llm::runtime::ITensor::ITensor"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5ShapeE", "tensorrt_llm::runtime::ITensor::Shape"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor14SharedConstPtrE", "tensorrt_llm::runtime::ITensor::SharedConstPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9SharedPtrE", "tensorrt_llm::runtime::ITensor::SharedPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9TensorMapE", "tensorrt_llm::runtime::ITensor::TensorMap"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor14UniqueConstPtrE", "tensorrt_llm::runtime::ITensor::UniqueConstPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9UniquePtrE", "tensorrt_llm::runtime::ITensor::UniquePtr"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::at"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atEN7ITensor14UniqueConstPtrERR9TConstPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::at"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor2atE9SharedPtrRK5Shape", "tensorrt_llm::runtime::ITensor::at"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor2atE9SharedPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::at"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::at::TConstPtr"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atEN7ITensor14UniqueConstPtrERR9TConstPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::at::TConstPtr"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::at::offsetDims"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atEN7ITensor14UniqueConstPtrERR9TConstPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::at::offsetDims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor2atE9SharedPtrRK5Shape", "tensorrt_llm::runtime::ITensor::at::offsetDims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor2atE9SharedPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::at::offsetDims"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::at::tensor"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor2atEN7ITensor14UniqueConstPtrERR9TConstPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::at::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor2atE9SharedPtrRK5Shape", "tensorrt_llm::runtime::ITensor::at::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor2atE9SharedPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::at::tensor"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor8castSizeE6size_t", "tensorrt_llm::runtime::ITensor::castSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor8castSizeE6size_t", "tensorrt_llm::runtime::ITensor::castSize::newSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor8flattenNE9SharedPtrNSt7int64_tE", "tensorrt_llm::runtime::ITensor::flattenN"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor8flattenNE9SharedPtrNSt7int64_tE", "tensorrt_llm::runtime::ITensor::flattenN::sliceN"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor8flattenNE9SharedPtrNSt7int64_tE", "tensorrt_llm::runtime::ITensor::flattenN::tensor"], [1, 3, 1, "_CPPv4I_10SizeType32ENK12tensorrt_llm7runtime7ITensor12getDimensionE9DimType64v", "tensorrt_llm::runtime::ITensor::getDimension"], [1, 8, 1, "_CPPv4I_10SizeType32ENK12tensorrt_llm7runtime7ITensor12getDimensionE9DimType64v", "tensorrt_llm::runtime::ITensor::getDimension::n"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7ITensor8getShapeEv", "tensorrt_llm::runtime::ITensor::getShape"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9makeShapeERKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::makeShape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9makeShapeERKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::makeShape::dims"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensoraSERK7ITensor", "tensorrt_llm::runtime::ITensor::operator="], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7reshapeERK5Shape", "tensorrt_llm::runtime::ITensor::reshape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7reshapeERK5Shape", "tensorrt_llm::runtime::ITensor::reshape::dims"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor6resizeENSt6size_tE", "tensorrt_llm::runtime::ITensor::resize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor6resizeENSt6size_tE", "tensorrt_llm::runtime::ITensor::resize::newSize"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor11shapeEqualsEbRK5ShapePK1T10SizeType32", "tensorrt_llm::runtime::ITensor::shapeEquals"], [1, 3, 1, "_CPPv4I0ENK12tensorrt_llm7runtime7ITensor11shapeEqualsEbPK1T10SizeType32", "tensorrt_llm::runtime::ITensor::shapeEquals"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor11shapeEqualsERK5ShapeRK5Shape", "tensorrt_llm::runtime::ITensor::shapeEquals"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7ITensor11shapeEqualsERK5Shape", "tensorrt_llm::runtime::ITensor::shapeEquals"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7ITensor11shapeEqualsERKNSt16initializer_listI10SizeType32EE", "tensorrt_llm::runtime::ITensor::shapeEquals"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor11shapeEqualsEbRK5ShapePK1T10SizeType32", "tensorrt_llm::runtime::ITensor::shapeEquals::T"], [1, 8, 1, "_CPPv4I0ENK12tensorrt_llm7runtime7ITensor11shapeEqualsEbPK1T10SizeType32", "tensorrt_llm::runtime::ITensor::shapeEquals::T"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor11shapeEqualsEbRK5ShapePK1T10SizeType32", "tensorrt_llm::runtime::ITensor::shapeEquals::count"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime7ITensor11shapeEqualsEbPK1T10SizeType32", "tensorrt_llm::runtime::ITensor::shapeEquals::count"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor11shapeEqualsEbRK5ShapePK1T10SizeType32", "tensorrt_llm::runtime::ITensor::shapeEquals::dims"], [1, 4, 1, "_CPPv4I0ENK12tensorrt_llm7runtime7ITensor11shapeEqualsEbPK1T10SizeType32", "tensorrt_llm::runtime::ITensor::shapeEquals::dims"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor11shapeEqualsEbRK5ShapePK1T10SizeType32", "tensorrt_llm::runtime::ITensor::shapeEquals::lhs"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor11shapeEqualsERK5ShapeRK5Shape", "tensorrt_llm::runtime::ITensor::shapeEquals::lhs"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7ITensor11shapeEqualsERK5Shape", "tensorrt_llm::runtime::ITensor::shapeEquals::other"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7ITensor11shapeEqualsERKNSt16initializer_listI10SizeType32EE", "tensorrt_llm::runtime::ITensor::shapeEquals::other"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor11shapeEqualsERK5ShapeRK5Shape", "tensorrt_llm::runtime::ITensor::shapeEquals::rhs"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRK5Shape", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRK5Shape9DimType64", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::slice"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRKNSt16initializer_listI9DimType64EE9DimType64", "tensorrt_llm::runtime::ITensor::slice"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::TConstPtr"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::TConstPtr"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::slice::TConstPtr"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::TConstPtr"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::slice::TConstPtr"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::TConstPtr"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::offset"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::offset"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::offset"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::offset"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::slice::offsetDims"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::offsetDims"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::slice::offsetDims"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::offsetDims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRK5Shape", "tensorrt_llm::runtime::ITensor::slice::offsetDims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRK5Shape9DimType64", "tensorrt_llm::runtime::ITensor::slice::offsetDims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::slice::offsetDims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRKNSt16initializer_listI9DimType64EE9DimType64", "tensorrt_llm::runtime::ITensor::slice::offsetDims"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::size"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::size"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRK5Shape9DimType64", "tensorrt_llm::runtime::ITensor::slice::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRKNSt16initializer_listI9DimType64EE9DimType64", "tensorrt_llm::runtime::ITensor::slice::size"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor5sliceE14UniqueConstPtrRR9TConstPtrRKNSt16initializer_listI9DimType64EENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrNSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrNSt6size_tENSt6size_tE", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRK5Shape", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRK5Shape9DimType64", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRKNSt16initializer_listI9DimType64EE", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor5sliceE9SharedPtrRKNSt16initializer_listI9DimType64EE9DimType64", "tensorrt_llm::runtime::ITensor::slice::tensor"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7squeezeE10SizeType32", "tensorrt_llm::runtime::ITensor::squeeze"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7squeezeERK5Shape10SizeType32", "tensorrt_llm::runtime::ITensor::squeeze"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7squeezeE10SizeType32", "tensorrt_llm::runtime::ITensor::squeeze::dim"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7squeezeERK5Shape10SizeType32", "tensorrt_llm::runtime::ITensor::squeeze::dim"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7squeezeERK5Shape10SizeType32", "tensorrt_llm::runtime::ITensor::squeeze::shape"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7stridesERK5Shape", "tensorrt_llm::runtime::ITensor::strides"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor7stridesERK5Shape", "tensorrt_llm::runtime::ITensor::strides::dims"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor8toStringERK5Shape", "tensorrt_llm::runtime::ITensor::toString"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor8toStringERK5Shape", "tensorrt_llm::runtime::ITensor::toString::dims"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9unsqueezeE10SizeType32", "tensorrt_llm::runtime::ITensor::unsqueeze"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9unsqueezeERK5Shape10SizeType32", "tensorrt_llm::runtime::ITensor::unsqueeze"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9unsqueezeE10SizeType32", "tensorrt_llm::runtime::ITensor::unsqueeze::dim"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9unsqueezeERK5Shape10SizeType32", "tensorrt_llm::runtime::ITensor::unsqueeze::dim"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor9unsqueezeERK5Shape10SizeType32", "tensorrt_llm::runtime::ITensor::unsqueeze::shape"], [1, 3, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor4viewE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::view"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4viewE9SharedPtr", "tensorrt_llm::runtime::ITensor::view"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4viewEN7IBuffer9SharedPtrERK5Shape", "tensorrt_llm::runtime::ITensor::view"], [1, 8, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor4viewE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::view::TConstPtr"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4viewEN7IBuffer9SharedPtrERK5Shape", "tensorrt_llm::runtime::ITensor::view::buffer"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor4viewE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::view::dims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4viewEN7IBuffer9SharedPtrERK5Shape", "tensorrt_llm::runtime::ITensor::view::dims"], [1, 4, 1, "_CPPv4I0_NSt11enable_if_tINSt10is_const_vI18PointerElementTypeI9TConstPtrEEEiEEEN12tensorrt_llm7runtime7ITensor4viewE14UniqueConstPtrRR9TConstPtrRK5Shape", "tensorrt_llm::runtime::ITensor::view::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4viewE9SharedPtr", "tensorrt_llm::runtime::ITensor::view::tensor"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor6volumeERK5Shape", "tensorrt_llm::runtime::ITensor::volume"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor6volumeERK5Shape", "tensorrt_llm::runtime::ITensor::volume::dims"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor17volumeNonNegativeERK5Shape", "tensorrt_llm::runtime::ITensor::volumeNonNegative"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor17volumeNonNegativeERK5Shape", "tensorrt_llm::runtime::ITensor::volumeNonNegative::shape"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5Shape", "tensorrt_llm::runtime::ITensor::wrap"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrRNSt6vectorI1TEERK5Shape", "tensorrt_llm::runtime::ITensor::wrap"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5Shape", "tensorrt_llm::runtime::ITensor::wrap"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5Shape", "tensorrt_llm::runtime::ITensor::wrap::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrRNSt6vectorI1TEERK5Shape", "tensorrt_llm::runtime::ITensor::wrap::T"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap::capacity"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap::capacity"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5Shape", "tensorrt_llm::runtime::ITensor::wrap::data"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap::data"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5Shape", "tensorrt_llm::runtime::ITensor::wrap::data"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap::data"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5Shape", "tensorrt_llm::runtime::ITensor::wrap::shape"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrP1TRK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap::shape"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrRNSt6vectorI1TEERK5Shape", "tensorrt_llm::runtime::ITensor::wrap::shape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5Shape", "tensorrt_llm::runtime::ITensor::wrap::shape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap::shape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5Shape", "tensorrt_llm::runtime::ITensor::wrap::type"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7ITensor4wrapEPvN8nvinfer18DataTypeERK5ShapeNSt6size_tE", "tensorrt_llm::runtime::ITensor::wrap::type"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime7ITensor4wrapE9UniquePtrRNSt6vectorI1TEERK5Shape", "tensorrt_llm::runtime::ITensor::wrap::v"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7ITensorD0Ev", "tensorrt_llm::runtime::ITensor::~ITensor"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemoryE", "tensorrt_llm::runtime::IpcMemory"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9BufferPtrE", "tensorrt_llm::runtime::IpcMemory::BufferPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory10FLAGS_SIZEE", "tensorrt_llm::runtime::IpcMemory::FLAGS_SIZE"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfigb", "tensorrt_llm::runtime::IpcMemory::IpcMemory"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryERK9IpcMemory", "tensorrt_llm::runtime::IpcMemory::IpcMemory"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryERR9IpcMemory", "tensorrt_llm::runtime::IpcMemory::IpcMemory"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfigb", "tensorrt_llm::runtime::IpcMemory::IpcMemory::bufferSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfigb", "tensorrt_llm::runtime::IpcMemory::IpcMemory::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfigb", "tensorrt_llm::runtime::IpcMemory::IpcMemory::openIpc"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9IpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfigb", "tensorrt_llm::runtime::IpcMemory::IpcMemory::worldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory17allocateIpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfig", "tensorrt_llm::runtime::IpcMemory::allocateIpcMemory"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory17allocateIpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfig", "tensorrt_llm::runtime::IpcMemory::allocateIpcMemory::bufferSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory17allocateIpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfig", "tensorrt_llm::runtime::IpcMemory::allocateIpcMemory::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory17allocateIpcMemoryENSt6size_tERK13BufferManagerRK11WorldConfig", "tensorrt_llm::runtime::IpcMemory::allocateIpcMemory::worldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory16destroyIpcMemoryEv", "tensorrt_llm::runtime::IpcMemory::destroyIpcMemory"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9IpcMemory11getCommPtrsEv", "tensorrt_llm::runtime::IpcMemory::getCommPtrs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory7mBufferE", "tensorrt_llm::runtime::IpcMemory::mBuffer"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory9mCommPtrsE", "tensorrt_llm::runtime::IpcMemory::mCommPtrs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory8mOpenIpcE", "tensorrt_llm::runtime::IpcMemory::mOpenIpc"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemory7mTpRankE", "tensorrt_llm::runtime::IpcMemory::mTpRank"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemoryaSERK9IpcMemory", "tensorrt_llm::runtime::IpcMemory::operator="], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemoryaSERR9IpcMemory", "tensorrt_llm::runtime::IpcMemory::operator="], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9IpcMemoryD0Ev", "tensorrt_llm::runtime::IpcMemory::~IpcMemory"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandleE", "tensorrt_llm::runtime::IpcNvlsHandle"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle14ipc_uc_handlesE", "tensorrt_llm::runtime::IpcNvlsHandle::ipc_uc_handles"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle11ipc_uc_ptrsE", "tensorrt_llm::runtime::IpcNvlsHandle::ipc_uc_ptrs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle10ipc_uc_vasE", "tensorrt_llm::runtime::IpcNvlsHandle::ipc_uc_vas"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle9mc_handleE", "tensorrt_llm::runtime::IpcNvlsHandle::mc_handle"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle6mc_ptrE", "tensorrt_llm::runtime::IpcNvlsHandle::mc_ptr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle5mc_vaE", "tensorrt_llm::runtime::IpcNvlsHandle::mc_va"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle4sizeE", "tensorrt_llm::runtime::IpcNvlsHandle::size"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle9uc_handleE", "tensorrt_llm::runtime::IpcNvlsHandle::uc_handle"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle6uc_ptrE", "tensorrt_llm::runtime::IpcNvlsHandle::uc_ptr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13IpcNvlsHandle5uc_vaE", "tensorrt_llm::runtime::IpcNvlsHandle::uc_va"], [1, 2, 1, "_CPPv4I_bEN12tensorrt_llm7runtime12LocalCreatorE", "tensorrt_llm::runtime::LocalCreator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator12LocalCreatorERK19CUmemAllocationProp6size_t", "tensorrt_llm::runtime::LocalCreator::LocalCreator"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator12LocalCreatorERK19CUmemAllocationProp6size_t", "tensorrt_llm::runtime::LocalCreator::LocalCreator::prop"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator12LocalCreatorERK19CUmemAllocationProp6size_t", "tensorrt_llm::runtime::LocalCreator::LocalCreator::size"], [1, 8, 1, "_CPPv4I_bEN12tensorrt_llm7runtime12LocalCreatorE", "tensorrt_llm::runtime::LocalCreator::count"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator6createEv", "tensorrt_llm::runtime::LocalCreator::create"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator5mPropE", "tensorrt_llm::runtime::LocalCreator::mProp"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator5mSizeE", "tensorrt_llm::runtime::LocalCreator::mSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator7releaseE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::LocalCreator::release"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator7releaseE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::LocalCreator::release::destructing"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12LocalCreator7releaseE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::LocalCreator::release::handle"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffersE", "tensorrt_llm::runtime::LookaheadDecodingBuffers"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers24LookaheadDecodingBuffersE10SizeType3210SizeType32RK13BufferManager", "tensorrt_llm::runtime::LookaheadDecodingBuffers::LookaheadDecodingBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers24LookaheadDecodingBuffersE10SizeType3210SizeType32RK13BufferManager", "tensorrt_llm::runtime::LookaheadDecodingBuffers::LookaheadDecodingBuffers::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers24LookaheadDecodingBuffersE10SizeType3210SizeType32RK13BufferManager", "tensorrt_llm::runtime::LookaheadDecodingBuffers::LookaheadDecodingBuffers::maxNumSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers24LookaheadDecodingBuffersE10SizeType3210SizeType32RK13BufferManager", "tensorrt_llm::runtime::LookaheadDecodingBuffers::LookaheadDecodingBuffers::maxTokensPerStep"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers9TensorPtrE", "tensorrt_llm::runtime::LookaheadDecodingBuffers::TensorPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers17generationLengthsE", "tensorrt_llm::runtime::LookaheadDecodingBuffers::generationLengths"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers11packedMasksE", "tensorrt_llm::runtime::LookaheadDecodingBuffers::packedMasks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers11positionIdsE", "tensorrt_llm::runtime::LookaheadDecodingBuffers::positionIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime24LookaheadDecodingBuffers15positionOffsetsE", "tensorrt_llm::runtime::LookaheadDecodingBuffers::positionOffsets"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModuleE", "tensorrt_llm::runtime::LookaheadModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule15LookaheadModuleE10SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadModule::LookaheadModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule15LookaheadModuleEv", "tensorrt_llm::runtime::LookaheadModule::LookaheadModule"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule15LookaheadModuleE10SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadModule::LookaheadModule::maxDecodingDraftTokens"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule15LookaheadModuleE10SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadModule::LookaheadModule::maxDraftPathLen"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime15LookaheadModule18getExecutionConfigEv", "tensorrt_llm::runtime::LookaheadModule::getExecutionConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule16mExecutionConfigE", "tensorrt_llm::runtime::LookaheadModule::mExecutionConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule18setExecutionConfigERKN8executor23LookaheadDecodingConfigE", "tensorrt_llm::runtime::LookaheadModule::setExecutionConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime15LookaheadModule18setExecutionConfigERKN8executor23LookaheadDecodingConfigE", "tensorrt_llm::runtime::LookaheadModule::setExecutionConfig::config"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffersE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23LookaheadRuntimeBuffersE10SizeType3210SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfigRKN8executor14DecodingConfigERK11TllmRuntime", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::LookaheadRuntimeBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23LookaheadRuntimeBuffersE10SizeType3210SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfigRKN8executor14DecodingConfigERK11TllmRuntime", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::LookaheadRuntimeBuffers::decodingConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23LookaheadRuntimeBuffersE10SizeType3210SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfigRKN8executor14DecodingConfigERK11TllmRuntime", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::LookaheadRuntimeBuffers::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23LookaheadRuntimeBuffersE10SizeType3210SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfigRKN8executor14DecodingConfigERK11TllmRuntime", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::LookaheadRuntimeBuffers::maxBatchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23LookaheadRuntimeBuffersE10SizeType3210SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfigRKN8executor14DecodingConfigERK11TllmRuntime", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::LookaheadRuntimeBuffers::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23LookaheadRuntimeBuffersE10SizeType3210SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfigRKN8executor14DecodingConfigERK11TllmRuntime", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::LookaheadRuntimeBuffers::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23LookaheadRuntimeBuffersE10SizeType3210SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfigRKN8executor14DecodingConfigERK11TllmRuntime", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::LookaheadRuntimeBuffers::runtime"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23LookaheadRuntimeBuffersE10SizeType3210SizeType32RK13BufferManagerRK11ModelConfigRK11WorldConfigRKN8executor14DecodingConfigERK11TllmRuntime", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::LookaheadRuntimeBuffers::worldConfig"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers9TensorMapE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::TensorMap"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers9TensorPtrE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::TensorPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers18batchSlotsHostCopyE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::batchSlotsHostCopy"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers12cumSumLengthE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::cumSumLength"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers24disableLookaheadDecodingEv", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::disableLookaheadDecoding"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23enableLookaheadDecodingE10SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::enableLookaheadDecoding"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23enableLookaheadDecodingE10SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::enableLookaheadDecoding::maxBatchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23enableLookaheadDecodingE10SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::enableLookaheadDecoding::tokensPerStep"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23generationLengthsDeviceE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::generationLengthsDevice"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers21generationLengthsHostE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::generationLengthsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers25generationLengthsHostCopyE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::generationLengthsHostCopy"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers18insertInputTensorsER9TensorMapR9TensorMapRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::insertInputTensors"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers18insertInputTensorsER9TensorMapR9TensorMapRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::insertInputTensors::inputBuffers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers18insertInputTensorsER9TensorMapR9TensorMapRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::insertInputTensors::outputBuffers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers18insertInputTensorsER9TensorMapR9TensorMapRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::insertInputTensors::worldConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers14packedMaskHostE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::packedMaskHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers18packedMaskHostCopyE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::packedMaskHostCopy"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers17packedMasksDeviceE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::packedMasksDevice"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers17positionIdsDeviceE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::positionIdsDevice"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers15positionIdsHostE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::positionIdsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers19positionIdsHostCopyE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::positionIdsHostCopy"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers21positionOffsetsDeviceE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::positionOffsetsDevice"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers19positionOffsetsHostE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::positionOffsetsHost"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers23positionOffsetsHostCopyE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::positionOffsetsHostCopy"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers7reshapeE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::reshape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers7reshapeE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::reshape::numCtxSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers7reshapeE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::reshape::numGenSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers7reshapeE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::reshape::tokensPerStep"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::setFromInputs"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::setFromInputs::decoderLookaheadBuffers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::setFromInputs::modelConfig"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::setFromInputs::numCtxSequences"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::setFromInputs::numGenSequences"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::setFromInputs::requestTypes"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::setFromInputs::runtime"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::setFromInputs::seqSlots"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23LookaheadRuntimeBuffers13setFromInputsE10SizeType3210SizeType32RK7ITensorRK7ITensorRK24LookaheadDecodingBuffersRK11TllmRuntimeRK11ModelConfigRK11WorldConfig", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::setFromInputs::worldConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23LookaheadRuntimeBuffers15useSpecDecodingE", "tensorrt_llm::runtime::LookaheadRuntimeBuffers::useSpecDecoding"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCacheE", "tensorrt_llm::runtime::LoraCache"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9LoraCacheERK26LoraCachePageManagerConfigRK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::LoraCache::LoraCache"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9LoraCacheERK26LoraCachePageManagerConfigRK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::LoraCache::LoraCache::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9LoraCacheERK26LoraCachePageManagerConfigRK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::LoraCache::LoraCache::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9LoraCacheERK26LoraCachePageManagerConfigRK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::LoraCache::LoraCache::pageManagerConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9LoraCacheERK26LoraCachePageManagerConfigRK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::LoraCache::LoraCache::worldConfig"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache10TaskIdTypeE", "tensorrt_llm::runtime::LoraCache::TaskIdType"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfigE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig11adapterSizeE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::adapterSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig6inSizeE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::inSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig7layerIdE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::layerId"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig8moduleIdE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::moduleId"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig8numSlotsE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::numSlots"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfigeqERKN9LoraCache21TaskLayerModuleConfigE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::operator=="], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfigeqERKN9LoraCache21TaskLayerModuleConfigE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::operator==::o"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig7outSizeE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::outSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig6pageIdE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::pageId"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig17scalingVecPointerE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::scalingVecPointer"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig7slotIdxE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::slotIdx"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig8toStringEv", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::toString"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig16weightsInPointerE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::weightsInPointer"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21TaskLayerModuleConfig17weightsOutPointerE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfig::weightsOutPointer"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache28TaskLayerModuleConfigListPtrE", "tensorrt_llm::runtime::LoraCache::TaskLayerModuleConfigListPtr"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValueE", "tensorrt_llm::runtime::LoraCache::TaskValue"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERKNSt6vectorINSt6size_tEEERK28TaskLayerModuleConfigListPtrNSt4listI10TaskIdTypeE8iteratorEbbbb", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERR9TaskValue", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueEv", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERKNSt6vectorINSt6size_tEEERK28TaskLayerModuleConfigListPtrNSt4listI10TaskIdTypeE8iteratorEbbbb", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue::configs"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERKNSt6vectorINSt6size_tEEERK28TaskLayerModuleConfigListPtrNSt4listI10TaskIdTypeE8iteratorEbbbb", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue::done"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERKNSt6vectorINSt6size_tEEERK28TaskLayerModuleConfigListPtrNSt4listI10TaskIdTypeE8iteratorEbbbb", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue::inProgress"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERKNSt6vectorINSt6size_tEEERK28TaskLayerModuleConfigListPtrNSt4listI10TaskIdTypeE8iteratorEbbbb", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue::it"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERKNSt6vectorINSt6size_tEEERK28TaskLayerModuleConfigListPtrNSt4listI10TaskIdTypeE8iteratorEbbbb", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue::loadInProgress"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERKNSt6vectorINSt6size_tEEERK28TaskLayerModuleConfigListPtrNSt4listI10TaskIdTypeE8iteratorEbbbb", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue::loaded"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERR9TaskValue", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue::o"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue9TaskValueERKNSt6vectorINSt6size_tEEERK28TaskLayerModuleConfigListPtrNSt4listI10TaskIdTypeE8iteratorEbbbb", "tensorrt_llm::runtime::LoraCache::TaskValue::TaskValue::pageIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue7configsE", "tensorrt_llm::runtime::LoraCache::TaskValue::configs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue4doneE", "tensorrt_llm::runtime::LoraCache::TaskValue::done"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue10inProgressE", "tensorrt_llm::runtime::LoraCache::TaskValue::inProgress"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue2itE", "tensorrt_llm::runtime::LoraCache::TaskValue::it"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue14loadInProgressE", "tensorrt_llm::runtime::LoraCache::TaskValue::loadInProgress"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue6loadedE", "tensorrt_llm::runtime::LoraCache::TaskValue::loaded"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValueaSERR9TaskValue", "tensorrt_llm::runtime::LoraCache::TaskValue::operator="], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValueaSERR9TaskValue", "tensorrt_llm::runtime::LoraCache::TaskValue::operator=::o"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValue7pageIdsE", "tensorrt_llm::runtime::LoraCache::TaskValue::pageIds"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TaskValueD0Ev", "tensorrt_llm::runtime::LoraCache::TaskValue::~TaskValue"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache12TaskValuePtrE", "tensorrt_llm::runtime::LoraCache::TaskValuePtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9TensorPtrE", "tensorrt_llm::runtime::LoraCache::TensorPtr"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatusE", "tensorrt_llm::runtime::LoraCache::ValueStatus"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatus20kVALUE_STATUS_LOADEDE", "tensorrt_llm::runtime::LoraCache::ValueStatus::kVALUE_STATUS_LOADED"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatus21kVALUE_STATUS_MISSINGE", "tensorrt_llm::runtime::LoraCache::ValueStatus::kVALUE_STATUS_MISSING"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatus24kVALUE_STATUS_PROCESSINGE", "tensorrt_llm::runtime::LoraCache::ValueStatus::kVALUE_STATUS_PROCESSING"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache4bumpE10TaskIdType", "tensorrt_llm::runtime::LoraCache::bump"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache4bumpE10TaskIdType", "tensorrt_llm::runtime::LoraCache::bump::taskId"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache18bumpTaskInProgressE10TaskIdType", "tensorrt_llm::runtime::LoraCache::bumpTaskInProgress"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache18bumpTaskInProgressE10TaskIdType", "tensorrt_llm::runtime::LoraCache::bumpTaskInProgress::taskId"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache19claimPagesWithEvictE10SizeType32", "tensorrt_llm::runtime::LoraCache::claimPagesWithEvict"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache19claimPagesWithEvictE10SizeType32", "tensorrt_llm::runtime::LoraCache::claimPagesWithEvict::numPages"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache8copyTaskE10TaskIdTypeR9LoraCacheb", "tensorrt_llm::runtime::LoraCache::copyTask"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache8copyTaskE10TaskIdTypeR9LoraCacheb", "tensorrt_llm::runtime::LoraCache::copyTask::deviceCache"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache8copyTaskE10TaskIdTypeR9LoraCacheb", "tensorrt_llm::runtime::LoraCache::copyTask::markDone"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache8copyTaskE10TaskIdTypeR9LoraCacheb", "tensorrt_llm::runtime::LoraCache::copyTask::taskId"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache16copyTaskMapPagesER9TaskValueRK9TaskValueRKNSt6vectorI6size_tEERK9LoraCache", "tensorrt_llm::runtime::LoraCache::copyTaskMapPages"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache16copyTaskMapPagesER9TaskValueRK9TaskValueRKNSt6vectorI6size_tEERK9LoraCache", "tensorrt_llm::runtime::LoraCache::copyTaskMapPages::sourceTaskValue"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache16copyTaskMapPagesER9TaskValueRK9TaskValueRKNSt6vectorI6size_tEERK9LoraCache", "tensorrt_llm::runtime::LoraCache::copyTaskMapPages::targetCache"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache16copyTaskMapPagesER9TaskValueRK9TaskValueRKNSt6vectorI6size_tEERK9LoraCache", "tensorrt_llm::runtime::LoraCache::copyTaskMapPages::targetPageIds"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache16copyTaskMapPagesER9TaskValueRK9TaskValueRKNSt6vectorI6size_tEERK9LoraCache", "tensorrt_llm::runtime::LoraCache::copyTaskMapPages::targetTaskValue"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCache::copyToPages"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCache::copyToPages::config"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCache::copyToPages::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCache::copyToPages::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCache::copyToPages::moduleIdToModel"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCache::copyToPages::pageIds"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCache::copyToPages::pages"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCache::copyToPages::weights"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11copyToPagesE9TensorPtr9TensorPtrRK11ModelConfigRK11WorldConfigNSt13unordered_mapI10SizeType3210LoraModuleEERK13BufferManagerRKNSt6vectorI9TensorPtrEERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCache::copyToPages::worldConfig"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache17determineNumPagesE10TaskIdType", "tensorrt_llm::runtime::LoraCache::determineNumPages"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache17determineNumPagesE9TensorPtr", "tensorrt_llm::runtime::LoraCache::determineNumPages"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache17determineNumPagesE9TensorPtr", "tensorrt_llm::runtime::LoraCache::determineNumPages::config"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache17determineNumPagesE10TaskIdType", "tensorrt_llm::runtime::LoraCache::determineNumPages::taskId"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache4fitsE9TensorPtr", "tensorrt_llm::runtime::LoraCache::fits"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache4fitsE9TensorPtr", "tensorrt_llm::runtime::LoraCache::fits::config"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache3getE10TaskIdType", "tensorrt_llm::runtime::LoraCache::get"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache3getE10TaskIdType", "tensorrt_llm::runtime::LoraCache::get::taskId"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache11getNumPagesEv", "tensorrt_llm::runtime::LoraCache::getNumPages"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache10getPagePtrE6size_t", "tensorrt_llm::runtime::LoraCache::getPagePtr"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache10getPagePtrE6size_t", "tensorrt_llm::runtime::LoraCache::getPagePtr::pageId"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache9getStatusE10TaskIdType", "tensorrt_llm::runtime::LoraCache::getStatus"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache9getStatusE10TaskIdType", "tensorrt_llm::runtime::LoraCache::getStatus::taskId"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache3hasE10TaskIdType", "tensorrt_llm::runtime::LoraCache::has"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache3hasE10TaskIdType", "tensorrt_llm::runtime::LoraCache::has::taskId"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache6isDoneE10TaskIdType", "tensorrt_llm::runtime::LoraCache::isDone"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache6isDoneE10TaskIdType", "tensorrt_llm::runtime::LoraCache::isDone::taskId"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache8isLoadedE10TaskIdType", "tensorrt_llm::runtime::LoraCache::isLoaded"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime9LoraCache8isLoadedE10TaskIdType", "tensorrt_llm::runtime::LoraCache::isLoaded::taskId"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatus20kVALUE_STATUS_LOADEDE", "tensorrt_llm::runtime::LoraCache::kVALUE_STATUS_LOADED"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatus21kVALUE_STATUS_MISSINGE", "tensorrt_llm::runtime::LoraCache::kVALUE_STATUS_MISSING"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11ValueStatus24kVALUE_STATUS_PROCESSINGE", "tensorrt_llm::runtime::LoraCache::kVALUE_STATUS_PROCESSING"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsE10TaskIdType9TensorPtr9TensorPtr", "tensorrt_llm::runtime::LoraCache::loadWeights"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsER9TaskValue9TensorPtr9TensorPtr", "tensorrt_llm::runtime::LoraCache::loadWeights"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsER9TaskValue9TensorPtr9TensorPtr", "tensorrt_llm::runtime::LoraCache::loadWeights::cacheValue"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsE10TaskIdType9TensorPtr9TensorPtr", "tensorrt_llm::runtime::LoraCache::loadWeights::config"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsER9TaskValue9TensorPtr9TensorPtr", "tensorrt_llm::runtime::LoraCache::loadWeights::config"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsE10TaskIdType9TensorPtr9TensorPtr", "tensorrt_llm::runtime::LoraCache::loadWeights::taskId"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsE10TaskIdType9TensorPtr9TensorPtr", "tensorrt_llm::runtime::LoraCache::loadWeights::weights"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11loadWeightsER9TaskValue9TensorPtr9TensorPtr", "tensorrt_llm::runtime::LoraCache::loadWeights::weights"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache14mBufferManagerE", "tensorrt_llm::runtime::LoraCache::mBufferManager"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache9mCacheMapE", "tensorrt_llm::runtime::LoraCache::mCacheMap"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11mCacheMutexE", "tensorrt_llm::runtime::LoraCache::mCacheMutex"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17mCachePageManagerE", "tensorrt_llm::runtime::LoraCache::mCachePageManager"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache21mDeviceBufferManagersE", "tensorrt_llm::runtime::LoraCache::mDeviceBufferManagers"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache10mDoneTasksE", "tensorrt_llm::runtime::LoraCache::mDoneTasks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache16mInProgressTasksE", "tensorrt_llm::runtime::LoraCache::mInProgressTasks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache12mModelConfigE", "tensorrt_llm::runtime::LoraCache::mModelConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17mModuleIdToModuleE", "tensorrt_llm::runtime::LoraCache::mModuleIdToModule"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache18mPageManagerConfigE", "tensorrt_llm::runtime::LoraCache::mPageManagerConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11mPagesMutexE", "tensorrt_llm::runtime::LoraCache::mPagesMutex"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache12mWorldConfigE", "tensorrt_llm::runtime::LoraCache::mWorldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache11markAllDoneEv", "tensorrt_llm::runtime::LoraCache::markAllDone"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache12markTaskDoneE10TaskIdType", "tensorrt_llm::runtime::LoraCache::markTaskDone"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache12markTaskDoneE10TaskIdType", "tensorrt_llm::runtime::LoraCache::markTaskDone::taskId"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache3putE10TaskIdType9TensorPtr9TensorPtrb", "tensorrt_llm::runtime::LoraCache::put"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache3putE10TaskIdType9TensorPtr9TensorPtrb", "tensorrt_llm::runtime::LoraCache::put::config"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache3putE10TaskIdType9TensorPtr9TensorPtrb", "tensorrt_llm::runtime::LoraCache::put::load"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache3putE10TaskIdType9TensorPtr9TensorPtrb", "tensorrt_llm::runtime::LoraCache::put::taskId"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache3putE10TaskIdType9TensorPtr9TensorPtrb", "tensorrt_llm::runtime::LoraCache::put::weights"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17splitTransposeCpuER7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpu"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17splitTransposeCpuER7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpu::input"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17splitTransposeCpuER7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpu::output"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17splitTransposeCpuER7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpu::tpRank"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9LoraCache17splitTransposeCpuER7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpu::tpSize"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime9LoraCache22splitTransposeCpuInnerEvR7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpuInner"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime9LoraCache22splitTransposeCpuInnerEvR7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpuInner::T"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime9LoraCache22splitTransposeCpuInnerEvR7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpuInner::input"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime9LoraCache22splitTransposeCpuInnerEvR7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpuInner::output"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime9LoraCache22splitTransposeCpuInnerEvR7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpuInner::tpRank"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime9LoraCache22splitTransposeCpuInnerEvR7ITensorRK7ITensor10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCache::splitTransposeCpuInner::tpSize"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime22LoraCacheFullExceptionE", "tensorrt_llm::runtime::LoraCacheFullException"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22LoraCacheFullException22LoraCacheFullExceptionERKNSt6stringE", "tensorrt_llm::runtime::LoraCacheFullException::LoraCacheFullException"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime22LoraCacheFullException22LoraCacheFullExceptionERKNSt6stringE", "tensorrt_llm::runtime::LoraCacheFullException::LoraCacheFullException::msg"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime22LoraCacheFullExceptionD0Ev", "tensorrt_llm::runtime::LoraCacheFullException::~LoraCacheFullException"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManagerE", "tensorrt_llm::runtime::LoraCachePageManager"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager20LoraCachePageManagerERK26LoraCachePageManagerConfigRK13BufferManager", "tensorrt_llm::runtime::LoraCachePageManager::LoraCachePageManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager20LoraCachePageManagerERK26LoraCachePageManagerConfigRK13BufferManager", "tensorrt_llm::runtime::LoraCachePageManager::LoraCachePageManager::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager20LoraCachePageManagerERK26LoraCachePageManagerConfigRK13BufferManager", "tensorrt_llm::runtime::LoraCachePageManager::LoraCachePageManager::config"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager9TensorPtrE", "tensorrt_llm::runtime::LoraCachePageManager::TensorPtr"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime20LoraCachePageManager8blockPtrE10SizeType32", "tensorrt_llm::runtime::LoraCachePageManager::blockPtr"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime20LoraCachePageManager8blockPtrE10SizeType32", "tensorrt_llm::runtime::LoraCachePageManager::blockPtr::blockIdx"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager10claimPagesE10SizeType32", "tensorrt_llm::runtime::LoraCachePageManager::claimPages"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager10claimPagesE10SizeType32", "tensorrt_llm::runtime::LoraCachePageManager::claimPages::numPages"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager10initializeERK13BufferManager", "tensorrt_llm::runtime::LoraCachePageManager::initialize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager10initializeERK13BufferManager", "tensorrt_llm::runtime::LoraCachePageManager::initialize::bufferManager"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager7mConfigE", "tensorrt_llm::runtime::LoraCachePageManager::mConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager12mFreePageIdsE", "tensorrt_llm::runtime::LoraCachePageManager::mFreePageIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager11mIsPageFreeE", "tensorrt_llm::runtime::LoraCachePageManager::mIsPageFree"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager11mPageBlocksE", "tensorrt_llm::runtime::LoraCachePageManager::mPageBlocks"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager14mutablePagePtrENSt6size_tE", "tensorrt_llm::runtime::LoraCachePageManager::mutablePagePtr"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager14mutablePagePtrENSt6size_tE", "tensorrt_llm::runtime::LoraCachePageManager::mutablePagePtr::pageIdx"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime20LoraCachePageManager17numAvailablePagesEv", "tensorrt_llm::runtime::LoraCachePageManager::numAvailablePages"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime20LoraCachePageManager7pagePtrENSt6size_tE", "tensorrt_llm::runtime::LoraCachePageManager::pagePtr"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime20LoraCachePageManager7pagePtrENSt6size_tE", "tensorrt_llm::runtime::LoraCachePageManager::pagePtr::pageIdx"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager12releasePagesERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCachePageManager::releasePages"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20LoraCachePageManager12releasePagesERKNSt6vectorINSt6size_tEEE", "tensorrt_llm::runtime::LoraCachePageManager::releasePages::pages"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfigE", "tensorrt_llm::runtime::LoraCachePageManagerConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig26LoraCachePageManagerConfigEN7runtime10MemoryTypeEN8nvinfer18DataTypeE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::LoraCachePageManagerConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig26LoraCachePageManagerConfigEN7runtime10MemoryTypeEN8nvinfer18DataTypeE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::LoraCachePageManagerConfig::dType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig26LoraCachePageManagerConfigEN7runtime10MemoryTypeEN8nvinfer18DataTypeE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::LoraCachePageManagerConfig::maxPagesPerBlock"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig26LoraCachePageManagerConfigEN7runtime10MemoryTypeEN8nvinfer18DataTypeE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::LoraCachePageManagerConfig::memType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig26LoraCachePageManagerConfigEN7runtime10MemoryTypeEN8nvinfer18DataTypeE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::LoraCachePageManagerConfig::numCopyStreams"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig26LoraCachePageManagerConfigEN7runtime10MemoryTypeEN8nvinfer18DataTypeE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::LoraCachePageManagerConfig::pageWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig26LoraCachePageManagerConfigEN7runtime10MemoryTypeEN8nvinfer18DataTypeE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::LoraCachePageManagerConfig::slotsPerPage"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig26LoraCachePageManagerConfigEN7runtime10MemoryTypeEN8nvinfer18DataTypeE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::LoraCachePageManagerConfig::totalNumPages"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig11getDataTypeEv", "tensorrt_llm::runtime::LoraCachePageManagerConfig::getDataType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig13getInitToZeroEv", "tensorrt_llm::runtime::LoraCachePageManagerConfig::getInitToZero"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig19getMaxPagesPerBlockEv", "tensorrt_llm::runtime::LoraCachePageManagerConfig::getMaxPagesPerBlock"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig13getMemoryTypeEv", "tensorrt_llm::runtime::LoraCachePageManagerConfig::getMemoryType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig17getNumCopyStreamsEv", "tensorrt_llm::runtime::LoraCachePageManagerConfig::getNumCopyStreams"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig12getPageWidthEv", "tensorrt_llm::runtime::LoraCachePageManagerConfig::getPageWidth"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig15getSlotsPerPageEv", "tensorrt_llm::runtime::LoraCachePageManagerConfig::getSlotsPerPage"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime26LoraCachePageManagerConfig16getTotalNumPagesEv", "tensorrt_llm::runtime::LoraCachePageManagerConfig::getTotalNumPages"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig9mDataTypeE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::mDataType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig11mInitToZeroE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::mInitToZero"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig17mMaxPagesPerBlockE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::mMaxPagesPerBlock"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig11mMemoryTypeE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::mMemoryType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig15mNumCopyStreamsE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::mNumCopyStreams"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig10mPageWidthE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::mPageWidth"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig13mSlotsPerPageE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::mSlotsPerPage"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig14mTotalNumPagesE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::mTotalNumPages"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig11setDataTypeERKN8nvinfer18DataTypeE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setDataType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig11setDataTypeERKN8nvinfer18DataTypeE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setDataType::dtype"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig13setInitToZeroEb", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setInitToZero"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig13setInitToZeroEb", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setInitToZero::initToZero"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig19setMaxPagesPerBlockERK10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setMaxPagesPerBlock"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig19setMaxPagesPerBlockERK10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setMaxPagesPerBlock::maxPagesPerBlock"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig13setMemoryTypeERKN7runtime10MemoryTypeE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setMemoryType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig13setMemoryTypeERKN7runtime10MemoryTypeE", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setMemoryType::memoryType"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig17setNumCopyStreamsE10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setNumCopyStreams"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig17setNumCopyStreamsE10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setNumCopyStreams::numCopyStreams"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig12setPageWidthERK10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setPageWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig12setPageWidthERK10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setPageWidth::pageWidth"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig15setSlotsPerPageERK10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setSlotsPerPage"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig15setSlotsPerPageERK10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setSlotsPerPage::slotsPerPage"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig15setTotalNumPageERK10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setTotalNumPage"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime26LoraCachePageManagerConfig15setTotalNumPageERK10SizeType32", "tensorrt_llm::runtime::LoraCachePageManagerConfig::setTotalNumPage::totalNumPages"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime21LoraExpectedExceptionE", "tensorrt_llm::runtime::LoraExpectedException"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime21LoraExpectedException21LoraExpectedExceptionERKNSt6stringE", "tensorrt_llm::runtime::LoraExpectedException::LoraExpectedException"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime21LoraExpectedException21LoraExpectedExceptionERKNSt6stringE", "tensorrt_llm::runtime::LoraExpectedException::LoraExpectedException::msg"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime21LoraExpectedExceptionD0Ev", "tensorrt_llm::runtime::LoraExpectedException::~LoraExpectedException"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModuleE", "tensorrt_llm::runtime::LoraModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10LoraModule", "tensorrt_llm::runtime::LoraModule::LoraModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10ModuleType10SizeType3210SizeType32bb10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::LoraModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleEv", "tensorrt_llm::runtime::LoraModule::LoraModule"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10ModuleType10SizeType3210SizeType32bb10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::LoraModule::inDim"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10ModuleType10SizeType3210SizeType32bb10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::LoraModule::inDimFirst"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10ModuleType10SizeType3210SizeType32bb10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::LoraModule::inTpSplitDim"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10LoraModule", "tensorrt_llm::runtime::LoraModule::LoraModule::o"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10ModuleType10SizeType3210SizeType32bb10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::LoraModule::outDim"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10ModuleType10SizeType3210SizeType32bb10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::LoraModule::outDimFirst"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10ModuleType10SizeType3210SizeType32bb10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::LoraModule::outTpSplitDim"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10LoraModuleERK10ModuleType10SizeType3210SizeType32bb10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::LoraModule::t"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleTypeE", "tensorrt_llm::runtime::LoraModule::ModuleType"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType11kATTN_DENSEE", "tensorrt_llm::runtime::LoraModule::ModuleType::kATTN_DENSE"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType7kATTN_KE", "tensorrt_llm::runtime::LoraModule::ModuleType::kATTN_K"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType7kATTN_QE", "tensorrt_llm::runtime::LoraModule::ModuleType::kATTN_Q"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType9kATTN_QKVE", "tensorrt_llm::runtime::LoraModule::ModuleType::kATTN_QKV"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType7kATTN_VE", "tensorrt_llm::runtime::LoraModule::ModuleType::kATTN_V"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType17kCROSS_ATTN_DENSEE", "tensorrt_llm::runtime::LoraModule::ModuleType::kCROSS_ATTN_DENSE"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType13kCROSS_ATTN_KE", "tensorrt_llm::runtime::LoraModule::ModuleType::kCROSS_ATTN_K"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType13kCROSS_ATTN_QE", "tensorrt_llm::runtime::LoraModule::ModuleType::kCROSS_ATTN_Q"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType15kCROSS_ATTN_QKVE", "tensorrt_llm::runtime::LoraModule::ModuleType::kCROSS_ATTN_QKV"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType13kCROSS_ATTN_VE", "tensorrt_llm::runtime::LoraModule::ModuleType::kCROSS_ATTN_V"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType8kINVALIDE", "tensorrt_llm::runtime::LoraModule::ModuleType::kINVALID"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMLP_4H_TO_HE", "tensorrt_llm::runtime::LoraModule::ModuleType::kMLP_4H_TO_H"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType9kMLP_GATEE", "tensorrt_llm::runtime::LoraModule::ModuleType::kMLP_GATE"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMLP_GATE_UPE", "tensorrt_llm::runtime::LoraModule::ModuleType::kMLP_GATE_UP"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMLP_H_TO_4HE", "tensorrt_llm::runtime::LoraModule::ModuleType::kMLP_H_TO_4H"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType11kMLP_ROUTERE", "tensorrt_llm::runtime::LoraModule::ModuleType::kMLP_ROUTER"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMOE_4H_TO_HE", "tensorrt_llm::runtime::LoraModule::ModuleType::kMOE_4H_TO_H"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType9kMOE_GATEE", "tensorrt_llm::runtime::LoraModule::ModuleType::kMOE_GATE"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType12kMOE_H_TO_4HE", "tensorrt_llm::runtime::LoraModule::ModuleType::kMOE_H_TO_4H"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule10ModuleType11kMOE_ROUTERE", "tensorrt_llm::runtime::LoraModule::ModuleType::kMOE_ROUTER"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule9TensorPtrE", "tensorrt_llm::runtime::LoraModule::TensorPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::createLoraModules"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::createLoraModules::attentionHeadSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::createLoraModules::hiddenSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::createLoraModules::loraModuleNames"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::createLoraModules::mlpHiddenSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::createLoraModules::numAttentionHeads"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::createLoraModules::numExperts"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::createLoraModules::numKvAttentionHeads"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule17createLoraModulesERKNSt6vectorINSt6stringEEE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::createLoraModules::tpSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule18flattenedInOutSizeE10SizeType32b", "tensorrt_llm::runtime::LoraModule::flattenedInOutSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule18flattenedInOutSizeE10SizeType32b", "tensorrt_llm::runtime::LoraModule::flattenedInOutSize::adapterSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule18flattenedInOutSizeE10SizeType32b", "tensorrt_llm::runtime::LoraModule::flattenedInOutSize::isDora"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule5inDimEv", "tensorrt_llm::runtime::LoraModule::inDim"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule10inDimFirstEv", "tensorrt_llm::runtime::LoraModule::inDimFirst"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule6inSizeE10SizeType32", "tensorrt_llm::runtime::LoraModule::inSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule6inSizeE10SizeType32", "tensorrt_llm::runtime::LoraModule::inSize::adapterSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule12inTpSplitDimEv", "tensorrt_llm::runtime::LoraModule::inTpSplitDim"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule18localInAdapterSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localInAdapterSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule18localInAdapterSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localInAdapterSize::adapterSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule18localInAdapterSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localInAdapterSize::tpSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule10localInDimE10SizeType32", "tensorrt_llm::runtime::LoraModule::localInDim"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule10localInDimE10SizeType32", "tensorrt_llm::runtime::LoraModule::localInDim::tpSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule14localInOutSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localInOutSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule14localInOutSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localInOutSize::adapterSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule14localInOutSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localInOutSize::tpSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule11localInSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localInSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule11localInSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localInSize::adapterSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule11localInSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localInSize::tpSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule19localOutAdapterSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localOutAdapterSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule19localOutAdapterSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localOutAdapterSize::adapterSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule19localOutAdapterSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localOutAdapterSize::tpSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule11localOutDimE10SizeType32", "tensorrt_llm::runtime::LoraModule::localOutDim"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule11localOutDimE10SizeType32", "tensorrt_llm::runtime::LoraModule::localOutDim::tpSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule12localOutSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localOutSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule12localOutSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localOutSize::adapterSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule12localOutSizeE10SizeType3210SizeType32", "tensorrt_llm::runtime::LoraModule::localOutSize::tpSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule15localScalesSizeE10SizeType32b", "tensorrt_llm::runtime::LoraModule::localScalesSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule15localScalesSizeE10SizeType32b", "tensorrt_llm::runtime::LoraModule::localScalesSize::isDora"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule15localScalesSizeE10SizeType32b", "tensorrt_llm::runtime::LoraModule::localScalesSize::tpSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule14localTotalSizeE10SizeType3210SizeType32b", "tensorrt_llm::runtime::LoraModule::localTotalSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule14localTotalSizeE10SizeType3210SizeType32b", "tensorrt_llm::runtime::LoraModule::localTotalSize::adapterSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule14localTotalSizeE10SizeType3210SizeType32b", "tensorrt_llm::runtime::LoraModule::localTotalSize::isDora"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule14localTotalSizeE10SizeType3210SizeType32b", "tensorrt_llm::runtime::LoraModule::localTotalSize::tpSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule6mInDimE", "tensorrt_llm::runtime::LoraModule::mInDim"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule11mInDimFirstE", "tensorrt_llm::runtime::LoraModule::mInDimFirst"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule13mInTpSplitDimE", "tensorrt_llm::runtime::LoraModule::mInTpSplitDim"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule7mOutDimE", "tensorrt_llm::runtime::LoraModule::mOutDim"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12mOutDimFirstE", "tensorrt_llm::runtime::LoraModule::mOutDimFirst"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule14mOutTpSplitDimE", "tensorrt_llm::runtime::LoraModule::mOutTpSplitDim"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule5mTypeE", "tensorrt_llm::runtime::LoraModule::mType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule4nameEv", "tensorrt_llm::runtime::LoraModule::name"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModuleaSERK10LoraModule", "tensorrt_llm::runtime::LoraModule::operator="], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModuleaSERK10LoraModule", "tensorrt_llm::runtime::LoraModule::operator=::o"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule6outDimEv", "tensorrt_llm::runtime::LoraModule::outDim"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule11outDimFirstEv", "tensorrt_llm::runtime::LoraModule::outDimFirst"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule7outSizeE10SizeType32", "tensorrt_llm::runtime::LoraModule::outSize"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule7outSizeE10SizeType32", "tensorrt_llm::runtime::LoraModule::outSize::adapterSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule13outTpSplitDimEv", "tensorrt_llm::runtime::LoraModule::outTpSplitDim"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12toModuleNameE10ModuleType", "tensorrt_llm::runtime::LoraModule::toModuleName"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12toModuleNameE10SizeType32", "tensorrt_llm::runtime::LoraModule::toModuleName"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12toModuleNameE10SizeType32", "tensorrt_llm::runtime::LoraModule::toModuleName::id"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12toModuleNameE10ModuleType", "tensorrt_llm::runtime::LoraModule::toModuleName::t"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12toModuleTypeERKNSt11string_viewE", "tensorrt_llm::runtime::LoraModule::toModuleType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10LoraModule12toModuleTypeERKNSt11string_viewE", "tensorrt_llm::runtime::LoraModule::toModuleType::name"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime10LoraModule5valueEv", "tensorrt_llm::runtime::LoraModule::value"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14LoraTaskIdTypeE", "tensorrt_llm::runtime::LoraTaskIdType"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime17MPI_group_barrierENSt3setIiEE", "tensorrt_llm::runtime::MPI_group_barrier"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime17MPI_group_barrierENSt3setIiEE", "tensorrt_llm::runtime::MPI_group_barrier::ranks"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime12MedusaModuleE", "tensorrt_llm::runtime::MedusaModule"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule13MedusaChoicesE", "tensorrt_llm::runtime::MedusaModule::MedusaChoices"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule12MedusaModuleE10SizeType3210SizeType32", "tensorrt_llm::runtime::MedusaModule::MedusaModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule12MedusaModuleEv", "tensorrt_llm::runtime::MedusaModule::MedusaModule"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule12MedusaModuleE10SizeType3210SizeType32", "tensorrt_llm::runtime::MedusaModule::MedusaModule::maxAcceptedTokens"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule12MedusaModuleE10SizeType3210SizeType32", "tensorrt_llm::runtime::MedusaModule::MedusaModule::maxDraftTokens"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule9TensorPtrE", "tensorrt_llm::runtime::MedusaModule::TensorPtr"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime12MedusaModule16getMedusaChoicesEv", "tensorrt_llm::runtime::MedusaModule::getMedusaChoices"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime12MedusaModule21mDefaultMedusaChoicesE", "tensorrt_llm::runtime::MedusaModule::mDefaultMedusaChoices"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCountersE", "tensorrt_llm::runtime::MemoryCounters"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8DiffTypeE", "tensorrt_llm::runtime::MemoryCounters::DiffType"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters14MemoryCountersEv", "tensorrt_llm::runtime::MemoryCounters::MemoryCounters"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters10SizeType32E", "tensorrt_llm::runtime::MemoryCounters::SizeType32"], [1, 3, 1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime14MemoryCounters8allocateEv10SizeType32", "tensorrt_llm::runtime::MemoryCounters::allocate"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8allocateE10MemoryType10SizeType32", "tensorrt_llm::runtime::MemoryCounters::allocate"], [1, 8, 1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime14MemoryCounters8allocateEv10SizeType32", "tensorrt_llm::runtime::MemoryCounters::allocate::T"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8allocateE10MemoryType10SizeType32", "tensorrt_llm::runtime::MemoryCounters::allocate::memoryType"], [1, 4, 1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime14MemoryCounters8allocateEv10SizeType32", "tensorrt_llm::runtime::MemoryCounters::allocate::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8allocateE10MemoryType10SizeType32", "tensorrt_llm::runtime::MemoryCounters::allocate::size"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters13bytesToStringE10SizeType32i", "tensorrt_llm::runtime::MemoryCounters::bytesToString"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters13bytesToStringE8DiffTypei", "tensorrt_llm::runtime::MemoryCounters::bytesToString"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters13bytesToStringE10SizeType32i", "tensorrt_llm::runtime::MemoryCounters::bytesToString::bytes"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters13bytesToStringE8DiffTypei", "tensorrt_llm::runtime::MemoryCounters::bytesToString::bytes"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters13bytesToStringE10SizeType32i", "tensorrt_llm::runtime::MemoryCounters::bytesToString::precision"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters13bytesToStringE8DiffTypei", "tensorrt_llm::runtime::MemoryCounters::bytesToString::precision"], [1, 3, 1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime14MemoryCounters10deallocateEv10SizeType32", "tensorrt_llm::runtime::MemoryCounters::deallocate"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters10deallocateE10MemoryType10SizeType32", "tensorrt_llm::runtime::MemoryCounters::deallocate"], [1, 8, 1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime14MemoryCounters10deallocateEv10SizeType32", "tensorrt_llm::runtime::MemoryCounters::deallocate::T"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters10deallocateE10MemoryType10SizeType32", "tensorrt_llm::runtime::MemoryCounters::deallocate::memoryType"], [1, 4, 1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime14MemoryCounters10deallocateEv10SizeType32", "tensorrt_llm::runtime::MemoryCounters::deallocate::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters10deallocateE10MemoryType10SizeType32", "tensorrt_llm::runtime::MemoryCounters::deallocate::size"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters6getCpuEv", "tensorrt_llm::runtime::MemoryCounters::getCpu"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters10getCpuDiffEv", "tensorrt_llm::runtime::MemoryCounters::getCpuDiff"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters6getGpuEv", "tensorrt_llm::runtime::MemoryCounters::getGpu"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters10getGpuDiffEv", "tensorrt_llm::runtime::MemoryCounters::getGpuDiff"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters11getInstanceEv", "tensorrt_llm::runtime::MemoryCounters::getInstance"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters9getPinnedEv", "tensorrt_llm::runtime::MemoryCounters::getPinned"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters13getPinnedDiffEv", "tensorrt_llm::runtime::MemoryCounters::getPinnedDiff"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters13getPinnedPoolEv", "tensorrt_llm::runtime::MemoryCounters::getPinnedPool"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters17getPinnedPoolDiffEv", "tensorrt_llm::runtime::MemoryCounters::getPinnedPoolDiff"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters6getUVMEv", "tensorrt_llm::runtime::MemoryCounters::getUVM"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters10getUVMDiffEv", "tensorrt_llm::runtime::MemoryCounters::getUVMDiff"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters4mCpuE", "tensorrt_llm::runtime::MemoryCounters::mCpu"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8mCpuDiffE", "tensorrt_llm::runtime::MemoryCounters::mCpuDiff"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters4mGpuE", "tensorrt_llm::runtime::MemoryCounters::mGpu"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8mGpuDiffE", "tensorrt_llm::runtime::MemoryCounters::mGpuDiff"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters7mPinnedE", "tensorrt_llm::runtime::MemoryCounters::mPinned"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters11mPinnedDiffE", "tensorrt_llm::runtime::MemoryCounters::mPinnedDiff"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters11mPinnedPoolE", "tensorrt_llm::runtime::MemoryCounters::mPinnedPool"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters15mPinnedPoolDiffE", "tensorrt_llm::runtime::MemoryCounters::mPinnedPoolDiff"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters4mUVME", "tensorrt_llm::runtime::MemoryCounters::mUVM"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14MemoryCounters8mUVMDiffE", "tensorrt_llm::runtime::MemoryCounters::mUVMDiff"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14MemoryCounters8toStringEv", "tensorrt_llm::runtime::MemoryCounters::toString"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime10MemoryTypeE", "tensorrt_llm::runtime::MemoryType"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10MemoryType4kCPUE", "tensorrt_llm::runtime::MemoryType::kCPU"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10MemoryType4kGPUE", "tensorrt_llm::runtime::MemoryType::kGPU"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10MemoryType7kPINNEDE", "tensorrt_llm::runtime::MemoryType::kPINNED"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10MemoryType11kPINNEDPOOLE", "tensorrt_llm::runtime::MemoryType::kPINNEDPOOL"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime10MemoryType4kUVME", "tensorrt_llm::runtime::MemoryType::kUVM"], [1, 2, 1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime16MemoryTypeStringE", "tensorrt_llm::runtime::MemoryTypeString"], [1, 8, 1, "_CPPv4I_10MemoryTypeEN12tensorrt_llm7runtime16MemoryTypeStringE", "tensorrt_llm::runtime::MemoryTypeString::T"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kCPUEEE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kCPU>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kCPUEE5valueE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kCPU>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kGPUEEE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kGPU>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kGPUEE5valueE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kGPU>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType7kPINNEDEEE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kPINNED>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType7kPINNEDEE5valueE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kPINNED>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType11kPINNEDPOOLEEE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kPINNEDPOOL>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType11kPINNEDPOOLEE5valueE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kPINNEDPOOL>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kUVMEEE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kUVM>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime16MemoryTypeStringIN10MemoryType4kUVMEE5valueE", "tensorrt_llm::runtime::MemoryTypeString<MemoryType::kUVM>::value"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfiguratorE", "tensorrt_llm::runtime::MemsetConfigurator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator18MemsetConfiguratorE11CUdeviceptr6size_t7uint8_t8CUstream", "tensorrt_llm::runtime::MemsetConfigurator::MemsetConfigurator"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator18MemsetConfiguratorE11CUdeviceptr6size_t7uint8_t8CUstream", "tensorrt_llm::runtime::MemsetConfigurator::MemsetConfigurator::address"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator18MemsetConfiguratorE11CUdeviceptr6size_t7uint8_t8CUstream", "tensorrt_llm::runtime::MemsetConfigurator::MemsetConfigurator::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator18MemsetConfiguratorE11CUdeviceptr6size_t7uint8_t8CUstream", "tensorrt_llm::runtime::MemsetConfigurator::MemsetConfigurator::stream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator18MemsetConfiguratorE11CUdeviceptr6size_t7uint8_t8CUstream", "tensorrt_llm::runtime::MemsetConfigurator::MemsetConfigurator::value"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator8mAddressE", "tensorrt_llm::runtime::MemsetConfigurator::mAddress"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator10mFirstTimeE", "tensorrt_llm::runtime::MemsetConfigurator::mFirstTime"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator5mSizeE", "tensorrt_llm::runtime::MemsetConfigurator::mSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator7mStreamE", "tensorrt_llm::runtime::MemsetConfigurator::mStream"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator6mValueE", "tensorrt_llm::runtime::MemsetConfigurator::mValue"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator5setupE28CUmemGenericAllocationHandle", "tensorrt_llm::runtime::MemsetConfigurator::setup"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18MemsetConfigurator8teardownE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::MemsetConfigurator::teardown"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfigE", "tensorrt_llm::runtime::ModelConfig"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11KVCacheTypeE", "tensorrt_llm::runtime::ModelConfig::KVCacheType"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11KVCacheType11kCONTINUOUSE", "tensorrt_llm::runtime::ModelConfig::KVCacheType::kCONTINUOUS"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11KVCacheType9kDISABLEDE", "tensorrt_llm::runtime::ModelConfig::KVCacheType::kDISABLED"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11KVCacheType6kPAGEDE", "tensorrt_llm::runtime::ModelConfig::KVCacheType::kPAGED"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21KVCacheTypeFromStringENSt6stringE", "tensorrt_llm::runtime::ModelConfig::KVCacheTypeFromString"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21KVCacheTypeFromStringENSt6stringE", "tensorrt_llm::runtime::ModelConfig::KVCacheTypeFromString::value"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerTypeE", "tensorrt_llm::runtime::ModelConfig::LayerType"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerType10kATTENTIONE", "tensorrt_llm::runtime::ModelConfig::LayerType::kATTENTION"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerType7kLINEARE", "tensorrt_llm::runtime::ModelConfig::LayerType::kLINEAR"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerType5kNOOPE", "tensorrt_llm::runtime::ModelConfig::LayerType::kNOOP"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9LayerType10kRECURRENTE", "tensorrt_llm::runtime::ModelConfig::LayerType::kRECURRENT"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17ManageWeightsTypeE", "tensorrt_llm::runtime::ModelConfig::ManageWeightsType"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17ManageWeightsType9kDisabledE", "tensorrt_llm::runtime::ModelConfig::ManageWeightsType::kDisabled"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17ManageWeightsType8kEnabledE", "tensorrt_llm::runtime::ModelConfig::ManageWeightsType::kEnabled"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11ModelConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::ModelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11ModelConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::ModelConfig::dtype"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11ModelConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::ModelConfig::hiddenSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11ModelConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::ModelConfig::nbAttentionLayers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11ModelConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::ModelConfig::nbHeads"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11ModelConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::ModelConfig::nbLayers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11ModelConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::ModelConfig::nbRnnLayers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11ModelConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::ModelConfig::vocabSize"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariantE", "tensorrt_llm::runtime::ModelConfig::ModelVariant"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant8kChatGlmE", "tensorrt_llm::runtime::ModelConfig::ModelVariant::kChatGlm"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant7kEncDecE", "tensorrt_llm::runtime::ModelConfig::ModelVariant::kEncDec"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant4kGlmE", "tensorrt_llm::runtime::ModelConfig::ModelVariant::kGlm"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant4kGptE", "tensorrt_llm::runtime::ModelConfig::ModelVariant::kGpt"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant6kMambaE", "tensorrt_llm::runtime::ModelConfig::ModelVariant::kMamba"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12ModelVariant15kRecurrentGemmaE", "tensorrt_llm::runtime::ModelConfig::ModelVariant::kRecurrentGemma"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfigE", "tensorrt_llm::runtime::ModelConfig::RnnConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig10convKernelE", "tensorrt_llm::runtime::ModelConfig::RnnConfig::convKernel"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig14rnnConvDimSizeE", "tensorrt_llm::runtime::ModelConfig::RnnConfig::rnnConvDimSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig11rnnHeadSizeE", "tensorrt_llm::runtime::ModelConfig::RnnConfig::rnnHeadSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig13rnnHiddenSizeE", "tensorrt_llm::runtime::ModelConfig::RnnConfig::rnnHiddenSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9RnnConfig9stateSizeE", "tensorrt_llm::runtime::ModelConfig::RnnConfig::stateSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20computeContextLogitsEb", "tensorrt_llm::runtime::ModelConfig::computeContextLogits"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20computeContextLogitsEv", "tensorrt_llm::runtime::ModelConfig::computeContextLogits"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20computeContextLogitsEb", "tensorrt_llm::runtime::ModelConfig::computeContextLogits::computeContextLogits"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig23computeGenerationLogitsEb", "tensorrt_llm::runtime::ModelConfig::computeGenerationLogits"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig23computeGenerationLogitsEv", "tensorrt_llm::runtime::ModelConfig::computeGenerationLogits"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig23computeGenerationLogitsEb", "tensorrt_llm::runtime::ModelConfig::computeGenerationLogits::computeGenerationLogits"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16countLocalLayersE9LayerType10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::countLocalLayers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16countLocalLayersE9LayerType10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::countLocalLayers::layerType"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16countLocalLayersE9LayerType10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::countLocalLayers::pipelineParallelism"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16countLocalLayersE9LayerType10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::countLocalLayers::pipelineParallelismRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20countLowerRankLayersE9LayerType10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::countLowerRankLayers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20countLowerRankLayersE9LayerType10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::countLowerRankLayers::layerType"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20countLowerRankLayersE9LayerType10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::countLowerRankLayers::pipelineParallelism"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20countLowerRankLayersE9LayerType10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::countLowerRankLayers::pipelineParallelismRank"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig32disableSeamlessLookaheadDecodingEv", "tensorrt_llm::runtime::ModelConfig::disableSeamlessLookaheadDecoding"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig31enableSeamlessLookaheadDecodingE10SizeType32", "tensorrt_llm::runtime::ModelConfig::enableSeamlessLookaheadDecoding"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig31enableSeamlessLookaheadDecodingE10SizeType32", "tensorrt_llm::runtime::ModelConfig::enableSeamlessLookaheadDecoding::maxDraftTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getContextFMHAEv", "tensorrt_llm::runtime::ModelConfig::getContextFMHA"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig11getDataTypeEv", "tensorrt_llm::runtime::ModelConfig::getDataType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getEncoderHiddenSizeEv", "tensorrt_llm::runtime::ModelConfig::getEncoderHiddenSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18getFirstLocalLayerE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getFirstLocalLayer"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18getFirstLocalLayerE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getFirstLocalLayer::pipelineParallelism"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18getFirstLocalLayerE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getFirstLocalLayer::pipelineParallelismRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21getGemmAllReduceDtypeEv", "tensorrt_llm::runtime::ModelConfig::getGemmAllReduceDtype"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13getHiddenSizeEv", "tensorrt_llm::runtime::ModelConfig::getHiddenSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getKVCacheTypeEv", "tensorrt_llm::runtime::ModelConfig::getKVCacheType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13getKvDataTypeEv", "tensorrt_llm::runtime::ModelConfig::getKvDataType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13getLayerTypesEv", "tensorrt_llm::runtime::ModelConfig::getLayerTypes"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getLogitsDtypeEv", "tensorrt_llm::runtime::ModelConfig::getLogitsDtype"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getLoraModulesEv", "tensorrt_llm::runtime::ModelConfig::getLoraModules"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getManageWeightsTypeEv", "tensorrt_llm::runtime::ModelConfig::getManageWeightsType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getMaxBatchSizeEv", "tensorrt_llm::runtime::ModelConfig::getMaxBatchSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getMaxBeamWidthEv", "tensorrt_llm::runtime::ModelConfig::getMaxBeamWidth"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig25getMaxDecodingDraftTokensEv", "tensorrt_llm::runtime::ModelConfig::getMaxDecodingDraftTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getMaxDecodingTokensEv", "tensorrt_llm::runtime::ModelConfig::getMaxDecodingTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16getMaxEncoderLenEv", "tensorrt_llm::runtime::ModelConfig::getMaxEncoderLen"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getMaxInputLenEv", "tensorrt_llm::runtime::ModelConfig::getMaxInputLen"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getMaxLoraRankEv", "tensorrt_llm::runtime::ModelConfig::getMaxLoraRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getMaxNumTokensEv", "tensorrt_llm::runtime::ModelConfig::getMaxNumTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig24getMaxPositionEmbeddingsEv", "tensorrt_llm::runtime::ModelConfig::getMaxPositionEmbeddings"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig30getMaxPromptEmbeddingTableSizeEv", "tensorrt_llm::runtime::ModelConfig::getMaxPromptEmbeddingTableSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig17getMaxSequenceLenEv", "tensorrt_llm::runtime::ModelConfig::getMaxSequenceLen"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16getMlpHiddenSizeEv", "tensorrt_llm::runtime::ModelConfig::getMlpHiddenSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getModelNameEv", "tensorrt_llm::runtime::ModelConfig::getModelName"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getModelVariantEv", "tensorrt_llm::runtime::ModelConfig::getModelVariant"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getNbAttentionLayersE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbAttentionLayers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getNbAttentionLayersE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbAttentionLayers::pipelineParallelism"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20getNbAttentionLayersE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbAttentionLayers::pipelineParallelismRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig10getNbHeadsEv", "tensorrt_llm::runtime::ModelConfig::getNbHeads"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getNbKvHeadsE10SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbKvHeads"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getNbKvHeadsE10SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbKvHeads::layerIdx"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig11getNbLayersE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbLayers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig11getNbLayersE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbLayers::pipelineParallelism"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig11getNbLayersE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbLayers::pipelineParallelismRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getNbRnnLayersE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbRnnLayers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getNbRnnLayersE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbRnnLayers::pipelineParallelism"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getNbRnnLayersE10SizeType3210SizeType32", "tensorrt_llm::runtime::ModelConfig::getNbRnnLayers::pipelineParallelismRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig27getNumKvHeadsForGivenLayersERKNSt6vectorI10SizeType32EEb", "tensorrt_llm::runtime::ModelConfig::getNumKvHeadsForGivenLayers"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig27getNumKvHeadsForGivenLayersERKNSt6vectorI10SizeType32EEb", "tensorrt_llm::runtime::ModelConfig::getNumKvHeadsForGivenLayers::isCrossAttention"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig27getNumKvHeadsForGivenLayersERKNSt6vectorI10SizeType32EEb", "tensorrt_llm::runtime::ModelConfig::getNumKvHeadsForGivenLayers::layers"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21getNumKvHeadsPerLayerEv", "tensorrt_llm::runtime::ModelConfig::getNumKvHeadsPerLayer"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig31getNumKvHeadsPerLayerLocalRangeE10SizeType3210SizeType32b", "tensorrt_llm::runtime::ModelConfig::getNumKvHeadsPerLayerLocalRange"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig31getNumKvHeadsPerLayerLocalRangeE10SizeType3210SizeType32b", "tensorrt_llm::runtime::ModelConfig::getNumKvHeadsPerLayerLocalRange::isCrossAttention"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig31getNumKvHeadsPerLayerLocalRangeE10SizeType3210SizeType32b", "tensorrt_llm::runtime::ModelConfig::getNumKvHeadsPerLayerLocalRange::pipelineParallelism"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig31getNumKvHeadsPerLayerLocalRangeE10SizeType3210SizeType32b", "tensorrt_llm::runtime::ModelConfig::getNumKvHeadsPerLayerLocalRange::pipelineParallelismRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15getNumLanguagesEv", "tensorrt_llm::runtime::ModelConfig::getNumLanguages"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig25getOptProfilesSplitPointsEv", "tensorrt_llm::runtime::ModelConfig::getOptProfilesSplitPoints"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig19getPagedContextFMHAEv", "tensorrt_llm::runtime::ModelConfig::getPagedContextFMHA"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18getPpReduceScatterEv", "tensorrt_llm::runtime::ModelConfig::getPpReduceScatter"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getQuantModeEv", "tensorrt_llm::runtime::ModelConfig::getQuantMode"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getRnnConfigEv", "tensorrt_llm::runtime::ModelConfig::getRnnConfig"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21getRotaryEmbeddingDimEv", "tensorrt_llm::runtime::ModelConfig::getRotaryEmbeddingDim"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14getSizePerHeadEv", "tensorrt_llm::runtime::ModelConfig::getSizePerHead"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig26getSpeculativeDecodingModeEv", "tensorrt_llm::runtime::ModelConfig::getSpeculativeDecodingMode"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig28getSpeculativeDecodingModuleEv", "tensorrt_llm::runtime::ModelConfig::getSpeculativeDecodingModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig31getSpeculativeDecodingModulePtrEv", "tensorrt_llm::runtime::ModelConfig::getSpeculativeDecodingModulePtr"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig31getSpeculativeDecodingModulePtrEv", "tensorrt_llm::runtime::ModelConfig::getSpeculativeDecodingModulePtr"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig17getTokensPerBlockEv", "tensorrt_llm::runtime::ModelConfig::getTokensPerBlock"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12getVocabSizeEv", "tensorrt_llm::runtime::ModelConfig::getVocabSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18getVocabSizePaddedE10SizeType32", "tensorrt_llm::runtime::ModelConfig::getVocabSizePadded"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18getVocabSizePaddedE10SizeType32", "tensorrt_llm::runtime::ModelConfig::getVocabSizePadded::worldSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12hasRnnConfigEv", "tensorrt_llm::runtime::ModelConfig::hasRnnConfig"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig28hasSpeculativeDecodingModuleEv", "tensorrt_llm::runtime::ModelConfig::hasSpeculativeDecodingModule"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig19isContinuousKVCacheEv", "tensorrt_llm::runtime::ModelConfig::isContinuousKVCache"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig16isKVCacheEnabledEv", "tensorrt_llm::runtime::ModelConfig::isKVCacheEnabled"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig12isMultiModalEv", "tensorrt_llm::runtime::ModelConfig::isMultiModal"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14isPagedKVCacheEv", "tensorrt_llm::runtime::ModelConfig::isPagedKVCache"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig10isRnnBasedEv", "tensorrt_llm::runtime::ModelConfig::isRnnBased"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18isTransformerBasedEv", "tensorrt_llm::runtime::ModelConfig::isTransformerBased"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig9isWhisperEv", "tensorrt_llm::runtime::ModelConfig::isWhisper"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig29kDEFAULT_NUM_TOKENS_PER_BLOCKE", "tensorrt_llm::runtime::ModelConfig::kDEFAULT_NUM_TOKENS_PER_BLOCK"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26kOPT_PROFILES_SPLIT_POINTSE", "tensorrt_llm::runtime::ModelConfig::kOPT_PROFILES_SPLIT_POINTS"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21mComputeContextLogitsE", "tensorrt_llm::runtime::ModelConfig::mComputeContextLogits"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24mComputeGenerationLogitsE", "tensorrt_llm::runtime::ModelConfig::mComputeGenerationLogits"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mContextFMHAE", "tensorrt_llm::runtime::ModelConfig::mContextFMHA"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9mDataTypeE", "tensorrt_llm::runtime::ModelConfig::mDataType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mEncoderHiddenSizeE", "tensorrt_llm::runtime::ModelConfig::mEncoderHiddenSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig19mGemmAllReduceDtypeE", "tensorrt_llm::runtime::ModelConfig::mGemmAllReduceDtype"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11mHiddenSizeE", "tensorrt_llm::runtime::ModelConfig::mHiddenSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mInputPackedE", "tensorrt_llm::runtime::ModelConfig::mInputPacked"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mKVCacheTypeE", "tensorrt_llm::runtime::ModelConfig::mKVCacheType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11mLayerTypesE", "tensorrt_llm::runtime::ModelConfig::mLayerTypes"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mLogitsDtypeE", "tensorrt_llm::runtime::ModelConfig::mLogitsDtype"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mLoraModulesE", "tensorrt_llm::runtime::ModelConfig::mLoraModules"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mManageWeightsTypeE", "tensorrt_llm::runtime::ModelConfig::mManageWeightsType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mMaxBatchSizeE", "tensorrt_llm::runtime::ModelConfig::mMaxBatchSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mMaxBeamWidthE", "tensorrt_llm::runtime::ModelConfig::mMaxBeamWidth"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14mMaxEncoderLenE", "tensorrt_llm::runtime::ModelConfig::mMaxEncoderLen"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mMaxInputLenE", "tensorrt_llm::runtime::ModelConfig::mMaxInputLen"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mMaxLoraRankE", "tensorrt_llm::runtime::ModelConfig::mMaxLoraRank"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mMaxNumTokensE", "tensorrt_llm::runtime::ModelConfig::mMaxNumTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22mMaxPositionEmbeddingsE", "tensorrt_llm::runtime::ModelConfig::mMaxPositionEmbeddings"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig28mMaxPromptEmbeddingTableSizeE", "tensorrt_llm::runtime::ModelConfig::mMaxPromptEmbeddingTableSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15mMaxSequenceLenE", "tensorrt_llm::runtime::ModelConfig::mMaxSequenceLen"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14mMlpHiddenSizeE", "tensorrt_llm::runtime::ModelConfig::mMlpHiddenSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig10mModelNameE", "tensorrt_llm::runtime::ModelConfig::mModelName"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mModelVariantE", "tensorrt_llm::runtime::ModelConfig::mModelVariant"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mNbAttentionLayersE", "tensorrt_llm::runtime::ModelConfig::mNbAttentionLayers"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig8mNbHeadsE", "tensorrt_llm::runtime::ModelConfig::mNbHeads"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9mNbLayersE", "tensorrt_llm::runtime::ModelConfig::mNbLayers"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mNbRnnLayersE", "tensorrt_llm::runtime::ModelConfig::mNbRnnLayers"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig28mNumKvHeadsPerAttentionLayerE", "tensorrt_llm::runtime::ModelConfig::mNumKvHeadsPerAttentionLayer"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig33mNumKvHeadsPerCrossAttentionLayerE", "tensorrt_llm::runtime::ModelConfig::mNumKvHeadsPerCrossAttentionLayer"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13mNumLanguagesE", "tensorrt_llm::runtime::ModelConfig::mNumLanguages"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17mPagedContextFMHAE", "tensorrt_llm::runtime::ModelConfig::mPagedContextFMHA"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11mPagedStateE", "tensorrt_llm::runtime::ModelConfig::mPagedState"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig16mPpReduceScatterE", "tensorrt_llm::runtime::ModelConfig::mPpReduceScatter"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig10mQuantModeE", "tensorrt_llm::runtime::ModelConfig::mQuantMode"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig10mRnnConfigE", "tensorrt_llm::runtime::ModelConfig::mRnnConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig19mRotaryEmbeddingDimE", "tensorrt_llm::runtime::ModelConfig::mRotaryEmbeddingDim"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12mSizePerHeadE", "tensorrt_llm::runtime::ModelConfig::mSizePerHead"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20mSkipCrossAttnBlocksE", "tensorrt_llm::runtime::ModelConfig::mSkipCrossAttnBlocks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24mSpeculativeDecodingModeE", "tensorrt_llm::runtime::ModelConfig::mSpeculativeDecodingMode"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26mSpeculativeDecodingModuleE", "tensorrt_llm::runtime::ModelConfig::mSpeculativeDecodingModule"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15mTokensPerBlockE", "tensorrt_llm::runtime::ModelConfig::mTokensPerBlock"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mUseCrossAttentionE", "tensorrt_llm::runtime::ModelConfig::mUseCrossAttention"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig23mUseGemmAllReducePluginE", "tensorrt_llm::runtime::ModelConfig::mUseGemmAllReducePlugin"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22mUseGptAttentionPluginE", "tensorrt_llm::runtime::ModelConfig::mUseGptAttentionPlugin"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14mUseLoraPluginE", "tensorrt_llm::runtime::ModelConfig::mUseLoraPlugin"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21mUseMambaConv1dPluginE", "tensorrt_llm::runtime::ModelConfig::mUseMambaConv1dPlugin"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig9mUseMropeE", "tensorrt_llm::runtime::ModelConfig::mUseMrope"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21mUsePositionEmbeddingE", "tensorrt_llm::runtime::ModelConfig::mUsePositionEmbedding"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18mUseShapeInferenceE", "tensorrt_llm::runtime::ModelConfig::mUseShapeInference"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22mUseTokenTypeEmbeddingE", "tensorrt_llm::runtime::ModelConfig::mUseTokenTypeEmbedding"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig10mVocabSizeE", "tensorrt_llm::runtime::ModelConfig::mVocabSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig30resetSpeculativeDecodingModuleEv", "tensorrt_llm::runtime::ModelConfig::resetSpeculativeDecodingModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setContextFMHAEb", "tensorrt_llm::runtime::ModelConfig::setContextFMHA"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setContextFMHAEb", "tensorrt_llm::runtime::ModelConfig::setContextFMHA::contextFMHA"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setEncoderHiddenSizeE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setEncoderHiddenSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setEncoderHiddenSizeE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setEncoderHiddenSize::encoderHiddenSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21setGemmAllReduceDtypeEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::setGemmAllReduceDtype"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21setGemmAllReduceDtypeEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::setGemmAllReduceDtype::inputDtype"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setKVCacheTypeE11KVCacheType", "tensorrt_llm::runtime::ModelConfig::setKVCacheType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setKVCacheTypeE11KVCacheType", "tensorrt_llm::runtime::ModelConfig::setKVCacheType::kvCacheType"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13setLayerTypesERKNSt6vectorI9LayerTypeEE", "tensorrt_llm::runtime::ModelConfig::setLayerTypes"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13setLayerTypesERKNSt6vectorI9LayerTypeEE", "tensorrt_llm::runtime::ModelConfig::setLayerTypes::layerTypes"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setLogitsDtypeEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::setLogitsDtype"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setLogitsDtypeEN8nvinfer18DataTypeE", "tensorrt_llm::runtime::ModelConfig::setLogitsDtype::inputDtype"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setLoraModulesERKNSt6vectorI10LoraModuleEE", "tensorrt_llm::runtime::ModelConfig::setLoraModules"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setLoraModulesERKNSt6vectorI10LoraModuleEE", "tensorrt_llm::runtime::ModelConfig::setLoraModules::loraModules"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setManageWeightsTypeEK17ManageWeightsType", "tensorrt_llm::runtime::ModelConfig::setManageWeightsType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setManageWeightsTypeEK17ManageWeightsType", "tensorrt_llm::runtime::ModelConfig::setManageWeightsType::manageWeightType"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setMaxBatchSizeE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxBatchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setMaxBatchSizeE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxBatchSize::maxBatchSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setMaxBeamWidthE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setMaxBeamWidthE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxBeamWidth::maxBeamWidth"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig16setMaxEncoderLenE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxEncoderLen"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig16setMaxEncoderLenE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxEncoderLen::maxEncoderLen"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setMaxInputLenE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxInputLen"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setMaxInputLenE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxInputLen::maxInputLen"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setMaxLoraRankE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxLoraRank"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setMaxLoraRankE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxLoraRank::maxLoraRank"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setMaxNumTokensENSt8optionalI10SizeType32EE", "tensorrt_llm::runtime::ModelConfig::setMaxNumTokens"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setMaxNumTokensENSt8optionalI10SizeType32EE", "tensorrt_llm::runtime::ModelConfig::setMaxNumTokens::maxNumTokens"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24setMaxPositionEmbeddingsE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxPositionEmbeddings"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24setMaxPositionEmbeddingsE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxPositionEmbeddings::maxPositionEmbeddings"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig30setMaxPromptEmbeddingTableSizeE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxPromptEmbeddingTableSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig30setMaxPromptEmbeddingTableSizeE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxPromptEmbeddingTableSize::maxPromptEmbeddingTableSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17setMaxSequenceLenE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxSequenceLen"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17setMaxSequenceLenE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMaxSequenceLen::maxSequenceLen"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig16setMlpHiddenSizeE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMlpHiddenSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig16setMlpHiddenSizeE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setMlpHiddenSize::mlpHiddenSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setModelNameERKNSt6stringE", "tensorrt_llm::runtime::ModelConfig::setModelName"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setModelNameERKNSt6stringE", "tensorrt_llm::runtime::ModelConfig::setModelName::modelName"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setModelVariantE12ModelVariant", "tensorrt_llm::runtime::ModelConfig::setModelVariant"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setModelVariantE12ModelVariant", "tensorrt_llm::runtime::ModelConfig::setModelVariant::modelVariant"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17setNbCrossKvHeadsE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setNbCrossKvHeads"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17setNbCrossKvHeadsE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setNbCrossKvHeads::nbKvHeads"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setNbKvHeadsE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setNbKvHeads"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setNbKvHeadsE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setNbKvHeads::nbKvHeads"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26setNumKvHeadsPerCrossLayerERKNSt6vectorI10SizeType32EE", "tensorrt_llm::runtime::ModelConfig::setNumKvHeadsPerCrossLayer"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26setNumKvHeadsPerCrossLayerERKNSt6vectorI10SizeType32EE", "tensorrt_llm::runtime::ModelConfig::setNumKvHeadsPerCrossLayer::headsPerLayer"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21setNumKvHeadsPerLayerERKNSt6vectorI10SizeType32EE", "tensorrt_llm::runtime::ModelConfig::setNumKvHeadsPerLayer"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21setNumKvHeadsPerLayerERKNSt6vectorI10SizeType32EE", "tensorrt_llm::runtime::ModelConfig::setNumKvHeadsPerLayer::headsPerLayer"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setNumLanguagesENSt8optionalI10SizeType32EE", "tensorrt_llm::runtime::ModelConfig::setNumLanguages"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig15setNumLanguagesENSt8optionalI10SizeType32EE", "tensorrt_llm::runtime::ModelConfig::setNumLanguages::numLanguages"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig19setPagedContextFMHAEb", "tensorrt_llm::runtime::ModelConfig::setPagedContextFMHA"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig19setPagedContextFMHAEb", "tensorrt_llm::runtime::ModelConfig::setPagedContextFMHA::pagedContextFMHA"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18setPpReduceScatterEb", "tensorrt_llm::runtime::ModelConfig::setPpReduceScatter"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig18setPpReduceScatterEb", "tensorrt_llm::runtime::ModelConfig::setPpReduceScatter::ppReduceScatter"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setQuantModeEN6common9QuantModeE", "tensorrt_llm::runtime::ModelConfig::setQuantMode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setQuantModeEN6common9QuantModeE", "tensorrt_llm::runtime::ModelConfig::setQuantMode::QuantMode"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setRnnConfigERK9RnnConfig", "tensorrt_llm::runtime::ModelConfig::setRnnConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig12setRnnConfigERK9RnnConfig", "tensorrt_llm::runtime::ModelConfig::setRnnConfig::rnnConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21setRotaryEmbeddingDimE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setRotaryEmbeddingDim"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21setRotaryEmbeddingDimE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setRotaryEmbeddingDim::rotaryEmbeddingDim"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setSizePerHeadE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setSizePerHead"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14setSizePerHeadE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setSizePerHead::sizePerHead"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22setSkipCrossAttnBlocksEb", "tensorrt_llm::runtime::ModelConfig::setSkipCrossAttnBlocks"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22setSkipCrossAttnBlocksEb", "tensorrt_llm::runtime::ModelConfig::setSkipCrossAttnBlocks::skipCrossAttnBlocks"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26setSpeculativeDecodingModeE23SpeculativeDecodingMode", "tensorrt_llm::runtime::ModelConfig::setSpeculativeDecodingMode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig26setSpeculativeDecodingModeE23SpeculativeDecodingMode", "tensorrt_llm::runtime::ModelConfig::setSpeculativeDecodingMode::mode"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig28setSpeculativeDecodingModuleERKNSt10shared_ptrI25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::ModelConfig::setSpeculativeDecodingModule"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig28setSpeculativeDecodingModuleERKNSt10shared_ptrI25SpeculativeDecodingModuleEE", "tensorrt_llm::runtime::ModelConfig::setSpeculativeDecodingModule::speculativeDecodingModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17setTokensPerBlockE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setTokensPerBlock"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig17setTokensPerBlockE10SizeType32", "tensorrt_llm::runtime::ModelConfig::setTokensPerBlock::TokensPerBlock"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setUseCrossAttentionEb", "tensorrt_llm::runtime::ModelConfig::setUseCrossAttention"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setUseCrossAttentionEb", "tensorrt_llm::runtime::ModelConfig::setUseCrossAttention::useCrossAttention"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11setUseMropeEb", "tensorrt_llm::runtime::ModelConfig::setUseMrope"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig11setUseMropeEb", "tensorrt_llm::runtime::ModelConfig::setUseMrope::useMrope"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig23setUsePositionEmbeddingEb", "tensorrt_llm::runtime::ModelConfig::setUsePositionEmbedding"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig23setUsePositionEmbeddingEb", "tensorrt_llm::runtime::ModelConfig::setUsePositionEmbedding::usePositionEmbedding"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setUseShapeInferenceEb", "tensorrt_llm::runtime::ModelConfig::setUseShapeInference"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20setUseShapeInferenceEb", "tensorrt_llm::runtime::ModelConfig::setUseShapeInference::useShapeInference"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24setUseTokenTypeEmbeddingEb", "tensorrt_llm::runtime::ModelConfig::setUseTokenTypeEmbedding"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig24setUseTokenTypeEmbeddingEb", "tensorrt_llm::runtime::ModelConfig::setUseTokenTypeEmbedding::useTokenTypeEmbedding"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig19skipCrossAttnBlocksEv", "tensorrt_llm::runtime::ModelConfig::skipCrossAttnBlocks"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig24supportsInflightBatchingEv", "tensorrt_llm::runtime::ModelConfig::supportsInflightBatching"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig17useCrossAttentionEv", "tensorrt_llm::runtime::ModelConfig::useCrossAttention"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22useGemmAllReducePluginEb", "tensorrt_llm::runtime::ModelConfig::useGemmAllReducePlugin"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig22useGemmAllReducePluginEv", "tensorrt_llm::runtime::ModelConfig::useGemmAllReducePlugin"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig22useGemmAllReducePluginEb", "tensorrt_llm::runtime::ModelConfig::useGemmAllReducePlugin::useGemmAllReducePlugin"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21useGptAttentionPluginEb", "tensorrt_llm::runtime::ModelConfig::useGptAttentionPlugin"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21useGptAttentionPluginEv", "tensorrt_llm::runtime::ModelConfig::useGptAttentionPlugin"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig21useGptAttentionPluginEb", "tensorrt_llm::runtime::ModelConfig::useGptAttentionPlugin::useGptAttentionPlugin"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig18useLanguageAdapterEv", "tensorrt_llm::runtime::ModelConfig::useLanguageAdapter"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13useLoraPluginEb", "tensorrt_llm::runtime::ModelConfig::useLoraPlugin"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13useLoraPluginEv", "tensorrt_llm::runtime::ModelConfig::useLoraPlugin"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13useLoraPluginEb", "tensorrt_llm::runtime::ModelConfig::useLoraPlugin::useLoraPlugin"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20useMambaConv1dPluginEb", "tensorrt_llm::runtime::ModelConfig::useMambaConv1dPlugin"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20useMambaConv1dPluginEv", "tensorrt_llm::runtime::ModelConfig::useMambaConv1dPlugin"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig20useMambaConv1dPluginEb", "tensorrt_llm::runtime::ModelConfig::useMambaConv1dPlugin::useMambaConv1dPlugin"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig8useMropeEv", "tensorrt_llm::runtime::ModelConfig::useMrope"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14usePackedInputEb", "tensorrt_llm::runtime::ModelConfig::usePackedInput"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig14usePackedInputEv", "tensorrt_llm::runtime::ModelConfig::usePackedInput"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig14usePackedInputEb", "tensorrt_llm::runtime::ModelConfig::usePackedInput::inputPacked"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13usePagedStateEb", "tensorrt_llm::runtime::ModelConfig::usePagedState"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig13usePagedStateEv", "tensorrt_llm::runtime::ModelConfig::usePagedState"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ModelConfig13usePagedStateEb", "tensorrt_llm::runtime::ModelConfig::usePagedState::pagedState"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig20usePositionEmbeddingEv", "tensorrt_llm::runtime::ModelConfig::usePositionEmbedding"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig15usePromptTuningEv", "tensorrt_llm::runtime::ModelConfig::usePromptTuning"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig17useShapeInferenceEv", "tensorrt_llm::runtime::ModelConfig::useShapeInference"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11ModelConfig21useTokenTypeEmbeddingEv", "tensorrt_llm::runtime::ModelConfig::useTokenTypeEmbedding"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfiguratorE", "tensorrt_llm::runtime::MulticastConfigurator"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator11mBindOffsetE", "tensorrt_llm::runtime::MulticastConfigurator::mBindOffset"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator7mDeviceE", "tensorrt_llm::runtime::MulticastConfigurator::mDevice"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator10mMulticastE", "tensorrt_llm::runtime::MulticastConfigurator::mMulticast"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator5mSizeE", "tensorrt_llm::runtime::MulticastConfigurator::mSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator5setupE28CUmemGenericAllocationHandle", "tensorrt_llm::runtime::MulticastConfigurator::setup"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator5setupE28CUmemGenericAllocationHandle", "tensorrt_llm::runtime::MulticastConfigurator::setup::handle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime21MulticastConfigurator8teardownE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::MulticastConfigurator::teardown"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfiguratorE", "tensorrt_llm::runtime::OffloadConfigurator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator19OffloadConfiguratorE11CUdeviceptr6size_t10MemoryType8CUstreamb", "tensorrt_llm::runtime::OffloadConfigurator::OffloadConfigurator"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator19OffloadConfiguratorE11CUdeviceptr6size_t10MemoryType8CUstreamb", "tensorrt_llm::runtime::OffloadConfigurator::OffloadConfigurator::address"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator19OffloadConfiguratorE11CUdeviceptr6size_t10MemoryType8CUstreamb", "tensorrt_llm::runtime::OffloadConfigurator::OffloadConfigurator::backType"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator19OffloadConfiguratorE11CUdeviceptr6size_t10MemoryType8CUstreamb", "tensorrt_llm::runtime::OffloadConfigurator::OffloadConfigurator::ondemand"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator19OffloadConfiguratorE11CUdeviceptr6size_t10MemoryType8CUstreamb", "tensorrt_llm::runtime::OffloadConfigurator::OffloadConfigurator::size"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator19OffloadConfiguratorE11CUdeviceptr6size_t10MemoryType8CUstreamb", "tensorrt_llm::runtime::OffloadConfigurator::OffloadConfigurator::stream"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator8mAddressE", "tensorrt_llm::runtime::OffloadConfigurator::mAddress"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator9mBackTypeE", "tensorrt_llm::runtime::OffloadConfigurator::mBackType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator14mBackedStorageE", "tensorrt_llm::runtime::OffloadConfigurator::mBackedStorage"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator9mOndemandE", "tensorrt_llm::runtime::OffloadConfigurator::mOndemand"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator5mSizeE", "tensorrt_llm::runtime::OffloadConfigurator::mSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator7mStreamE", "tensorrt_llm::runtime::OffloadConfigurator::mStream"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator5setupE28CUmemGenericAllocationHandle", "tensorrt_llm::runtime::OffloadConfigurator::setup"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator5setupE28CUmemGenericAllocationHandle", "tensorrt_llm::runtime::OffloadConfigurator::setup::handle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator8teardownE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::OffloadConfigurator::teardown"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator8teardownE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::OffloadConfigurator::teardown::destructing"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19OffloadConfigurator8teardownE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::OffloadConfigurator::teardown::handle"], [1, 1, 1, "_CPPv4I0EN12tensorrt_llm7runtime18PointerElementTypeE", "tensorrt_llm::runtime::PointerElementType"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime18PointerElementTypeE", "tensorrt_llm::runtime::PointerElementType::T"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParamsE", "tensorrt_llm::runtime::PromptTuningParams"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams18PromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", "tensorrt_llm::runtime::PromptTuningParams::PromptTuningParams"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams18PromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", "tensorrt_llm::runtime::PromptTuningParams::PromptTuningParams::embeddingTable"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams18PromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", "tensorrt_llm::runtime::PromptTuningParams::PromptTuningParams::tasks"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams18PromptTuningParamsE9TensorPtr9TensorPtr9TensorPtr", "tensorrt_llm::runtime::PromptTuningParams::PromptTuningParams::vocabSize"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams10SizeType32E", "tensorrt_llm::runtime::PromptTuningParams::SizeType32"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams9TensorPtrE", "tensorrt_llm::runtime::PromptTuningParams::TensorPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams15fillTasksTensorE9TensorPtr10SizeType3210SizeType32RKNSt6vectorI10SizeType32EERKNSt6vectorI10SizeType32EERK13BufferManagerb", "tensorrt_llm::runtime::PromptTuningParams::fillTasksTensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams15fillTasksTensorE9TensorPtr10SizeType3210SizeType32RKNSt6vectorI10SizeType32EERKNSt6vectorI10SizeType32EERK13BufferManagerb", "tensorrt_llm::runtime::PromptTuningParams::fillTasksTensor::batchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams15fillTasksTensorE9TensorPtr10SizeType3210SizeType32RKNSt6vectorI10SizeType32EERKNSt6vectorI10SizeType32EERK13BufferManagerb", "tensorrt_llm::runtime::PromptTuningParams::fillTasksTensor::manager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams15fillTasksTensorE9TensorPtr10SizeType3210SizeType32RKNSt6vectorI10SizeType32EERKNSt6vectorI10SizeType32EERK13BufferManagerb", "tensorrt_llm::runtime::PromptTuningParams::fillTasksTensor::numContextRequests"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams15fillTasksTensorE9TensorPtr10SizeType3210SizeType32RKNSt6vectorI10SizeType32EERKNSt6vectorI10SizeType32EERK13BufferManagerb", "tensorrt_llm::runtime::PromptTuningParams::fillTasksTensor::packedInput"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams15fillTasksTensorE9TensorPtr10SizeType3210SizeType32RKNSt6vectorI10SizeType32EERKNSt6vectorI10SizeType32EERK13BufferManagerb", "tensorrt_llm::runtime::PromptTuningParams::fillTasksTensor::reqBeamWidths"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams15fillTasksTensorE9TensorPtr10SizeType3210SizeType32RKNSt6vectorI10SizeType32EERKNSt6vectorI10SizeType32EERK13BufferManagerb", "tensorrt_llm::runtime::PromptTuningParams::fillTasksTensor::reqPromptLengths"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime18PromptTuningParams15fillTasksTensorE9TensorPtr10SizeType3210SizeType32RKNSt6vectorI10SizeType32EERKNSt6vectorI10SizeType32EERK13BufferManagerb", "tensorrt_llm::runtime::PromptTuningParams::fillTasksTensor::tasksHost"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngineE", "tensorrt_llm::runtime::RawEngine"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4Type15AddressWithSizeE", "tensorrt_llm::runtime::RawEngine::AddressWithSize"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4Type8FilePathE", "tensorrt_llm::runtime::RawEngine::FilePath"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4Type10HostMemoryE", "tensorrt_llm::runtime::RawEngine::HostMemory"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineENSt10filesystem4pathE", "tensorrt_llm::runtime::RawEngine::RawEngine"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineEPKN8nvinfer111IHostMemoryE", "tensorrt_llm::runtime::RawEngine::RawEngine"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineEPKvNSt6size_tE", "tensorrt_llm::runtime::RawEngine::RawEngine"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineEPKvNSt6size_tE", "tensorrt_llm::runtime::RawEngine::RawEngine::engineAddr"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineEPKN8nvinfer111IHostMemoryE", "tensorrt_llm::runtime::RawEngine::RawEngine::engineBuffer"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineENSt10filesystem4pathE", "tensorrt_llm::runtime::RawEngine::RawEngine::enginePath"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine9RawEngineEPKvNSt6size_tE", "tensorrt_llm::runtime::RawEngine::RawEngine::engineSize"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4TypeE", "tensorrt_llm::runtime::RawEngine::Type"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4Type15AddressWithSizeE", "tensorrt_llm::runtime::RawEngine::Type::AddressWithSize"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4Type8FilePathE", "tensorrt_llm::runtime::RawEngine::Type::FilePath"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine4Type10HostMemoryE", "tensorrt_llm::runtime::RawEngine::Type::HostMemory"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine10getAddressEv", "tensorrt_llm::runtime::RawEngine::getAddress"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine13getHostMemoryEv", "tensorrt_llm::runtime::RawEngine::getHostMemory"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine23getManagedWeightsMapOptEv", "tensorrt_llm::runtime::RawEngine::getManagedWeightsMapOpt"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine7getPathEv", "tensorrt_llm::runtime::RawEngine::getPath"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine10getPathOptEv", "tensorrt_llm::runtime::RawEngine::getPathOpt"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine7getSizeEv", "tensorrt_llm::runtime::RawEngine::getSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime9RawEngine7getTypeEv", "tensorrt_llm::runtime::RawEngine::getType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine11mEngineAddrE", "tensorrt_llm::runtime::RawEngine::mEngineAddr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine13mEngineBufferE", "tensorrt_llm::runtime::RawEngine::mEngineBuffer"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine11mEnginePathE", "tensorrt_llm::runtime::RawEngine::mEnginePath"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine11mEngineSizeE", "tensorrt_llm::runtime::RawEngine::mEngineSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine18mManagedWeightsMapE", "tensorrt_llm::runtime::RawEngine::mManagedWeightsMap"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine5mTypeE", "tensorrt_llm::runtime::RawEngine::mType"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine20setManagedWeightsMapENSt3mapINSt6stringEN12tensorrt_llm8executor6TensorEEE", "tensorrt_llm::runtime::RawEngine::setManagedWeightsMap"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine20setManagedWeightsMapENSt3mapINSt6stringEN12tensorrt_llm8executor6TensorEEE", "tensorrt_llm::runtime::RawEngine::setManagedWeightsMap::managedWeightsMap"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine7setPathENSt10filesystem4pathE", "tensorrt_llm::runtime::RawEngine::setPath"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9RawEngine7setPathENSt10filesystem4pathE", "tensorrt_llm::runtime::RawEngine::setPath::enginePath"], [1, 6, 1, "_CPPv4N12tensorrt_llm7runtime11RequestTypeE", "tensorrt_llm::runtime::RequestType"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11RequestType8kCONTEXTE", "tensorrt_llm::runtime::RequestType::kCONTEXT"], [1, 7, 1, "_CPPv4N12tensorrt_llm7runtime11RequestType11kGENERATIONE", "tensorrt_llm::runtime::RequestType::kGENERATION"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaultsE", "tensorrt_llm::runtime::RuntimeDefaults"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults15RuntimeDefaultsENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalI10SizeType32EE", "tensorrt_llm::runtime::RuntimeDefaults::RuntimeDefaults"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults15RuntimeDefaultsEv", "tensorrt_llm::runtime::RuntimeDefaults::RuntimeDefaults"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults15RuntimeDefaultsENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalI10SizeType32EE", "tensorrt_llm::runtime::RuntimeDefaults::RuntimeDefaults::maxAttentionWindowVec"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults15RuntimeDefaultsENSt8optionalINSt6vectorI10SizeType32EEEENSt8optionalI10SizeType32EE", "tensorrt_llm::runtime::RuntimeDefaults::RuntimeDefaults::sinkTokenLength"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults21maxAttentionWindowVecE", "tensorrt_llm::runtime::RuntimeDefaults::maxAttentionWindowVec"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime15RuntimeDefaults15sinkTokenLengthE", "tensorrt_llm::runtime::RuntimeDefaults::sinkTokenLength"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfigE", "tensorrt_llm::runtime::SamplingConfig"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig9FloatTypeE", "tensorrt_llm::runtime::SamplingConfig::FloatType"], [1, 1, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig6OptVecE", "tensorrt_llm::runtime::SamplingConfig::OptVec"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig6OptVecE", "tensorrt_llm::runtime::SamplingConfig::OptVec::T"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigE10SizeType32", "tensorrt_llm::runtime::SamplingConfig::SamplingConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigERKN8executor14SamplingConfigERKNSt8optionalIN8executor25ExternalDraftTokensConfigEEE", "tensorrt_llm::runtime::SamplingConfig::SamplingConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigERKNSt6vectorI14SamplingConfigEE", "tensorrt_llm::runtime::SamplingConfig::SamplingConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigE10SizeType32", "tensorrt_llm::runtime::SamplingConfig::SamplingConfig::beamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigERKNSt6vectorI14SamplingConfigEE", "tensorrt_llm::runtime::SamplingConfig::SamplingConfig::configs"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigERKN8executor14SamplingConfigERKNSt8optionalIN8executor25ExternalDraftTokensConfigEEE", "tensorrt_llm::runtime::SamplingConfig::SamplingConfig::externalDraftTokensConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14SamplingConfigERKN8executor14SamplingConfigERKNSt8optionalIN8executor25ExternalDraftTokensConfigEEE", "tensorrt_llm::runtime::SamplingConfig::SamplingConfig::samplingConfig"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig23beamSearchDiversityRateE", "tensorrt_llm::runtime::SamplingConfig::beamSearchDiversityRate"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig9beamWidthE", "tensorrt_llm::runtime::SamplingConfig::beamWidth"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14beamWidthArrayE", "tensorrt_llm::runtime::SamplingConfig::beamWidthArray"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig11cumLogProbsE", "tensorrt_llm::runtime::SamplingConfig::cumLogProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig24draftAcceptanceThresholdE", "tensorrt_llm::runtime::SamplingConfig::draftAcceptanceThreshold"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig13earlyStoppingE", "tensorrt_llm::runtime::SamplingConfig::earlyStopping"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig16frequencyPenaltyE", "tensorrt_llm::runtime::SamplingConfig::frequencyPenalty"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig10fuseValuesE6OptVecI1TERKNSt6vectorI14SamplingConfigEENSt8functionIF6OptVecI1TE6size_tEEE1T", "tensorrt_llm::runtime::SamplingConfig::fuseValues"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig10fuseValuesE6OptVecI1TERKNSt6vectorI14SamplingConfigEENSt8functionIF6OptVecI1TE6size_tEEE1T", "tensorrt_llm::runtime::SamplingConfig::fuseValues::T"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig10fuseValuesE6OptVecI1TERKNSt6vectorI14SamplingConfigEENSt8functionIF6OptVecI1TE6size_tEEE1T", "tensorrt_llm::runtime::SamplingConfig::fuseValues::accessor"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig10fuseValuesE6OptVecI1TERKNSt6vectorI14SamplingConfigEENSt8functionIF6OptVecI1TE6size_tEEE1T", "tensorrt_llm::runtime::SamplingConfig::fuseValues::configs"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig10fuseValuesE6OptVecI1TERKNSt6vectorI14SamplingConfigEENSt8functionIF6OptVecI1TE6size_tEEE1T", "tensorrt_llm::runtime::SamplingConfig::fuseValues::defaultValue"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14SamplingConfig15getMaxBeamWidthEv", "tensorrt_llm::runtime::SamplingConfig::getMaxBeamWidth"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14SamplingConfig17getNumReturnBeamsEv", "tensorrt_llm::runtime::SamplingConfig::getNumReturnBeams"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig13lengthPenaltyE", "tensorrt_llm::runtime::SamplingConfig::lengthPenalty"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig9minLengthE", "tensorrt_llm::runtime::SamplingConfig::minLength"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig4minPE", "tensorrt_llm::runtime::SamplingConfig::minP"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig17noRepeatNgramSizeE", "tensorrt_llm::runtime::SamplingConfig::noRepeatNgramSize"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig17normalizeLogProbsE", "tensorrt_llm::runtime::SamplingConfig::normalizeLogProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig18numReturnSequencesE", "tensorrt_llm::runtime::SamplingConfig::numReturnSequences"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime14SamplingConfigeqERK14SamplingConfig", "tensorrt_llm::runtime::SamplingConfig::operator=="], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime14SamplingConfigeqERK14SamplingConfig", "tensorrt_llm::runtime::SamplingConfig::operator==::other"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig19originalTemperatureE", "tensorrt_llm::runtime::SamplingConfig::originalTemperature"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig14outputLogProbsE", "tensorrt_llm::runtime::SamplingConfig::outputLogProbs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig15presencePenaltyE", "tensorrt_llm::runtime::SamplingConfig::presencePenalty"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig10randomSeedE", "tensorrt_llm::runtime::SamplingConfig::randomSeed"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig17repetitionPenaltyE", "tensorrt_llm::runtime::SamplingConfig::repetitionPenalty"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig11temperatureE", "tensorrt_llm::runtime::SamplingConfig::temperature"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig4topKE", "tensorrt_llm::runtime::SamplingConfig::topK"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig15topKMedusaHeadsE", "tensorrt_llm::runtime::SamplingConfig::topKMedusaHeads"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig4topPE", "tensorrt_llm::runtime::SamplingConfig::topP"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig9topPDecayE", "tensorrt_llm::runtime::SamplingConfig::topPDecay"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig7topPMinE", "tensorrt_llm::runtime::SamplingConfig::topPMin"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig12topPResetIdsE", "tensorrt_llm::runtime::SamplingConfig::topPResetIds"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig16useDefaultValuesEbRK6OptVecI1TE1T", "tensorrt_llm::runtime::SamplingConfig::useDefaultValues"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig16useDefaultValuesEbRK6OptVecI1TE1T", "tensorrt_llm::runtime::SamplingConfig::useDefaultValues::T"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig16useDefaultValuesEbRK6OptVecI1TE1T", "tensorrt_llm::runtime::SamplingConfig::useDefaultValues::defaultValue"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig16useDefaultValuesEbRK6OptVecI1TE1T", "tensorrt_llm::runtime::SamplingConfig::useDefaultValues::vec"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime14SamplingConfig8validateEv", "tensorrt_llm::runtime::SamplingConfig::validate"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig11validateVecEbNSt6stringERK6OptVecI1TE1TNSt8optionalI1TEE", "tensorrt_llm::runtime::SamplingConfig::validateVec"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig11validateVecEbNSt6stringERK6OptVecI1TE1TNSt8optionalI1TEE", "tensorrt_llm::runtime::SamplingConfig::validateVec::T"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig11validateVecEbNSt6stringERK6OptVecI1TE1TNSt8optionalI1TEE", "tensorrt_llm::runtime::SamplingConfig::validateVec::max"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig11validateVecEbNSt6stringERK6OptVecI1TE1TNSt8optionalI1TEE", "tensorrt_llm::runtime::SamplingConfig::validateVec::min"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig11validateVecEbNSt6stringERK6OptVecI1TE1TNSt8optionalI1TEE", "tensorrt_llm::runtime::SamplingConfig::validateVec::name"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime14SamplingConfig11validateVecEbNSt6stringERK6OptVecI1TE1TNSt8optionalI1TEE", "tensorrt_llm::runtime::SamplingConfig::validateVec::vec"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime10SizeType32E", "tensorrt_llm::runtime::SizeType32"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime10SizeType64E", "tensorrt_llm::runtime::SizeType64"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingModeE", "tensorrt_llm::runtime::SpeculativeDecodingMode"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode19DraftTokensExternalEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::DraftTokensExternal"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode5EagleEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::Eagle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode19ExplicitDraftTokensEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::ExplicitDraftTokens"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode17LookaheadDecodingEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::LookaheadDecoding"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode6MedusaEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::Medusa"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode4NoneEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::None"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode23SpeculativeDecodingModeE14UnderlyingType", "tensorrt_llm::runtime::SpeculativeDecodingMode::SpeculativeDecodingMode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode23SpeculativeDecodingModeE14UnderlyingType", "tensorrt_llm::runtime::SpeculativeDecodingMode::SpeculativeDecodingMode::state"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode14UnderlyingTypeE", "tensorrt_llm::runtime::SpeculativeDecodingMode::UnderlyingType"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode9allBitSetE14UnderlyingType", "tensorrt_llm::runtime::SpeculativeDecodingMode::allBitSet"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode9allBitSetE14UnderlyingType", "tensorrt_llm::runtime::SpeculativeDecodingMode::allBitSet::bits"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode9anyBitSetE14UnderlyingType", "tensorrt_llm::runtime::SpeculativeDecodingMode::anyBitSet"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode9anyBitSetE14UnderlyingType", "tensorrt_llm::runtime::SpeculativeDecodingMode::anyBitSet::bits"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode14hasDraftLogitsEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::hasDraftLogits"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode21isDraftTokensExternalEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::isDraftTokensExternal"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode7isEagleEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::isEagle"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode21isExplicitDraftTokensEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::isExplicitDraftTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode19isLookaheadDecodingEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::isLookaheadDecoding"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode8isMedusaEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::isMedusa"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode6isNoneEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::isNone"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode20kDraftTokensExternalE", "tensorrt_llm::runtime::SpeculativeDecodingMode::kDraftTokensExternal"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode6kEagleE", "tensorrt_llm::runtime::SpeculativeDecodingMode::kEagle"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode20kExplicitDraftTokensE", "tensorrt_llm::runtime::SpeculativeDecodingMode::kExplicitDraftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode18kLookaheadDecodingE", "tensorrt_llm::runtime::SpeculativeDecodingMode::kLookaheadDecoding"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode7kMedusaE", "tensorrt_llm::runtime::SpeculativeDecodingMode::kMedusa"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode5kNoneE", "tensorrt_llm::runtime::SpeculativeDecodingMode::kNone"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime23SpeculativeDecodingMode6mStateE", "tensorrt_llm::runtime::SpeculativeDecodingMode::mState"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode20needsDecoderPrologueEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::needsDecoderPrologue"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode18needsKVCacheRewindEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::needsKVCacheRewind"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingModeeqERK23SpeculativeDecodingMode", "tensorrt_llm::runtime::SpeculativeDecodingMode::operator=="], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingModeeqERK23SpeculativeDecodingMode", "tensorrt_llm::runtime::SpeculativeDecodingMode::operator==::other"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode19predictsDraftTokensEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::predictsDraftTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode21requiresAttentionMaskEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::requiresAttentionMask"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode18updatesPositionIdsEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::updatesPositionIds"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime23SpeculativeDecodingMode19variableDraftLengthEv", "tensorrt_llm::runtime::SpeculativeDecodingMode::variableDraftLength"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModuleE", "tensorrt_llm::runtime::SpeculativeDecodingModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::SpeculativeDecodingModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleERK25SpeculativeDecodingModule", "tensorrt_llm::runtime::SpeculativeDecodingModule::SpeculativeDecodingModule"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleEv", "tensorrt_llm::runtime::SpeculativeDecodingModule::SpeculativeDecodingModule"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::SpeculativeDecodingModule::maxDecodingDraftTokens"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::SpeculativeDecodingModule::maxDraftPathLen"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::SpeculativeDecodingModule::maxNumPaths"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule25SpeculativeDecodingModuleERK25SpeculativeDecodingModule", "tensorrt_llm::runtime::SpeculativeDecodingModule::SpeculativeDecodingModule::o"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule21computeNumPackedMasksEv", "tensorrt_llm::runtime::SpeculativeDecodingModule::computeNumPackedMasks"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule25getMaxDecodingDraftTokensEv", "tensorrt_llm::runtime::SpeculativeDecodingModule::getMaxDecodingDraftTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule20getMaxDecodingTokensEv", "tensorrt_llm::runtime::SpeculativeDecodingModule::getMaxDecodingTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule18getMaxDraftPathLenEv", "tensorrt_llm::runtime::SpeculativeDecodingModule::getMaxDraftPathLen"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule14getMaxNumPathsEv", "tensorrt_llm::runtime::SpeculativeDecodingModule::getMaxNumPaths"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule13getMaxPathLenEv", "tensorrt_llm::runtime::SpeculativeDecodingModule::getMaxPathLen"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime25SpeculativeDecodingModule17getNumPackedMasksEv", "tensorrt_llm::runtime::SpeculativeDecodingModule::getNumPackedMasks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule23mMaxDecodingDraftTokensE", "tensorrt_llm::runtime::SpeculativeDecodingModule::mMaxDecodingDraftTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule16mMaxDraftPathLenE", "tensorrt_llm::runtime::SpeculativeDecodingModule::mMaxDraftPathLen"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule18mMaxNumPackedMasksE", "tensorrt_llm::runtime::SpeculativeDecodingModule::mMaxNumPackedMasks"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule12mMaxNumPathsE", "tensorrt_llm::runtime::SpeculativeDecodingModule::mMaxNumPaths"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModuleaSERK25SpeculativeDecodingModule", "tensorrt_llm::runtime::SpeculativeDecodingModule::operator="], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModuleaSERK25SpeculativeDecodingModule", "tensorrt_llm::runtime::SpeculativeDecodingModule::operator=::o"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule18setMaxDraftPathLenE10SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::setMaxDraftPathLen"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule18setMaxDraftPathLenE10SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::setMaxDraftPathLen::maxDraftPathLen"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule17setMaxDraftTokensE10SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::setMaxDraftTokens"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule17setMaxDraftTokensE10SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::setMaxDraftTokens::maxDraftTokens"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule14setMaxNumPathsE10SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::setMaxNumPaths"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModule14setMaxNumPathsE10SizeType32", "tensorrt_llm::runtime::SpeculativeDecodingModule::setMaxNumPaths::maxNumPaths"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25SpeculativeDecodingModuleD0Ev", "tensorrt_llm::runtime::SpeculativeDecodingModule::~SpeculativeDecodingModule"], [1, 1, 1, "_CPPv4I0EN12tensorrt_llm7runtime12StringPtrMapE", "tensorrt_llm::runtime::StringPtrMap"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime12StringPtrMapE", "tensorrt_llm::runtime::StringPtrMap::T"], [1, 2, 1, "_CPPv4I0_bEN12tensorrt_llm7runtime11TRTDataTypeE", "tensorrt_llm::runtime::TRTDataType"], [1, 8, 1, "_CPPv4I0_bEN12tensorrt_llm7runtime11TRTDataTypeE", "tensorrt_llm::runtime::TRTDataType::T"], [1, 2, 1, "_CPPv4I0EN12tensorrt_llm7runtime11TRTDataTypeIP1TEE", "tensorrt_llm::runtime::TRTDataType<T*>"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime11TRTDataTypeIP1TEE", "tensorrt_llm::runtime::TRTDataType<T*>::T"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIP1TE15kUnderlyingTypeE", "tensorrt_llm::runtime::TRTDataType<T*>::kUnderlyingType"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIP1TE5valueE", "tensorrt_llm::runtime::TRTDataType<T*>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIbEE", "tensorrt_llm::runtime::TRTDataType<bool>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIbE5valueE", "tensorrt_llm::runtime::TRTDataType<bool>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIfEE", "tensorrt_llm::runtime::TRTDataType<float>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIfE5valueE", "tensorrt_llm::runtime::TRTDataType<float>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeI4halfEE", "tensorrt_llm::runtime::TRTDataType<half>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeI4halfE5valueE", "tensorrt_llm::runtime::TRTDataType<half>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIN7kernels13FinishedStateEEE", "tensorrt_llm::runtime::TRTDataType<kernels::FinishedState>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIN7kernels13FinishedStateEE5valueE", "tensorrt_llm::runtime::TRTDataType<kernels::FinishedState>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIN7kernels12KVCacheIndexEEE", "tensorrt_llm::runtime::TRTDataType<kernels::KVCacheIndex>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIN7kernels12KVCacheIndexEE5valueE", "tensorrt_llm::runtime::TRTDataType<kernels::KVCacheIndex>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIN7runtime11RequestTypeEEE", "tensorrt_llm::runtime::TRTDataType<runtime::RequestType>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIN7runtime11RequestTypeEE5valueE", "tensorrt_llm::runtime::TRTDataType<runtime::RequestType>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt7int32_tEEE", "tensorrt_llm::runtime::TRTDataType<std::int32_t>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt7int32_tEE5valueE", "tensorrt_llm::runtime::TRTDataType<std::int32_t>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt7int64_tEEE", "tensorrt_llm::runtime::TRTDataType<std::int64_t>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt7int64_tEE5valueE", "tensorrt_llm::runtime::TRTDataType<std::int64_t>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt6int8_tEEE", "tensorrt_llm::runtime::TRTDataType<std::int8_t>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt6int8_tEE5valueE", "tensorrt_llm::runtime::TRTDataType<std::int8_t>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt8uint32_tEEE", "tensorrt_llm::runtime::TRTDataType<std::uint32_t>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt8uint32_tEE5valueE", "tensorrt_llm::runtime::TRTDataType<std::uint32_t>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt8uint64_tEEE", "tensorrt_llm::runtime::TRTDataType<std::uint64_t>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt8uint64_tEE5valueE", "tensorrt_llm::runtime::TRTDataType<std::uint64_t>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeINSt7uint8_tEEE", "tensorrt_llm::runtime::TRTDataType<std::uint8_t>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeINSt7uint8_tEE5valueE", "tensorrt_llm::runtime::TRTDataType<std::uint8_t>::value"], [1, 2, 1, "_CPPv4IEN12tensorrt_llm7runtime11TRTDataTypeIPvEE", "tensorrt_llm::runtime::TRTDataType<void*>"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11TRTDataTypeIPvE5valueE", "tensorrt_llm::runtime::TRTDataType<void*>::value"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime10TllmLoggerE", "tensorrt_llm::runtime::TllmLogger"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10TllmLogger8getLevelEv", "tensorrt_llm::runtime::TllmLogger::getLevel"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10TllmLogger3logE8SeverityPKN8nvinfer19AsciiCharE", "tensorrt_llm::runtime::TllmLogger::log"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10TllmLogger3logE8SeverityPKN8nvinfer19AsciiCharE", "tensorrt_llm::runtime::TllmLogger::log::msg"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10TllmLogger3logE8SeverityPKN8nvinfer19AsciiCharE", "tensorrt_llm::runtime::TllmLogger::log::severity"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime10TllmLogger8setLevelE8Severity", "tensorrt_llm::runtime::TllmLogger::setLevel"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime10TllmLogger8setLevelE8Severity", "tensorrt_llm::runtime::TllmLogger::setLevel::level"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime16TokenExtraIdTypeE", "tensorrt_llm::runtime::TokenExtraIdType"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime11TokenIdTypeE", "tensorrt_llm::runtime::TokenIdType"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfiguratorE", "tensorrt_llm::runtime::UnicastConfigurator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator19UnicastConfiguratorE11CUdeviceptr6size_tRK15CUmemAccessDesc", "tensorrt_llm::runtime::UnicastConfigurator::UnicastConfigurator"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator19UnicastConfiguratorE11CUdeviceptr6size_tRK15CUmemAccessDesc", "tensorrt_llm::runtime::UnicastConfigurator::UnicastConfigurator::address"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator19UnicastConfiguratorE11CUdeviceptr6size_tRK15CUmemAccessDesc", "tensorrt_llm::runtime::UnicastConfigurator::UnicastConfigurator::desc"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator19UnicastConfiguratorE11CUdeviceptr6size_tRK15CUmemAccessDesc", "tensorrt_llm::runtime::UnicastConfigurator::UnicastConfigurator::size"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator8mAddressE", "tensorrt_llm::runtime::UnicastConfigurator::mAddress"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator5mDescE", "tensorrt_llm::runtime::UnicastConfigurator::mDesc"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator5mSizeE", "tensorrt_llm::runtime::UnicastConfigurator::mSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator5setupE28CUmemGenericAllocationHandle", "tensorrt_llm::runtime::UnicastConfigurator::setup"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator5setupE28CUmemGenericAllocationHandle", "tensorrt_llm::runtime::UnicastConfigurator::setup::handle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime19UnicastConfigurator8teardownE28CUmemGenericAllocationHandleb", "tensorrt_llm::runtime::UnicastConfigurator::teardown"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime11UniqueTokenE", "tensorrt_llm::runtime::UniqueToken"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11UniqueTokeneqERK11UniqueToken", "tensorrt_llm::runtime::UniqueToken::operator=="], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11UniqueTokeneqERK11UniqueToken", "tensorrt_llm::runtime::UniqueToken::operator==::other"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11UniqueToken12tokenExtraIdE", "tensorrt_llm::runtime::UniqueToken::tokenExtraId"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11UniqueToken7tokenIdE", "tensorrt_llm::runtime::UniqueToken::tokenId"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime16VecTokenExtraIdsE", "tensorrt_llm::runtime::VecTokenExtraIds"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime15VecUniqueTokensE", "tensorrt_llm::runtime::VecUniqueTokens"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfigE", "tensorrt_llm::runtime::WorldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig11WorldConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::WorldConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig11WorldConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::WorldConfig::contextParallelism"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig11WorldConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::WorldConfig::deviceIds"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig11WorldConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::WorldConfig::enableAttentionDP"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig11WorldConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::WorldConfig::gpusPerNode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig11WorldConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::WorldConfig::pipelineParallelism"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig11WorldConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::WorldConfig::rank"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig11WorldConfigE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::WorldConfig::tensorParallelism"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig17enableAttentionDPEv", "tensorrt_llm::runtime::WorldConfig::enableAttentionDP"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig23getContextParallelGroupEv", "tensorrt_llm::runtime::WorldConfig::getContextParallelGroup"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig22getContextParallelRankEv", "tensorrt_llm::runtime::WorldConfig::getContextParallelRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig21getContextParallelismEv", "tensorrt_llm::runtime::WorldConfig::getContextParallelism"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig9getDeviceEv", "tensorrt_llm::runtime::WorldConfig::getDevice"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig11getDeviceOfE10SizeType32", "tensorrt_llm::runtime::WorldConfig::getDeviceOf"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig11getDeviceOfE10SizeType32", "tensorrt_llm::runtime::WorldConfig::getDeviceOf::rank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig15getGpusPerGroupEv", "tensorrt_llm::runtime::WorldConfig::getGpusPerGroup"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig14getGpusPerNodeEv", "tensorrt_llm::runtime::WorldConfig::getGpusPerNode"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig11getLastRankEv", "tensorrt_llm::runtime::WorldConfig::getLastRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig12getLocalRankEv", "tensorrt_llm::runtime::WorldConfig::getLocalRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig11getNodeRankEv", "tensorrt_llm::runtime::WorldConfig::getNodeRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig13getNodeRankOfE10SizeType32", "tensorrt_llm::runtime::WorldConfig::getNodeRankOf"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig13getNodeRankOfE10SizeType32", "tensorrt_llm::runtime::WorldConfig::getNodeRankOf::rank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig24getPipelineParallelGroupEv", "tensorrt_llm::runtime::WorldConfig::getPipelineParallelGroup"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig23getPipelineParallelRankEv", "tensorrt_llm::runtime::WorldConfig::getPipelineParallelRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig22getPipelineParallelismEv", "tensorrt_llm::runtime::WorldConfig::getPipelineParallelism"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig7getRankEv", "tensorrt_llm::runtime::WorldConfig::getRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig7getSizeEv", "tensorrt_llm::runtime::WorldConfig::getSize"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig22getTensorParallelGroupEv", "tensorrt_llm::runtime::WorldConfig::getTensorParallelGroup"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig21getTensorParallelRankEv", "tensorrt_llm::runtime::WorldConfig::getTensorParallelRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig20getTensorParallelismEv", "tensorrt_llm::runtime::WorldConfig::getTensorParallelism"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig17isContextParallelEv", "tensorrt_llm::runtime::WorldConfig::isContextParallel"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig26isFirstContextParallelRankEv", "tensorrt_llm::runtime::WorldConfig::isFirstContextParallelRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig27isFirstPipelineParallelRankEv", "tensorrt_llm::runtime::WorldConfig::isFirstPipelineParallelRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig25isFirstTensorParallelRankEv", "tensorrt_llm::runtime::WorldConfig::isFirstTensorParallelRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig26isLastPipelineParallelRankEv", "tensorrt_llm::runtime::WorldConfig::isLastPipelineParallelRank"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig18isPipelineParallelEv", "tensorrt_llm::runtime::WorldConfig::isPipelineParallel"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig16isTensorParallelEv", "tensorrt_llm::runtime::WorldConfig::isTensorParallel"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig19kDefaultGpusPerNodeE", "tensorrt_llm::runtime::WorldConfig::kDefaultGpusPerNode"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig19mContextParallelismE", "tensorrt_llm::runtime::WorldConfig::mContextParallelism"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig10mDeviceIdsE", "tensorrt_llm::runtime::WorldConfig::mDeviceIds"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig18mEnableAttentionDPE", "tensorrt_llm::runtime::WorldConfig::mEnableAttentionDP"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig12mGpusPerNodeE", "tensorrt_llm::runtime::WorldConfig::mGpusPerNode"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig20mPipelineParallelismE", "tensorrt_llm::runtime::WorldConfig::mPipelineParallelism"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig5mRankE", "tensorrt_llm::runtime::WorldConfig::mRank"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig18mTensorParallelismE", "tensorrt_llm::runtime::WorldConfig::mTensorParallelism"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig3mpiE10SizeType32NSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::mpi"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig3mpiE10SizeType32NSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::mpi::contextParallelism"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig3mpiE10SizeType32NSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::mpi::deviceIds"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig3mpiE10SizeType32NSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::mpi::enableAttentionDP"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig3mpiE10SizeType32NSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::mpi::gpusPerNode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig3mpiE10SizeType32NSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::mpi::pipelineParallelism"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11WorldConfig3mpiE10SizeType32NSt8optionalI10SizeType32EENSt8optionalI10SizeType32EENSt8optionalI10SizeType32EERKNSt8optionalINSt6vectorI10SizeType32EEEEb", "tensorrt_llm::runtime::WorldConfig::mpi::tensorParallelism"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime11WorldConfig14validMpiConfigEv", "tensorrt_llm::runtime::WorldConfig::validMpiConfig"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime10bufferCastEP1TR7IBuffer", "tensorrt_llm::runtime::bufferCast"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime10bufferCastEPK1TRK7IBuffer", "tensorrt_llm::runtime::bufferCast"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime10bufferCastEP1TR7IBuffer", "tensorrt_llm::runtime::bufferCast::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime10bufferCastEPK1TRK7IBuffer", "tensorrt_llm::runtime::bufferCast::T"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime10bufferCastEP1TR7IBuffer", "tensorrt_llm::runtime::bufferCast::buffer"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime10bufferCastEPK1TRK7IBuffer", "tensorrt_llm::runtime::bufferCast::buffer"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKN7IBuffer9SharedPtrE", "tensorrt_llm::runtime::bufferCastOrNull"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKN7ITensor9SharedPtrE", "tensorrt_llm::runtime::bufferCastOrNull"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKNSt8optionalIN7IBuffer9SharedPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKNSt8optionalIN7ITensor9SharedPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKN7IBuffer14SharedConstPtrE", "tensorrt_llm::runtime::bufferCastOrNull"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKN7ITensor14SharedConstPtrE", "tensorrt_llm::runtime::bufferCastOrNull"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKNSt8optionalIN7IBuffer14SharedConstPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKNSt8optionalIN7ITensor14SharedConstPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKN7IBuffer9SharedPtrE", "tensorrt_llm::runtime::bufferCastOrNull::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKN7ITensor9SharedPtrE", "tensorrt_llm::runtime::bufferCastOrNull::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKNSt8optionalIN7IBuffer9SharedPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKNSt8optionalIN7ITensor9SharedPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKN7IBuffer14SharedConstPtrE", "tensorrt_llm::runtime::bufferCastOrNull::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKN7ITensor14SharedConstPtrE", "tensorrt_llm::runtime::bufferCastOrNull::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKNSt8optionalIN7IBuffer14SharedConstPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKNSt8optionalIN7ITensor14SharedConstPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull::T"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKN7IBuffer9SharedPtrE", "tensorrt_llm::runtime::bufferCastOrNull::bufferPtr"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKN7IBuffer14SharedConstPtrE", "tensorrt_llm::runtime::bufferCastOrNull::bufferPtr"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKNSt8optionalIN7IBuffer9SharedPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull::optionalBufferPtr"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKNSt8optionalIN7IBuffer14SharedConstPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull::optionalBufferPtr"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKNSt8optionalIN7ITensor9SharedPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull::optionalTensorPtr"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKNSt8optionalIN7ITensor14SharedConstPtrEEE", "tensorrt_llm::runtime::bufferCastOrNull::optionalTensorPtr"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEP1TRKN7ITensor9SharedPtrE", "tensorrt_llm::runtime::bufferCastOrNull::tensorPtr"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime16bufferCastOrNullEPK1TRKN7ITensor14SharedConstPtrE", "tensorrt_llm::runtime::bufferCastOrNull::tensorPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13canAccessPeerERK11WorldConfig", "tensorrt_llm::runtime::canAccessPeer"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13canAccessPeerERK11WorldConfig", "tensorrt_llm::runtime::canAccessPeer::worldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime27clearVirtualMemoryAllocatorEv", "tensorrt_llm::runtime::clearVirtualMemoryAllocator"], [1, 3, 1, "_CPPv4I00EN12tensorrt_llm7runtime16constPointerCastENSt10shared_ptrINSt14remove_const_tI1TEEEERRNSt10unique_ptrI1T1DEE", "tensorrt_llm::runtime::constPointerCast"], [1, 3, 1, "_CPPv4I0EN12tensorrt_llm7runtime16constPointerCastENSt10shared_ptrINSt14remove_const_tI1TEEEERKNSt10shared_ptrI1TEE", "tensorrt_llm::runtime::constPointerCast"], [1, 8, 1, "_CPPv4I00EN12tensorrt_llm7runtime16constPointerCastENSt10shared_ptrINSt14remove_const_tI1TEEEERRNSt10unique_ptrI1T1DEE", "tensorrt_llm::runtime::constPointerCast::D"], [1, 8, 1, "_CPPv4I00EN12tensorrt_llm7runtime16constPointerCastENSt10shared_ptrINSt14remove_const_tI1TEEEERRNSt10unique_ptrI1T1DEE", "tensorrt_llm::runtime::constPointerCast::T"], [1, 8, 1, "_CPPv4I0EN12tensorrt_llm7runtime16constPointerCastENSt10shared_ptrINSt14remove_const_tI1TEEEERKNSt10shared_ptrI1TEE", "tensorrt_llm::runtime::constPointerCast::T"], [1, 4, 1, "_CPPv4I00EN12tensorrt_llm7runtime16constPointerCastENSt10shared_ptrINSt14remove_const_tI1TEEEERRNSt10unique_ptrI1T1DEE", "tensorrt_llm::runtime::constPointerCast::ptr"], [1, 4, 1, "_CPPv4I0EN12tensorrt_llm7runtime16constPointerCastENSt10shared_ptrINSt14remove_const_tI1TEEEERKNSt10shared_ptrI1TEE", "tensorrt_llm::runtime::constPointerCast::ptr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7decoderE", "tensorrt_llm::runtime::decoder"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7decoderE", "tensorrt_llm::runtime::decoder"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffersE", "tensorrt_llm::runtime::decoder::BeamSearchBuffers"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers17BeamSearchBuffersERK13BufferManager", "tensorrt_llm::runtime::decoder::BeamSearchBuffers::BeamSearchBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers17BeamSearchBuffersERK13BufferManager", "tensorrt_llm::runtime::decoder::BeamSearchBuffers::BeamSearchBuffers::bufferManager"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers15mCumLogProbsTmpE", "tensorrt_llm::runtime::decoder::BeamSearchBuffers::mCumLogProbsTmp"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers7mNumSMsE", "tensorrt_llm::runtime::decoder::BeamSearchBuffers::mNumSMs"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers21mOutputBeamHypothesesE", "tensorrt_llm::runtime::decoder::BeamSearchBuffers::mOutputBeamHypotheses"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers7reshapeE10SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::BeamSearchBuffers::reshape"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers7reshapeE10SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::BeamSearchBuffers::reshape::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder17BeamSearchBuffers7reshapeE10SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::BeamSearchBuffers::reshape::maxSequenceLength"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderStateE", "tensorrt_llm::runtime::decoder::DecoderState"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12DecoderStateEv", "tensorrt_llm::runtime::decoder::DecoderState::DecoderState"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState16DecodingInputPtrE", "tensorrt_llm::runtime::decoder::DecoderState::DecodingInputPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState17DecodingOutputPtrE", "tensorrt_llm::runtime::decoder::DecoderState::DecodingOutputPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState13LlmRequestPtrE", "tensorrt_llm::runtime::decoder::DecoderState::LlmRequestPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState13RequestVectorE", "tensorrt_llm::runtime::decoder::DecoderState::RequestVector"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState9TensorPtrE", "tensorrt_llm::runtime::decoder::DecoderState::TensorPtr"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState16disableLookaheadERK13RequestVector", "tensorrt_llm::runtime::decoder::DecoderState::disableLookahead"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState16disableLookaheadERK13RequestVector", "tensorrt_llm::runtime::decoder::DecoderState::disableLookahead::genRequests"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState24getAcceptedLengthsCumSumEv", "tensorrt_llm::runtime::decoder::DecoderState::getAcceptedLengthsCumSum"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState22getAcceptedPackedPathsEv", "tensorrt_llm::runtime::decoder::DecoderState::getAcceptedPackedPaths"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState15getAllNewTokensEv", "tensorrt_llm::runtime::decoder::DecoderState::getAllNewTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState20getBeamSearchBuffersEv", "tensorrt_llm::runtime::decoder::DecoderState::getBeamSearchBuffers"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState24getCacheIndirectionInputEv", "tensorrt_llm::runtime::decoder::DecoderState::getCacheIndirectionInput"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState25getCacheIndirectionOutputEv", "tensorrt_llm::runtime::decoder::DecoderState::getCacheIndirectionOutput"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getCumLogProbsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getCumLogProbs"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getCumLogProbsEv", "tensorrt_llm::runtime::decoder::DecoderState::getCumLogProbs"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getCumLogProbsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getCumLogProbs::batchIdx"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState15getEagleBuffersEv", "tensorrt_llm::runtime::decoder::DecoderState::getEagleBuffers"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState29getExplicitDraftTokensBuffersEv", "tensorrt_llm::runtime::decoder::DecoderState::getExplicitDraftTokensBuffers"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState16getFinishReasonsEv", "tensorrt_llm::runtime::decoder::DecoderState::getFinishReasons"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getFinishedSumEv", "tensorrt_llm::runtime::decoder::DecoderState::getFinishedSum"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getGatheredIdsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getGatheredIds"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getGatheredIdsEv", "tensorrt_llm::runtime::decoder::DecoderState::getGatheredIds"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState14getGatheredIdsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getGatheredIds::batchIdx"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getGenerationStepsEv", "tensorrt_llm::runtime::decoder::DecoderState::getGenerationSteps"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState6getIdsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getIds"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState6getIdsEv", "tensorrt_llm::runtime::decoder::DecoderState::getIds"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState6getIdsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getIds::batchIdx"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState21getJointDecodingInputEv", "tensorrt_llm::runtime::decoder::DecoderState::getJointDecodingInput"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState22getJointDecodingOutputEv", "tensorrt_llm::runtime::decoder::DecoderState::getJointDecodingOutput"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState11getLogProbsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getLogProbs"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState11getLogProbsEv", "tensorrt_llm::runtime::decoder::DecoderState::getLogProbs"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState11getLogProbsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getLogProbs::batchIdx"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState19getLookaheadBuffersEv", "tensorrt_llm::runtime::decoder::DecoderState::getLookaheadBuffers"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState15getMaxBeamWidthEv", "tensorrt_llm::runtime::decoder::DecoderState::getMaxBeamWidth"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState27getMaxDecodingDecoderTokensEv", "tensorrt_llm::runtime::decoder::DecoderState::getMaxDecodingDecoderTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState26getMaxDecodingEngineTokensEv", "tensorrt_llm::runtime::decoder::DecoderState::getMaxDecodingEngineTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getMaxNumSequencesEv", "tensorrt_llm::runtime::decoder::DecoderState::getMaxNumSequences"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState20getMaxSequenceLengthEv", "tensorrt_llm::runtime::decoder::DecoderState::getMaxSequenceLength"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getNextDraftTokensEv", "tensorrt_llm::runtime::decoder::DecoderState::getNextDraftTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState25getNextDraftTokensLengthsEv", "tensorrt_llm::runtime::decoder::DecoderState::getNextDraftTokensLengths"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState26getNumDecodingEngineTokensE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getNumDecodingEngineTokens"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState26getNumDecodingEngineTokensEv", "tensorrt_llm::runtime::decoder::DecoderState::getNumDecodingEngineTokens"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState26getNumDecodingEngineTokensE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getNumDecodingEngineTokens::batchIdx"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState12getParentIdsEv", "tensorrt_llm::runtime::decoder::DecoderState::getParentIds"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState25getPrevDraftTokensLengthsEv", "tensorrt_llm::runtime::decoder::DecoderState::getPrevDraftTokensLengths"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getSequenceLengthsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getSequenceLengths"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getSequenceLengthsEv", "tensorrt_llm::runtime::decoder::DecoderState::getSequenceLengths"], [1, 4, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState18getSequenceLengthsE10SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::getSequenceLengths::batchIdx"], [1, 3, 1, "_CPPv4NK12tensorrt_llm7runtime7decoder12DecoderState26getSpeculativeDecodingModeEv", "tensorrt_llm::runtime::decoder::DecoderState::getSpeculativeDecodingMode"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState18mBeamSearchBuffersE", "tensorrt_llm::runtime::decoder::DecoderState::mBeamSearchBuffers"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState19mJointDecodingInputE", "tensorrt_llm::runtime::decoder::DecoderState::mJointDecodingInput"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState20mJointDecodingOutputE", "tensorrt_llm::runtime::decoder::DecoderState::mJointDecodingOutput"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState13mMaxBeamWidthE", "tensorrt_llm::runtime::decoder::DecoderState::mMaxBeamWidth"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState25mMaxDecodingDecoderTokensE", "tensorrt_llm::runtime::decoder::DecoderState::mMaxDecodingDecoderTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24mMaxDecodingEngineTokensE", "tensorrt_llm::runtime::decoder::DecoderState::mMaxDecodingEngineTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState16mMaxNumSequencesE", "tensorrt_llm::runtime::decoder::DecoderState::mMaxNumSequences"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState18mMaxSequenceLengthE", "tensorrt_llm::runtime::decoder::DecoderState::mMaxSequenceLength"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24mNumDecodingEngineTokensE", "tensorrt_llm::runtime::decoder::DecoderState::mNumDecodingEngineTokens"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24mSpeculativeDecodingModeE", "tensorrt_llm::runtime::decoder::DecoderState::mSpeculativeDecodingMode"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeBuffers::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeBuffers::maxAttentionWindow"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeBuffers::maxBatchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeBuffers::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeBuffers::maxSequenceLength"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeBuffers::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeBuffers::sinkTokenLength"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState14reshapeBuffersE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeBuffers::worldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState30reshapeCacheIndirectionBuffersE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::reshapeCacheIndirectionBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState30reshapeCacheIndirectionBuffersE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::reshapeCacheIndirectionBuffers::maxAttentionWindow"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState30reshapeCacheIndirectionBuffersE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::reshapeCacheIndirectionBuffers::maxBatchSize"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState30reshapeCacheIndirectionBuffersE10SizeType3210SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::reshapeCacheIndirectionBuffers::maxBeamWidth"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState33reshapeSpeculativeDecodingBuffersERK23SpeculativeDecodingMode10SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeSpeculativeDecodingBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState33reshapeSpeculativeDecodingBuffersERK23SpeculativeDecodingMode10SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeSpeculativeDecodingBuffers::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState33reshapeSpeculativeDecodingBuffersERK23SpeculativeDecodingMode10SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeSpeculativeDecodingBuffers::maxTokensPerEngineStep"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState33reshapeSpeculativeDecodingBuffersERK23SpeculativeDecodingMode10SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeSpeculativeDecodingBuffers::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState33reshapeSpeculativeDecodingBuffersERK23SpeculativeDecodingMode10SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeSpeculativeDecodingBuffers::speculativeDecodingMode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState33reshapeSpeculativeDecodingBuffersERK23SpeculativeDecodingMode10SizeType32RK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::reshapeSpeculativeDecodingBuffers::worldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12setBeamWidthE10SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::setBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12setBeamWidthE10SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::setBeamWidth::batchIdx"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12setBeamWidthE10SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::setBeamWidth::beamWidth"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState18setGenerationStepsERKNSt6vectorI10SizeType32EE", "tensorrt_llm::runtime::decoder::DecoderState::setGenerationSteps"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState18setGenerationStepsERKNSt6vectorI10SizeType32EE", "tensorrt_llm::runtime::decoder::DecoderState::setGenerationSteps::generationSteps"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState26setNumDecodingEngineTokensE10SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::setNumDecodingEngineTokens"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState26setNumDecodingEngineTokensE10SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::setNumDecodingEngineTokens::batchIdx"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState26setNumDecodingEngineTokensE10SizeType3210SizeType32", "tensorrt_llm::runtime::decoder::DecoderState::setNumDecodingEngineTokens::numTokens"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup::dtype"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup::maxAttentionWindow"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup::maxNumSequences"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup::maxSequenceLength"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup::sinkTokenLength"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState5setupE10SizeType3210SizeType3210SizeType3210SizeType3210SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setup::worldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12setupBuffersEN8nvinfer18DataTypeERK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12setupBuffersEN8nvinfer18DataTypeERK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupBuffers::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState12setupBuffersEN8nvinfer18DataTypeERK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupBuffers::dtype"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState21setupCacheIndirectionE10SizeType3210SizeType3210SizeType32RK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupCacheIndirection"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState21setupCacheIndirectionE10SizeType3210SizeType3210SizeType32RK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupCacheIndirection::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState21setupCacheIndirectionE10SizeType3210SizeType3210SizeType32RK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupCacheIndirection::maxAttentionWindow"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState21setupCacheIndirectionE10SizeType3210SizeType3210SizeType32RK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupCacheIndirection::maxBeamWidth"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState21setupCacheIndirectionE10SizeType3210SizeType3210SizeType32RK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupCacheIndirection::maxNumSequences"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState28setupCacheIndirectionBuffersERK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupCacheIndirectionBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState28setupCacheIndirectionBuffersERK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupCacheIndirectionBuffers::bufferManager"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24setupSpeculativeDecodingERK23SpeculativeDecodingMode10SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecoding"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24setupSpeculativeDecodingERK23SpeculativeDecodingMode10SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecoding::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24setupSpeculativeDecodingERK23SpeculativeDecodingMode10SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecoding::dtype"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24setupSpeculativeDecodingERK23SpeculativeDecodingMode10SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecoding::maxTokensPerEngineStep"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24setupSpeculativeDecodingERK23SpeculativeDecodingMode10SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecoding::modelConfig"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24setupSpeculativeDecodingERK23SpeculativeDecodingMode10SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecoding::speculativeDecodingMode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState24setupSpeculativeDecodingERK23SpeculativeDecodingMode10SizeType32N8nvinfer18DataTypeERK11ModelConfigRK11WorldConfigRK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecoding::worldConfig"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState31setupSpeculativeDecodingBuffersE23SpeculativeDecodingModeN8nvinfer18DataTypeERK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecodingBuffers"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState31setupSpeculativeDecodingBuffersE23SpeculativeDecodingModeN8nvinfer18DataTypeERK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecodingBuffers::bufferManager"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState31setupSpeculativeDecodingBuffersE23SpeculativeDecodingModeN8nvinfer18DataTypeERK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecodingBuffers::dtype"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime7decoder12DecoderState31setupSpeculativeDecodingBuffersE23SpeculativeDecodingModeN8nvinfer18DataTypeERK13BufferManager", "tensorrt_llm::runtime::decoder::DecoderState::setupSpeculativeDecodingBuffers::speculativeDecodingMode"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batchE", "tensorrt_llm::runtime::decoder_batch"], [1, 2, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5InputE", "tensorrt_llm::runtime::decoder_batch::Input"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input5InputERKNSt6vectorI14TensorConstPtrEE", "tensorrt_llm::runtime::decoder_batch::Input::Input"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input5InputERKNSt6vectorINSt6vectorI14TensorConstPtrEEEE10SizeType32", "tensorrt_llm::runtime::decoder_batch::Input::Input"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input5InputERKNSt6vectorI14TensorConstPtrEE", "tensorrt_llm::runtime::decoder_batch::Input::Input::logits"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input5InputERKNSt6vectorINSt6vectorI14TensorConstPtrEEEE10SizeType32", "tensorrt_llm::runtime::decoder_batch::Input::Input::logits"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input5InputERKNSt6vectorINSt6vectorI14TensorConstPtrEEEE10SizeType32", "tensorrt_llm::runtime::decoder_batch::Input::Input::maxDecoderSteps"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input14TensorConstPtrE", "tensorrt_llm::runtime::decoder_batch::Input::TensorConstPtr"], [1, 1, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input9TensorPtrE", "tensorrt_llm::runtime::decoder_batch::Input::TensorPtr"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input10batchSlotsE", "tensorrt_llm::runtime::decoder_batch::Input::batchSlots"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input6logitsE", "tensorrt_llm::runtime::decoder_batch::Input::logits"], [1, 5, 1, "_CPPv4N12tensorrt_llm7runtime13decoder_batch5Input15maxDecoderStepsE", "tensorrt_llm::runtime::decoder_batch::Input::maxDecoderSteps"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20getDefaultBatchSlotsEN7runtime10SizeType32E", "tensorrt_llm::runtime::getDefaultBatchSlots"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20getDefaultBatchSlotsEN7runtime10SizeType32E", "tensorrt_llm::runtime::getDefaultBatchSlots::batchSize"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25getVirtualMemoryAllocatorEv", "tensorrt_llm::runtime::getVirtualMemoryAllocator"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime23getVirtualMemoryManagerEv", "tensorrt_llm::runtime::getVirtualMemoryManager"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime15ipcNvlsAllocateE6size_tNSt3setIiEE", "tensorrt_llm::runtime::ipcNvlsAllocate"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime15ipcNvlsAllocateE6size_tNSt3setIiEE", "tensorrt_llm::runtime::ipcNvlsAllocate::ranks"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime15ipcNvlsAllocateE6size_tNSt3setIiEE", "tensorrt_llm::runtime::ipcNvlsAllocate::size"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime11ipcNvlsFreeEP13IpcNvlsHandle", "tensorrt_llm::runtime::ipcNvlsFree"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime11ipcNvlsFreeEP13IpcNvlsHandle", "tensorrt_llm::runtime::ipcNvlsFree::handle"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime16ipcNvlsSupportedEv", "tensorrt_llm::runtime::ipcNvlsSupported"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime20lamportInitializeAllEPvPvPv6size_t", "tensorrt_llm::runtime::lamportInitializeAll"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20lamportInitializeAllEPvPvPv6size_t", "tensorrt_llm::runtime::lamportInitializeAll::buffer_0"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20lamportInitializeAllEPvPvPv6size_t", "tensorrt_llm::runtime::lamportInitializeAll::buffer_1"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20lamportInitializeAllEPvPvPv6size_t", "tensorrt_llm::runtime::lamportInitializeAll::buffer_2"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime20lamportInitializeAllEPvPvPv6size_t", "tensorrt_llm::runtime::lamportInitializeAll::size"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK10LoraModule", "tensorrt_llm::runtime::operator<<"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK26LoraCachePageManagerConfig", "tensorrt_llm::runtime::operator<<"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK7IBuffer", "tensorrt_llm::runtime::operator<<"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK7ITensor", "tensorrt_llm::runtime::operator<<"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERKN7ITensor5ShapeE", "tensorrt_llm::runtime::operator<<"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERKN9LoraCache21TaskLayerModuleConfigE", "tensorrt_llm::runtime::operator<<"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK7IBuffer", "tensorrt_llm::runtime::operator<<::buffer"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK26LoraCachePageManagerConfig", "tensorrt_llm::runtime::operator<<::c"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERKN7ITensor5ShapeE", "tensorrt_llm::runtime::operator<<::dims"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK10LoraModule", "tensorrt_llm::runtime::operator<<::module"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK26LoraCachePageManagerConfig", "tensorrt_llm::runtime::operator<<::os"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERKN9LoraCache21TaskLayerModuleConfigE", "tensorrt_llm::runtime::operator<<::os"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK10LoraModule", "tensorrt_llm::runtime::operator<<::output"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK7IBuffer", "tensorrt_llm::runtime::operator<<::output"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK7ITensor", "tensorrt_llm::runtime::operator<<::output"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERKN7ITensor5ShapeE", "tensorrt_llm::runtime::operator<<::output"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERK7ITensor", "tensorrt_llm::runtime::operator<<::tensor"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtimelsERNSt7ostreamERKN9LoraCache21TaskLayerModuleConfigE", "tensorrt_llm::runtime::operator<<::v"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime25setVirtualMemoryAllocatorERKNSt6stringEN26CudaVirtualMemoryAllocator11RestoreModeENSt10shared_ptrI10CudaStreamEE", "tensorrt_llm::runtime::setVirtualMemoryAllocator"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25setVirtualMemoryAllocatorERKNSt6stringEN26CudaVirtualMemoryAllocator11RestoreModeENSt10shared_ptrI10CudaStreamEE", "tensorrt_llm::runtime::setVirtualMemoryAllocator::backStream"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25setVirtualMemoryAllocatorERKNSt6stringEN26CudaVirtualMemoryAllocator11RestoreModeENSt10shared_ptrI10CudaStreamEE", "tensorrt_llm::runtime::setVirtualMemoryAllocator::mode"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime25setVirtualMemoryAllocatorERKNSt6stringEN26CudaVirtualMemoryAllocator11RestoreModeENSt10shared_ptrI10CudaStreamEE", "tensorrt_llm::runtime::setVirtualMemoryAllocator::tag"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9to_stringERK26LoraCachePageManagerConfig", "tensorrt_llm::runtime::to_string"], [1, 3, 1, "_CPPv4N12tensorrt_llm7runtime9to_stringERKN9LoraCache21TaskLayerModuleConfigE", "tensorrt_llm::runtime::to_string"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9to_stringERK26LoraCachePageManagerConfig", "tensorrt_llm::runtime::to_string::c"], [1, 4, 1, "_CPPv4N12tensorrt_llm7runtime9to_stringERKN9LoraCache21TaskLayerModuleConfigE", "tensorrt_llm::runtime::to_string::v"], [138, 9, 0, "-", "tensorrt_llm"]], "tensorrt_llm": [[133, 9, 0, "-", "functional"], [135, 9, 0, "-", "models"], [136, 9, 0, "-", "plugin"], [137, 9, 0, "-", "quantization"], [138, 9, 0, "-", "runtime"]], "tensorrt_llm.functional": [[133, 10, 1, "", "AllReduceFusionOp"], [133, 10, 1, "", "AllReduceParams"], [133, 10, 1, "", "AllReduceStrategy"], [133, 10, 1, "", "AttentionMaskType"], [133, 10, 1, "", "Conditional"], [133, 10, 1, "", "DimRange"], [133, 10, 1, "", "LayerNormPositionType"], [133, 10, 1, "", "LayerNormType"], [133, 10, 1, "", "MLPType"], [133, 10, 1, "", "MoEAllReduceParams"], [133, 10, 1, "", "PositionEmbeddingType"], [133, 10, 1, "", "RopeEmbeddingUtils"], [133, 10, 1, "", "RotaryScalingType"], [133, 10, 1, "", "SideStreamIDType"], [133, 10, 1, "", "SliceInputType"], [133, 10, 1, "", "Tensor"], [133, 14, 1, "", "abs"], [133, 14, 1, "", "activation"], [133, 14, 1, "", "add"], [133, 14, 1, "", "allgather"], [133, 14, 1, "", "allreduce"], [133, 14, 1, "", "arange"], [133, 14, 1, "", "argmax"], [133, 14, 1, "", "assertion"], [133, 14, 1, "", "avg_pool2d"], [133, 14, 1, "", "bert_attention"], [133, 14, 1, "", "broadcast_helper"], [133, 14, 1, "", "cast"], [133, 14, 1, "", "categorical_sample"], [133, 14, 1, "", "chunk"], [133, 14, 1, "", "clip"], [133, 14, 1, "", "concat"], [133, 14, 1, "", "constant"], [133, 14, 1, "", "constant_to_tensor_"], [133, 14, 1, "", "constants_to_tensors_"], [133, 14, 1, "", "conv1d"], [133, 14, 1, "", "conv2d"], [133, 14, 1, "", "conv3d"], [133, 14, 1, "", "conv_transpose2d"], [133, 14, 1, "", "cos"], [133, 14, 1, "", "cp_split_plugin"], [133, 14, 1, "", "create_allreduce_plugin"], [133, 14, 1, "", "cuda_stream_sync"], [133, 14, 1, "", "cumsum"], [133, 14, 1, "", "div"], [133, 14, 1, "", "dora_plugin"], [133, 14, 1, "", "einsum"], [133, 14, 1, "", "elementwise_binary"], [133, 14, 1, "", "embedding"], [133, 14, 1, "", "eq"], [133, 14, 1, "", "exp"], [133, 14, 1, "", "expand"], [133, 14, 1, "", "expand_dims"], [133, 14, 1, "", "expand_dims_like"], [133, 14, 1, "", "expand_mask"], [133, 14, 1, "", "flatten"], [133, 14, 1, "", "flip"], [133, 14, 1, "", "floordiv"], [133, 14, 1, "", "gather"], [133, 14, 1, "", "gather_last_token_logits"], [133, 14, 1, "", "gather_nd"], [133, 14, 1, "", "gegelu"], [133, 14, 1, "", "geglu"], [133, 14, 1, "", "gelu"], [133, 14, 1, "", "gemm_allreduce"], [133, 14, 1, "", "gemm_swiglu"], [133, 14, 1, "", "generate_alibi_biases"], [133, 14, 1, "", "generate_alibi_slopes"], [133, 14, 1, "", "generate_logn_scaling"], [133, 14, 1, "", "gpt_attention"], [133, 14, 1, "", "group_norm"], [133, 14, 1, "", "gt"], [133, 14, 1, "", "identity"], [133, 14, 1, "", "index_select"], [133, 14, 1, "", "int_clip"], [133, 14, 1, "", "interpolate"], [133, 14, 1, "", "is_gated_activation"], [133, 14, 1, "", "layer_norm"], [133, 14, 1, "", "log"], [133, 14, 1, "", "log_softmax"], [133, 14, 1, "", "lora_plugin"], [133, 14, 1, "", "low_latency_gemm"], [133, 14, 1, "", "low_latency_gemm_swiglu"], [133, 14, 1, "", "lt"], [133, 14, 1, "", "mamba_conv1d"], [133, 14, 1, "", "masked_scatter"], [133, 14, 1, "", "masked_select"], [133, 14, 1, "", "matmul"], [133, 14, 1, "", "max"], [133, 14, 1, "", "maximum"], [133, 14, 1, "", "mean"], [133, 14, 1, "", "meshgrid2d"], [133, 14, 1, "", "min"], [133, 14, 1, "", "minimum"], [133, 14, 1, "", "modulo"], [133, 14, 1, "", "mul"], [133, 14, 1, "", "non_gated_version"], [133, 14, 1, "", "nonzero"], [133, 14, 1, "", "not_op"], [133, 14, 1, "", "op_and"], [133, 14, 1, "", "op_or"], [133, 14, 1, "", "op_xor"], [133, 14, 1, "", "outer"], [133, 14, 1, "", "pad"], [133, 14, 1, "", "permute"], [133, 14, 1, "", "pow"], [133, 14, 1, "", "prod"], [133, 14, 1, "", "quick_gelu"], [133, 14, 1, "", "rand"], [133, 14, 1, "", "rearrange"], [133, 14, 1, "", "recv"], [133, 14, 1, "", "reduce"], [133, 14, 1, "", "reduce_scatter"], [133, 14, 1, "", "relu"], [133, 14, 1, "", "repeat"], [133, 14, 1, "", "repeat_interleave"], [133, 14, 1, "", "rg_lru"], [133, 14, 1, "", "rms_norm"], [133, 14, 1, "", "round"], [133, 14, 1, "", "scatter"], [133, 14, 1, "", "scatter_nd"], [133, 14, 1, "", "select"], [133, 14, 1, "", "selective_scan"], [133, 14, 1, "", "send"], [133, 14, 1, "", "shape"], [133, 14, 1, "", "sigmoid"], [133, 14, 1, "", "silu"], [133, 14, 1, "", "sin"], [133, 14, 1, "", "slice"], [133, 14, 1, "", "softmax"], [133, 14, 1, "", "softplus"], [133, 14, 1, "", "split"], [133, 14, 1, "", "sqrt"], [133, 14, 1, "", "squared_relu"], [133, 14, 1, "", "squeeze"], [133, 14, 1, "", "stack"], [133, 14, 1, "", "sub"], [133, 14, 1, "", "sum"], [133, 14, 1, "", "swiglu"], [133, 14, 1, "", "tanh"], [133, 14, 1, "", "topk"], [133, 14, 1, "", "transpose"], [133, 14, 1, "", "unary"], [133, 14, 1, "", "unbind"], [133, 14, 1, "", "unsqueeze"], [133, 14, 1, "", "view"], [133, 14, 1, "", "where"]], "tensorrt_llm.functional.AllReduceFusionOp": [[133, 11, 1, "", "LAST_PROCESS_FOR_UB"], [133, 11, 1, "", "MOE_FINALIZE_ALLREDUCE_RESIDUAL_RMS_NORM"], [133, 11, 1, "", "NONE"], [133, 11, 1, "", "RESIDUAL_RMS_NORM"], [133, 11, 1, "", "RESIDUAL_RMS_NORM_OUT_QUANT_FP8"], [133, 11, 1, "", "RESIDUAL_RMS_NORM_OUT_QUANT_NVFP4"], [133, 11, 1, "", "RESIDUAL_RMS_NORM_QUANT_FP8"], [133, 11, 1, "", "RESIDUAL_RMS_NORM_QUANT_NVFP4"], [133, 11, 1, "", "RESIDUAL_RMS_PREPOST_NORM"]], "tensorrt_llm.functional.AllReduceParams": [[133, 12, 1, "", "has_affine"], [133, 12, 1, "", "has_bias"], [133, 12, 1, "", "has_scale"], [133, 12, 1, "", "update_strategy"]], "tensorrt_llm.functional.AllReduceStrategy": [[133, 11, 1, "", "AUTO"], [133, 11, 1, "", "LOWPRECISION"], [133, 11, 1, "", "MIN_LATENCY"], [133, 11, 1, "", "MNNVL"], [133, 11, 1, "", "NCCL"], [133, 11, 1, "", "NCCL_SYMMETRIC"], [133, 11, 1, "", "ONESHOT"], [133, 11, 1, "", "TWOSHOT"], [133, 11, 1, "", "UB"]], "tensorrt_llm.functional.AttentionMaskType": [[133, 11, 1, "", "bidirectional"], [133, 11, 1, "", "bidirectionalglm"], [133, 11, 1, "", "blocksparse"], [133, 11, 1, "", "causal"], [133, 11, 1, "", "custom_mask"], [133, 11, 1, "", "padding"], [133, 11, 1, "", "sliding_window_causal"]], "tensorrt_llm.functional.Conditional": [[133, 12, 1, "", "add_input"], [133, 12, 1, "", "add_output"]], "tensorrt_llm.functional.LayerNormPositionType": [[133, 11, 1, "", "post_layernorm"], [133, 11, 1, "", "pre_layernorm"]], "tensorrt_llm.functional.LayerNormType": [[133, 11, 1, "", "GroupNorm"], [133, 11, 1, "", "LayerNorm"], [133, 11, 1, "", "RmsNorm"]], "tensorrt_llm.functional.MLPType": [[133, 11, 1, "", "FusedGatedMLP"], [133, 11, 1, "", "GatedMLP"], [133, 11, 1, "", "MLP"]], "tensorrt_llm.functional.MoEAllReduceParams": [[133, 12, 1, "", "is_valid"]], "tensorrt_llm.functional.PositionEmbeddingType": [[133, 11, 1, "", "alibi"], [133, 11, 1, "", "alibi_with_scale"], [133, 11, 1, "", "chatglm"], [133, 12, 1, "", "choices"], [133, 11, 1, "", "deferred"], [133, 12, 1, "", "from_string"], [133, 12, 1, "", "is_alibi"], [133, 12, 1, "", "is_deferred"], [133, 12, 1, "", "is_mrope"], [133, 12, 1, "", "is_rope"], [133, 11, 1, "", "learned_absolute"], [133, 11, 1, "", "long_rope"], [133, 11, 1, "", "mrope"], [133, 11, 1, "", "relative"], [133, 11, 1, "", "rope_gpt_neox"], [133, 11, 1, "", "rope_gptj"], [133, 11, 1, "", "yarn"]], "tensorrt_llm.functional.RopeEmbeddingUtils": [[133, 12, 1, "", "apply_llama3_scaling"], [133, 12, 1, "", "apply_rotary_pos_emb"], [133, 12, 1, "", "apply_rotary_pos_emb_chatglm"], [133, 12, 1, "", "apply_rotary_pos_emb_cogvlm"], [133, 12, 1, "", "create_fake_weight"], [133, 12, 1, "", "create_sinusoidal_positions"], [133, 12, 1, "", "create_sinusoidal_positions_for_attention_plugin"], [133, 12, 1, "", "create_sinusoidal_positions_for_cogvlm_attention_plugin"], [133, 12, 1, "", "create_sinusoidal_positions_long_rope"], [133, 12, 1, "", "create_sinusoidal_positions_long_rope_for_attention_plugin"], [133, 12, 1, "", "create_sinusoidal_positions_yarn"], [133, 12, 1, "", "rotate_every_two"], [133, 12, 1, "", "rotate_half"]], "tensorrt_llm.functional.RotaryScalingType": [[133, 11, 1, "", "dynamic"], [133, 12, 1, "", "from_string"], [133, 11, 1, "", "linear"], [133, 11, 1, "", "llama3"], [133, 11, 1, "", "longrope"], [133, 11, 1, "", "mrope"], [133, 11, 1, "", "none"], [133, 11, 1, "", "yarn"]], "tensorrt_llm.functional.SideStreamIDType": [[133, 11, 1, "", "disable"], [133, 11, 1, "", "moe"]], "tensorrt_llm.functional.SliceInputType": [[133, 11, 1, "", "axes"], [133, 11, 1, "", "data"], [133, 11, 1, "", "fill_value"], [133, 11, 1, "", "size"], [133, 11, 1, "", "start"], [133, 11, 1, "", "stride"]], "tensorrt_llm.functional.Tensor": [[133, 12, 1, "", "abs"], [133, 12, 1, "", "cast"], [133, 13, 1, "", "dtype"], [133, 12, 1, "", "flatten"], [133, 12, 1, "", "get_parent"], [133, 12, 1, "", "get_users"], [133, 12, 1, "", "is_dynamic"], [133, 12, 1, "", "is_trt_wrapper"], [133, 13, 1, "", "location"], [133, 12, 1, "", "log"], [133, 12, 1, "", "mark_output"], [133, 12, 1, "", "max"], [133, 12, 1, "", "mean"], [133, 13, 1, "", "name"], [133, 12, 1, "", "ndim"], [133, 13, 1, "", "network"], [133, 12, 1, "", "permute"], [133, 12, 1, "", "rank"], [133, 12, 1, "", "repeat"], [133, 12, 1, "", "replace_all_uses_with"], [133, 12, 1, "", "select"], [133, 13, 1, "", "shape"], [133, 12, 1, "", "size"], [133, 12, 1, "", "split"], [133, 12, 1, "", "sqrt"], [133, 12, 1, "", "squeeze"], [133, 12, 1, "", "transpose"], [133, 12, 1, "", "unbind"], [133, 12, 1, "", "unsqueeze"], [133, 12, 1, "", "view"]], "tensorrt_llm.layers": [[134, 9, 0, "-", "activation"], [134, 9, 0, "-", "attention"], [134, 9, 0, "-", "cast"], [134, 9, 0, "-", "conv"], [134, 9, 0, "-", "embedding"], [134, 9, 0, "-", "linear"], [134, 9, 0, "-", "mlp"], [134, 9, 0, "-", "normalization"], [134, 9, 0, "-", "pooling"]], "tensorrt_llm.layers.activation": [[134, 10, 1, "", "Mish"]], "tensorrt_llm.layers.activation.Mish": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.attention": [[134, 10, 1, "", "Attention"], [134, 10, 1, "", "AttentionMaskParams"], [134, 10, 1, "", "AttentionParams"], [134, 10, 1, "", "BertAttention"], [134, 10, 1, "", "BlockSparseAttnParams"], [134, 10, 1, "", "CogVLMAttention"], [134, 10, 1, "", "DeepseekV2Attention"], [134, 10, 1, "", "DiffusersAttention"], [134, 10, 1, "", "KeyValueCacheParams"], [134, 10, 1, "", "MropeParams"], [134, 10, 1, "", "SpecDecodingParams"], [134, 14, 1, "", "compute_relative_bias"], [134, 14, 1, "", "make_causal_mask"]], "tensorrt_llm.layers.attention.Attention": [[134, 12, 1, "", "create_attention_const_params"], [134, 12, 1, "", "fill_attention_params"], [134, 12, 1, "", "forward"], [134, 12, 1, "", "postprocess"], [134, 12, 1, "", "set_rel_attn_table"]], "tensorrt_llm.layers.attention.AttentionParams": [[134, 12, 1, "", "fill_attention_const_params_for_long_rope"], [134, 12, 1, "", "fill_attention_const_params_for_rope"], [134, 12, 1, "", "is_valid"], [134, 12, 1, "", "is_valid_cross_attn"]], "tensorrt_llm.layers.attention.BertAttention": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.attention.CogVLMAttention": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.attention.DeepseekV2Attention": [[134, 12, 1, "", "forward"], [134, 12, 1, "", "postprocess"], [134, 12, 1, "", "weight_loader"]], "tensorrt_llm.layers.attention.DiffusersAttention": [[134, 12, 1, "", "forward"], [134, 12, 1, "", "joint_attn_forward"]], "tensorrt_llm.layers.attention.KeyValueCacheParams": [[134, 12, 1, "", "fill_none_tensor_list"], [134, 12, 1, "", "get_first_past_key_value"], [134, 12, 1, "", "is_valid"]], "tensorrt_llm.layers.cast": [[134, 10, 1, "", "Cast"]], "tensorrt_llm.layers.cast.Cast": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.conv": [[134, 10, 1, "", "Conv1d"], [134, 10, 1, "", "Conv2d"], [134, 10, 1, "", "Conv3d"], [134, 10, 1, "", "ConvTranspose2d"]], "tensorrt_llm.layers.conv.Conv1d": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.conv.Conv2d": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.conv.Conv3d": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.conv.ConvTranspose2d": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.embedding": [[134, 10, 1, "", "CombinedTimestepLabelEmbeddings"], [134, 10, 1, "", "CombinedTimestepTextProjEmbeddings"], [134, 10, 1, "", "Embedding"], [134, 10, 1, "", "LabelEmbedding"], [134, 10, 1, "", "PixArtAlphaTextProjection"], [134, 10, 1, "", "PromptTuningEmbedding"], [134, 10, 1, "", "SD3PatchEmbed"], [134, 10, 1, "", "TimestepEmbedding"], [134, 10, 1, "", "Timesteps"], [134, 14, 1, "", "get_1d_sincos_pos_embed_from_grid"], [134, 14, 1, "", "get_2d_sincos_pos_embed"], [134, 14, 1, "", "get_2d_sincos_pos_embed_from_grid"], [134, 14, 1, "", "get_timestep_embedding"]], "tensorrt_llm.layers.embedding.CombinedTimestepLabelEmbeddings": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.embedding.CombinedTimestepTextProjEmbeddings": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.embedding.Embedding": [[134, 12, 1, "", "forward"], [134, 12, 1, "", "postprocess"], [134, 12, 1, "", "weight_loader"]], "tensorrt_llm.layers.embedding.LabelEmbedding": [[134, 12, 1, "", "forward"], [134, 12, 1, "", "token_drop"]], "tensorrt_llm.layers.embedding.PixArtAlphaTextProjection": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.embedding.PromptTuningEmbedding": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.embedding.SD3PatchEmbed": [[134, 12, 1, "", "cropped_pos_embed"], [134, 12, 1, "", "forward"]], "tensorrt_llm.layers.embedding.TimestepEmbedding": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.embedding.Timesteps": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.linear": [[134, 11, 1, "", "ColumnLinear"], [134, 10, 1, "", "Linear"], [134, 10, 1, "", "LinearBase"], [134, 10, 1, "", "RowLinear"]], "tensorrt_llm.layers.linear.Linear": [[134, 12, 1, "", "collect_and_bias"], [134, 12, 1, "", "postprocess"], [134, 12, 1, "", "tp_split_dim"]], "tensorrt_llm.layers.linear.LinearBase": [[134, 12, 1, "", "collect_and_bias"], [134, 12, 1, "", "forward"], [134, 12, 1, "", "get_weight"], [134, 12, 1, "", "multiply_and_lora"], [134, 12, 1, "", "multiply_collect"], [134, 12, 1, "", "tp_split_dim"], [134, 12, 1, "", "weight_loader"]], "tensorrt_llm.layers.linear.RowLinear": [[134, 12, 1, "", "collect_and_bias"], [134, 12, 1, "", "multiply_collect"], [134, 12, 1, "", "tp_split_dim"]], "tensorrt_llm.layers.mlp": [[134, 10, 1, "", "FusedGatedMLP"], [134, 10, 1, "", "GatedMLP"], [134, 10, 1, "", "LinearActivation"], [134, 10, 1, "", "LinearApproximateGELU"], [134, 10, 1, "", "LinearGEGLU"], [134, 10, 1, "", "LinearGELU"], [134, 10, 1, "", "LinearSwiGLU"], [134, 10, 1, "", "MLP"], [134, 14, 1, "", "fc_gate_dora"], [134, 14, 1, "", "fc_gate_lora"]], "tensorrt_llm.layers.mlp.FusedGatedMLP": [[134, 12, 1, "", "fc_gate"], [134, 12, 1, "", "fc_gate_plugin"], [134, 12, 1, "", "forward"]], "tensorrt_llm.layers.mlp.GatedMLP": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.mlp.LinearActivation": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.mlp.LinearApproximateGELU": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.mlp.LinearGEGLU": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.mlp.LinearGELU": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.mlp.LinearSwiGLU": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.mlp.MLP": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.normalization": [[134, 10, 1, "", "AdaLayerNorm"], [134, 10, 1, "", "AdaLayerNormContinuous"], [134, 10, 1, "", "AdaLayerNormZero"], [134, 10, 1, "", "AdaLayerNormZeroSingle"], [134, 10, 1, "", "GroupNorm"], [134, 10, 1, "", "LayerNorm"], [134, 10, 1, "", "RmsNorm"], [134, 10, 1, "", "SD35AdaLayerNormZeroX"]], "tensorrt_llm.layers.normalization.AdaLayerNorm": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.normalization.AdaLayerNormContinuous": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.normalization.AdaLayerNormZero": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.normalization.AdaLayerNormZeroSingle": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.normalization.GroupNorm": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.normalization.LayerNorm": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.normalization.RmsNorm": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.normalization.SD35AdaLayerNormZeroX": [[134, 12, 1, "", "forward"]], "tensorrt_llm.layers.pooling": [[134, 10, 1, "", "AvgPool2d"]], "tensorrt_llm.layers.pooling.AvgPool2d": [[134, 12, 1, "", "forward"]], "tensorrt_llm.llmapi": [[147, 10, 1, "", "AttentionDpConfig"], [147, 10, 1, "", "AutoDecodingConfig"], [147, 10, 1, "", "BatchingType"], [147, 10, 1, "", "BuildCacheConfig"], [147, 10, 1, "", "BuildConfig"], [147, 10, 1, "", "CacheTransceiverConfig"], [147, 10, 1, "", "CalibConfig"], [147, 10, 1, "", "CapacitySchedulerPolicy"], [147, 10, 1, "", "CompletionOutput"], [147, 10, 1, "", "ContextChunkingPolicy"], [147, 10, 1, "", "CudaGraphConfig"], [147, 10, 1, "", "DisaggregatedParams"], [147, 10, 1, "", "DraftTargetDecodingConfig"], [147, 10, 1, "", "DynamicBatchConfig"], [147, 10, 1, "", "EagleDecodingConfig"], [147, 10, 1, "", "ExtendedRuntimePerfKnobConfig"], [147, 10, 1, "", "GuidedDecodingParams"], [147, 10, 1, "", "KvCacheConfig"], [147, 10, 1, "", "KvCacheRetentionConfig"], [147, 10, 1, "", "LLM"], [147, 11, 1, "", "LlmArgs"], [147, 10, 1, "", "LoRARequest"], [147, 10, 1, "", "LookaheadDecodingConfig"], [147, 10, 1, "", "MTPDecodingConfig"], [147, 10, 1, "", "MedusaDecodingConfig"], [147, 10, 1, "", "MoeConfig"], [147, 10, 1, "", "MpiCommSession"], [147, 10, 1, "", "MultimodalEncoder"], [147, 10, 1, "", "NGramDecodingConfig"], [147, 10, 1, "", "QuantAlgo"], [147, 10, 1, "", "QuantConfig"], [147, 10, 1, "", "RequestError"], [147, 10, 1, "", "RequestOutput"], [147, 10, 1, "", "RocketSparseAttentionConfig"], [147, 10, 1, "", "SamplingParams"], [147, 10, 1, "", "SaveHiddenStatesDecodingConfig"], [147, 10, 1, "", "SchedulerConfig"], [147, 10, 1, "", "TorchCompileConfig"], [147, 10, 1, "", "TorchLlmArgs"], [147, 10, 1, "", "TrtLlmArgs"], [147, 10, 1, "", "UserProvidedDecodingConfig"]], "tensorrt_llm.llmapi.AttentionDpConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "batching_wait_iters"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 12, 1, "", "dict"], [147, 15, 1, "", "enable_balance"], [147, 12, 1, "", "from_dict"], [147, 12, 1, "", "from_orm"], [147, 12, 1, "", "json"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 15, 1, "", "timeout_iters"], [147, 12, 1, "", "update_forward_refs"], [147, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.AttentionDpConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.AutoDecodingConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "acceptance_length_threshold"], [147, 15, 1, "", "acceptance_window"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 11, 1, "", "decoding_type"], [147, 12, 1, "", "dict"], [147, 12, 1, "", "from_dict"], [147, 12, 1, "", "from_orm"], [147, 12, 1, "", "json"], [147, 15, 1, "", "load_format"], [147, 15, 1, "", "max_concurrency"], [147, 15, 1, "", "max_draft_len"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 13, 1, "", "spec_dec_mode"], [147, 15, 1, "", "speculative_model_dir"], [147, 12, 1, "", "supports_backend"], [147, 12, 1, "", "update_forward_refs"], [147, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.AutoDecodingConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.BatchingType": [[147, 11, 1, "", "INFLIGHT"], [147, 11, 1, "", "STATIC"], [147, 12, 1, "", "__init__"], [147, 12, 1, "", "capitalize"], [147, 12, 1, "", "casefold"], [147, 12, 1, "", "center"], [147, 12, 1, "", "count"], [147, 12, 1, "", "encode"], [147, 12, 1, "", "endswith"], [147, 12, 1, "", "expandtabs"], [147, 12, 1, "", "find"], [147, 12, 1, "", "format"], [147, 12, 1, "", "format_map"], [147, 12, 1, "", "index"], [147, 12, 1, "", "isalnum"], [147, 12, 1, "", "isalpha"], [147, 12, 1, "", "isascii"], [147, 12, 1, "", "isdecimal"], [147, 12, 1, "", "isdigit"], [147, 12, 1, "", "isidentifier"], [147, 12, 1, "", "islower"], [147, 12, 1, "", "isnumeric"], [147, 12, 1, "", "isprintable"], [147, 12, 1, "", "isspace"], [147, 12, 1, "", "istitle"], [147, 12, 1, "", "isupper"], [147, 12, 1, "", "join"], [147, 12, 1, "", "ljust"], [147, 12, 1, "", "lower"], [147, 12, 1, "", "lstrip"], [147, 12, 1, "", "maketrans"], [147, 12, 1, "", "partition"], [147, 12, 1, "", "removeprefix"], [147, 12, 1, "", "removesuffix"], [147, 12, 1, "", "replace"], [147, 12, 1, "", "rfind"], [147, 12, 1, "", "rindex"], [147, 12, 1, "", "rjust"], [147, 12, 1, "", "rpartition"], [147, 12, 1, "", "rsplit"], [147, 12, 1, "", "rstrip"], [147, 12, 1, "", "split"], [147, 12, 1, "", "splitlines"], [147, 12, 1, "", "startswith"], [147, 12, 1, "", "strip"], [147, 12, 1, "", "swapcase"], [147, 12, 1, "", "title"], [147, 12, 1, "", "translate"], [147, 12, 1, "", "upper"], [147, 12, 1, "", "zfill"]], "tensorrt_llm.llmapi.BuildCacheConfig": [[147, 12, 1, "", "__init__"], [147, 13, 1, "id13", "cache_root"], [147, 13, 1, "id14", "max_cache_storage_gb"], [147, 13, 1, "id15", "max_records"]], "tensorrt_llm.llmapi.BuildConfig": [[147, 12, 1, "", "__init__"], [147, 11, 1, "", "auto_parallel_config"], [147, 11, 1, "", "dry_run"], [147, 11, 1, "", "enable_debug_output"], [147, 11, 1, "", "force_num_profiles"], [147, 12, 1, "", "from_dict"], [147, 12, 1, "", "from_json_file"], [147, 11, 1, "", "gather_context_logits"], [147, 11, 1, "", "gather_generation_logits"], [147, 12, 1, "", "get_build_config_defaults"], [147, 11, 1, "", "input_timing_cache"], [147, 11, 1, "", "kv_cache_type"], [147, 11, 1, "", "lora_config"], [147, 11, 1, "", "max_batch_size"], [147, 11, 1, "", "max_beam_width"], [147, 11, 1, "", "max_draft_len"], [147, 11, 1, "", "max_encoder_input_len"], [147, 11, 1, "", "max_input_len"], [147, 11, 1, "", "max_num_tokens"], [147, 11, 1, "", "max_prompt_embedding_table_size"], [147, 11, 1, "", "max_seq_len"], [147, 11, 1, "", "monitor_memory"], [147, 11, 1, "", "opt_batch_size"], [147, 11, 1, "", "opt_num_tokens"], [147, 11, 1, "", "output_timing_cache"], [147, 11, 1, "", "plugin_config"], [147, 11, 1, "", "profiling_verbosity"], [147, 11, 1, "", "speculative_decoding_mode"], [147, 11, 1, "", "strongly_typed"], [147, 12, 1, "", "to_dict"], [147, 12, 1, "", "update"], [147, 12, 1, "", "update_from_dict"], [147, 12, 1, "", "update_kv_cache_type"], [147, 11, 1, "", "use_mrope"], [147, 11, 1, "", "use_refit"], [147, 11, 1, "", "use_strip_plan"], [147, 11, 1, "", "visualize_network"], [147, 11, 1, "", "weight_sparsity"], [147, 11, 1, "", "weight_streaming"]], "tensorrt_llm.llmapi.CacheTransceiverConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "backend"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 12, 1, "", "dict"], [147, 12, 1, "", "from_orm"], [147, 12, 1, "", "from_pybind"], [147, 12, 1, "", "get_pybind_enum_fields"], [147, 12, 1, "", "get_pybind_variable_fields"], [147, 12, 1, "", "json"], [147, 15, 1, "", "max_tokens_in_buffer"], [147, 12, 1, "", "maybe_to_pybind"], [147, 12, 1, "", "mirror_pybind_enum"], [147, 12, 1, "", "mirror_pybind_fields"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 12, 1, "", "pybind_equals"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 12, 1, "", "update_forward_refs"], [147, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.CacheTransceiverConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.CalibConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "calib_batch_size"], [147, 15, 1, "", "calib_batches"], [147, 15, 1, "", "calib_dataset"], [147, 15, 1, "", "calib_max_seq_length"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 15, 1, "", "device"], [147, 12, 1, "", "dict"], [147, 12, 1, "", "from_dict"], [147, 12, 1, "", "from_orm"], [147, 12, 1, "", "json"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 15, 1, "", "random_seed"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 12, 1, "", "to_dict"], [147, 15, 1, "", "tokenizer_max_seq_length"], [147, 12, 1, "", "update_forward_refs"], [147, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.CalibConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.CapacitySchedulerPolicy": [[147, 11, 1, "", "GUARANTEED_NO_EVICT"], [147, 11, 1, "", "MAX_UTILIZATION"], [147, 11, 1, "", "STATIC_BATCH"], [147, 12, 1, "", "__init__"], [147, 12, 1, "", "capitalize"], [147, 12, 1, "", "casefold"], [147, 12, 1, "", "center"], [147, 12, 1, "", "count"], [147, 12, 1, "", "encode"], [147, 12, 1, "", "endswith"], [147, 12, 1, "", "expandtabs"], [147, 12, 1, "", "find"], [147, 12, 1, "", "format"], [147, 12, 1, "", "format_map"], [147, 12, 1, "", "index"], [147, 12, 1, "", "isalnum"], [147, 12, 1, "", "isalpha"], [147, 12, 1, "", "isascii"], [147, 12, 1, "", "isdecimal"], [147, 12, 1, "", "isdigit"], [147, 12, 1, "", "isidentifier"], [147, 12, 1, "", "islower"], [147, 12, 1, "", "isnumeric"], [147, 12, 1, "", "isprintable"], [147, 12, 1, "", "isspace"], [147, 12, 1, "", "istitle"], [147, 12, 1, "", "isupper"], [147, 12, 1, "", "join"], [147, 12, 1, "", "ljust"], [147, 12, 1, "", "lower"], [147, 12, 1, "", "lstrip"], [147, 12, 1, "", "maketrans"], [147, 12, 1, "", "partition"], [147, 12, 1, "", "removeprefix"], [147, 12, 1, "", "removesuffix"], [147, 12, 1, "", "replace"], [147, 12, 1, "", "rfind"], [147, 12, 1, "", "rindex"], [147, 12, 1, "", "rjust"], [147, 12, 1, "", "rpartition"], [147, 12, 1, "", "rsplit"], [147, 12, 1, "", "rstrip"], [147, 12, 1, "", "split"], [147, 12, 1, "", "splitlines"], [147, 12, 1, "", "startswith"], [147, 12, 1, "", "strip"], [147, 12, 1, "", "swapcase"], [147, 12, 1, "", "title"], [147, 12, 1, "", "translate"], [147, 12, 1, "", "upper"], [147, 12, 1, "", "zfill"]], "tensorrt_llm.llmapi.CompletionOutput": [[147, 12, 1, "", "__init__"], [147, 11, 1, "", "additional_context_outputs"], [147, 11, 1, "", "additional_generation_outputs"], [147, 11, 1, "", "cumulative_logprob"], [147, 11, 1, "", "disaggregated_params"], [147, 11, 1, "", "finish_reason"], [147, 11, 1, "", "generation_logits"], [147, 11, 1, "", "index"], [147, 13, 1, "id2", "length"], [147, 11, 1, "", "logprobs"], [147, 13, 1, "id3", "logprobs_diff"], [147, 11, 1, "", "prompt_logprobs"], [147, 11, 1, "", "request_perf_metrics"], [147, 11, 1, "", "stop_reason"], [147, 11, 1, "", "text"], [147, 13, 1, "id4", "text_diff"], [147, 11, 1, "", "token_ids"], [147, 13, 1, "id5", "token_ids_diff"]], "tensorrt_llm.llmapi.ContextChunkingPolicy": [[147, 11, 1, "", "EQUAL_PROGRESS"], [147, 11, 1, "", "FIRST_COME_FIRST_SERVED"], [147, 12, 1, "", "__init__"], [147, 12, 1, "", "capitalize"], [147, 12, 1, "", "casefold"], [147, 12, 1, "", "center"], [147, 12, 1, "", "count"], [147, 12, 1, "", "encode"], [147, 12, 1, "", "endswith"], [147, 12, 1, "", "expandtabs"], [147, 12, 1, "", "find"], [147, 12, 1, "", "format"], [147, 12, 1, "", "format_map"], [147, 12, 1, "", "index"], [147, 12, 1, "", "isalnum"], [147, 12, 1, "", "isalpha"], [147, 12, 1, "", "isascii"], [147, 12, 1, "", "isdecimal"], [147, 12, 1, "", "isdigit"], [147, 12, 1, "", "isidentifier"], [147, 12, 1, "", "islower"], [147, 12, 1, "", "isnumeric"], [147, 12, 1, "", "isprintable"], [147, 12, 1, "", "isspace"], [147, 12, 1, "", "istitle"], [147, 12, 1, "", "isupper"], [147, 12, 1, "", "join"], [147, 12, 1, "", "ljust"], [147, 12, 1, "", "lower"], [147, 12, 1, "", "lstrip"], [147, 12, 1, "", "maketrans"], [147, 12, 1, "", "partition"], [147, 12, 1, "", "removeprefix"], [147, 12, 1, "", "removesuffix"], [147, 12, 1, "", "replace"], [147, 12, 1, "", "rfind"], [147, 12, 1, "", "rindex"], [147, 12, 1, "", "rjust"], [147, 12, 1, "", "rpartition"], [147, 12, 1, "", "rsplit"], [147, 12, 1, "", "rstrip"], [147, 12, 1, "", "split"], [147, 12, 1, "", "splitlines"], [147, 12, 1, "", "startswith"], [147, 12, 1, "", "strip"], [147, 12, 1, "", "swapcase"], [147, 12, 1, "", "title"], [147, 12, 1, "", "translate"], [147, 12, 1, "", "upper"], [147, 12, 1, "", "zfill"]], "tensorrt_llm.llmapi.CudaGraphConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "batch_sizes"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 12, 1, "", "dict"], [147, 15, 1, "", "enable_padding"], [147, 12, 1, "", "from_orm"], [147, 12, 1, "", "json"], [147, 15, 1, "", "max_batch_size"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 12, 1, "", "update_forward_refs"], [147, 12, 1, "", "validate"], [147, 16, 1, "", "validate_cuda_graph_max_batch_size"]], "tensorrt_llm.llmapi.CudaGraphConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.DisaggregatedParams": [[147, 12, 1, "", "__init__"], [147, 11, 1, "", "ctx_request_id"], [147, 11, 1, "", "draft_tokens"], [147, 11, 1, "", "first_gen_tokens"], [147, 12, 1, "", "get_context_phase_params"], [147, 12, 1, "", "get_request_type"], [147, 11, 1, "", "multimodal_embedding_handles"], [147, 11, 1, "", "multimodal_hashes"], [147, 11, 1, "", "opaque_state"], [147, 11, 1, "", "request_type"]], "tensorrt_llm.llmapi.DraftTargetDecodingConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "acceptance_length_threshold"], [147, 15, 1, "", "acceptance_window"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 11, 1, "", "decoding_type"], [147, 12, 1, "", "dict"], [147, 12, 1, "", "from_dict"], [147, 12, 1, "", "from_orm"], [147, 12, 1, "", "json"], [147, 15, 1, "", "load_format"], [147, 15, 1, "", "max_concurrency"], [147, 15, 1, "", "max_draft_len"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 13, 1, "", "spec_dec_mode"], [147, 15, 1, "", "speculative_model_dir"], [147, 12, 1, "", "supports_backend"], [147, 12, 1, "", "update_forward_refs"], [147, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.DraftTargetDecodingConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.DynamicBatchConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 12, 1, "", "dict"], [147, 15, 1, "", "dynamic_batch_moving_average_window"], [147, 15, 1, "", "enable_batch_size_tuning"], [147, 15, 1, "", "enable_max_num_tokens_tuning"], [147, 12, 1, "", "from_orm"], [147, 12, 1, "", "from_pybind"], [147, 12, 1, "", "get_pybind_enum_fields"], [147, 12, 1, "", "get_pybind_variable_fields"], [147, 12, 1, "", "json"], [147, 12, 1, "", "maybe_to_pybind"], [147, 12, 1, "", "mirror_pybind_enum"], [147, 12, 1, "", "mirror_pybind_fields"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 12, 1, "", "pybind_equals"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 12, 1, "", "update_forward_refs"], [147, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.DynamicBatchConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.EagleDecodingConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "acceptance_length_threshold"], [147, 15, 1, "", "acceptance_window"], [147, 12, 1, "", "check_eagle_choices"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 11, 1, "", "decoding_type"], [147, 12, 1, "", "dict"], [147, 15, 1, "", "dynamic_tree_max_topK"], [147, 15, 1, "", "eagle3_layers_to_capture"], [147, 15, 1, "", "eagle3_one_model"], [147, 15, 1, "", "eagle_choices"], [147, 12, 1, "", "from_dict"], [147, 12, 1, "", "from_orm"], [147, 15, 1, "", "greedy_sampling"], [147, 12, 1, "", "json"], [147, 15, 1, "", "load_format"], [147, 15, 1, "", "max_concurrency"], [147, 15, 1, "", "max_draft_len"], [147, 15, 1, "", "max_non_leaves_per_layer"], [147, 15, 1, "", "max_total_draft_tokens"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 13, 1, "", "num_capture_layers"], [147, 15, 1, "", "num_eagle_layers"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 15, 1, "", "posterior_threshold"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 13, 1, "", "spec_dec_mode"], [147, 15, 1, "", "speculative_model_dir"], [147, 12, 1, "", "supports_backend"], [147, 12, 1, "", "update_forward_refs"], [147, 15, 1, "", "use_dynamic_tree"], [147, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.EagleDecodingConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.ExtendedRuntimePerfKnobConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 15, 1, "", "cuda_graph_cache_size"], [147, 15, 1, "", "cuda_graph_mode"], [147, 12, 1, "", "dict"], [147, 15, 1, "", "enable_context_fmha_fp32_acc"], [147, 12, 1, "", "from_orm"], [147, 12, 1, "", "from_pybind"], [147, 12, 1, "", "get_pybind_enum_fields"], [147, 12, 1, "", "get_pybind_variable_fields"], [147, 12, 1, "", "json"], [147, 12, 1, "", "maybe_to_pybind"], [147, 12, 1, "", "mirror_pybind_enum"], [147, 12, 1, "", "mirror_pybind_fields"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 15, 1, "", "multi_block_mode"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 12, 1, "", "pybind_equals"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 12, 1, "", "update_forward_refs"], [147, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.ExtendedRuntimePerfKnobConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.GuidedDecodingParams": [[147, 12, 1, "", "__init__"], [147, 11, 1, "", "grammar"], [147, 11, 1, "", "json"], [147, 11, 1, "", "json_object"], [147, 11, 1, "", "regex"], [147, 11, 1, "", "structural_tag"]], "tensorrt_llm.llmapi.KvCacheConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "attention_dp_events_gather_period_ms"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 15, 1, "", "copy_on_partial_reuse"], [147, 15, 1, "", "cross_kv_cache_fraction"], [147, 12, 1, "", "dict"], [147, 15, 1, "", "dtype"], [147, 15, 1, "", "enable_block_reuse"], [147, 15, 1, "", "enable_partial_reuse"], [147, 15, 1, "", "event_buffer_max_size"], [147, 15, 1, "", "free_gpu_memory_fraction"], [147, 12, 1, "", "from_orm"], [147, 12, 1, "", "from_pybind"], [147, 12, 1, "", "get_pybind_enum_fields"], [147, 12, 1, "", "get_pybind_variable_fields"], [147, 15, 1, "", "host_cache_size"], [147, 12, 1, "", "json"], [147, 15, 1, "", "mamba_ssm_cache_dtype"], [147, 15, 1, "", "max_attention_window"], [147, 15, 1, "", "max_gpu_total_bytes"], [147, 15, 1, "", "max_tokens"], [147, 12, 1, "", "maybe_to_pybind"], [147, 12, 1, "", "mirror_pybind_enum"], [147, 12, 1, "", "mirror_pybind_fields"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 15, 1, "", "onboard_blocks"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 12, 1, "", "pybind_equals"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 15, 1, "", "secondary_offload_min_priority"], [147, 15, 1, "", "sink_token_length"], [147, 15, 1, "", "tokens_per_block"], [147, 12, 1, "", "update_forward_refs"], [147, 15, 1, "", "use_uvm"], [147, 12, 1, "", "validate"], [147, 16, 1, "", "validate_max_attention_window"], [147, 16, 1, "", "validate_max_gpu_total_bytes"]], "tensorrt_llm.llmapi.KvCacheConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.KvCacheRetentionConfig": [[147, 10, 1, "", "TokenRangeRetentionConfig"], [147, 11, 1, "", "__init__"], [147, 13, 1, "", "decode_duration_ms"], [147, 13, 1, "", "decode_retention_priority"], [147, 13, 1, "", "directory"], [147, 13, 1, "", "token_range_retention_configs"], [147, 13, 1, "", "transfer_mode"]], "tensorrt_llm.llmapi.KvCacheRetentionConfig.TokenRangeRetentionConfig": [[147, 11, 1, "", "__init__"], [147, 13, 1, "", "duration_ms"], [147, 13, 1, "", "priority"], [147, 13, 1, "", "token_end"], [147, 13, 1, "", "token_start"]], "tensorrt_llm.llmapi.LLM": [[147, 12, 1, "", "__init__"], [147, 12, 1, "", "generate"], [147, 12, 1, "", "generate_async"], [147, 12, 1, "", "get_kv_cache_events"], [147, 12, 1, "", "get_kv_cache_events_async"], [147, 12, 1, "", "get_stats"], [147, 12, 1, "", "get_stats_async"], [147, 13, 1, "id0", "llm_id"], [147, 12, 1, "", "shutdown"], [147, 13, 1, "id1", "tokenizer"]], "tensorrt_llm.llmapi.LoRARequest": [[147, 12, 1, "", "__init__"], [147, 13, 1, "", "adapter_id"], [147, 13, 1, "", "ckpt_source"], [147, 11, 1, "", "lora_ckpt_source"], [147, 11, 1, "", "lora_int_id"], [147, 11, 1, "", "lora_name"], [147, 11, 1, "", "lora_path"], [147, 13, 1, "", "name"], [147, 13, 1, "", "path"]], "tensorrt_llm.llmapi.LookaheadDecodingConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "acceptance_length_threshold"], [147, 15, 1, "", "acceptance_window"], [147, 12, 1, "", "calculate_speculative_resource"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 11, 1, "", "decoding_type"], [147, 12, 1, "", "dict"], [147, 12, 1, "", "from_dict"], [147, 12, 1, "", "from_orm"], [147, 12, 1, "", "from_pybind"], [147, 12, 1, "", "get_pybind_enum_fields"], [147, 12, 1, "", "get_pybind_variable_fields"], [147, 12, 1, "", "json"], [147, 15, 1, "", "load_format"], [147, 15, 1, "", "max_concurrency"], [147, 15, 1, "", "max_draft_len"], [147, 15, 1, "", "max_ngram_size"], [147, 15, 1, "", "max_verification_set_size"], [147, 15, 1, "", "max_window_size"], [147, 12, 1, "", "maybe_to_pybind"], [147, 12, 1, "", "mirror_pybind_enum"], [147, 12, 1, "", "mirror_pybind_fields"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 12, 1, "", "pybind_equals"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 13, 1, "", "spec_dec_mode"], [147, 15, 1, "", "speculative_model_dir"], [147, 12, 1, "", "supports_backend"], [147, 12, 1, "", "update_forward_refs"], [147, 12, 1, "", "validate"], [147, 16, 1, "", "validate_positive_values"]], "tensorrt_llm.llmapi.LookaheadDecodingConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.MTPDecodingConfig": [[147, 15, 1, "", "BEGIN_THINKING_PHASE_TOKEN"], [147, 10, 1, "", "Config"], [147, 15, 1, "", "END_THINKING_PHASE_TOKEN"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "acceptance_length_threshold"], [147, 15, 1, "", "acceptance_window"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 11, 1, "", "decoding_type"], [147, 12, 1, "", "dict"], [147, 12, 1, "", "from_dict"], [147, 12, 1, "", "from_orm"], [147, 12, 1, "", "json"], [147, 15, 1, "", "load_format"], [147, 15, 1, "", "max_concurrency"], [147, 15, 1, "", "max_draft_len"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 15, 1, "", "mtp_eagle_one_model"], [147, 13, 1, "", "num_capture_layers"], [147, 15, 1, "", "num_nextn_predict_layers"], [147, 15, 1, "", "num_nextn_predict_layers_from_model_config"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 15, 1, "", "relaxed_delta"], [147, 15, 1, "", "relaxed_topk"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 13, 1, "", "spec_dec_mode"], [147, 15, 1, "", "speculative_model_dir"], [147, 12, 1, "", "supports_backend"], [147, 12, 1, "", "update_forward_refs"], [147, 15, 1, "", "use_mtp_vanilla"], [147, 15, 1, "", "use_relaxed_acceptance_for_thinking"], [147, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.MTPDecodingConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.MedusaDecodingConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "acceptance_length_threshold"], [147, 15, 1, "", "acceptance_window"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 11, 1, "", "decoding_type"], [147, 12, 1, "", "dict"], [147, 12, 1, "", "from_dict"], [147, 12, 1, "", "from_orm"], [147, 12, 1, "", "json"], [147, 15, 1, "", "load_format"], [147, 15, 1, "", "max_concurrency"], [147, 15, 1, "", "max_draft_len"], [147, 15, 1, "", "medusa_choices"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 15, 1, "", "num_medusa_heads"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 13, 1, "", "spec_dec_mode"], [147, 15, 1, "", "speculative_model_dir"], [147, 12, 1, "", "supports_backend"], [147, 12, 1, "", "update_forward_refs"], [147, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.MedusaDecodingConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.MoeConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "backend"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 12, 1, "", "dict"], [147, 15, 1, "", "disable_finalize_fusion"], [147, 12, 1, "", "from_dict"], [147, 12, 1, "", "from_orm"], [147, 12, 1, "", "json"], [147, 15, 1, "", "load_balancer"], [147, 15, 1, "", "max_num_tokens"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 12, 1, "", "update_forward_refs"], [147, 15, 1, "", "use_low_precision_moe_combine"], [147, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.MoeConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.MpiCommSession": [[147, 12, 1, "", "__init__"], [147, 12, 1, "", "abort"], [147, 12, 1, "", "get_comm"], [147, 12, 1, "", "is_comm_session"], [147, 12, 1, "", "shutdown"], [147, 12, 1, "", "shutdown_abort"], [147, 12, 1, "", "submit"], [147, 12, 1, "", "submit_sync"]], "tensorrt_llm.llmapi.MultimodalEncoder": [[147, 12, 1, "", "__init__"], [147, 12, 1, "", "generate"], [147, 12, 1, "", "generate_async"], [147, 12, 1, "", "get_kv_cache_events"], [147, 12, 1, "", "get_kv_cache_events_async"], [147, 12, 1, "", "get_stats"], [147, 12, 1, "", "get_stats_async"], [147, 13, 1, "", "llm_id"], [147, 12, 1, "", "shutdown"], [147, 13, 1, "", "tokenizer"]], "tensorrt_llm.llmapi.NGramDecodingConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "acceptance_length_threshold"], [147, 15, 1, "", "acceptance_window"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 11, 1, "", "decoding_type"], [147, 12, 1, "", "dict"], [147, 12, 1, "", "from_dict"], [147, 12, 1, "", "from_orm"], [147, 15, 1, "", "is_keep_all"], [147, 15, 1, "", "is_public_pool"], [147, 15, 1, "", "is_use_oldest"], [147, 12, 1, "", "json"], [147, 15, 1, "", "load_format"], [147, 15, 1, "", "max_concurrency"], [147, 15, 1, "", "max_draft_len"], [147, 15, 1, "", "max_matching_ngram_size"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 13, 1, "", "spec_dec_mode"], [147, 15, 1, "", "speculative_model_dir"], [147, 12, 1, "", "supports_backend"], [147, 12, 1, "", "update_forward_refs"], [147, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.NGramDecodingConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.QuantAlgo": [[147, 11, 1, "", "FP8"], [147, 11, 1, "", "FP8_BLOCK_SCALES"], [147, 11, 1, "", "FP8_PER_CHANNEL_PER_TOKEN"], [147, 11, 1, "", "INT8"], [147, 11, 1, "", "MIXED_PRECISION"], [147, 11, 1, "", "NO_QUANT"], [147, 11, 1, "", "NVFP4"], [147, 11, 1, "", "W4A16"], [147, 11, 1, "", "W4A16_AWQ"], [147, 11, 1, "", "W4A16_GPTQ"], [147, 11, 1, "", "W4A16_MXFP4"], [147, 11, 1, "", "W4A8_AWQ"], [147, 11, 1, "", "W4A8_MXFP4_FP8"], [147, 11, 1, "", "W4A8_MXFP4_MXFP8"], [147, 11, 1, "", "W4A8_NVFP4_FP8"], [147, 11, 1, "", "W4A8_QSERVE_PER_CHANNEL"], [147, 11, 1, "", "W4A8_QSERVE_PER_GROUP"], [147, 11, 1, "", "W8A16"], [147, 11, 1, "", "W8A16_GPTQ"], [147, 11, 1, "", "W8A8_SQ_PER_CHANNEL"], [147, 11, 1, "", "W8A8_SQ_PER_CHANNEL_PER_TENSOR_PLUGIN"], [147, 11, 1, "", "W8A8_SQ_PER_CHANNEL_PER_TOKEN_PLUGIN"], [147, 11, 1, "", "W8A8_SQ_PER_TENSOR_PER_TOKEN_PLUGIN"], [147, 11, 1, "", "W8A8_SQ_PER_TENSOR_PLUGIN"], [147, 12, 1, "", "__init__"], [147, 12, 1, "", "capitalize"], [147, 12, 1, "", "casefold"], [147, 12, 1, "", "center"], [147, 12, 1, "", "count"], [147, 12, 1, "", "encode"], [147, 12, 1, "", "endswith"], [147, 12, 1, "", "expandtabs"], [147, 12, 1, "", "find"], [147, 12, 1, "", "format"], [147, 12, 1, "", "format_map"], [147, 12, 1, "", "index"], [147, 12, 1, "", "isalnum"], [147, 12, 1, "", "isalpha"], [147, 12, 1, "", "isascii"], [147, 12, 1, "", "isdecimal"], [147, 12, 1, "", "isdigit"], [147, 12, 1, "", "isidentifier"], [147, 12, 1, "", "islower"], [147, 12, 1, "", "isnumeric"], [147, 12, 1, "", "isprintable"], [147, 12, 1, "", "isspace"], [147, 12, 1, "", "istitle"], [147, 12, 1, "", "isupper"], [147, 12, 1, "", "join"], [147, 12, 1, "", "ljust"], [147, 12, 1, "", "lower"], [147, 12, 1, "", "lstrip"], [147, 12, 1, "", "maketrans"], [147, 12, 1, "", "partition"], [147, 12, 1, "", "removeprefix"], [147, 12, 1, "", "removesuffix"], [147, 12, 1, "", "replace"], [147, 12, 1, "", "rfind"], [147, 12, 1, "", "rindex"], [147, 12, 1, "", "rjust"], [147, 12, 1, "", "rpartition"], [147, 12, 1, "", "rsplit"], [147, 12, 1, "", "rstrip"], [147, 12, 1, "", "split"], [147, 12, 1, "", "splitlines"], [147, 12, 1, "", "startswith"], [147, 12, 1, "", "strip"], [147, 12, 1, "", "swapcase"], [147, 12, 1, "", "title"], [147, 12, 1, "", "translate"], [147, 12, 1, "", "upper"], [147, 12, 1, "", "zfill"]], "tensorrt_llm.llmapi.QuantConfig": [[147, 12, 1, "", "__init__"], [147, 11, 1, "", "clamp_val"], [147, 11, 1, "", "exclude_modules"], [147, 12, 1, "", "from_dict"], [147, 11, 1, "", "group_size"], [147, 11, 1, "", "has_zero_point"], [147, 12, 1, "", "is_module_excluded_from_quantization"], [147, 11, 1, "", "kv_cache_quant_algo"], [147, 13, 1, "", "layer_quant_mode"], [147, 11, 1, "", "mamba_ssm_cache_dtype"], [147, 11, 1, "", "pre_quant_scale"], [147, 11, 1, "", "quant_algo"], [147, 13, 1, "", "quant_mode"], [147, 11, 1, "", "smoothquant_val"], [147, 12, 1, "", "to_dict"], [147, 11, 1, "", "use_meta_recipe"]], "tensorrt_llm.llmapi.RequestError": [[147, 12, 1, "", "__init__"], [147, 12, 1, "", "add_note"], [147, 11, 1, "", "args"], [147, 12, 1, "", "with_traceback"]], "tensorrt_llm.llmapi.RequestOutput": [[147, 10, 1, "", "PostprocWorker"], [147, 12, 1, "", "__init__"], [147, 12, 1, "", "abort"], [147, 12, 1, "", "aborted"], [147, 12, 1, "", "aresult"], [147, 12, 1, "", "clear_logprob_params"], [147, 13, 1, "id6", "context_logits"], [147, 13, 1, "id7", "finished"], [147, 13, 1, "id8", "mm_embedding_handle"], [147, 13, 1, "id9", "outputs"], [147, 13, 1, "id10", "prompt"], [147, 13, 1, "id11", "prompt_token_ids"], [147, 12, 1, "", "record_stats"], [147, 13, 1, "id12", "request_id"], [147, 12, 1, "", "result"]], "tensorrt_llm.llmapi.RequestOutput.PostprocWorker": [[147, 10, 1, "", "Input"], [147, 10, 1, "", "Output"], [147, 12, 1, "", "__init__"], [147, 12, 1, "", "default_record_creator"], [147, 12, 1, "", "start"]], "tensorrt_llm.llmapi.RequestOutput.PostprocWorker.Input": [[147, 12, 1, "", "__init__"], [147, 11, 1, "", "postproc_params"], [147, 11, 1, "", "rsp"], [147, 11, 1, "", "sampling_params"], [147, 11, 1, "", "streaming"]], "tensorrt_llm.llmapi.RequestOutput.PostprocWorker.Output": [[147, 11, 1, "", "client_id"], [147, 12, 1, "", "count"], [147, 11, 1, "", "disaggregated_params"], [147, 11, 1, "", "error"], [147, 12, 1, "", "index"], [147, 11, 1, "", "is_final"], [147, 11, 1, "", "metrics"], [147, 11, 1, "", "request_perf_metrics"], [147, 11, 1, "", "res"]], "tensorrt_llm.llmapi.RocketSparseAttentionConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "algorithm"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 12, 1, "", "dict"], [147, 12, 1, "", "from_dict"], [147, 12, 1, "", "from_orm"], [147, 12, 1, "", "json"], [147, 15, 1, "", "kernel_size"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 15, 1, "", "page_size"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 15, 1, "", "prompt_budget"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 12, 1, "", "supports_backend"], [147, 15, 1, "", "topk"], [147, 15, 1, "", "topr"], [147, 12, 1, "", "update_forward_refs"], [147, 12, 1, "", "validate"], [147, 15, 1, "", "window_size"]], "tensorrt_llm.llmapi.RocketSparseAttentionConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.SamplingParams": [[147, 12, 1, "", "__init__"], [147, 11, 1, "", "add_special_tokens"], [147, 11, 1, "", "additional_model_outputs"], [147, 11, 1, "", "apply_batched_logits_processor"], [147, 11, 1, "", "bad"], [147, 11, 1, "", "bad_token_ids"], [147, 11, 1, "", "beam_search_diversity_rate"], [147, 11, 1, "", "beam_width_array"], [147, 11, 1, "", "best_of"], [147, 11, 1, "", "detokenize"], [147, 11, 1, "", "early_stopping"], [147, 11, 1, "", "embedding_bias"], [147, 11, 1, "", "end_id"], [147, 11, 1, "", "exclude_input_from_output"], [147, 11, 1, "", "frequency_penalty"], [147, 11, 1, "", "guided_decoding"], [147, 11, 1, "", "ignore_eos"], [147, 11, 1, "", "include_stop_str_in_output"], [147, 11, 1, "", "length_penalty"], [147, 11, 1, "", "logits_processor"], [147, 11, 1, "", "logprobs"], [147, 11, 1, "", "lookahead_config"], [147, 11, 1, "", "max_tokens"], [147, 11, 1, "", "min_p"], [147, 11, 1, "", "min_tokens"], [147, 11, 1, "", "n"], [147, 11, 1, "", "no_repeat_ngram_size"], [147, 11, 1, "", "pad_id"], [147, 12, 1, "", "params_imply_greedy_decoding"], [147, 11, 1, "", "presence_penalty"], [147, 11, 1, "", "prompt_logprobs"], [147, 11, 1, "", "repetition_penalty"], [147, 11, 1, "", "return_context_logits"], [147, 11, 1, "", "return_encoder_output"], [147, 11, 1, "", "return_generation_logits"], [147, 11, 1, "", "return_perf_metrics"], [147, 11, 1, "", "seed"], [147, 11, 1, "", "skip_special_tokens"], [147, 11, 1, "", "spaces_between_special_tokens"], [147, 11, 1, "", "stop"], [147, 11, 1, "", "stop_token_ids"], [147, 11, 1, "", "temperature"], [147, 11, 1, "", "top_k"], [147, 11, 1, "", "top_p"], [147, 11, 1, "", "top_p_decay"], [147, 11, 1, "", "top_p_min"], [147, 11, 1, "", "top_p_reset_ids"], [147, 11, 1, "", "truncate_prompt_tokens"], [147, 11, 1, "", "use_beam_search"]], "tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "acceptance_length_threshold"], [147, 15, 1, "", "acceptance_window"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 11, 1, "", "decoding_type"], [147, 12, 1, "", "dict"], [147, 15, 1, "", "eagle3_layers_to_capture"], [147, 15, 1, "", "eagle_choices"], [147, 15, 1, "", "file_prefix"], [147, 12, 1, "", "from_dict"], [147, 12, 1, "", "from_orm"], [147, 12, 1, "", "json"], [147, 15, 1, "", "load_format"], [147, 15, 1, "", "max_concurrency"], [147, 15, 1, "", "max_draft_len"], [147, 15, 1, "", "max_total_draft_tokens"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 13, 1, "", "num_capture_layers"], [147, 15, 1, "", "output_directory"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 13, 1, "", "spec_dec_mode"], [147, 15, 1, "", "speculative_model_dir"], [147, 12, 1, "", "supports_backend"], [147, 12, 1, "", "update_forward_refs"], [147, 12, 1, "", "validate"], [147, 15, 1, "", "write_interval"]], "tensorrt_llm.llmapi.SaveHiddenStatesDecodingConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.SchedulerConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "capacity_scheduler_policy"], [147, 12, 1, "", "construct"], [147, 15, 1, "", "context_chunking_policy"], [147, 12, 1, "", "copy"], [147, 12, 1, "", "dict"], [147, 15, 1, "", "dynamic_batch_config"], [147, 12, 1, "", "from_orm"], [147, 12, 1, "", "from_pybind"], [147, 12, 1, "", "get_pybind_enum_fields"], [147, 12, 1, "", "get_pybind_variable_fields"], [147, 12, 1, "", "json"], [147, 12, 1, "", "maybe_to_pybind"], [147, 12, 1, "", "mirror_pybind_enum"], [147, 12, 1, "", "mirror_pybind_fields"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 12, 1, "", "pybind_equals"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 12, 1, "", "update_forward_refs"], [147, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.SchedulerConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.TorchCompileConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "capture_num_tokens"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 12, 1, "", "dict"], [147, 15, 1, "", "enable_fullgraph"], [147, 15, 1, "", "enable_inductor"], [147, 15, 1, "", "enable_piecewise_cuda_graph"], [147, 15, 1, "", "enable_userbuffers"], [147, 12, 1, "", "from_orm"], [147, 12, 1, "", "json"], [147, 15, 1, "", "max_num_streams"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 12, 1, "", "update_forward_refs"], [147, 12, 1, "", "validate"], [147, 16, 1, "", "validate_capture_num_tokens"], [147, 16, 1, "", "validate_torch_compile_max_num_streams"]], "tensorrt_llm.llmapi.TorchCompileConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.TorchLlmArgs": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "allreduce_strategy"], [147, 15, 1, "", "attention_dp_config"], [147, 15, 1, "", "attn_backend"], [147, 15, 1, "", "backend"], [147, 15, 1, "", "batch_wait_max_tokens_ratio"], [147, 15, 1, "", "batch_wait_timeout_iters"], [147, 15, 1, "", "batch_wait_timeout_ms"], [147, 15, 1, "", "batched_logits_processor"], [147, 15, 1, "", "build_config"], [147, 15, 1, "", "cache_transceiver_config"], [147, 15, 1, "", "checkpoint_format"], [147, 15, 1, "", "checkpoint_loader"], [147, 15, 1, "", "context_parallel_size"], [147, 16, 1, "", "convert_load_format"], [147, 15, 1, "", "cp_config"], [147, 15, 1, "", "cuda_graph_config"], [147, 11, 1, "", "decoding_config"], [147, 15, 1, "", "disable_overlap_scheduler"], [147, 15, 1, "", "dtype"], [147, 15, 1, "", "enable_attention_dp"], [147, 15, 1, "", "enable_autotuner"], [147, 15, 1, "", "enable_chunked_prefill"], [147, 15, 1, "", "enable_iter_perf_stats"], [147, 15, 1, "", "enable_iter_req_stats"], [147, 15, 1, "", "enable_layerwise_nvtx_marker"], [147, 15, 1, "", "enable_lm_head_tp_in_adp"], [147, 15, 1, "", "enable_lora"], [147, 15, 1, "", "enable_min_latency"], [147, 13, 1, "", "extra_resource_managers"], [147, 15, 1, "", "fail_fast_on_attention_window_too_large"], [147, 11, 1, "", "field_name"], [147, 15, 1, "", "force_dynamic_quantization"], [147, 12, 1, "", "from_kwargs"], [147, 15, 1, "", "garbage_collection_gen0_threshold"], [147, 15, 1, "", "gather_generation_logits"], [147, 12, 1, "", "get_executor_config"], [147, 12, 1, "", "get_pytorch_backend_config"], [147, 12, 1, "", "get_runtime_sizes"], [147, 15, 1, "", "gpus_per_node"], [147, 15, 1, "", "guided_decoding_backend"], [147, 16, 1, "", "init_backend"], [147, 16, 1, "", "init_build_config"], [147, 15, 1, "", "iter_stats_max_iterations"], [147, 15, 1, "", "kv_cache_config"], [147, 15, 1, "", "kv_connector_config"], [147, 15, 1, "", "load_format"], [147, 15, 1, "", "lora_config"], [147, 15, 1, "", "max_batch_size"], [147, 15, 1, "", "max_beam_width"], [147, 15, 1, "", "max_input_len"], [147, 15, 1, "", "max_num_tokens"], [147, 15, 1, "", "max_seq_len"], [147, 15, 1, "", "mm_encoder_only"], [147, 15, 1, "", "model"], [147, 13, 1, "", "model_format"], [147, 15, 1, "", "moe_cluster_parallel_size"], [147, 15, 1, "", "moe_config"], [147, 15, 1, "", "moe_expert_parallel_size"], [147, 15, 1, "", "moe_tensor_parallel_size"], [147, 15, 1, "", "mpi_session"], [147, 11, 1, "", "msg"], [147, 15, 1, "", "num_postprocess_workers"], [147, 15, 1, "", "orchestrator_type"], [147, 13, 1, "", "parallel_config"], [147, 15, 1, "", "peft_cache_config"], [147, 15, 1, "", "perf_metrics_max_requests"], [147, 15, 1, "", "pipeline_parallel_size"], [147, 15, 1, "", "postprocess_tokenizer_dir"], [147, 15, 1, "", "print_iter_log"], [147, 13, 1, "", "quant_config"], [147, 15, 1, "", "reasoning_parser"], [147, 15, 1, "", "request_stats_max_iterations"], [147, 15, 1, "", "return_perf_metrics"], [147, 15, 1, "", "revision"], [147, 15, 1, "", "sampler_type"], [147, 15, 1, "", "scheduler_config"], [147, 16, 1, "", "set_default_max_input_len"], [147, 16, 1, "", "set_runtime_knobs_from_build_config"], [147, 15, 1, "", "skip_tokenizer_init"], [147, 15, 1, "", "sparse_attention_config"], [147, 15, 1, "", "speculative_config"], [147, 13, 1, "", "speculative_model_dir"], [147, 13, 1, "", "speculative_model_format"], [147, 15, 1, "", "stream_interval"], [147, 16, 1, "", "sync_quant_config_with_kv_cache_config_dtype"], [147, 15, 1, "", "tensor_parallel_size"], [147, 12, 1, "", "to_dict"], [147, 15, 1, "", "tokenizer"], [147, 15, 1, "", "tokenizer_mode"], [147, 15, 1, "", "tokenizer_revision"], [147, 15, 1, "", "torch_compile_config"], [147, 15, 1, "", "trust_remote_code"], [147, 16, 1, "", "validate_and_init_tokenizer"], [147, 16, 1, "", "validate_attention_dp_config"], [147, 16, 1, "", "validate_batch_wait_max_tokens_ratio"], [147, 16, 1, "", "validate_batch_wait_timeout_iters"], [147, 16, 1, "", "validate_batch_wait_timeout_ms"], [147, 16, 1, "", "validate_build_config_remaining"], [147, 16, 1, "", "validate_build_config_with_runtime_params"], [147, 16, 1, "", "validate_checkpoint_format"], [147, 16, 1, "", "validate_cuda_graph_config"], [147, 16, 1, "", "validate_dtype"], [147, 16, 1, "", "validate_gpus_per_node"], [147, 16, 1, "", "validate_load_balancer"], [147, 16, 1, "", "validate_lora_config_consistency"], [147, 16, 1, "", "validate_model"], [147, 16, 1, "", "validate_model_format_misc"], [147, 16, 1, "", "validate_parallel_config"], [147, 16, 1, "", "validate_peft_cache_config"], [147, 16, 1, "", "validate_runtime_args"], [147, 16, 1, "", "validate_speculative_config"], [147, 16, 1, "", "validate_stream_interval"], [147, 12, 1, "", "warn_on_unstable_feature_usage"], [147, 11, 1, "", "wrapped_property"]], "tensorrt_llm.llmapi.TorchLlmArgs.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.TrtLlmArgs": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 11, 1, "", "auto_parallel"], [147, 13, 1, "", "auto_parallel_config"], [147, 11, 1, "", "auto_parallel_world_size"], [147, 15, 1, "", "backend"], [147, 15, 1, "", "batched_logits_processor"], [147, 15, 1, "", "batching_type"], [147, 15, 1, "", "build_config"], [147, 15, 1, "", "cache_transceiver_config"], [147, 15, 1, "", "calib_config"], [147, 15, 1, "", "context_parallel_size"], [147, 15, 1, "", "cp_config"], [147, 11, 1, "", "decoding_config"], [147, 15, 1, "", "dtype"], [147, 15, 1, "", "embedding_parallel_mode"], [147, 15, 1, "", "enable_attention_dp"], [147, 15, 1, "", "enable_build_cache"], [147, 15, 1, "", "enable_chunked_prefill"], [147, 15, 1, "", "enable_lm_head_tp_in_adp"], [147, 15, 1, "", "enable_lora"], [147, 15, 1, "", "enable_prompt_adapter"], [147, 15, 1, "", "enable_tqdm"], [147, 15, 1, "", "extended_runtime_perf_knob_config"], [147, 15, 1, "", "fail_fast_on_attention_window_too_large"], [147, 15, 1, "", "fast_build"], [147, 11, 1, "id21", "field_name"], [147, 12, 1, "", "from_kwargs"], [147, 15, 1, "", "gather_generation_logits"], [147, 12, 1, "", "get_runtime_sizes"], [147, 15, 1, "", "gpus_per_node"], [147, 15, 1, "", "guided_decoding_backend"], [147, 16, 1, "", "init_build_config"], [147, 16, 1, "", "init_calib_config"], [147, 15, 1, "", "iter_stats_max_iterations"], [147, 15, 1, "", "kv_cache_config"], [147, 15, 1, "", "load_format"], [147, 15, 1, "", "lora_config"], [147, 15, 1, "", "max_batch_size"], [147, 15, 1, "", "max_beam_width"], [147, 15, 1, "", "max_input_len"], [147, 15, 1, "", "max_num_tokens"], [147, 15, 1, "", "max_prompt_adapter_token"], [147, 15, 1, "", "max_seq_len"], [147, 15, 1, "", "model"], [147, 13, 1, "", "model_format"], [147, 15, 1, "", "moe_cluster_parallel_size"], [147, 15, 1, "", "moe_expert_parallel_size"], [147, 15, 1, "", "moe_tensor_parallel_size"], [147, 15, 1, "", "mpi_session"], [147, 11, 1, "id19", "msg"], [147, 15, 1, "", "normalize_log_probs"], [147, 15, 1, "", "num_postprocess_workers"], [147, 15, 1, "", "orchestrator_type"], [147, 13, 1, "", "parallel_config"], [147, 15, 1, "", "peft_cache_config"], [147, 15, 1, "", "pipeline_parallel_size"], [147, 15, 1, "", "postprocess_tokenizer_dir"], [147, 15, 1, "", "quant_config"], [147, 15, 1, "", "reasoning_parser"], [147, 15, 1, "", "request_stats_max_iterations"], [147, 15, 1, "", "return_perf_metrics"], [147, 15, 1, "", "revision"], [147, 15, 1, "", "scheduler_config"], [147, 16, 1, "", "set_default_max_input_len"], [147, 16, 1, "", "set_runtime_knobs_from_build_config"], [147, 16, 1, "", "setup_embedding_parallel_mode"], [147, 15, 1, "", "skip_tokenizer_init"], [147, 15, 1, "", "sparse_attention_config"], [147, 15, 1, "", "speculative_config"], [147, 13, 1, "", "speculative_model_dir"], [147, 13, 1, "", "speculative_model_format"], [147, 15, 1, "", "tensor_parallel_size"], [147, 12, 1, "", "to_dict"], [147, 15, 1, "", "tokenizer"], [147, 15, 1, "", "tokenizer_mode"], [147, 15, 1, "", "tokenizer_revision"], [147, 15, 1, "", "trust_remote_code"], [147, 16, 1, "", "validate_and_init_tokenizer"], [147, 16, 1, "", "validate_auto_parallel"], [147, 16, 1, "", "validate_build_config_remaining"], [147, 16, 1, "", "validate_build_config_with_runtime_params"], [147, 16, 1, "", "validate_dtype"], [147, 16, 1, "", "validate_enable_build_cache"], [147, 16, 1, "", "validate_gpus_per_node"], [147, 16, 1, "", "validate_kv_cache_dtype"], [147, 16, 1, "", "validate_lora_config_consistency"], [147, 16, 1, "", "validate_model"], [147, 16, 1, "", "validate_model_format_misc"], [147, 16, 1, "", "validate_parallel_config"], [147, 16, 1, "", "validate_peft_cache_config"], [147, 16, 1, "", "validate_quant_config"], [147, 16, 1, "", "validate_runtime_args"], [147, 16, 1, "", "validate_speculative_config"], [147, 15, 1, "", "workspace"], [147, 11, 1, "id20", "wrapped_property"]], "tensorrt_llm.llmapi.TrtLlmArgs.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.llmapi.UserProvidedDecodingConfig": [[147, 10, 1, "", "Config"], [147, 12, 1, "", "__init__"], [147, 15, 1, "", "acceptance_length_threshold"], [147, 15, 1, "", "acceptance_window"], [147, 12, 1, "", "construct"], [147, 12, 1, "", "copy"], [147, 11, 1, "", "decoding_type"], [147, 12, 1, "", "dict"], [147, 15, 1, "", "drafter"], [147, 12, 1, "", "from_dict"], [147, 12, 1, "", "from_orm"], [147, 12, 1, "", "json"], [147, 15, 1, "", "load_format"], [147, 15, 1, "", "max_concurrency"], [147, 15, 1, "", "max_draft_len"], [147, 11, 1, "", "model_computed_fields"], [147, 11, 1, "", "model_config"], [147, 12, 1, "", "model_construct"], [147, 12, 1, "", "model_copy"], [147, 12, 1, "", "model_dump"], [147, 12, 1, "", "model_dump_json"], [147, 13, 1, "", "model_extra"], [147, 11, 1, "", "model_fields"], [147, 13, 1, "", "model_fields_set"], [147, 12, 1, "", "model_json_schema"], [147, 12, 1, "", "model_parametrized_name"], [147, 12, 1, "", "model_post_init"], [147, 12, 1, "", "model_rebuild"], [147, 12, 1, "", "model_validate"], [147, 12, 1, "", "model_validate_json"], [147, 12, 1, "", "model_validate_strings"], [147, 12, 1, "", "parse_file"], [147, 12, 1, "", "parse_obj"], [147, 12, 1, "", "parse_raw"], [147, 15, 1, "", "resource_manager"], [147, 12, 1, "", "schema"], [147, 12, 1, "", "schema_json"], [147, 13, 1, "", "spec_dec_mode"], [147, 15, 1, "", "speculative_model_dir"], [147, 12, 1, "", "supports_backend"], [147, 12, 1, "", "update_forward_refs"], [147, 12, 1, "", "validate"]], "tensorrt_llm.llmapi.UserProvidedDecodingConfig.Config": [[147, 11, 1, "", "extra"]], "tensorrt_llm.models": [[135, 10, 1, "", "BaichuanForCausalLM"], [135, 10, 1, "", "BertForQuestionAnswering"], [135, 10, 1, "", "BertForSequenceClassification"], [135, 10, 1, "", "BertModel"], [135, 10, 1, "", "BloomForCausalLM"], [135, 10, 1, "", "BloomModel"], [135, 10, 1, "", "CLIPVisionTransformer"], [135, 10, 1, "", "ChatGLMConfig"], [135, 10, 1, "", "ChatGLMForCausalLM"], [135, 10, 1, "", "ChatGLMModel"], [135, 10, 1, "", "CogVLMConfig"], [135, 10, 1, "", "CogVLMForCausalLM"], [135, 10, 1, "", "CohereForCausalLM"], [135, 10, 1, "", "DbrxConfig"], [135, 10, 1, "", "DbrxForCausalLM"], [135, 10, 1, "", "DecoderModel"], [135, 10, 1, "", "DeepseekForCausalLM"], [135, 10, 1, "", "DeepseekV2ForCausalLM"], [135, 10, 1, "", "DiT"], [135, 10, 1, "", "EagleForCausalLM"], [135, 10, 1, "", "EncoderModel"], [135, 10, 1, "", "FalconConfig"], [135, 10, 1, "", "FalconForCausalLM"], [135, 10, 1, "", "FalconModel"], [135, 10, 1, "", "GPTConfig"], [135, 10, 1, "", "GPTForCausalLM"], [135, 10, 1, "", "GPTJConfig"], [135, 10, 1, "", "GPTJForCausalLM"], [135, 10, 1, "", "GPTJModel"], [135, 10, 1, "", "GPTModel"], [135, 10, 1, "", "GPTNeoXForCausalLM"], [135, 10, 1, "", "GPTNeoXModel"], [135, 10, 1, "", "GemmaConfig"], [135, 10, 1, "", "GemmaForCausalLM"], [135, 10, 1, "", "LLaMAConfig"], [135, 10, 1, "", "LLaMAForCausalLM"], [135, 10, 1, "", "LLaMAModel"], [135, 10, 1, "", "LlavaNextVisionConfig"], [135, 10, 1, "", "LlavaNextVisionWrapper"], [135, 10, 1, "", "MLLaMAForCausalLM"], [135, 10, 1, "", "MPTForCausalLM"], [135, 10, 1, "", "MPTModel"], [135, 10, 1, "", "MambaForCausalLM"], [135, 10, 1, "", "MedusaConfig"], [135, 10, 1, "", "MedusaForCausalLm"], [135, 10, 1, "", "OPTForCausalLM"], [135, 10, 1, "", "OPTModel"], [135, 10, 1, "", "Phi3ForCausalLM"], [135, 10, 1, "", "Phi3Model"], [135, 10, 1, "", "PhiForCausalLM"], [135, 10, 1, "", "PhiModel"], [135, 10, 1, "", "PretrainedConfig"], [135, 10, 1, "", "PretrainedModel"], [135, 10, 1, "", "ReDrafterForLLaMALM"], [135, 10, 1, "", "ReDrafterForQWenLM"], [135, 10, 1, "", "RecurrentGemmaForCausalLM"], [135, 11, 1, "", "RobertaForQuestionAnswering"], [135, 11, 1, "", "RobertaForSequenceClassification"], [135, 11, 1, "", "RobertaModel"], [135, 10, 1, "", "SD3Transformer2DModel"], [135, 10, 1, "", "SpeculativeDecodingMode"], [135, 10, 1, "", "WhisperEncoder"]], "tensorrt_llm.models.BaichuanForCausalLM": [[135, 11, 1, "", "config_class"], [135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "quantize"]], "tensorrt_llm.models.BertForQuestionAnswering": [[135, 12, 1, "", "forward"]], "tensorrt_llm.models.BertForSequenceClassification": [[135, 12, 1, "", "forward"]], "tensorrt_llm.models.BertModel": [[135, 12, 1, "", "forward"]], "tensorrt_llm.models.BloomModel": [[135, 12, 1, "", "forward"]], "tensorrt_llm.models.CLIPVisionTransformer": [[135, 12, 1, "", "forward"]], "tensorrt_llm.models.ChatGLMConfig": [[135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "to_dict"]], "tensorrt_llm.models.ChatGLMForCausalLM": [[135, 11, 1, "", "config_class"], [135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "prepare_inputs"], [135, 12, 1, "", "quantize"]], "tensorrt_llm.models.ChatGLMModel": [[135, 12, 1, "", "forward"]], "tensorrt_llm.models.CogVLMConfig": [[135, 12, 1, "", "to_dict"]], "tensorrt_llm.models.CogVLMForCausalLM": [[135, 11, 1, "", "config_class"], [135, 12, 1, "", "default_plugin_config"], [135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "quantize"]], "tensorrt_llm.models.CohereForCausalLM": [[135, 11, 1, "", "config_class"], [135, 12, 1, "", "from_hugging_face"]], "tensorrt_llm.models.DbrxConfig": [[135, 12, 1, "", "to_dict"]], "tensorrt_llm.models.DbrxForCausalLM": [[135, 11, 1, "", "config_class"]], "tensorrt_llm.models.DecoderModel": [[135, 12, 1, "", "check_config"], [135, 12, 1, "", "forward"], [135, 12, 1, "", "precompute_relative_attention_bias"], [135, 12, 1, "", "prepare_inputs"], [135, 12, 1, "", "use_lora"]], "tensorrt_llm.models.DeepseekForCausalLM": [[135, 11, 1, "", "config_class"], [135, 12, 1, "", "from_hugging_face"]], "tensorrt_llm.models.DeepseekV2ForCausalLM": [[135, 11, 1, "", "config_class"], [135, 12, 1, "", "from_hugging_face"]], "tensorrt_llm.models.DiT": [[135, 12, 1, "", "check_config"], [135, 12, 1, "", "forward"], [135, 12, 1, "", "forward_with_cfg"], [135, 12, 1, "", "forward_without_cfg"], [135, 12, 1, "", "prepare_inputs"], [135, 12, 1, "", "unpatchify"]], "tensorrt_llm.models.EagleForCausalLM": [[135, 11, 1, "", "config_class"], [135, 12, 1, "", "forward"], [135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "prepare_inputs"]], "tensorrt_llm.models.EncoderModel": [[135, 12, 1, "", "check_config"], [135, 12, 1, "", "forward"], [135, 12, 1, "", "precompute_relative_attention_bias"], [135, 12, 1, "", "prepare_inputs"], [135, 12, 1, "", "use_lora"], [135, 12, 1, "", "use_prompt_tuning"]], "tensorrt_llm.models.FalconConfig": [[135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "to_dict"]], "tensorrt_llm.models.FalconForCausalLM": [[135, 12, 1, "", "check_config"], [135, 11, 1, "", "config_class"], [135, 12, 1, "", "from_hugging_face"]], "tensorrt_llm.models.FalconModel": [[135, 12, 1, "", "forward"]], "tensorrt_llm.models.GPTConfig": [[135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "from_nemo"], [135, 12, 1, "", "to_dict"]], "tensorrt_llm.models.GPTForCausalLM": [[135, 11, 1, "", "config_class"], [135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "from_nemo"], [135, 12, 1, "", "quantize"], [135, 12, 1, "", "use_lora"]], "tensorrt_llm.models.GPTJConfig": [[135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "to_dict"]], "tensorrt_llm.models.GPTJForCausalLM": [[135, 11, 1, "", "config_class"], [135, 12, 1, "", "from_hugging_face"]], "tensorrt_llm.models.GPTJModel": [[135, 12, 1, "", "forward"]], "tensorrt_llm.models.GPTModel": [[135, 12, 1, "", "forward"]], "tensorrt_llm.models.GPTNeoXModel": [[135, 12, 1, "", "forward"]], "tensorrt_llm.models.GemmaConfig": [[135, 11, 1, "", "GEMMA2_ADDED_FIELDS"], [135, 11, 1, "", "GEMMA3_ADDED_FIELDS"], [135, 11, 1, "", "GEMMA_ADDED_FIELDS"], [135, 11, 1, "", "VERBATIM"], [135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "gemma2_config"], [135, 12, 1, "", "gemma3_config"], [135, 12, 1, "", "get_hf_config"], [135, 13, 1, "", "is_gemma_2"], [135, 13, 1, "", "is_gemma_3"], [135, 12, 1, "", "to_dict"]], "tensorrt_llm.models.GemmaForCausalLM": [[135, 11, 1, "", "NATIVE_QUANT_FLOW"], [135, 12, 1, "", "assert_valid_quant_algo"], [135, 11, 1, "", "config_class"], [135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "quantize"], [135, 12, 1, "", "use_lora"]], "tensorrt_llm.models.LLaMAConfig": [[135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "from_meta_ckpt"], [135, 12, 1, "", "to_dict"]], "tensorrt_llm.models.LLaMAForCausalLM": [[135, 11, 1, "", "config_class"], [135, 12, 1, "", "default_plugin_config"], [135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "from_meta_ckpt"], [135, 12, 1, "", "quantize"], [135, 12, 1, "", "use_lora"]], "tensorrt_llm.models.LLaMAModel": [[135, 12, 1, "", "forward"]], "tensorrt_llm.models.LlavaNextVisionConfig": [[135, 12, 1, "", "from_hugging_face"]], "tensorrt_llm.models.LlavaNextVisionWrapper": [[135, 12, 1, "", "forward"], [135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "prepare_inputs"], [135, 12, 1, "", "save_checkpoint"]], "tensorrt_llm.models.MLLaMAForCausalLM": [[135, 11, 1, "", "config_class"], [135, 12, 1, "", "forward"], [135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "prepare_inputs"], [135, 12, 1, "", "use_lora"]], "tensorrt_llm.models.MPTForCausalLM": [[135, 12, 1, "", "check_config"]], "tensorrt_llm.models.MPTModel": [[135, 12, 1, "", "forward"]], "tensorrt_llm.models.MambaForCausalLM": [[135, 11, 1, "", "config_class"], [135, 12, 1, "", "forward"], [135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "prepare_inputs"]], "tensorrt_llm.models.MedusaConfig": [[135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "to_dict"]], "tensorrt_llm.models.MedusaForCausalLm": [[135, 11, 1, "", "config_class"], [135, 12, 1, "", "from_hugging_face"]], "tensorrt_llm.models.OPTForCausalLM": [[135, 12, 1, "", "check_config"]], "tensorrt_llm.models.OPTModel": [[135, 12, 1, "", "forward"]], "tensorrt_llm.models.Phi3ForCausalLM": [[135, 11, 1, "", "config_class"], [135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "use_lora"]], "tensorrt_llm.models.Phi3Model": [[135, 12, 1, "", "forward"]], "tensorrt_llm.models.PhiForCausalLM": [[135, 12, 1, "", "check_config"], [135, 11, 1, "", "config_class"], [135, 12, 1, "", "from_hugging_face"], [135, 12, 1, "", "use_lora"]], "tensorrt_llm.models.PhiModel": [[135, 12, 1, "", "forward"]], "tensorrt_llm.models.PretrainedConfig": [[135, 12, 1, "", "create_runtime_defaults"], [135, 12, 1, "", "for_each_rank"], [135, 12, 1, "", "from_checkpoint"], [135, 12, 1, "", "from_dict"], [135, 12, 1, "", "from_json_file"], [135, 12, 1, "", "get_config_group"], [135, 12, 1, "", "has_config_group"], [135, 13, 1, "", "kv_dtype"], [135, 13, 1, "", "quant_algo"], [135, 13, 1, "", "quant_mode"], [135, 12, 1, "", "set_if_not_exist"], [135, 12, 1, "", "set_rank"], [135, 12, 1, "", "to_dict"], [135, 12, 1, "", "to_json_file"], [135, 12, 1, "", "to_layer_quant_config"]], "tensorrt_llm.models.PretrainedModel": [[135, 12, 1, "", "check_config"], [135, 12, 1, "", "from_checkpoint"], [135, 12, 1, "", "from_config"], [135, 12, 1, "", "load"], [135, 12, 1, "", "prepare_inputs"], [135, 12, 1, "", "quantize"], [135, 12, 1, "", "release"], [135, 12, 1, "", "save_checkpoint"]], "tensorrt_llm.models.RecurrentGemmaForCausalLM": [[135, 12, 1, "", "forward"], [135, 12, 1, "", "prepare_inputs"], [135, 12, 1, "", "prepare_recurrent_inputs"]], "tensorrt_llm.models.SD3Transformer2DModel": [[135, 13, 1, "", "attn_processors"], [135, 11, 1, "", "config_class"], [135, 12, 1, "", "disable_forward_chunking"], [135, 12, 1, "", "enable_forward_chunking"], [135, 12, 1, "", "forward"], [135, 12, 1, "", "from_pretrained"], [135, 12, 1, "", "fuse_qkv_projections"], [135, 12, 1, "", "load"], [135, 12, 1, "", "prepare_inputs"], [135, 12, 1, "", "set_attn_processor"], [135, 12, 1, "", "unfuse_qkv_projections"]], "tensorrt_llm.models.SpeculativeDecodingMode": [[135, 11, 1, "", "AUTO"], [135, 11, 1, "", "DRAFT_TOKENS_EXTERNAL"], [135, 11, 1, "", "EAGLE"], [135, 11, 1, "", "EXPLICIT_DRAFT_TOKENS"], [135, 11, 1, "", "LOOKAHEAD_DECODING"], [135, 11, 1, "", "MEDUSA"], [135, 11, 1, "", "NGRAM"], [135, 11, 1, "", "NONE"], [135, 11, 1, "", "SAVE_HIDDEN_STATES"], [135, 11, 1, "", "USER_PROVIDED"], [135, 12, 1, "", "from_arguments"]], "tensorrt_llm.models.WhisperEncoder": [[135, 12, 1, "", "forward"], [135, 12, 1, "", "precompute_relative_attention_bias"], [135, 12, 1, "", "prepare_inputs"]], "tensorrt_llm.plugin": [[136, 17, 1, "", "PluginConfig"]], "tensorrt_llm.plugin.PluginConfig": [[136, 15, 1, "", "bert_attention_plugin"], [136, 15, 1, "", "bert_context_fmha_fp32_acc"], [136, 15, 1, "", "context_fmha"], [136, 13, 1, "", "context_fmha_type"], [136, 16, 1, "", "convert_enable_disable"], [136, 15, 1, "", "dora_plugin"], [136, 15, 1, "", "dtype"], [136, 12, 1, "", "enable_paged_kv_cache"], [136, 15, 1, "", "fp8_rowwise_gemm_plugin"], [136, 12, 1, "", "from_arguments"], [136, 15, 1, "", "fuse_fp4_quant"], [136, 15, 1, "", "gemm_allreduce_plugin"], [136, 15, 1, "", "gemm_plugin"], [136, 15, 1, "", "gemm_swiglu_plugin"], [136, 15, 1, "", "gpt_attention_plugin"], [136, 15, 1, "", "identity_plugin"], [136, 12, 1, "", "is_context_fmha_enabled"], [136, 15, 1, "", "layernorm_quantization_plugin"], [136, 16, 1, "", "log_field_changes"], [136, 15, 1, "", "lora_plugin"], [136, 15, 1, "", "low_latency_gemm_plugin"], [136, 15, 1, "", "low_latency_gemm_swiglu_plugin"], [136, 15, 1, "", "mamba_conv1d_plugin"], [136, 15, 1, "", "manage_weights"], [136, 12, 1, "", "model_post_init"], [136, 15, 1, "", "moe_plugin"], [136, 15, 1, "", "multiple_profiles"], [136, 15, 1, "", "nccl_plugin"], [136, 15, 1, "", "norm_quant_fusion"], [136, 15, 1, "", "paged_kv_cache"], [136, 15, 1, "", "paged_state"], [136, 15, 1, "", "pp_reduce_scatter"], [136, 15, 1, "", "qserve_gemm_plugin"], [136, 15, 1, "", "quantize_per_token_plugin"], [136, 15, 1, "", "quantize_tensor_plugin"], [136, 15, 1, "", "reduce_fusion"], [136, 15, 1, "", "remove_input_padding"], [136, 15, 1, "", "rmsnorm_quantization_plugin"], [136, 12, 1, "", "set_context_fmha"], [136, 12, 1, "", "set_dora_plugin"], [136, 12, 1, "", "set_fp8_rowwise_quant_plugins"], [136, 12, 1, "", "set_lora_plugin"], [136, 12, 1, "", "set_nccl_plugin"], [136, 12, 1, "", "set_qserve_plugins"], [136, 12, 1, "", "set_smooth_quant_plugins"], [136, 15, 1, "", "smooth_quant_gemm_plugin"], [136, 15, 1, "", "smooth_quant_plugins"], [136, 15, 1, "", "streamingllm"], [136, 12, 1, "", "to_legacy_setting"], [136, 15, 1, "", "tokens_per_block"], [136, 15, 1, "", "use_fp8_context_fmha"], [136, 15, 1, "", "use_fused_mlp"], [136, 15, 1, "", "use_paged_context_fmha"], [136, 15, 1, "", "user_buffer"], [136, 12, 1, "", "validate"], [136, 16, 1, "", "validate_dtype_not_auto"], [136, 15, 1, "", "weight_only_groupwise_quant_matmul_plugin"], [136, 15, 1, "", "weight_only_quant_matmul_plugin"]], "tensorrt_llm.quantization": [[137, 10, 1, "", "QuantAlgo"], [137, 10, 1, "", "QuantMode"], [137, 14, 1, "", "quantize_and_export"]], "tensorrt_llm.runtime": [[138, 10, 1, "", "ChatGLMGenerationSession"], [138, 10, 1, "", "EncDecModelRunner"], [138, 10, 1, "", "GenerationSequence"], [138, 10, 1, "", "GenerationSession"], [138, 10, 1, "", "KVCacheManager"], [138, 10, 1, "", "LogitsProcessor"], [138, 10, 1, "", "LogitsProcessorList"], [138, 10, 1, "", "ModelConfig"], [138, 10, 1, "", "ModelRunner"], [138, 10, 1, "", "ModelRunnerCpp"], [138, 10, 1, "", "MultimodalModelRunner"], [138, 10, 1, "", "QWenForCausalLMGenerationSession"], [138, 10, 1, "", "SamplingConfig"], [138, 10, 1, "", "Session"], [138, 10, 1, "", "StoppingCriteria"], [138, 10, 1, "", "StoppingCriteriaList"], [138, 10, 1, "", "TensorInfo"], [138, 14, 1, "", "decode_words_list"]], "tensorrt_llm.runtime.EncDecModelRunner": [[138, 12, 1, "", "encoder_run"], [138, 12, 1, "", "from_engine"], [138, 12, 1, "", "generate"], [138, 12, 1, "", "process_input"]], "tensorrt_llm.runtime.GenerationSequence": [[138, 12, 1, "", "get_batch_idx"], [138, 12, 1, "", "get_seq_idx"]], "tensorrt_llm.runtime.GenerationSession": [[138, 11, 1, "", "batch_size"], [138, 11, 1, "", "buffer_allocated"], [138, 13, 1, "", "context_mem_size"], [138, 13, 1, "", "conv_kernel"], [138, 13, 1, "", "cross_attention"], [138, 11, 1, "", "cuda_graph_mode"], [138, 12, 1, "", "cuda_stream_guard"], [138, 11, 1, "", "debug_mode"], [138, 11, 1, "", "debug_tensors_to_save"], [138, 12, 1, "", "decode"], [138, 12, 1, "", "decode_batch"], [138, 12, 1, "", "decode_regular"], [138, 12, 1, "", "decode_stream"], [138, 11, 1, "", "device"], [138, 13, 1, "", "dtype"], [138, 12, 1, "", "dump_debug_buffers"], [138, 12, 1, "", "early_stop_criteria"], [138, 13, 1, "", "engine_inspector"], [138, 12, 1, "", "filter_medusa_logits"], [138, 12, 1, "", "finalize_decoder"], [138, 12, 1, "", "find_best_medusa_path"], [138, 13, 1, "", "first_layer"], [138, 13, 1, "", "gather_context_logits"], [138, 13, 1, "", "gather_generation_logits"], [138, 13, 1, "", "gemm_allreduce_plugin"], [138, 12, 1, "", "get_next_medusa_tokens"], [138, 12, 1, "", "get_num_heads_kv"], [138, 12, 1, "", "handle_per_step"], [138, 13, 1, "", "has_position_embedding"], [138, 13, 1, "", "has_token_type_embedding"], [138, 13, 1, "", "head_size"], [138, 13, 1, "", "hidden_size"], [138, 13, 1, "", "is_medusa_mode"], [138, 13, 1, "", "is_redrafter_mode"], [138, 13, 1, "", "kv_cache_type"], [138, 13, 1, "", "last_layer"], [138, 12, 1, "", "locate_accepted_draft_tokens"], [138, 11, 1, "", "mapping"], [138, 13, 1, "", "max_draft_tokens"], [138, 13, 1, "", "max_prompt_embedding_table_size"], [138, 12, 1, "", "medusa_decode_and_verify"], [138, 11, 1, "", "medusa_paths"], [138, 11, 1, "", "medusa_position_offsets"], [138, 11, 1, "", "medusa_temperature"], [138, 11, 1, "", "medusa_topks"], [138, 11, 1, "", "medusa_tree_ids"], [138, 12, 1, "", "next_medusa_input_ids"], [138, 11, 1, "", "num_draft_tokens"], [138, 13, 1, "", "num_heads"], [138, 13, 1, "", "num_layers"], [138, 13, 1, "", "num_medusa_heads"], [138, 13, 1, "", "paged_kv_cache"], [138, 13, 1, "", "paged_state"], [138, 12, 1, "", "pp_communicate_final_output_ids"], [138, 12, 1, "", "pp_communicate_new_tokens"], [138, 12, 1, "", "process_logits_including_draft"], [138, 13, 1, "", "profiler"], [138, 13, 1, "", "quant_mode"], [138, 13, 1, "", "remove_input_padding"], [138, 12, 1, "", "reorder_kv_cache_for_beam_search"], [138, 13, 1, "", "rnn_conv_dim_size"], [138, 13, 1, "", "rnn_head_size"], [138, 13, 1, "", "rnn_hidden_size"], [138, 11, 1, "", "runtime"], [138, 12, 1, "", "setup"], [138, 13, 1, "", "state_dtype"], [138, 13, 1, "", "state_size"], [138, 13, 1, "", "tokens_per_block"], [138, 12, 1, "", "update_output_ids_by_offset"], [138, 13, 1, "", "use_gemm_allreduce_plugin"], [138, 13, 1, "", "use_gpt_attention_plugin"], [138, 13, 1, "", "use_kv_cache"], [138, 13, 1, "", "use_lora_plugin"], [138, 13, 1, "", "use_mamba_conv1d_plugin"], [138, 13, 1, "", "vocab_size"]], "tensorrt_llm.runtime.KVCacheManager": [[138, 12, 1, "", "add_sequence"], [138, 12, 1, "", "get_block_offsets"], [138, 12, 1, "", "step"]], "tensorrt_llm.runtime.ModelConfig": [[138, 11, 1, "", "conv_kernel"], [138, 11, 1, "", "cross_attention"], [138, 11, 1, "", "dtype"], [138, 12, 1, "", "from_model_config_cpp"], [138, 11, 1, "", "gather_context_logits"], [138, 11, 1, "", "gather_generation_logits"], [138, 11, 1, "", "gemm_allreduce_plugin"], [138, 11, 1, "", "gpt_attention_plugin"], [138, 11, 1, "", "gpu_weights_percent"], [138, 11, 1, "", "has_position_embedding"], [138, 11, 1, "", "has_token_type_embedding"], [138, 11, 1, "", "head_size"], [138, 11, 1, "", "hidden_size"], [138, 11, 1, "", "kv_cache_type"], [138, 11, 1, "", "language_adapter_config"], [138, 11, 1, "", "layer_types"], [138, 11, 1, "", "lora_plugin"], [138, 11, 1, "", "lora_target_modules"], [138, 11, 1, "", "mamba_conv1d_plugin"], [138, 11, 1, "", "max_batch_size"], [138, 11, 1, "", "max_beam_width"], [138, 11, 1, "", "max_medusa_tokens"], [138, 11, 1, "", "max_prompt_embedding_table_size"], [138, 11, 1, "", "model_name"], [138, 11, 1, "", "num_heads"], [138, 11, 1, "", "num_kv_heads"], [138, 11, 1, "", "num_kv_heads_per_cross_attn_layer"], [138, 11, 1, "", "num_kv_heads_per_layer"], [138, 11, 1, "", "num_layers"], [138, 11, 1, "", "num_medusa_heads"], [138, 11, 1, "", "paged_state"], [138, 11, 1, "", "quant_mode"], [138, 11, 1, "", "redrafter_draft_len_per_beam"], [138, 11, 1, "", "redrafter_num_beams"], [138, 11, 1, "", "remove_input_padding"], [138, 11, 1, "", "rnn_conv_dim_size"], [138, 11, 1, "", "rnn_head_size"], [138, 11, 1, "", "rnn_hidden_size"], [138, 11, 1, "", "skip_cross_attn_blocks"], [138, 11, 1, "", "skip_cross_kv"], [138, 11, 1, "", "state_dtype"], [138, 11, 1, "", "state_size"], [138, 11, 1, "", "tokens_per_block"], [138, 11, 1, "", "trtllm_modules_to_hf_modules"], [138, 11, 1, "", "vocab_size"]], "tensorrt_llm.runtime.ModelRunner": [[138, 13, 1, "", "dtype"], [138, 12, 1, "", "from_dir"], [138, 12, 1, "", "from_engine"], [138, 13, 1, "", "gather_context_logits"], [138, 13, 1, "", "gather_generation_logits"], [138, 12, 1, "", "generate"], [138, 13, 1, "", "hidden_size"], [138, 13, 1, "", "mapping"], [138, 13, 1, "", "max_prompt_embedding_table_size"], [138, 13, 1, "", "max_sequence_length"], [138, 13, 1, "", "num_heads"], [138, 13, 1, "", "num_layers"], [138, 13, 1, "", "remove_input_padding"], [138, 12, 1, "", "serialize_engine"], [138, 13, 1, "", "use_lora_plugin"], [138, 13, 1, "", "vocab_size"], [138, 13, 1, "", "vocab_size_padded"]], "tensorrt_llm.runtime.ModelRunnerCpp": [[138, 13, 1, "", "dtype"], [138, 12, 1, "", "from_dir"], [138, 13, 1, "", "gather_context_logits"], [138, 13, 1, "", "gather_generation_logits"], [138, 12, 1, "", "generate"], [138, 13, 1, "", "hidden_size"], [138, 13, 1, "", "max_prompt_embedding_table_size"], [138, 13, 1, "", "max_sequence_length"], [138, 13, 1, "", "num_heads"], [138, 13, 1, "", "num_layers"], [138, 13, 1, "", "remove_input_padding"], [138, 13, 1, "", "vocab_size"], [138, 13, 1, "", "vocab_size_padded"]], "tensorrt_llm.runtime.MultimodalModelRunner": [[138, 13, 1, "", "audio_engine_dir"], [138, 13, 1, "", "cpp_e2e"], [138, 13, 1, "", "cpp_llm_only"], [138, 12, 1, "", "generate"], [138, 12, 1, "", "get_audio_features"], [138, 12, 1, "", "get_rope_index"], [138, 12, 1, "", "get_visual_features"], [138, 12, 1, "", "init_audio_encoder"], [138, 12, 1, "", "init_image_encoder"], [138, 12, 1, "", "init_llm"], [138, 12, 1, "", "init_processor"], [138, 12, 1, "", "init_tokenizer"], [138, 13, 1, "", "llm_engine_dir"], [138, 12, 1, "", "load_test_audio"], [138, 12, 1, "", "load_test_data"], [138, 12, 1, "", "prepare_position_ids_for_cogvlm"], [138, 12, 1, "", "preprocess"], [138, 12, 1, "", "ptuning_setup"], [138, 12, 1, "", "ptuning_setup_fuyu"], [138, 12, 1, "", "ptuning_setup_llava_next"], [138, 12, 1, "", "ptuning_setup_phi3"], [138, 12, 1, "", "ptuning_setup_pixtral"], [138, 13, 1, "", "python_e2e"], [138, 12, 1, "", "run"], [138, 12, 1, "", "setup_fake_prompts"], [138, 12, 1, "", "setup_fake_prompts_qwen2vl"], [138, 12, 1, "", "setup_fake_prompts_vila"], [138, 12, 1, "", "setup_inputs"], [138, 12, 1, "", "split_prompt_by_images"], [138, 12, 1, "", "tokenizer_image_token"], [138, 12, 1, "", "video_preprocess"], [138, 13, 1, "", "visual_engine_dir"]], "tensorrt_llm.runtime.QWenForCausalLMGenerationSession": [[138, 12, 1, "", "generate"]], "tensorrt_llm.runtime.SamplingConfig": [[138, 11, 1, "", "bad_words_list"], [138, 11, 1, "", "beam_search_diversity_rate"], [138, 11, 1, "", "early_stopping"], [138, 11, 1, "", "end_id"], [138, 11, 1, "", "frequency_penalty"], [138, 11, 1, "", "length_penalty"], [138, 11, 1, "", "max_attention_window_size"], [138, 11, 1, "", "max_new_tokens"], [138, 11, 1, "", "min_length"], [138, 11, 1, "", "min_p"], [138, 11, 1, "", "no_repeat_ngram_size"], [138, 11, 1, "", "num_beams"], [138, 11, 1, "", "num_return_sequences"], [138, 11, 1, "", "output_cum_log_probs"], [138, 11, 1, "", "output_log_probs"], [138, 11, 1, "", "output_sequence_lengths"], [138, 11, 1, "", "pad_id"], [138, 11, 1, "", "presence_penalty"], [138, 11, 1, "", "random_seed"], [138, 11, 1, "", "repetition_penalty"], [138, 11, 1, "", "return_dict"], [138, 11, 1, "", "sink_token_length"], [138, 11, 1, "", "stop_words_list"], [138, 11, 1, "", "temperature"], [138, 11, 1, "", "top_k"], [138, 11, 1, "", "top_p"], [138, 11, 1, "", "top_p_decay"], [138, 11, 1, "", "top_p_min"], [138, 11, 1, "", "top_p_reset_ids"], [138, 12, 1, "", "update"], [138, 11, 1, "", "use_beam_hyps"]], "tensorrt_llm.runtime.Session": [[138, 13, 1, "", "context"], [138, 13, 1, "", "context_mem_size"], [138, 13, 1, "", "engine"], [138, 12, 1, "", "from_engine"], [138, 12, 1, "", "from_serialized_engine"], [138, 12, 1, "", "infer_shapes"], [138, 12, 1, "", "run"], [138, 13, 1, "", "runtime"], [138, 12, 1, "", "set_shapes"]], "tensorrt_llm.runtime.TensorInfo": [[138, 11, 1, "", "dtype"], [138, 11, 1, "", "name"], [138, 12, 1, "", "numel"], [138, 11, 1, "", "shape"], [138, 12, 1, "", "squeeze"], [138, 12, 1, "", "view"]], "trtllm-bench": [[22, 18, 1, "cmdoption-trtllm-bench-log_level", "--log_level"], [22, 18, 1, "cmdoption-trtllm-bench-m", "--model"], [22, 18, 1, "cmdoption-trtllm-bench-model_path", "--model_path"], [22, 18, 1, "cmdoption-trtllm-bench-w", "--workspace"], [22, 18, 1, "cmdoption-trtllm-bench-m", "-m"], [22, 18, 1, "cmdoption-trtllm-bench-w", "-w"]], "trtllm-bench-build": [[22, 18, 1, "cmdoption-trtllm-bench-build-dataset", "--dataset"], [22, 18, 1, "cmdoption-trtllm-bench-build-max_batch_size", "--max_batch_size"], [22, 18, 1, "cmdoption-trtllm-bench-build-max_num_tokens", "--max_num_tokens"], [22, 18, 1, "cmdoption-trtllm-bench-build-max_seq_len", "--max_seq_len"], [22, 18, 1, "cmdoption-trtllm-bench-build-no_weights_loading", "--no_weights_loading"], [22, 18, 1, "cmdoption-trtllm-bench-build-pp", "--pp_size"], [22, 18, 1, "cmdoption-trtllm-bench-build-q", "--quantization"], [22, 18, 1, "cmdoption-trtllm-bench-build-target_input_len", "--target_input_len"], [22, 18, 1, "cmdoption-trtllm-bench-build-target_output_len", "--target_output_len"], [22, 18, 1, "cmdoption-trtllm-bench-build-tp", "--tp_size"], [22, 18, 1, "cmdoption-trtllm-bench-build-trust_remote_code", "--trust_remote_code"], [22, 18, 1, "cmdoption-trtllm-bench-build-pp", "-pp"], [22, 18, 1, "cmdoption-trtllm-bench-build-q", "-q"], [22, 18, 1, "cmdoption-trtllm-bench-build-tp", "-tp"]], "trtllm-bench-latency": [[22, 18, 1, "cmdoption-trtllm-bench-latency-backend", "--backend"], [22, 18, 1, "cmdoption-trtllm-bench-latency-beam_width", "--beam_width"], [22, 18, 1, "cmdoption-trtllm-bench-latency-concurrency", "--concurrency"], [22, 18, 1, "cmdoption-trtllm-bench-latency-dataset", "--dataset"], [22, 18, 1, "cmdoption-trtllm-bench-latency-engine_dir", "--engine_dir"], [22, 18, 1, "cmdoption-trtllm-bench-latency-ep", "--ep"], [22, 18, 1, "cmdoption-trtllm-bench-latency-extra_llm_api_options", "--extra_llm_api_options"], [22, 18, 1, "cmdoption-trtllm-bench-latency-iteration_log", "--iteration_log"], [22, 18, 1, "cmdoption-trtllm-bench-latency-kv_cache_free_gpu_mem_fraction", "--kv_cache_free_gpu_mem_fraction"], [22, 18, 1, "cmdoption-trtllm-bench-latency-max_input_len", "--max_input_len"], [22, 18, 1, "cmdoption-trtllm-bench-latency-max_seq_len", "--max_seq_len"], [22, 18, 1, "cmdoption-trtllm-bench-latency-medusa_choices", "--medusa_choices"], [22, 18, 1, "cmdoption-trtllm-bench-latency-modality", "--modality"], [22, 18, 1, "cmdoption-trtllm-bench-latency-num_requests", "--num_requests"], [22, 18, 1, "cmdoption-trtllm-bench-latency-pp", "--pp"], [22, 18, 1, "cmdoption-trtllm-bench-latency-report_json", "--report_json"], [22, 18, 1, "cmdoption-trtllm-bench-latency-sampler_options", "--sampler_options"], [22, 18, 1, "cmdoption-trtllm-bench-latency-tp", "--tp"], [22, 18, 1, "cmdoption-trtllm-bench-latency-warmup", "--warmup"]], "trtllm-bench-throughput": [[22, 18, 1, "cmdoption-trtllm-bench-throughput-backend", "--backend"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-beam_width", "--beam_width"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-cluster_size", "--cluster_size"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-concurrency", "--concurrency"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-custom_module_dirs", "--custom_module_dirs"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-data_device", "--data_device"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-dataset", "--dataset"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-enable_chunked_context", "--disable_chunked_context"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-enable_chunked_context", "--enable_chunked_context"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-engine_dir", "--engine_dir"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-eos_id", "--eos_id"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-ep", "--ep"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-extra_llm_api_options", "--extra_llm_api_options"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-image_data_format", "--image_data_format"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-iteration_log", "--iteration_log"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-kv_cache_free_gpu_mem_fraction", "--kv_cache_free_gpu_mem_fraction"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-max_batch_size", "--max_batch_size"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-max_input_len", "--max_input_len"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-max_num_tokens", "--max_num_tokens"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-max_seq_len", "--max_seq_len"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-modality", "--modality"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-no_skip_tokenizer_init", "--no_skip_tokenizer_init"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-num_requests", "--num_requests"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-output_json", "--output_json"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-pp", "--pp"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-report_json", "--report_json"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-request_json", "--request_json"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-sampler_options", "--sampler_options"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-scheduler_policy", "--scheduler_policy"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-streaming", "--streaming"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-target_input_len", "--target_input_len"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-target_output_len", "--target_output_len"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-tp", "--tp"], [22, 18, 1, "cmdoption-trtllm-bench-throughput-warmup", "--warmup"]], "trtllm-eval": [[24, 18, 1, "cmdoption-trtllm-eval-backend", "--backend"], [24, 18, 1, "cmdoption-trtllm-eval-disable_kv_cache_reuse", "--disable_kv_cache_reuse"], [24, 18, 1, "cmdoption-trtllm-eval-ep_size", "--ep_size"], [24, 18, 1, "cmdoption-trtllm-eval-extra_llm_api_options", "--extra_llm_api_options"], [24, 18, 1, "cmdoption-trtllm-eval-gpus_per_node", "--gpus_per_node"], [24, 18, 1, "cmdoption-trtllm-eval-kv_cache_free_gpu_memory_fraction", "--kv_cache_free_gpu_memory_fraction"], [24, 18, 1, "cmdoption-trtllm-eval-log_level", "--log_level"], [24, 18, 1, "cmdoption-trtllm-eval-max_batch_size", "--max_batch_size"], [24, 18, 1, "cmdoption-trtllm-eval-max_beam_width", "--max_beam_width"], [24, 18, 1, "cmdoption-trtllm-eval-max_num_tokens", "--max_num_tokens"], [24, 18, 1, "cmdoption-trtllm-eval-max_seq_len", "--max_seq_len"], [24, 18, 1, "cmdoption-trtllm-eval-model", "--model"], [24, 18, 1, "cmdoption-trtllm-eval-pp_size", "--pp_size"], [24, 18, 1, "cmdoption-trtllm-eval-tokenizer", "--tokenizer"], [24, 18, 1, "cmdoption-trtllm-eval-tp_size", "--tp_size"], [24, 18, 1, "cmdoption-trtllm-eval-trust_remote_code", "--trust_remote_code"]], "trtllm-eval-cnn_dailymail": [[24, 18, 1, "cmdoption-trtllm-eval-cnn_dailymail-apply_chat_template", "--apply_chat_template"], [24, 18, 1, "cmdoption-trtllm-eval-cnn_dailymail-dataset_path", "--dataset_path"], [24, 18, 1, "cmdoption-trtllm-eval-cnn_dailymail-max_input_length", "--max_input_length"], [24, 18, 1, "cmdoption-trtllm-eval-cnn_dailymail-max_output_length", "--max_output_length"], [24, 18, 1, "cmdoption-trtllm-eval-cnn_dailymail-num_samples", "--num_samples"], [24, 18, 1, "cmdoption-trtllm-eval-cnn_dailymail-random_seed", "--random_seed"], [24, 18, 1, "cmdoption-trtllm-eval-cnn_dailymail-rouge_path", "--rouge_path"], [24, 18, 1, "cmdoption-trtllm-eval-cnn_dailymail-system_prompt", "--system_prompt"]], "trtllm-eval-gpqa_diamond": [[24, 18, 1, "cmdoption-trtllm-eval-gpqa_diamond-apply_chat_template", "--apply_chat_template"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_diamond-chat_template_kwargs", "--chat_template_kwargs"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_diamond-dataset_path", "--dataset_path"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_diamond-max_input_length", "--max_input_length"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_diamond-max_output_length", "--max_output_length"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_diamond-num_samples", "--num_samples"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_diamond-random_seed", "--random_seed"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_diamond-system_prompt", "--system_prompt"]], "trtllm-eval-gpqa_extended": [[24, 18, 1, "cmdoption-trtllm-eval-gpqa_extended-apply_chat_template", "--apply_chat_template"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_extended-chat_template_kwargs", "--chat_template_kwargs"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_extended-dataset_path", "--dataset_path"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_extended-max_input_length", "--max_input_length"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_extended-max_output_length", "--max_output_length"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_extended-num_samples", "--num_samples"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_extended-random_seed", "--random_seed"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_extended-system_prompt", "--system_prompt"]], "trtllm-eval-gpqa_main": [[24, 18, 1, "cmdoption-trtllm-eval-gpqa_main-apply_chat_template", "--apply_chat_template"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_main-chat_template_kwargs", "--chat_template_kwargs"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_main-dataset_path", "--dataset_path"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_main-max_input_length", "--max_input_length"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_main-max_output_length", "--max_output_length"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_main-num_samples", "--num_samples"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_main-random_seed", "--random_seed"], [24, 18, 1, "cmdoption-trtllm-eval-gpqa_main-system_prompt", "--system_prompt"]], "trtllm-eval-gsm8k": [[24, 18, 1, "cmdoption-trtllm-eval-gsm8k-apply_chat_template", "--apply_chat_template"], [24, 18, 1, "cmdoption-trtllm-eval-gsm8k-chat_template_kwargs", "--chat_template_kwargs"], [24, 18, 1, "cmdoption-trtllm-eval-gsm8k-dataset_path", "--dataset_path"], [24, 18, 1, "cmdoption-trtllm-eval-gsm8k-fewshot_as_multiturn", "--fewshot_as_multiturn"], [24, 18, 1, "cmdoption-trtllm-eval-gsm8k-max_input_length", "--max_input_length"], [24, 18, 1, "cmdoption-trtllm-eval-gsm8k-max_output_length", "--max_output_length"], [24, 18, 1, "cmdoption-trtllm-eval-gsm8k-num_samples", "--num_samples"], [24, 18, 1, "cmdoption-trtllm-eval-gsm8k-random_seed", "--random_seed"], [24, 18, 1, "cmdoption-trtllm-eval-gsm8k-system_prompt", "--system_prompt"]], "trtllm-eval-json_mode_eval": [[24, 18, 1, "cmdoption-trtllm-eval-json_mode_eval-dataset_path", "--dataset_path"], [24, 18, 1, "cmdoption-trtllm-eval-json_mode_eval-max_input_length", "--max_input_length"], [24, 18, 1, "cmdoption-trtllm-eval-json_mode_eval-max_output_length", "--max_output_length"], [24, 18, 1, "cmdoption-trtllm-eval-json_mode_eval-num_samples", "--num_samples"], [24, 18, 1, "cmdoption-trtllm-eval-json_mode_eval-random_seed", "--random_seed"], [24, 18, 1, "cmdoption-trtllm-eval-json_mode_eval-system_prompt", "--system_prompt"]], "trtllm-eval-mmlu": [[24, 18, 1, "cmdoption-trtllm-eval-mmlu-accuracy_threshold", "--accuracy_threshold"], [24, 18, 1, "cmdoption-trtllm-eval-mmlu-apply_chat_template", "--apply_chat_template"], [24, 18, 1, "cmdoption-trtllm-eval-mmlu-chat_template_kwargs", "--chat_template_kwargs"], [24, 18, 1, "cmdoption-trtllm-eval-mmlu-check_accuracy", "--check_accuracy"], [24, 18, 1, "cmdoption-trtllm-eval-mmlu-dataset_path", "--dataset_path"], [24, 18, 1, "cmdoption-trtllm-eval-mmlu-max_input_length", "--max_input_length"], [24, 18, 1, "cmdoption-trtllm-eval-mmlu-max_output_length", "--max_output_length"], [24, 18, 1, "cmdoption-trtllm-eval-mmlu-num_fewshot", "--num_fewshot"], [24, 18, 1, "cmdoption-trtllm-eval-mmlu-num_samples", "--num_samples"], [24, 18, 1, "cmdoption-trtllm-eval-mmlu-random_seed", "--random_seed"], [24, 18, 1, "cmdoption-trtllm-eval-mmlu-system_prompt", "--system_prompt"]], "trtllm-eval-mmmu": [[24, 18, 1, "cmdoption-trtllm-eval-mmmu-chat_template_kwargs", "--chat_template_kwargs"], [24, 18, 1, "cmdoption-trtllm-eval-mmmu-dataset_path", "--dataset_path"], [24, 18, 1, "cmdoption-trtllm-eval-mmmu-max_input_length", "--max_input_length"], [24, 18, 1, "cmdoption-trtllm-eval-mmmu-max_output_length", "--max_output_length"], [24, 18, 1, "cmdoption-trtllm-eval-mmmu-num_samples", "--num_samples"], [24, 18, 1, "cmdoption-trtllm-eval-mmmu-random_seed", "--random_seed"], [24, 18, 1, "cmdoption-trtllm-eval-mmmu-system_prompt", "--system_prompt"]], "trtllm-serve-disaggregated": [[27, 18, 1, "cmdoption-trtllm-serve-disaggregated-c", "--config_file"], [27, 18, 1, "cmdoption-trtllm-serve-disaggregated-l", "--log_level"], [27, 18, 1, "cmdoption-trtllm-serve-disaggregated-m", "--metadata_server_config_file"], [27, 18, 1, "cmdoption-trtllm-serve-disaggregated-metrics-log-interval", "--metrics-log-interval"], [27, 18, 1, "cmdoption-trtllm-serve-disaggregated-r", "--request_timeout"], [27, 18, 1, "cmdoption-trtllm-serve-disaggregated-t", "--server_start_timeout"], [27, 18, 1, "cmdoption-trtllm-serve-disaggregated-c", "-c"], [27, 18, 1, "cmdoption-trtllm-serve-disaggregated-l", "-l"], [27, 18, 1, "cmdoption-trtllm-serve-disaggregated-m", "-m"], [27, 18, 1, "cmdoption-trtllm-serve-disaggregated-r", "-r"], [27, 18, 1, "cmdoption-trtllm-serve-disaggregated-t", "-t"]], "trtllm-serve-disaggregated_mpi_worker": [[27, 18, 1, "cmdoption-trtllm-serve-disaggregated_mpi_worker-c", "--config_file"], [27, 18, 1, "cmdoption-trtllm-serve-disaggregated_mpi_worker-log_level", "--log_level"], [27, 18, 1, "cmdoption-trtllm-serve-disaggregated_mpi_worker-c", "-c"]], "trtllm-serve-mm_embedding_serve": [[27, 18, 1, "cmdoption-trtllm-serve-mm_embedding_serve-extra_encoder_options", "--extra_encoder_options"], [27, 18, 1, "cmdoption-trtllm-serve-mm_embedding_serve-gpus_per_node", "--gpus_per_node"], [27, 18, 1, "cmdoption-trtllm-serve-mm_embedding_serve-host", "--host"], [27, 18, 1, "cmdoption-trtllm-serve-mm_embedding_serve-log_level", "--log_level"], [27, 18, 1, "cmdoption-trtllm-serve-mm_embedding_serve-max_batch_size", "--max_batch_size"], [27, 18, 1, "cmdoption-trtllm-serve-mm_embedding_serve-max_num_tokens", "--max_num_tokens"], [27, 18, 1, "cmdoption-trtllm-serve-mm_embedding_serve-metadata_server_config_file", "--metadata_server_config_file"], [27, 18, 1, "cmdoption-trtllm-serve-mm_embedding_serve-port", "--port"], [27, 18, 1, "cmdoption-trtllm-serve-mm_embedding_serve-trust_remote_code", "--trust_remote_code"], [27, 18, 1, "cmdoption-trtllm-serve-mm_embedding_serve-arg-MODEL", "MODEL"]], "trtllm-serve-serve": [[27, 18, 1, "cmdoption-trtllm-serve-serve-backend", "--backend"], [27, 18, 1, "cmdoption-trtllm-serve-serve-cluster_size", "--cluster_size"], [27, 18, 1, "cmdoption-trtllm-serve-serve-enable_chunked_prefill", "--enable_chunked_prefill"], [27, 18, 1, "cmdoption-trtllm-serve-serve-ep_size", "--ep_size"], [27, 18, 1, "cmdoption-trtllm-serve-serve-extra_llm_api_options", "--extra_llm_api_options"], [27, 18, 1, "cmdoption-trtllm-serve-serve-fail_fast_on_attention_window_too_large", "--fail_fast_on_attention_window_too_large"], [27, 18, 1, "cmdoption-trtllm-serve-serve-gpus_per_node", "--gpus_per_node"], [27, 18, 1, "cmdoption-trtllm-serve-serve-host", "--host"], [27, 18, 1, "cmdoption-trtllm-serve-serve-kv_cache_free_gpu_memory_fraction", "--kv_cache_free_gpu_memory_fraction"], [27, 18, 1, "cmdoption-trtllm-serve-serve-log_level", "--log_level"], [27, 18, 1, "cmdoption-trtllm-serve-serve-max_batch_size", "--max_batch_size"], [27, 18, 1, "cmdoption-trtllm-serve-serve-max_beam_width", "--max_beam_width"], [27, 18, 1, "cmdoption-trtllm-serve-serve-max_num_tokens", "--max_num_tokens"], [27, 18, 1, "cmdoption-trtllm-serve-serve-max_seq_len", "--max_seq_len"], [27, 18, 1, "cmdoption-trtllm-serve-serve-metadata_server_config_file", "--metadata_server_config_file"], [27, 18, 1, "cmdoption-trtllm-serve-serve-num_postprocess_workers", "--num_postprocess_workers"], [27, 18, 1, "cmdoption-trtllm-serve-serve-port", "--port"], [27, 18, 1, "cmdoption-trtllm-serve-serve-pp_size", "--pp_size"], [27, 18, 1, "cmdoption-trtllm-serve-serve-reasoning_parser", "--reasoning_parser"], [27, 18, 1, "cmdoption-trtllm-serve-serve-server_role", "--server_role"], [27, 18, 1, "cmdoption-trtllm-serve-serve-tokenizer", "--tokenizer"], [27, 18, 1, "cmdoption-trtllm-serve-serve-tp_size", "--tp_size"], [27, 18, 1, "cmdoption-trtllm-serve-serve-trust_remote_code", "--trust_remote_code"], [27, 18, 1, "cmdoption-trtllm-serve-serve-arg-MODEL", "MODEL"]]}, "objnames": {"0": ["c", "macro", "C macro"], "1": ["cpp", "type", "C++ type"], "2": ["cpp", "class", "C++ class"], "3": ["cpp", "function", "C++ function"], "4": ["cpp", "functionParam", "C++ function parameter"], "5": ["cpp", "member", "C++ member"], "6": ["cpp", "enum", "C++ enum"], "7": ["cpp", "enumerator", "C++ enumerator"], "8": ["cpp", "templateParam", "C++ template parameter"], "9": ["py", "module", "Python module"], "10": ["py", "class", "Python class"], "11": ["py", "attribute", "Python attribute"], "12": ["py", "method", "Python method"], "13": ["py", "property", "Python property"], "14": ["py", "function", "Python function"], "15": ["py", "pydantic_field", "Python field"], "16": ["py", "pydantic_validator", "Python validator"], "17": ["py", "pydantic_model", "Python model"], "18": ["std", "cmdoption", "program option"]}, "objtypes": {"0": "c:macro", "1": "cpp:type", "2": "cpp:class", "3": "cpp:function", "4": "cpp:functionParam", "5": "cpp:member", "6": "cpp:enum", "7": "cpp:enumerator", "8": "cpp:templateParam", "9": "py:module", "10": "py:class", "11": "py:attribute", "12": "py:method", "13": "py:property", "14": "py:function", "15": "py:pydantic_field", "16": "py:pydantic_validator", "17": "py:pydantic_model", "18": "std:cmdoption"}, "terms": {"": [0, 1, 2, 3, 5, 6, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 26, 27, 29, 30, 31, 32, 33, 34, 35, 37, 39, 49, 53, 54, 57, 59, 60, 64, 76, 78, 80, 81, 83, 84, 87, 89, 90, 91, 92, 93, 94, 96, 98, 101, 102, 103, 104, 106, 107, 108, 113, 115, 116, 117, 118, 119, 120, 124, 125, 127, 128, 129, 130, 131, 133, 134, 135, 136, 138, 139, 141, 143, 144, 146, 147, 148, 150, 151, 152, 153, 154, 155, 156, 158, 160, 161, 162, 164, 169], "0": [0, 1, 2, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 34, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 47, 48, 49, 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 62, 63, 64, 65, 66, 67, 68, 69, 71, 72, 73, 75, 76, 77, 78, 80, 81, 82, 84, 86, 88, 92, 95, 96, 98, 101, 102, 103, 105, 106, 107, 109, 110, 113, 114, 116, 117, 118, 119, 123, 124, 125, 130, 131, 132, 133, 134, 135, 138, 139, 142, 143, 144, 145, 147, 148, 149, 150, 151, 153, 156, 157, 158, 160, 161, 162, 163, 165, 168, 170], "00": [8, 13, 39, 61, 62, 63, 117, 124, 125, 143], "000": [2, 8, 11, 39, 97, 124, 150], "0000": [39, 124, 125, 147], "0012": 124, "00128": 27, "0017": 40, "003": 40, "0047": 143, "0058": 29, "0060": [29, 30], "0062": 29, "0063": 29, "0068": 31, "00688362121582": 27, "007": 40, "0070": 143, "0071": 143, "0075": 32, "007f": 147, "0080": 32, "0083": 32, "0086": 32, "0087": 31, "0096": 143, "00978": 141, "01": [7, 13, 61, 62, 63, 68, 124, 139, 152], "0101": 31, "0105": 2, "014": 5, "0158": 125, "0162": 127, "0165": 130, "02": 152, "020": 40, "0235": 143, "024": 24, "026": 40, "0260": 143, "027": 40, "0273": 143, "028": 40, "0294": 143, "03": [26, 130, 143, 152], "032": 13, "0339": 40, "035": 40, "03762": 133, "03961": 104, "03x": 14, "04": [90, 101, 145, 152, 166], "0449": 143, "04532": 136, "045471": 16, "046": 40, "0461": 2, "0463": 40, "048": 40, "049": 40, "05": [133, 134, 135, 143, 152], "051": 40, "05100": 133, "0523": 143, "0528": [12, 29, 40], "0554": 125, "0560": 143, "0563": 40, "057": 40, "06": [13, 19, 124, 133, 134, 152], "061": 40, "0630": 143, "0669": 2, "0675": 2, "0682": 143, "0689e": 124, "07": [7, 13, 152], "0704": 125, "0713": 143, "0723": 143, "0732": 143, "074": 40, "0772": 2, "0776": 143, "078": 40, "079": 19, "08": [13, 26, 130, 142], "0804": 143, "081947": 16, "082": 40, "0838": [2, 40], "088": 40, "0881": 131, "09": [13, 143], "0903": 143, "0910": 143, "092": 19, "092314": 16, "092623": 16, "093256": 16, "09353": 110, "094": 24, "096": 24, "0964": 40, "09685": 110, "09f": [0, 1], "0cf2f6f154b4a5765d89945b20aa3449b2be7933": 20, "0e": 106, "0f": [0, 106], "0rc0": 97, "0rc1": [21, 26, 39, 99, 124, 151], "0rc6": [29, 30, 31, 32], "0u": 1, "0x": 4, "0x0000000000000000": 152, "1": [0, 1, 3, 4, 5, 6, 7, 10, 11, 12, 13, 14, 15, 17, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 35, 36, 37, 38, 39, 41, 43, 44, 45, 47, 48, 49, 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 64, 65, 66, 67, 68, 69, 71, 72, 73, 75, 76, 77, 78, 81, 82, 84, 86, 87, 88, 90, 91, 93, 95, 96, 97, 99, 101, 102, 103, 105, 106, 107, 109, 110, 113, 114, 116, 118, 119, 123, 124, 125, 126, 127, 130, 132, 133, 134, 135, 137, 138, 140, 142, 143, 144, 147, 149, 150, 151, 156, 157, 158, 160, 162, 163, 165, 166, 167, 168, 169], "10": [0, 2, 7, 8, 9, 10, 13, 14, 16, 17, 19, 20, 21, 29, 39, 40, 44, 47, 60, 67, 68, 78, 86, 88, 97, 109, 110, 113, 124, 125, 127, 131, 133, 139, 142, 143, 147, 158, 165], "100": [0, 2, 8, 10, 16, 20, 24, 26, 38, 39, 47, 51, 62, 65, 86, 109, 112, 123, 124, 125], "1000": [0, 18, 38, 39, 40, 123, 124, 125], "10000": [133, 134, 135], "1003": 152, "100gb": 15, "100m": 97, "101": [20, 109], "101029": 16, "101253": 27, "101256": 27, "101978": 40, "102": [4, 20, 109], "1024": [1, 2, 5, 7, 11, 16, 18, 21, 23, 26, 29, 30, 31, 32, 33, 40, 44, 59, 65, 78, 88, 106, 116, 124, 125, 130, 133, 134, 143, 144, 147, 158, 165], "102415": [39, 124], "103": [16, 20, 109], "1039": 33, "104": [20, 152], "1041": 21, "10438": 141, "1045": 143, "1047": [39, 124], "105": [20, 40], "1050": 143, "1051": 125, "1059": [39, 124], "106": [20, 40], "106563": 40, "107": [20, 40], "1072": 143, "107501": 40, "10774": 0, "1079": 119, "107u": 12, "108": [20, 40], "1082": 143, "10858": 44, "109": [16, 20, 40], "10b": [133, 152], "10m": 4, "11": [0, 2, 5, 7, 8, 16, 17, 19, 20, 40, 91, 110, 113, 124, 127, 133, 143], "110": 20, "11023": [39, 124], "110804": 40, "110b": 152, "111": [4, 13, 20], "111302": 40, "111618": 40, "111668": 40, "1118": 152, "112": [20, 40], "1123": 152, "113": 20, "1134": 139, "113420": 16, "1135": 143, "114": [16, 20, 40], "1141": 143, "114688": 2, "1148": 152, "11489": 2, "11490": 124, "115": [16, 20], "1151": 2, "115378": 16, "115716": 40, "116": [20, 40], "1160": [27, 45], "117": [20, 40], "1178": [39, 124], "118": 20, "1181": 152, "1183": 152, "119": [16, 20, 39, 124], "11943": [39, 124], "11947": 44, "1196": 2, "119648": 16, "11b": [142, 149, 152], "11x": 17, "12": [0, 4, 8, 13, 16, 17, 18, 19, 20, 26, 40, 44, 78, 90, 91, 97, 98, 101, 110, 116, 124, 127, 130, 133, 143, 158, 166], "120": [12, 16, 20], "120b": [30, 97], "121": 20, "1212": 143, "121847": 124, "1219": 2, "122": [20, 124], "1225": 133, "12288": [39, 124], "123": [20, 47, 48], "1234": [135, 147], "1239": 152, "124": 20, "1242": 152, "1245": 26, "1248": 152, "125": [16, 20, 124], "1252": [39, 119, 124], "1256": 152, "1257": 2, "125m": [113, 116], "126": [20, 39, 124], "1266": 147, "1267": 152, "127": [20, 133], "1272": 143, "128": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 12, 13, 16, 20, 26, 29, 30, 31, 32, 33, 39, 40, 44, 47, 48, 62, 67, 75, 76, 105, 109, 110, 114, 117, 124, 135, 147, 152, 156, 160], "1284": 152, "1287": 127, "128798": 147, "128799": 147, "128e": [18, 40, 82, 163], "128k": [10, 82, 163], "129": [16, 20, 40], "1290": 143, "1291504": 125, "1293": 119, "12945": 2, "129498": 2, "13": [6, 15, 16, 17, 19, 20, 40, 98, 101, 105, 110, 124, 125, 133, 142, 143], "130": [20, 40], "1300": 53, "131": [20, 40], "131072": [9, 124, 125], "13195": [39, 124], "132": [20, 124], "1323": 152, "1328": 152, "1329": 152, "133": [20, 152], "133120": 30, "13368": 124, "1337": 152, "134": [20, 40], "1341": 2, "1343": 152, "1344": 152, "135": [20, 40], "13525": 124, "13598": [39, 124], "136": [20, 40], "137": [20, 39, 40, 124], "1378": 143, "138": [16, 20], "139": [20, 40], "1392": 152, "13b": [4, 82, 83, 97, 163, 164], "14": [7, 16, 17, 19, 20, 26, 39, 40, 110, 116, 124, 127, 130, 131, 143], "140": [8, 20], "140g": 119, "141": [5, 20], "1418": [39, 124], "141gb": [3, 40], "142": [15, 16, 20, 40], "1424": 152, "143": 20, "1436": [2, 152], "1437": 143, "144": 127, "1446": 152, "1447": 152, "14480": [39, 124], "1449": 152, "145": [130, 131], "1459": 143, "146": [40, 130, 131], "1467": 152, "147": [125, 127, 130, 131], "14702": 10, "148": 40, "1480": 152, "1486": 152, "149": [143, 152], "15": [13, 16, 17, 18, 19, 20, 26, 33, 40, 110, 124, 131, 133, 143], "150": [20, 38, 78, 123, 158], "1500": 40, "15043": 44, "151": 40, "1514": 152, "152": [16, 39, 40, 124], "1529": 152, "153": 26, "1534": 152, "1535": 152, "1536": 2, "1537": 152, "1539": 152, "154": [13, 26, 27], "1552": 152, "1556": 143, "15585": [39, 124], "1562": 152, "1564": [125, 130, 131], "157": 40, "158": 2, "1583": 152, "1584": 2, "1585": 125, "1589": 152, "1590": 152, "1597": 127, "15b": [82, 163], "15u": 15, "16": [0, 2, 4, 7, 8, 11, 12, 13, 16, 17, 20, 26, 27, 29, 30, 31, 32, 33, 39, 40, 41, 43, 59, 61, 62, 63, 75, 76, 78, 84, 88, 98, 105, 110, 111, 113, 117, 124, 125, 126, 133, 134, 135, 139, 141, 143, 156, 158, 160, 165], "160": [16, 152], "1607": [39, 124], "161": [27, 39, 45, 124], "162": 16, "1625": 127, "1626": 152, "163": 3, "163061": 10, "163062": 10, "1637": 152, "16384": [29, 30, 31, 32, 127, 130], "164": [13, 16], "1642": 152, "1643": 26, "165": 40, "1650": 152, "1651165696": 20, "166": 40, "1660": 152, "1664": 30, "1669": 152, "167": [39, 124], "1671517696": 20, "1672": 143, "1674": 152, "1675": 152, "167507": 16, "1676": 152, "168": 13, "16e": [19, 32, 82, 142, 149, 163], "16x": [14, 139], "17": [0, 2, 16, 18, 19, 20, 21, 29, 30, 31, 32, 39, 40, 110, 124, 130, 143, 145], "170": 40, "1706": 133, "171": 16, "1721": 143, "1723": 152, "172321": 2, "17233": 2, "173": [13, 40], "1732": 152, "17323": 141, "1738": 152, "1741966075": 151, "1742": 152, "17453": [23, 136], "17453v3": 1, "1748018634": 18, "175": 40, "1753843963": [31, 32], "1754294810": 29, "1754358426": 21, "1755815898": 30, "1759022940": 33, "175b": 5, "176": 124, "1762": 152, "1774995776": 20, "1776": [82, 163], "1799": 152, "17b": [18, 19, 28, 40, 82, 97, 142, 149, 163], "17x": 20, "18": [15, 16, 19, 20, 40, 110, 124, 143], "180": [11, 13, 139], "1806": 2, "180b": [7, 39, 124], "180gb": 40, "181": 40, "1815": 152, "181540": 2, "182": 40, "1822": 44, "1834": 152, "185": [4, 39, 124], "1851": 152, "18527": 44, "18563": [39, 124], "1861": 131, "1866": 131, "187": 16, "188415": 20, "188416": 20, "1885": 125, "1886": 152, "1897": 152, "19": [2, 16, 19, 20, 21, 40, 131, 143], "190": 40, "1909": 152, "192": [3, 12, 16], "1926": 152, "192gb": 40, "1937": 152, "1939": 152, "194": 40, "1944": 130, "195": 40, "1950": 16, "1953": 152, "1959": [39, 124], "1963": 16, "198": [13, 19], "1985": 152, "1987": 152, "199": 40, "1993": [58, 143], "1999": 152, "1_1b": [82, 163], "1_405b": 117, "1_70b": [26, 117], "1_output": 26, "1b": [17, 27, 37, 41, 43, 47, 49, 50, 51, 53, 54, 55, 56, 57, 60, 64, 65, 66, 69, 71, 73, 77, 78, 81, 82, 84, 101, 142, 144, 146, 149, 151, 152, 157, 158, 162, 163], "1d": [75, 105, 133, 138], "1e": [116, 133, 134, 135], "1e20f": 1, "1g": 143, "1gb": [84, 88, 102, 165], "1k": [2, 12, 13, 14, 15, 20, 21, 29], "1k2k": 21, "1m": 131, "1st": [4, 26, 29, 30, 31, 32, 133, 139], "1u": [0, 1], "1x": [13, 21], "1xgpu": 21, "1xh200": 3, "1xtep": 17, "1ytic": 152, "2": [0, 1, 3, 4, 5, 7, 10, 11, 12, 13, 14, 15, 17, 21, 24, 26, 27, 30, 31, 32, 33, 35, 39, 40, 56, 57, 59, 60, 61, 62, 63, 64, 65, 75, 76, 77, 78, 80, 82, 84, 87, 88, 91, 93, 94, 97, 99, 101, 102, 103, 105, 106, 107, 108, 109, 110, 111, 113, 114, 116, 117, 119, 124, 126, 127, 130, 131, 133, 135, 138, 141, 142, 143, 147, 149, 151, 156, 157, 158, 160, 161, 163, 165, 169], "20": [1, 10, 11, 15, 16, 17, 19, 20, 26, 29, 30, 31, 32, 33, 40, 66, 69, 71, 72, 88, 106, 113, 114, 124, 125, 130, 133, 138, 143, 147, 165], "200": [5, 9, 16, 18, 21, 29, 30, 31, 32, 33, 60, 78, 138, 147, 158], "2000": [15, 40], "20000": [34, 40, 147], "200mb": 15, "2017": 130, "2018": 152, "202": 16, "2023": [3, 143], "2024": [13, 90, 166], "2025": [2, 11, 13, 124], "2028": 152, "2033": 131, "2039": 152, "204": [13, 40], "2040": 152, "2042": 2, "2044": [130, 131], "2045": 130, "2048": [2, 3, 5, 6, 15, 22, 23, 26, 29, 30, 31, 32, 39, 40, 67, 78, 84, 116, 124, 125, 127, 128, 129, 130, 131, 135, 138, 143, 147, 152, 158], "205": [16, 40], "2056": 152, "206": 40, "20627": 44, "20685": [39, 124], "2079": 143, "208": 40, "2081": [127, 130, 152], "2087": 152, "2089": 40, "209": [16, 40], "20b": 152, "20k": 21, "21": [2, 7, 13, 16, 19, 20, 40, 113, 130, 143], "2101": 104, "2102": 40, "2106": 110, "2107": 143, "210g": 119, "211": 13, "2113": 152, "212": [16, 40], "2135": 152, "214": 19, "215": 30, "2152": 152, "2158": 40, "2168": 2, "2169": 152, "21747": [39, 124], "2176": 40, "21764": [39, 124], "2182": 152, "2191": 152, "22": [15, 16, 20, 29, 31, 32, 37, 40, 121, 133, 143], "220": 40, "22000": 40, "22056": [39, 124], "221": [40, 124], "2210": 141, "2211": [133, 141], "2219": 152, "222": 40, "22213": [39, 124], "2225": 143, "2232": 152, "224": 134, "2243": 152, "225": [30, 40], "2263": 152, "227": 6, "2288": 152, "2294": 152, "22b": [82, 163], "22x": 14, "23": [8, 16, 19, 20, 39, 40, 124, 143, 152], "2305": 143, "2306": 141, "2309": [1, 23, 136], "232": [6, 16], "234": 16, "2352": 152, "2357": 152, "235b": [82, 163], "236": 13, "2366": 152, "2370": 152, "2373": 152, "2379": 152, "238": 40, "2388": 152, "239": 13, "2397": [39, 124], "24": [0, 16, 20, 39, 40, 101, 124, 143, 145, 152], "2401": 0, "2402": 110, "2405": 136, "24189": 40, "2419": 152, "2425": 152, "243": 16, "2439": 152, "244": 40, "245": 13, "2458": 152, "246": 16, "2461": 130, "2466": 130, "2473": 152, "2474": [127, 130], "2484": 152, "2485": 152, "2487": 40, "249": 13, "24b": [142, 152], "24mib": 16, "25": [6, 8, 13, 16, 17, 20, 39, 40, 124, 142, 152], "250": [2, 13, 16], "2500": 40, "2503": 142, "25032": [39, 124], "251": [16, 40], "252u": 15, "253": [13, 40], "253b": [82, 163], "2552": 152, "256": [1, 2, 3, 6, 12, 15, 16, 20, 24, 26, 29, 30, 31, 32, 33, 40, 59, 67, 76, 86, 124, 133, 135, 143, 144, 147, 152, 156, 160], "25603": [39, 124], "257": 40, "2573": 152, "2581": [127, 130], "2590780": 124, "259840": 139, "26": [16, 20, 40, 124, 127, 151], "2602": 44, "2628": [130, 131], "263": [3, 44], "2640": 131, "2649": 143, "266": 40, "2671": 2, "2677": 152, "26778": [39, 124], "2679": 127, "2685": 152, "2691": 152, "27": [20, 40, 152], "270m": [82, 163], "271": 40, "2712": 152, "274": [2, 40, 152], "2742": 125, "275": 152, "2755": 2, "2766684": 2, "278": 44, "2782": 152, "2787": 152, "2796": 152, "27b": [82, 142, 163], "27th": 33, "28": [13, 20, 40, 124, 143], "2820": 143, "283": 40, "28390": 124, "287113": 124, "288": [16, 29, 152], "29": [20, 139, 152], "290": 16, "2939": 143, "295": 40, "296": 40, "297": 44, "29892": 44, "299": [13, 39, 124], "29962": 44, "2998": 143, "2b": [39, 82, 119, 124, 163], "2cta": 15, "2d": [113, 133, 134, 141], "2gib": 152, "2k": [2, 13, 14, 15, 21], "2m": 131, "2nd": 133, "2u": 1, "2x": [4, 5, 10, 17], "2xdep": 17, "3": [0, 1, 3, 4, 5, 7, 8, 9, 10, 11, 13, 14, 15, 19, 21, 24, 26, 27, 28, 29, 32, 36, 39, 49, 54, 56, 59, 64, 66, 67, 68, 75, 76, 78, 82, 84, 85, 87, 91, 93, 95, 97, 101, 103, 105, 107, 109, 110, 118, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 138, 140, 143, 144, 146, 147, 148, 149, 151, 152, 153, 156, 158, 160, 163, 167, 168], "30": [0, 2, 10, 13, 17, 18, 19, 20, 113, 125, 127, 131, 133, 139, 147, 150], "300": [6, 16, 39, 124], "3000": [19, 39, 40, 124], "30000": 40, "30065": [39, 124], "300k": 19, "3019": [39, 124], "3021": 2, "3022": [39, 124], "303": 5, "3031": 130, "304": 44, "3040": [125, 130, 131], "305": 40, "306": 44, "306u": 12, "307": [26, 40], "3072": 2, "309": 40, "3095": 152, "30990": [39, 124], "30b": [7, 82, 142, 149, 163], "30x": 7, "31": [8, 20, 40, 111, 125, 130, 131, 152], "3106": 152, "311": 40, "312": 8, "3121": 27, "3126": 27, "3132": [39, 124], "315": 13, "3159": 11, "316": 40, "319": 40, "31st": 12, "32": [1, 2, 4, 5, 11, 12, 16, 17, 19, 20, 23, 24, 26, 27, 29, 30, 31, 32, 33, 40, 44, 50, 51, 58, 67, 75, 76, 78, 105, 109, 111, 124, 125, 133, 134, 135, 136, 138, 139, 141, 143, 147, 151, 152, 156, 158, 160], "3200": 29, "3201": 125, "321": [40, 124], "322": 44, "323": 40, "325": 40, "326": 40, "3276": [125, 130, 131], "32768": [30, 133], "329": 40, "3291": 143, "32b": [82, 142, 149, 152, 163], "32k": [10, 152], "32x": 7, "33": [8, 10, 19, 20, 40, 143, 152], "330": 40, "331": 40, "332": 40, "3328": 143, "332826": 2, "3338": 125, "336": 40, "338": [13, 44], "3389": 127, "339": 40, "339447": 16, "339981": 16, "33x": 14, "34": [2, 8, 16, 20, 40], "340": [11, 13, 40], "341": [5, 16], "342": 40, "3442": 143, "3445": 143, "345": 40, "3452": [39, 124, 143], "3476": 2, "348": 40, "348gib": 16, "349": [5, 40], "34b": [82, 152, 163], "35": [0, 8, 20, 40, 51, 86], "350": 30, "3504": 16, "351": 40, "352": 40, "3555": 143, "356": 40, "36": [13, 16, 19, 20, 126, 127], "360": 30, "36384": 2, "3671": [39, 124], "367714": 16, "368": 13, "369": 40, "37": [8, 16, 19, 20, 40, 124], "370318": 20, "3763": 13, "3764": 152, "378": 40, "38": [8, 20, 39, 124], "381": 16, "384": [2, 40], "3863": 40, "387b12598a9e": 124, "3887": 143, "389": 40, "39": [8, 13, 20, 40], "391": 40, "3914": 40, "393": 40, "3936": [39, 124], "3977": 143, "399": 40, "3_1": [82, 142, 149, 163], "3_3": [82, 142, 149, 163], "3_output": 31, "3b": [42, 48, 70, 82, 142, 163], "3d": [75, 105, 133, 138], "3rd": 133, "3u": 1, "3x": [7, 13, 15], "4": [0, 1, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 19, 21, 24, 26, 27, 30, 31, 32, 33, 39, 44, 51, 59, 61, 62, 63, 65, 76, 78, 82, 84, 88, 90, 91, 92, 93, 95, 96, 97, 107, 109, 110, 111, 113, 117, 119, 124, 125, 127, 128, 129, 130, 131, 132, 133, 135, 136, 138, 139, 140, 141, 142, 143, 147, 149, 150, 152, 156, 158, 160, 163, 165, 166], "40": [19, 20, 30, 33, 40, 66, 106, 127, 133, 150, 152], "400": [15, 16], "4000": [15, 19], "401": 40, "403": 152, "405b": [39, 82, 124, 126, 152, 163], "4060": 139, "4066": 44, "407": 40, "408348": 16, "4089": 131, "4096": [3, 11, 15, 26, 29, 30, 33, 39, 40, 44, 86, 124, 127, 133, 134, 138], "40b": 7, "40gb": 23, "40x": 7, "41": 20, "410": 40, "4101": 152, "41020": 124, "411": [39, 124], "4117e": [39, 124], "4133": 131, "4135": 152, "41375": 124, "414": 2, "4141": 152, "41607": 124, "4168": 2, "4192": 143, "42": [16, 20, 30, 33, 40, 124], "420": [21, 22], "421": 40, "422": 40, "4224": 40, "4227": 152, "4248": 127, "4265": 124, "427": [39, 124], "4280": 13, "43": [19, 20, 40, 139, 151], "431": 40, "43146": 2, "434": 40, "435": 40, "437": 40, "4384": 16, "44": [10, 16, 19, 20, 26, 40, 139], "4408": 44, "442": 40, "4439": [39, 124], "4456": 40, "447": 40, "449": 152, "4493": [130, 131], "4495": 16, "4497": 40, "44x": 7, "45": [16, 20, 33, 109, 142, 152], "450": 40, "45000000000": 109, "450m": [82, 163], "453": 40, "4532": 152, "4548": 2, "456": 40, "4566": 40, "457": 40, "458676": 16, "459": 40, "46": [7, 20], "4600": 15, "461014": 16, "463": 40, "464": [16, 40], "465004": 20, "4653": 44, "4656": 40, "4667": 40, "4678": 152, "47": [7, 16, 20, 127], "4701": [39, 124], "472": 44, "476": 40, "4767": 152, "478": 152, "4798": 152, "47x": 7, "48": [16, 20, 127, 139, 152], "4809": 152, "480gb": [16, 20], "481": 4, "482": 152, "4853": 152, "489795": 20, "489935": 20, "49": [16, 20, 127], "491": [16, 40], "49152": 2, "494": 40, "495": 40, "496": [40, 111], "4963": [39, 124], "4963654": 35, "498043": 16, "499": [8, 40], "4993": 152, "49b": [82, 142, 149, 163], "4b": [82, 152, 163], "4bit": 3, "4gb": 15, "4gpu": 21, "4k": [20, 82, 163], "4u": 1, "4x": [3, 4, 5, 17, 21, 97], "4xgpu": 21, "5": [0, 1, 3, 4, 5, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 26, 29, 30, 31, 32, 33, 39, 40, 42, 48, 53, 65, 66, 70, 82, 84, 91, 93, 95, 109, 110, 113, 114, 116, 124, 130, 133, 135, 138, 140, 142, 143, 147, 149, 152, 163, 168], "50": [0, 7, 8, 10, 11, 16, 17, 20, 30, 39, 53, 65, 66, 78, 88, 124, 147, 152, 158, 165], "500": [13, 15, 30, 40], "5000": 40, "500000": 135, "5007": 44, "500m": 7, "50272": 116, "5029": 152, "505": 40, "5064": 40, "5073": 143, "50m": 16, "51": 20, "511": 40, "512": [1, 2, 5, 6, 20, 22, 24, 26, 29, 30, 40, 65, 110, 114, 124, 127, 130, 135, 147], "5120": 2, "512mb": [84, 102], "513": 40, "5141": 20, "518": 44, "51b": [82, 142, 149, 152, 163], "51x": 7, "52": [16, 20, 40], "5213": 20, "5215": 20, "5224": 20, "52269": 40, "524288": 20, "526": 152, "52667": 40, "5284": 20, "529514": 16, "5299": 127, "53": [16, 20, 124, 130, 131, 152], "530": 40, "5305": 127, "535": 18, "5371": 152, "5373": 152, "537602": 16, "538": 40, "5393": 2, "54": [7, 8, 20], "540": [39, 40, 124], "5417": 152, "5436": 152, "5443839": 2, "54576": 2, "5496": 127, "5497": 40, "55": [7, 10, 19, 20, 124], "5500": 40, "551": 40, "5510": [39, 124], "5514": [39, 124, 152], "5519": 152, "552": [8, 18], "5520": 152, "5530": 40, "5531": 152, "5534": 152, "554": 40, "5558": 152, "556": 40, "5563": 152, "5564": 152, "5568": 152, "5570": 20, "558": 40, "56": [7, 16, 20, 26], "560": 3, "562": [110, 114], "5636": 152, "564": 16, "5642": 152, "564272": 16, "566": 40, "5669": 152, "568": 124, "5698": 152, "57": [16, 20, 40, 124], "570": 16, "572": 40, "5739": 2, "5742": [127, 130], "575": [29, 30, 31, 32, 33], "5761": 152, "5772": 152, "5779": 152, "5782": 152, "579": 40, "58": [13, 16, 20, 130], "5800": 152, "5801": 152, "5809": 152, "581": 40, "5815": 152, "5816": 152, "5821": 40, "5823": 152, "5825": 152, "5830": 143, "5835": 152, "5855": 152, "5874": 143, "5877": 127, "5879": 143, "58x": 13, "59": [10, 16, 19, 20, 26, 124], "590": 44, "5900": 152, "5902": 152, "5904": 152, "591": 40, "5918": 143, "5921": 152, "5925": 152, "5931": 152, "594": 40, "5941": 152, "5947": 152, "5949": 152, "5954": 152, "5957": 143, "5963": 20, "5975": 152, "5976": 127, "5980": 127, "599": 40, "5992": 152, "5b": [82, 152, 163], "5k": 21, "5m": 21, "5th": [15, 133], "5u": 1, "5vl": [26, 152], "5x": [4, 7, 13, 17], "6": [0, 1, 5, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 26, 29, 31, 32, 33, 40, 84, 106, 109, 110, 113, 133, 138, 142, 143, 147, 152], "60": [0, 8, 10, 16, 19, 20, 147], "600": 120, "6000": [10, 39, 40, 124, 150, 152], "6014": 152, "602": 40, "603": 40, "6049": 127, "605": 40, "6059": [39, 124], "6064": 143, "6065": 152, "6075": 152, "608": 40, "6080": 152, "61": [20, 26, 86], "610": 40, "6100": 2, "6103": 152, "612328": 16, "613": 40, "6136": 152, "6140": 152, "615": 40, "6157": 143, "616": 40, "617": 40, "61954812": 36, "62": [13, 19, 20, 40, 130], "622": 26, "623": [26, 40], "623219": 16, "6255": 143, "626": 44, "6299": 143, "63": [19, 20, 40, 67, 98, 124, 130, 135, 139], "630": 40, "6300": 10, "63266": 125, "633": 30, "63307": 125, "63308": 125, "63331": 125, "63374": 125, "634": 40, "6344": 152, "63456": 125, "6345624": 125, "636": 40, "6372": 127, "6376": 2, "639": 152, "6393": 12, "64": [0, 1, 2, 4, 5, 11, 12, 16, 20, 23, 26, 29, 30, 31, 32, 33, 39, 40, 42, 48, 64, 70, 75, 76, 80, 87, 88, 105, 106, 116, 124, 130, 133, 134, 135, 139, 152, 156, 160, 161, 165], "640": [3, 21], "640gb": 15, "6429": 152, "643": 40, "6437": 152, "644": 40, "6447": 152, "6452": 131, "646": 40, "6475": 130, "649": 152, "6499": 10, "64ac201c77bf46a7a3a4eca7759b1fd8": 33, "64x": 13, "65": [10, 20], "650": 40, "65024": 143, "651": 40, "65100": 2, "651199": 16, "6523": 131, "653": 8, "6538": 12, "654": 5, "6550": 127, "65536": 20, "6554": 130, "658": 40, "6591": [39, 124], "66": [13, 19, 20, 29, 30, 31, 32, 40], "660": 40, "662": 40, "6628": [130, 131], "664": 8, "665": 40, "666": 40, "6678": 139, "6684": 131, "6695": 139, "67": [7, 13, 16, 20], "6704": 10, "671": 2, "67108864": [9, 18, 21, 26, 98, 151], "671b": 14, "672": 40, "673": 152, "675": 124, "6753e": [39, 124], "676": 40, "6769": 130, "677": 40, "6774": 10, "678": 40, "679": [4, 40], "68": [7, 13, 20, 40, 131], "6825": [39, 124], "684": 13, "6852": [127, 130], "6862": 124, "6890": 143, "69": [7, 13, 16, 20, 131, 151], "6925": [39, 124], "6938": 44, "6948": 10, "695": [40, 152], "697": [15, 40], "6973": 12, "6975": 127, "6976": [125, 130, 131], "699": 40, "6a": 3, "6b": [4, 39, 82, 124, 133, 152, 163], "6x": [5, 17, 97], "7": [0, 1, 3, 4, 7, 8, 10, 12, 13, 14, 15, 16, 17, 19, 20, 21, 27, 29, 30, 32, 33, 39, 40, 50, 66, 78, 84, 96, 98, 101, 109, 110, 124, 125, 133, 138, 143, 158], "70": [0, 7, 8, 16, 19, 20, 131, 139], "700": 120, "7000": [39, 124], "701": 152, "703": 10, "7031": 127, "705": [15, 30, 152], "7063": [39, 124], "707": 40, "7072": 40, "708": 19, "709": 124, "7090": 143, "70b": [5, 7, 10, 24, 26, 28, 39, 75, 82, 96, 97, 119, 125, 127, 128, 129, 130, 131, 132, 142, 149, 152, 163], "70g": 119, "71": [13, 20, 124], "712": 40, "7128": 30, "7134": 143, "7136": 125, "714": 40, "7144": 143, "715": 40, "7155": 12, "7168": [2, 13, 15], "717498": 16, "7187": 40, "7188": 2, "72": [10, 16, 20, 30, 126, 152], "720": [30, 33], "7206": 2, "722": 40, "724": 40, "727": 40, "728516": 16, "72b": [142, 149, 152], "73": [13, 19, 20], "734": 40, "738": 40, "7382": 40, "739": [40, 152], "73x": 17, "74": [13, 20, 30, 33], "741": [40, 152], "7422": 12, "744": 40, "7456": 2, "74561": 2, "7480": 125, "7481": 10, "749": 40, "74x": 11, "75": [7, 16, 18, 20, 30, 124, 152], "750": 5, "7502": 125, "7520": 2, "755": [40, 120], "7571": 12, "7584": 2, "75903": 40, "76": [20, 147], "7607": 130, "7610": 12, "7612": 12, "762": 40, "7621": 40, "7638": [125, 130, 131], "7657": 2, "766": 40, "76665782272": 27, "767": 40, "768": [24, 116, 134], "77": [16, 19, 20, 26], "770": 40, "7743": 125, "7770": 125, "78": [13, 20, 127], "780": [39, 124], "781": 40, "782": 40, "783": 40, "7842": 127, "785": 40, "78509": 40, "787": 40, "7871": 10, "7876": 130, "789": 40, "7891": 12, "7898": 12, "79": [20, 40, 124, 139], "790": 10, "7900": 143, "791": 16, "792": 16, "7933": 130, "794": 152, "7949": 143, "795": 40, "797": 40, "7977": [12, 127], "798": 40, "799": 40, "7a": 3, "7b": [7, 11, 26, 27, 39, 40, 72, 82, 89, 104, 110, 113, 114, 124, 142, 149, 152, 163], "7b_model": 26, "7k": 11, "7x": [4, 13, 17], "8": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 26, 27, 29, 30, 31, 32, 33, 40, 44, 45, 49, 54, 55, 56, 57, 60, 61, 62, 63, 65, 66, 67, 68, 75, 76, 78, 80, 88, 92, 95, 101, 104, 105, 109, 110, 111, 116, 117, 119, 124, 125, 126, 127, 132, 133, 134, 135, 136, 139, 141, 143, 144, 146, 147, 151, 156, 158, 160, 161, 165, 168], "80": [0, 5, 12, 13, 15, 20, 98, 106, 152], "800": [3, 30, 152], "8000": [9, 17, 18, 21, 26, 27, 29, 30, 31, 32, 33, 41, 42, 43, 45, 47, 48, 69, 70, 71, 72, 73, 84, 88, 151, 160, 165], "8001": [17, 84], "8002": [17, 84, 124], "8003": [17, 84], "8004": 17, "8005": 40, "803": [3, 8], "804": 40, "8044": 12, "8048": 124, "80833": 26, "80b": [33, 149], "80gb": [4, 7, 23, 40, 125, 127, 128, 129], "81": [13, 16, 20, 127], "812": 40, "813": 40, "8140": 2, "8149": 143, "817": 40, "8179": 143, "819": 5, "8192": [11, 12, 23, 26, 29, 40, 67, 91, 124, 125, 130, 133, 134, 143, 147, 152], "82": [13, 20, 127], "820": 124, "821": 40, "8212": [0, 1], "8218": 143, "8225": 127, "825": 152, "8259": [39, 124], "828": 40, "829": 40, "83": [20, 26], "830": 11, "8307": 131, "831": 40, "833": 40, "835": 40, "8351": [39, 124], "838": 40, "8393": 31, "84": [8, 13, 20, 26], "844": 40, "8441": [39, 124], "849": 40, "84d2f12": 12, "85": [2, 7, 20, 26, 124, 152], "852": 40, "854": 40, "856": 40, "86": [10, 20, 98], "863": [39, 124], "8672": 143, "87": [7, 8, 10, 16, 20, 30], "870": 40, "871": 40, "872": 40, "876": 40, "8779": 143, "878": 19, "88": [20, 26, 127, 131], "8804": 125, "880676": 16, "881": 40, "88226": [39, 124], "8828": 143, "8841": 127, "8870": 31, "89": [7, 10, 13, 20, 98, 142], "8908": 32, "893": 40, "8932": [39, 124], "895": 40, "8958": 131, "896": 2, "898": 40, "8984": 32, "899": 40, "8a": 6, "8b": [10, 24, 39, 49, 54, 59, 67, 68, 76, 78, 82, 89, 93, 95, 96, 101, 124, 142, 144, 146, 149, 151, 156, 158, 160, 163, 167, 168], "8bit": 4, "8k": [12, 20, 29, 152], "8tb": 5, "8x": [15, 17, 18, 21], "8x22b": [82, 163], "8x7b": [39, 104, 124, 142, 149, 152], "8xb200": [13, 18], "8xgpu": [15, 21], "8xh100": 6, "8xh200": 3, "9": [0, 1, 4, 11, 13, 14, 16, 17, 19, 20, 21, 26, 30, 31, 32, 36, 40, 50, 66, 76, 86, 98, 101, 110, 113, 119, 127, 133, 143, 156, 160], "90": [0, 2, 16, 19, 20, 30, 39, 40, 50, 66, 86, 98, 112, 124, 125, 127, 132, 139, 147], "9007": 2, "902": 40, "9028": 143, "907": 4, "9075": 32, "908": 40, "9087": 131, "909": 40, "91": 20, "910": 40, "9101": 40, "9115": 131, "912": 8, "912656": 2, "916": 40, "9184": 127, "9189": 32, "919": 40, "9197": 2, "92": [13, 20, 26], "9203": 127, "9214": 40, "9216": 29, "922": 40, "9223372036854775807": 147, "924": [24, 40, 116], "925": 19, "9263": 2, "9274": [10, 125], "93": [2, 16, 20], "931": 40, "932": 40, "933": 40, "9348": 31, "935": 152, "9353e": 125, "9356": 31, "937": 40, "9379": 2, "939": 40, "94": 20, "94022": 40, "941": [3, 6, 40], "943": 40, "944": 40, "9447": 29, "946": 3, "9462": 29, "948": [16, 40], "949": 40, "9494": 130, "95": [11, 20, 27, 40, 45, 49, 54, 55, 56, 57, 60, 65, 66, 67, 101, 125, 132, 144, 146, 151], "9500": 29, "9521": 143, "953": 40, "9537": 127, "9538": 29, "954": [15, 40], "955200": 16, "957": 40, "96": [3, 13, 15, 19, 20, 127, 152], "960": 3, "9606": 15, "960gb": 16, "961": 40, "9613": 15, "9623": 130, "9629": 15, "9639": 40, "965": 40, "96583": 40, "967": 152, "9692": 143, "97": [15, 20, 124, 127], "972": 40, "976442": 16, "977": 40, "978": 40, "98": 20, "981": 40, "983": 152, "9845": 26, "987": 152, "9898": 2, "99": [13, 16, 19, 20, 109, 120], "992": 152, "9928": 131, "993": 40, "9938": 2, "994": 40, "9980": 26, "9982": [130, 131], "9b": [82, 163], "9f": 0, "9x": [5, 6], "A": [0, 1, 2, 7, 8, 10, 13, 16, 18, 19, 26, 29, 30, 31, 32, 33, 35, 38, 39, 40, 59, 60, 61, 62, 63, 64, 75, 82, 84, 86, 91, 92, 96, 97, 102, 103, 105, 106, 108, 110, 113, 116, 117, 119, 123, 124, 133, 138, 140, 142, 144, 146, 147, 148, 149, 152, 153, 155, 163], "AND": 133, "And": [11, 14, 15, 16, 33, 96, 113, 119, 133, 134, 139], "As": [10, 11, 12, 14, 16, 17, 19, 20, 21, 29, 44, 76, 84, 87, 98, 104, 105, 107, 110, 113, 117, 127, 131, 132, 133, 139, 141, 143, 152, 155, 156, 169], "At": [10, 15, 20, 21, 24, 64, 90, 115, 127, 134, 139, 166], "Being": 35, "But": [11, 16, 20, 33, 75, 105, 108], "By": [0, 1, 8, 10, 11, 12, 13, 15, 16, 17, 20, 24, 30, 36, 37, 44, 75, 83, 84, 98, 102, 106, 112, 113, 124, 127, 131, 133, 143, 147, 155, 164], "For": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 24, 26, 27, 29, 30, 31, 32, 33, 34, 35, 37, 39, 40, 44, 49, 50, 58, 61, 62, 63, 74, 75, 76, 77, 78, 80, 83, 84, 86, 87, 88, 89, 91, 92, 94, 96, 97, 98, 100, 102, 103, 105, 106, 107, 108, 109, 110, 112, 113, 116, 117, 118, 119, 124, 125, 126, 127, 130, 131, 132, 133, 136, 138, 139, 143, 144, 146, 147, 148, 151, 152, 153, 154, 155, 156, 157, 158, 160, 161, 164, 165, 169, 170], "If": [0, 1, 2, 7, 9, 10, 11, 12, 14, 16, 18, 19, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 36, 37, 39, 58, 60, 75, 83, 84, 86, 87, 91, 93, 95, 96, 98, 99, 101, 102, 103, 104, 105, 106, 107, 109, 110, 111, 112, 113, 116, 117, 118, 119, 120, 121, 124, 125, 126, 127, 130, 131, 132, 133, 135, 136, 138, 139, 142, 143, 146, 147, 148, 151, 152, 153, 155, 164, 169, 170], "In": [0, 1, 2, 4, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 24, 33, 36, 39, 40, 44, 64, 82, 83, 84, 87, 89, 94, 96, 97, 98, 99, 107, 108, 111, 113, 117, 118, 119, 121, 124, 125, 126, 127, 130, 131, 133, 139, 141, 142, 143, 147, 150, 151, 152, 155, 163, 164, 169, 170], "It": [0, 1, 2, 3, 6, 7, 9, 10, 11, 12, 13, 14, 15, 16, 19, 20, 21, 22, 23, 24, 29, 30, 31, 32, 33, 36, 39, 40, 44, 58, 60, 64, 66, 75, 83, 84, 86, 87, 91, 92, 94, 96, 97, 98, 103, 105, 106, 107, 110, 113, 115, 117, 118, 124, 127, 128, 129, 130, 131, 132, 133, 136, 141, 143, 146, 147, 148, 150, 151, 153, 154, 155, 164, 170], "Its": [11, 59, 75, 91, 105, 133, 150, 155], "NOT": [58, 133], "Near": 8, "No": [0, 16, 27, 33, 34, 39, 64, 84, 85, 102, 109, 124, 125, 140, 149], "Not": [1, 2, 7, 21, 86], "ON": [124, 127, 130, 131], "OR": 133, "Of": [13, 29, 152], "On": [12, 16, 19, 20, 30, 38, 75, 98, 101, 105, 109, 123, 126, 131, 133, 152], "One": [10, 11, 16, 20, 85, 86, 116, 117, 130, 133, 143, 149, 154], "Or": [33, 98, 133, 138, 167], "TO": 8, "That": [11, 33, 35, 40, 75, 91, 103, 105, 106, 109, 117, 127, 133, 147], "The": [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 12, 13, 14, 15, 17, 18, 19, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 44, 49, 50, 51, 53, 54, 55, 56, 57, 59, 60, 61, 62, 63, 64, 65, 67, 76, 77, 78, 79, 80, 81, 82, 83, 84, 88, 89, 90, 92, 93, 94, 95, 96, 97, 98, 100, 101, 102, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 115, 116, 117, 118, 119, 123, 124, 125, 126, 127, 130, 131, 132, 133, 134, 135, 136, 138, 139, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170], "Their": [15, 18], "Then": [11, 12, 14, 16, 20, 26, 27, 33, 39, 78, 87, 110, 119, 120, 124, 125, 133, 146, 148, 153, 158, 169], "There": [5, 10, 13, 14, 15, 16, 17, 19, 29, 30, 44, 75, 83, 84, 91, 96, 98, 100, 101, 102, 105, 106, 107, 108, 109, 110, 116, 119, 133, 136, 139, 141, 143, 146, 152, 154, 155, 164, 169, 170], "These": [3, 5, 6, 8, 10, 11, 12, 13, 15, 16, 17, 20, 29, 30, 31, 32, 33, 34, 36, 39, 44, 84, 93, 96, 113, 119, 124, 125, 126, 134, 136, 152], "To": [2, 5, 8, 10, 11, 12, 13, 14, 16, 17, 18, 20, 21, 24, 27, 29, 30, 31, 32, 33, 35, 37, 38, 39, 40, 75, 77, 81, 83, 84, 86, 87, 90, 91, 92, 94, 95, 96, 98, 103, 105, 109, 110, 112, 113, 114, 117, 118, 119, 123, 124, 127, 130, 131, 132, 133, 139, 141, 145, 146, 147, 148, 150, 151, 152, 153, 155, 157, 162, 164, 166, 169, 170], "WITH": 66, "Will": [0, 1, 12], "With": [10, 11, 12, 16, 17, 19, 21, 30, 44, 59, 75, 84, 87, 91, 97, 102, 105, 106, 113, 117, 120, 124, 147], "_": [0, 10, 11, 34, 37, 90, 91, 103, 118], "_1": [26, 29, 30, 31, 32], "_2": [26, 29, 30, 31, 32], "__all__": [148, 153], "__call__": [60, 95], "__dict__": 147, "__file__": 58, "__getitem__": 147, "__global__": 12, "__init__": [11, 34, 58, 60, 75, 83, 107, 115, 117, 118, 124, 143, 147, 148, 152, 153, 155, 164, 170], "__main__": [49, 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 64, 65, 66, 67, 68, 101, 125, 127, 131, 132, 144, 146, 148, 151, 152, 153], "__name__": [49, 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 64, 65, 66, 67, 68, 101, 125, 127, 131, 132, 144, 146, 148, 151, 152, 153], "__post_init__": 152, "__pydantic_extra__": 147, "__pydantic_fields_set__": 147, "__repr__": 152, "__traceback__": 147, "__version__": 99, "_autodeploi": [22, 27, 76, 156, 160], "_autodeployllmarg": 152, "_callback": [83, 164], "_capac": 1, "_checkpoint_format": [83, 164], "_chunk_token": 58, "_config_load": [83, 164], "_context_logits_auto_en": 147, "_cpp_gen": 103, "_create_tensor": 117, "_custom_transform_funct": [83, 164], "_executor_loop": 96, "_fields_set": 147, "_file_path": 58, "_forward_step": [37, 90], "_generation_logits_auto_en": 147, "_get_config_dict": [78, 158], "_get_pretrained_config": [83, 164], "_handl": 1, "_hash_token": 58, "_hf_model_dir": 147, "_llm_arg": 58, "_load_pretrained_config": [83, 164], "_map": [83, 164], "_mark_output": 143, "_metadata": 58, "_modelformatkind": 147, "_mpi_sess": 147, "_n": [26, 29, 30, 31, 32], "_note": [75, 105], "_parallelconfig": 147, "_parent_namespace_depth": 147, "_path": 2, "_postproc_param": 147, "_postprocess_result": 147, "_process_previous_batch": [37, 90], "_releas": 1, "_return_log_prob": 147, "_run": 143, "_runtim": 138, "_sample_async": [37, 90], "_savehiddenstatesdecodingconfig__context": 147, "_schedul": [37, 90], "_sliding_window_pattern": 135, "_static": 117, "_str_to_trt_dtype_dict": 133, "_tensorrt_engin": 144, "_torch": [10, 39, 58, 67, 75, 78, 80, 83, 96, 124, 145, 147, 148, 152, 153, 155, 158, 161, 164, 167], "_torchllm": 147, "_types_namespac": 147, "_unsign": 1, "_util": 133, "_was_": 147, "_weight_load": [83, 164], "_weight_mapp": [83, 164], "a10": 23, "a100": [21, 23, 35, 97, 106, 119, 150], "a100x": 35, "a10g": 23, "a2": 152, "a22b": [82, 163], "a2a": [12, 152], "a30": 23, "a3b": [33, 82, 142, 149, 163], "a40": 23, "a8": 141, "a_": 133, "a_1": 133, "a_2": 133, "a_n": 133, "a_sf": 133, "aarch64": 142, "ab": [23, 58, 110, 133, 136, 141, 147], "abbrevi": [12, 27], "abc": [11, 14], "abcd": 14, "abi": [98, 152], "abil": [1, 39, 124, 152], "abl": [4, 10, 13, 16, 20, 39, 75, 91, 96, 101, 105, 124, 130, 133, 152], "ablat": [10, 14, 15], "abnorm": [16, 152], "abort": [147, 152], "about": [0, 1, 2, 3, 4, 6, 7, 11, 15, 16, 17, 20, 21, 30, 33, 39, 58, 64, 66, 80, 84, 97, 103, 119, 124, 125, 127, 128, 129, 131, 133, 139, 143, 147, 151, 152, 161], "abov": [2, 7, 11, 12, 15, 16, 17, 18, 20, 21, 27, 29, 30, 31, 32, 33, 36, 39, 40, 44, 75, 84, 92, 98, 101, 110, 111, 117, 119, 124, 125, 127, 131, 139], "absenc": [17, 106], "absolut": 9, "absorb": 13, "abstract": [37, 96, 131, 134, 147], "abstractsetintstr": 147, "ac": 152, "acc": [133, 136], "acceler": [4, 5, 6, 7, 11, 12, 16, 21, 23, 29, 30, 31, 32, 75, 81, 91, 96, 105, 111, 113, 136, 150, 162], "accept": [0, 1, 2, 10, 11, 12, 16, 18, 19, 20, 21, 22, 24, 34, 35, 44, 49, 54, 55, 56, 57, 75, 84, 96, 97, 98, 101, 113, 125, 127, 133, 138, 144, 146, 147, 151, 152, 155], "accept_length": 138, "acceptance_length_threshold": 147, "acceptance_window": 147, "acceptancelength": 0, "acceptancer": 0, "acceptancethreshold": 0, "acceptedlen": 1, "acceptedlengthscumsum": 1, "acceptedpath": 1, "acceptedpathid": 1, "acceptedtoken": 1, "acceptedtokenslen": 1, "access": [10, 16, 18, 20, 21, 29, 30, 36, 53, 103, 121, 124, 125, 133, 147, 151, 152], "accessor": 1, "accommod": [17, 84, 104, 154, 169], "accomplish": 126, "accord": [10, 12, 18, 19, 21, 75, 105, 118, 133, 134, 155], "accordingli": [10, 18, 19, 118, 147], "account": [2, 8, 17, 61, 62, 63, 96, 98, 117, 120], "accumul": [0, 8, 11, 16, 20, 23, 60, 105, 106, 133, 136, 138, 147], "accur": [3, 14, 39, 53, 84, 102, 124, 125, 150, 152], "accuraci": [2, 3, 11, 12, 13, 15, 21, 23, 24, 33, 127, 132, 133, 136, 141, 151, 152], "accuracy_threshold": 24, "achiev": [2, 3, 7, 8, 10, 11, 12, 13, 15, 16, 17, 19, 20, 21, 30, 33, 40, 51, 87, 96, 97, 98, 113, 125, 127, 130, 132, 147, 148, 153], "acknowledg": 97, "acquir": [10, 20], "across": [0, 5, 8, 10, 11, 12, 13, 16, 17, 18, 20, 21, 26, 27, 29, 30, 31, 32, 33, 34, 35, 40, 67, 75, 84, 87, 92, 97, 102, 104, 105, 106, 107, 117, 118, 126, 127, 130, 131, 133, 138, 147, 150], "act": [13, 16, 17, 84, 93], "act_fn": 134, "act_typ": [117, 133], "action": 59, "activ": [0, 1, 3, 4, 7, 13, 15, 16, 17, 21, 23, 30, 34, 37, 75, 81, 84, 87, 92, 93, 94, 105, 107, 117, 126, 133, 136, 141, 142, 152, 162, 170], "activation_scaling_factor": 116, "activationtyp": [117, 133], "active_request": [96, 170], "actor": 94, "actual": [2, 7, 8, 13, 14, 16, 18, 20, 22, 23, 26, 30, 33, 34, 83, 91, 96, 107, 108, 113, 127, 130, 131, 132, 147, 152, 164, 169], "ad": [1, 8, 11, 12, 14, 15, 17, 19, 20, 24, 30, 38, 50, 75, 78, 84, 86, 88, 91, 96, 97, 105, 106, 107, 109, 113, 114, 119, 121, 123, 126, 130, 131, 133, 135, 138, 145, 147, 152, 154, 158, 165], "ada": [7, 75, 93, 98, 105, 127, 142, 150, 152], "adalayernorm": 134, "adalayernormcontinu": 134, "adalayernormzero": 134, "adalayernormzerosingl": 134, "adapt": [0, 14, 16, 20, 22, 52, 81, 92, 94, 97, 110, 133, 134, 147, 148, 150, 152, 153, 162], "adapter1": [88, 165], "adapter2": [88, 165], "adapter_id": 147, "adapter_s": 110, "adapters": 1, "add": [1, 9, 11, 14, 19, 21, 24, 29, 30, 31, 32, 33, 35, 36, 39, 75, 83, 98, 103, 105, 107, 110, 115, 116, 117, 119, 120, 121, 124, 125, 127, 131, 133, 138, 143, 146, 147, 148, 151, 152, 153, 164, 169], "add_": 10, "add_activ": 117, "add_argu": [59, 64, 65, 67], "add_bias_linear": 135, "add_generation_prompt": 13, "add_input": 133, "add_not": 147, "add_output": 133, "add_padding_request": 169, "add_prefix_spac": 60, "add_qkv_bia": 135, "add_rmsnorm": 13, "add_sequ": 138, "add_special_token": [11, 13, 29, 31, 60, 67, 138, 147, 152], "addbadhandl": 1, "addcumlogprob": 152, "added_kv_proj_dim": 134, "added_proj_bia": 134, "addit": [0, 5, 8, 10, 11, 14, 15, 16, 17, 19, 20, 27, 29, 30, 31, 32, 33, 34, 39, 40, 44, 53, 75, 77, 78, 82, 83, 84, 89, 91, 96, 98, 100, 101, 105, 106, 110, 113, 117, 119, 124, 126, 127, 130, 133, 134, 141, 142, 143, 147, 152, 155, 157, 158, 163, 164, 169], "addition": [12, 20, 75, 96, 124, 125, 127, 131, 148, 153, 155], "additional_context_output": 147, "additional_generation_output": 147, "additional_model_output": 147, "additional_opt": 63, "additionalmodeloutput": [0, 103], "additionaloutput": [0, 103], "addr": 0, "address": [1, 2, 7, 8, 10, 13, 15, 16, 92, 118, 131, 139, 146, 147, 152], "addresswiths": 1, "adequ": [134, 147], "adher": 53, "adirondack": [30, 33], "adjust": [9, 16, 17, 18, 21, 39, 60, 61, 84, 102, 124, 125, 139, 147, 170], "admin": 101, "adopt": [10, 16, 19, 106, 119, 152], "adp": [10, 97, 152], "advanc": [6, 10, 12, 13, 14, 15, 16, 17, 23, 49, 54, 55, 57, 75, 78, 84, 92, 97, 101, 113, 117, 133, 136, 144, 146, 151, 152, 155, 158], "advantag": [16, 17, 19, 20, 21, 84, 86, 106], "advers": [3, 23, 136], "advertis": [39, 124], "affect": [2, 8, 10, 12, 23, 78, 91, 111, 119, 125, 127, 130, 131, 136, 139, 152, 158], "affin": 134, "aforement": [10, 16, 36], "after": [0, 1, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 35, 36, 50, 75, 81, 82, 84, 86, 91, 92, 96, 98, 102, 103, 105, 107, 108, 109, 110, 113, 117, 118, 120, 124, 127, 130, 131, 132, 133, 134, 136, 139, 147, 151, 152, 155, 162, 163, 170], "afterward": 12, "again": [10, 16, 59, 117, 125, 127, 131, 143], "against": [19, 39, 98, 124, 147, 152], "agent": [5, 10, 11], "agentdesc": 0, "agentnam": 0, "agentst": 0, "aggreg": [12, 15, 16, 17, 46, 84], "aggress": [12, 14, 50, 116, 127, 132], "agnost": [21, 152], "agre": [31, 32, 146], "agreement": 146, "ahead": [0, 18, 105, 113], "ai": [2, 4, 11, 12, 13, 16, 18, 20, 21, 27, 29, 31, 38, 45, 49, 50, 51, 54, 55, 56, 57, 60, 65, 68, 82, 101, 123, 125, 132, 133, 142, 144, 146, 149, 150, 151, 152, 163], "aidc": 152, "aim": [2, 13, 16, 21, 37, 39, 104, 116, 124, 125, 127, 152], "aime2025": 30, "aime24": 11, "aime25": 30, "ainsli": 3, "air": [30, 152], "airport": 30, "aka": 133, "akhoroshev": 152, "al": [3, 19], "albani": [30, 33], "albeit": 113, "alessionetti": 152, "algo": [67, 68], "algorithm": [0, 7, 8, 10, 11, 12, 13, 14, 15, 16, 22, 39, 67, 68, 75, 96, 97, 105, 106, 113, 116, 117, 119, 124, 127, 133, 147, 150, 152], "alia": [34, 134, 135, 147], "alias": 147, "alibi": 133, "alibi_bias_max": [133, 134], "alibi_scal": 133, "alibi_slop": 133, "alibi_with_scal": 133, "align": [8, 12, 19, 20, 89, 152, 170], "align_corn": 133, "aligneddata": 12, "all": [0, 1, 2, 5, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 26, 29, 30, 31, 32, 33, 34, 35, 39, 40, 51, 58, 61, 62, 63, 64, 65, 66, 67, 75, 78, 82, 83, 84, 86, 88, 91, 92, 96, 98, 99, 101, 102, 103, 104, 105, 106, 107, 108, 110, 113, 117, 118, 119, 124, 125, 126, 127, 130, 131, 132, 133, 134, 136, 138, 139, 141, 142, 143, 147, 151, 152, 155, 158, 163, 164, 165, 169, 170], "all2al": [16, 20, 152], "all_clos": 152, "all_reduc": 152, "all_reduce_param": [133, 134], "allbitset": [0, 1], "allgath": [12, 15, 20, 23, 117, 131, 133, 136, 152], "allgeneratedtoken": 0, "alllayersdrafttokenid": 1, "alllayersdrafttokenidspredecessor": 1, "alllayersscor": 1, "alloc": [0, 1, 18, 20, 24, 27, 29, 37, 44, 50, 75, 77, 84, 87, 91, 96, 98, 102, 105, 108, 109, 132, 133, 138, 139, 143, 147, 152, 154, 155, 157, 169, 170], "allocateipcmemori": 1, "allocnewblock": [0, 27], "allocnewblocksperrequest": 0, "alloctotalblock": [0, 27], "alloctotalblocksperrequest": 0, "allot": 0, "allottedtimem": [0, 152], "allow": [0, 1, 3, 6, 10, 11, 12, 15, 16, 17, 20, 21, 23, 29, 30, 31, 32, 33, 34, 36, 37, 38, 39, 40, 83, 84, 86, 87, 88, 90, 91, 92, 93, 95, 102, 103, 105, 106, 109, 113, 116, 123, 124, 125, 126, 127, 130, 131, 133, 136, 143, 146, 147, 150, 151, 152, 154, 164, 165, 166, 170], "allow_pickl": 147, "allreduc": [13, 15, 23, 117, 131, 133, 136, 147, 152], "allreduce_gemm": 112, "allreduce_strategi": [111, 147, 152], "allreducebuff": 1, "allreducefusionkernel": 13, "allreducefusionop": 133, "allreduceparam": [133, 134], "allreducestrategi": [111, 133], "alltoal": [20, 152], "alltoall_prepare_maybe_dispatch": 152, "alltoallprepar": 20, "almost": [10, 15, 16, 117, 127, 130, 139], "alon": [8, 104], "along": [10, 14, 20, 75, 82, 98, 105, 113, 133, 152, 163], "alongsid": [16, 81, 82, 162, 163], "alpaca": 110, "alpha": [133, 134, 147, 152], "alphabet": [133, 147], "alreadi": [0, 2, 11, 12, 13, 14, 15, 16, 17, 19, 21, 37, 58, 75, 83, 91, 105, 107, 109, 127, 130, 132, 133, 147, 148, 152, 153, 169], "also": [0, 2, 5, 6, 7, 10, 11, 12, 13, 14, 15, 16, 17, 20, 21, 23, 24, 27, 29, 30, 31, 32, 33, 35, 36, 39, 40, 44, 50, 51, 60, 75, 83, 84, 86, 87, 91, 96, 98, 99, 101, 102, 103, 105, 107, 113, 116, 117, 118, 119, 124, 125, 126, 127, 128, 129, 130, 133, 134, 136, 139, 141, 146, 147, 148, 150, 151, 152, 153, 154, 155, 169], "altair": 152, "alter": [86, 103, 107], "altern": [10, 13, 20, 21, 24, 35, 39, 59, 81, 83, 98, 103, 112, 124, 148, 153, 154, 162, 164], "although": [10, 17, 20, 39, 78, 84, 102, 107, 117, 124, 127, 131, 158], "alwai": [0, 1, 10, 15, 16, 19, 20, 34, 58, 75, 78, 91, 96, 103, 105, 106, 109, 116, 117, 119, 130, 131, 133, 143, 147, 158], "always_share_across_beam": 138, "am": [49, 54, 55, 57, 101, 125, 132, 138, 144, 146, 151], "ambigu": 1, "amc23": 11, "amd": 152, "amen": [0, 10, 103, 147], "american": 58, "among": [10, 17, 19, 86, 121, 133], "amongst": 133, "amort": 19, "amount": [0, 15, 16, 20, 23, 39, 87, 91, 109, 117, 124, 130, 132, 136, 138, 139, 143, 147], "amp": 21, "amper": [4, 21, 93, 98, 142, 150, 152], "amplifi": 10, "amsterdam": 30, "amtrak": 30, "an": [0, 1, 3, 5, 7, 8, 9, 10, 12, 13, 14, 15, 16, 17, 19, 20, 21, 22, 23, 26, 27, 29, 30, 31, 32, 33, 35, 36, 37, 39, 40, 44, 49, 53, 54, 55, 56, 57, 58, 59, 60, 75, 78, 80, 81, 82, 83, 84, 86, 87, 88, 89, 91, 92, 93, 94, 95, 96, 97, 98, 101, 102, 103, 105, 106, 107, 109, 110, 112, 113, 114, 115, 116, 117, 118, 119, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 136, 138, 139, 141, 143, 144, 146, 147, 148, 151, 152, 153, 154, 155, 158, 160, 161, 162, 163, 164, 165, 169, 170], "analog": 36, "analys": 16, "analysi": [11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 97, 107, 139], "analysispatternmanag": 107, "analysisth": 30, "analyt": 4, "analyz": [12, 20, 60, 107, 125], "andlength": 22, "ani": [0, 1, 2, 8, 10, 12, 14, 15, 16, 20, 27, 30, 34, 35, 36, 37, 39, 40, 58, 75, 78, 83, 86, 91, 98, 103, 107, 108, 113, 118, 119, 124, 130, 131, 132, 133, 135, 136, 138, 143, 147, 148, 153, 154, 155, 158, 164], "annot": [34, 147], "announc": [2, 3, 4, 6], "anoth": [0, 1, 4, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 27, 29, 30, 31, 32, 33, 75, 84, 87, 91, 105, 107, 110, 119, 130, 133, 143, 146, 155, 170], "answer": [11, 14, 19, 21, 30, 33, 53, 60, 65], "answer_suffix": 11, "answer_suffix_with_mark": 11, "antialia": 133, "antonin": [49, 101, 144, 146, 151], "anybitset": [0, 1], "anymor": [16, 96], "anyof": 136, "anyth": [40, 58, 64, 147], "aotman": 152, "apart": 44, "api": [2, 9, 10, 11, 12, 14, 15, 16, 17, 18, 20, 21, 22, 24, 26, 37, 38, 39, 40, 44, 45, 49, 52, 58, 62, 63, 80, 81, 82, 83, 84, 87, 89, 94, 98, 102, 106, 109, 113, 115, 116, 117, 123, 124, 127, 128, 129, 131, 132, 133, 139, 143, 145, 150, 153, 161, 162, 163, 164, 165], "api_kei": [69, 70, 71, 72, 73, 88, 165], "api_st": 34, "app": [98, 152], "appar": [8, 84, 102], "appeal": 19, "appear": [0, 20, 75, 91, 96, 101, 105, 106, 133, 143, 147, 152], "append": [11, 14, 19, 38, 58, 67, 83, 123, 133, 164, 170], "append_paged_kv_cach": [75, 155], "appl": [82, 152, 163], "appli": [0, 12, 13, 14, 15, 16, 20, 21, 23, 24, 36, 37, 39, 75, 81, 82, 83, 86, 87, 88, 98, 103, 105, 107, 110, 113, 116, 117, 118, 124, 133, 134, 138, 141, 147, 152, 155, 162, 163, 164, 165], "applic": [0, 4, 7, 8, 9, 13, 15, 16, 17, 18, 21, 26, 29, 30, 31, 32, 33, 36, 38, 41, 42, 43, 58, 80, 84, 96, 101, 109, 113, 123, 138, 143, 146, 149, 150, 151, 152, 160, 161, 170], "apply_batched_logits_processor": 147, "apply_callback": [83, 164], "apply_chat_templ": [13, 24, 53], "apply_llama3_sc": 133, "apply_query_key_layer_sc": [134, 135], "apply_residual_connection_post_layernorm": 135, "apply_rotary_pos_emb": 133, "apply_rotary_pos_emb_chatglm": 133, "apply_rotary_pos_emb_cogvlm": 133, "apply_router_weight_on_input": 152, "apply_silu": 133, "applybiasropeupdatekvcach": 152, "applyrop": 13, "appreci": [10, 11, 15], "approach": [0, 8, 11, 12, 13, 14, 15, 16, 17, 37, 39, 83, 84, 91, 96, 100, 104, 107, 109, 111, 113, 124, 132, 147, 164], "appropri": [2, 7, 11, 17, 34, 35, 37, 44, 78, 80, 84, 91, 143, 158, 161], "approx": 30, "approxim": [15, 16, 19, 33, 98, 134], "apt": [2, 81, 98, 101, 120, 162], "ar": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 49, 51, 53, 54, 55, 57, 58, 59, 60, 61, 62, 63, 64, 69, 70, 73, 75, 77, 78, 81, 83, 84, 86, 87, 88, 89, 91, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 138, 139, 141, 143, 144, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 157, 158, 160, 162, 164, 165, 167, 168, 169, 170], "arang": 133, "arbitrag": [39, 124], "arbitrari": [10, 96, 118, 152], "arbitrarili": 12, "arch": 96, "architectur": [4, 8, 14, 15, 16, 17, 20, 21, 29, 30, 31, 32, 33, 76, 77, 81, 82, 83, 84, 97, 98, 102, 104, 106, 109, 116, 135, 138, 142, 145, 149, 152, 156, 157, 162, 163, 164], "archiv": [83, 164], "arctic": [142, 152], "area": [15, 16, 30, 33], "aresult": [44, 147], "arg": [0, 11, 22, 24, 27, 34, 59, 64, 65, 67, 77, 78, 83, 107, 119, 134, 135, 136, 138, 147, 152, 157, 158, 164], "arglist": 107, "argmax": [12, 96, 133], "argpars": [59, 64, 65, 67, 136], "argument": [2, 10, 24, 27, 29, 30, 31, 32, 33, 35, 39, 44, 50, 51, 58, 68, 75, 84, 96, 97, 98, 102, 103, 124, 126, 133, 136, 139, 147, 152, 155], "argumentpars": [59, 64, 65, 67], "aris": [8, 10, 98], "arithmet": [10, 21, 117], "arm": 35, "arm64": 21, "aros": 20, "around": [1, 20, 83, 87, 116, 119, 125, 131, 164], "arrai": [0, 1, 35, 133, 138, 147], "arrayview": [0, 1], "arriv": [0, 8, 11, 59, 104], "arrivaltim": 0, "arrow": 133, "art": [2, 13, 16, 17], "articl": [13, 14, 105, 113], "artifact": [36, 97, 98], "artifici": [18, 66], "arxiv": [0, 1, 23, 104, 110, 133, 136, 141], "as_dtyp": 133, "as_lay": 107, "as_shap": 133, "ascii": [133, 147], "asciichar": 1, "ask": [11, 19, 21, 30, 33, 64, 143], "aspect": [91, 105], "asscoiat": 83, "assembl": [11, 117], "assert": [58, 72, 73, 107, 133, 143, 152, 170], "assert_valid_quant_algo": 135, "asset": 9, "assign": [0, 10, 16, 20, 39, 86, 119, 124, 134, 136, 148, 153], "assist": [18, 21, 30, 33, 41, 42, 53, 69, 70, 73, 106, 146, 151, 160], "assistant_model": 106, "assistantfin": 30, "associ": [1, 8, 15, 17, 36, 37, 84, 96, 98, 103, 104, 110, 125, 133, 164], "asssembl": 113, "assum": [1, 2, 9, 14, 15, 16, 33, 39, 51, 103, 109, 110, 113, 114, 124, 133, 138, 147], "assumpt": [17, 23, 113], "assur": 16, "async": [1, 11, 44, 55, 56, 124, 138, 147], "asynchron": [1, 10, 11, 16, 37, 44, 52, 58, 89, 94, 103, 147], "asynchroni": 16, "asyncio": [11, 55, 56], "asyncllmengin": 152, "aten": [81, 82, 162, 163], "atlant": [30, 33], "atom": 1, "attach": [2, 96, 147, 151], "attack": 86, "attempt": [0, 1, 36, 40, 84, 102, 125, 127, 147], "attend": [87, 132], "attent": [0, 1, 2, 3, 8, 10, 12, 16, 17, 20, 21, 23, 26, 27, 29, 30, 31, 32, 33, 37, 52, 76, 77, 80, 81, 84, 85, 96, 97, 102, 106, 108, 109, 110, 113, 115, 117, 118, 133, 136, 138, 139, 143, 145, 147, 148, 149, 150, 152, 153, 156, 157, 160, 161, 162, 169], "attention_backend": [67, 75, 148, 153, 155], "attention_bia": 152, "attention_chunk_s": 87, "attention_dp": 152, "attention_dp_config": [8, 30, 147], "attention_dp_events_gather_period_m": 147, "attention_head_s": [133, 134], "attention_mask": [75, 133, 134, 135, 138, 155], "attention_mask_param": 135, "attention_mask_typ": 134, "attention_multipli": 135, "attention_output": 143, "attention_output_orig_quant_scal": 133, "attention_output_sf_scal": 133, "attention_packed_mask": [133, 134], "attention_param": [134, 135], "attention_qk_half_accumul": 152, "attention_window_s": [75, 108], "attentionconfig": 0, "attentiondpconfig": [97, 147], "attentiondpeventsgatherperiodm": 0, "attentiondprank": 0, "attentionheads": 1, "attentionlayernumperpp": 0, "attentionmask": [75, 155], "attentionmaskparam": 134, "attentionmasktyp": [133, 134], "attentionmetadata": [96, 148, 153], "attentionparam": [134, 135], "attentiontyp": 0, "attn": [77, 78, 157, 158], "attn_backend": [67, 75, 80, 82, 147, 155, 160, 163], "attn_bia": 135, "attn_dens": [23, 110], "attn_forward_funcnam": 134, "attn_k": [23, 39, 88, 110, 124, 165], "attn_logit_softcap": 135, "attn_logit_softcapping_scal": 133, "attn_metadata": [148, 153], "attn_page_s": [80, 161], "attn_processor": 135, "attn_q": [23, 39, 88, 110, 124, 165], "attn_qkv": [23, 110], "attn_v": [23, 39, 88, 110, 124, 165], "attribut": [0, 1, 96, 103, 107, 118, 119, 136, 138, 147], "audienc": 60, "audio": [27, 138, 142, 149, 152], "audio_engine_dir": 138, "audio_featur": 138, "audio_path": 138, "audio_url": 27, "aug": 12, "augment": 160, "august": 12, "authent": [9, 125, 146], "author": [10, 96], "authorized_kei": [120, 121], "auto": [0, 1, 16, 21, 22, 26, 29, 31, 32, 36, 39, 67, 77, 84, 86, 97, 102, 103, 105, 106, 111, 114, 117, 124, 131, 133, 135, 136, 137, 147, 152, 157], "auto_deploi": [77, 78, 81, 152, 157, 158, 162], "auto_deploy_log_level": [79, 159], "auto_parallel": [23, 147, 152], "auto_parallel_config": 147, "auto_parallel_world_s": 147, "auto_quantize_bit": 137, "autoawq": 152, "autodecodingconfig": [97, 147], "autodeploi": [76, 77, 78, 80, 82, 97, 145, 152, 156, 157, 158, 160, 161, 163], "autodeploy_config": [78, 158, 160], "autodeploy_overrid": [78, 158], "autodeployconfi": [78, 158], "autodeployconfig": [78, 80, 158, 161], "autogptq": 152, "autom": [17, 35, 53, 60, 81, 82, 152, 162, 163], "automat": [0, 1, 10, 12, 13, 16, 21, 24, 27, 34, 35, 37, 39, 44, 76, 78, 81, 96, 103, 107, 111, 117, 118, 124, 125, 133, 139, 141, 146, 147, 150, 152, 156, 158, 160, 162], "automodelforcausallm": [77, 80, 82, 157, 161, 163], "automodelforimagetexttotext": [82, 163], "automot": 58, "autonom": 18, "autoparallelconfig": 147, "autopp": 152, "autoq": 152, "autoregress": [0, 37, 75, 113, 155, 169], "autotoken": [11, 44], "autotun": [147, 152], "autotuner_en": 152, "aux": 139, "auxiliari": [10, 83, 113, 164], "avaiable_block": 170, "avail": [0, 1, 3, 5, 8, 9, 11, 16, 17, 20, 26, 27, 29, 30, 31, 32, 36, 37, 39, 44, 49, 54, 55, 57, 66, 75, 76, 77, 80, 84, 86, 89, 91, 93, 94, 95, 96, 98, 99, 101, 103, 107, 109, 112, 113, 117, 124, 130, 131, 132, 138, 139, 141, 144, 145, 146, 147, 151, 152, 155, 156, 157, 168, 169], "averag": [0, 2, 8, 10, 14, 16, 17, 19, 20, 22, 26, 29, 30, 31, 32, 39, 40, 113, 124, 125, 127, 130, 131, 147], "avg": [8, 26, 29, 30, 31, 32, 39, 124, 125, 133], "avg_decoded_tokens_per_it": [30, 33], "avg_pool2d": 133, "avgdecodedtokensperit": 0, "avgnumdecodedtokensperit": 0, "avgpool2d": 134, "avoid": [1, 8, 10, 11, 12, 13, 14, 15, 16, 21, 29, 31, 86, 96, 98, 119, 138, 139, 146, 152], "awai": [91, 130, 131], "await": [0, 11, 44, 55, 56, 103], "await_respons": 147, "awaitcontextrespons": 0, "awaitgenerationrespons": 0, "awaitrespons": [0, 103], "awar": [3, 12, 17, 19, 21, 75, 96, 105, 143, 152], "awq": [7, 44, 93, 142, 152], "awq_block_s": 137, "ax": 133, "axi": [6, 16, 133], "b": [1, 3, 4, 5, 6, 35, 38, 59, 78, 107, 110, 117, 123, 133, 135, 138, 147, 152, 158], "b200": [14, 15, 16, 18, 19, 21, 29, 30, 40, 97, 150, 152], "b6261862419c33d6ce2313aff1e7116067d6037d": 2, "b_sf": 133, "back": [0, 1, 10, 12, 16, 37, 40, 50, 58, 59, 86, 109, 111, 113, 152], "backend": [0, 2, 9, 10, 11, 14, 15, 16, 22, 24, 26, 27, 31, 32, 33, 35, 38, 39, 40, 52, 53, 58, 66, 67, 76, 77, 78, 80, 81, 83, 87, 88, 89, 93, 95, 96, 97, 103, 113, 117, 124, 146, 147, 148, 152, 154, 156, 157, 158, 160, 161, 162, 164, 165, 167, 168, 169, 170], "backend_token": [0, 103], "backendagentdesc": 0, "backendtyp": [0, 84], "background": [1, 16, 26, 37, 97], "backgroundconfigur": 1, "backlog": 35, "backstream": 1, "backtyp": 1, "backu": [0, 95, 103, 147], "backup": 1, "backward": [81, 83, 119, 162, 164], "bad": [0, 95, 103, 147, 150, 152, 168], "bad_token_id": 147, "bad_words_data": 138, "bad_words_list": 138, "badword": 0, "badwordslen": 1, "badwordslist": 1, "badwordsptr": 1, "baichuan": [141, 142, 152], "baichuan2": 142, "baichuanconfig": 135, "baichuanforcausallm": 135, "balanc": [11, 15, 17, 29, 66, 86, 91, 97, 104, 106, 113, 117, 130, 132, 147, 152], "band": 53, "bandwidth": [3, 4, 5, 7, 12, 15, 16, 20, 21, 40, 53, 84, 102, 106, 117], "bangbang": 4, "bank": 12, "bantoken": 0, "banword": 0, "bar": [12, 147], "bare": [145, 152], "barissglc": 64, "barrier": [19, 150], "bart": [142, 152], "base": [0, 1, 2, 3, 4, 7, 8, 10, 12, 13, 15, 16, 17, 20, 21, 23, 26, 31, 32, 36, 37, 39, 55, 56, 60, 78, 82, 83, 84, 88, 91, 92, 96, 97, 98, 103, 109, 110, 111, 113, 115, 118, 119, 124, 130, 132, 133, 134, 135, 136, 137, 138, 139, 142, 145, 147, 148, 149, 152, 153, 154, 158, 163, 164, 165, 169, 170], "base64": [27, 70], "base_checkpoint_load": [83, 164], "base_config_load": [83, 164], "base_model": 110, "base_s": 134, "base_url": [29, 31, 32, 69, 70, 71, 72, 73, 88, 165], "base_weight_load": [83, 164], "base_weight_mapp": [83, 164], "baseagentconfig": 0, "basecheckpointload": 147, "basekvcachemanag": 0, "baselin": [7, 11, 12, 13, 14, 15, 20, 75, 78, 125, 130, 131, 155, 158], "baseline_fp8_engin": 127, "basellmarg": 147, "baseloopbackag": 0, "basemodel": [136, 147], "baseresourcemanag": [154, 169], "basesparseattentionconfig": 147, "basetransferag": 0, "bash": [9, 21, 26, 27, 29, 30, 31, 32, 33, 38, 41, 42, 43, 45, 47, 48, 59, 61, 62, 63, 67, 117, 121, 123, 151], "basi": 17, "basic": [38, 97, 115, 123, 133, 152], "basic_string_view": 0, "batch": [0, 1, 2, 4, 5, 7, 8, 10, 11, 12, 13, 14, 15, 16, 18, 19, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 37, 39, 40, 59, 67, 76, 77, 78, 81, 84, 87, 89, 90, 92, 96, 97, 106, 109, 110, 111, 113, 114, 117, 122, 124, 125, 127, 128, 129, 131, 132, 133, 134, 136, 138, 139, 143, 147, 148, 150, 152, 153, 154, 155, 156, 157, 158, 162, 166, 169, 170], "batch_beam_s": [105, 133], "batch_dim": 133, "batch_idx": 138, "batch_input_id": 138, "batch_manag": [0, 1, 58, 152, 169], "batch_output": [37, 59, 90], "batch_schedul": 152, "batch_siz": [2, 3, 6, 10, 12, 19, 26, 29, 31, 32, 40, 65, 75, 78, 96, 105, 107, 114, 116, 133, 134, 137, 138, 139, 147, 155, 158], "batch_wait_max_tokens_ratio": 147, "batch_wait_timeout_it": 147, "batch_wait_timeout_m": 147, "batchdon": 1, "batched_logits_processor": 147, "batchedlogitsprocessor": 147, "batchidx": 1, "batchindex": 1, "batching_typ": 147, "batching_wait_it": [8, 30, 147], "batchingtyp": [0, 97, 147], "batchsiz": [0, 1, 4, 106], "batchsizelimit": 0, "batchsizet": 0, "batchslot": 1, "batchslotshostcopi": 1, "bc": 133, "bc1393d529ce485c961d9ffee5b25d72": [31, 32], "beam": [0, 1, 6, 22, 23, 24, 27, 37, 44, 66, 84, 87, 97, 106, 113, 133, 138, 139, 143, 147, 152], "beam_search_diversity_r": [138, 147], "beam_width": [22, 44, 75, 91, 105, 106, 133, 138, 152], "beam_width_arrai": 147, "beamhypothes": 1, "beamsearch": [0, 147], "beamsearchbuff": 1, "beamsearchdiversityr": [0, 1, 106], "beamsiz": 0, "beamtoken": [0, 103], "beamwidth": [0, 1, 102, 103, 106, 147, 152], "beamwidtharrai": [0, 1, 106], "beat": [78, 158], "beauti": 66, "becam": [0, 12, 20], "becaus": [0, 7, 8, 10, 11, 13, 14, 15, 16, 20, 23, 39, 40, 44, 64, 75, 84, 91, 92, 96, 99, 102, 103, 109, 124, 125, 126, 127, 130, 132, 133, 136, 139, 146, 147], "becom": [7, 8, 10, 11, 12, 13, 15, 16, 19, 20, 75, 78, 87, 89, 91, 92, 104, 105, 106, 107, 109, 110, 117, 118, 158], "been": [0, 4, 5, 10, 11, 12, 13, 15, 16, 20, 39, 64, 75, 82, 86, 91, 94, 98, 99, 101, 103, 105, 119, 121, 124, 127, 130, 133, 143, 147, 152, 163], "befor": [0, 1, 8, 10, 11, 12, 13, 14, 16, 18, 20, 21, 22, 29, 31, 34, 38, 58, 60, 61, 62, 63, 75, 76, 84, 86, 91, 92, 95, 96, 97, 98, 100, 101, 102, 103, 105, 107, 109, 110, 111, 116, 117, 118, 123, 126, 127, 130, 132, 133, 135, 138, 139, 143, 146, 147, 148, 152, 153, 154, 155, 156, 169, 170], "beforehand": 125, "began": 20, "begin": [29, 30, 31, 32, 33, 58, 81, 82, 113, 126, 147, 148, 152, 153, 162, 163], "begin_thinking_phase_token": 147, "behav": [0, 136, 139, 147], "behavior": [8, 11, 20, 36, 39, 40, 59, 75, 76, 78, 84, 91, 95, 97, 102, 105, 130, 133, 138, 139, 147, 152, 156, 158], "behaviour": [0, 16, 133], "behind": [4, 11, 12, 15, 37], "being": [0, 10, 15, 18, 19, 20, 21, 29, 30, 31, 32, 33, 64, 75, 77, 83, 86, 91, 94, 105, 109, 117, 119, 130, 143, 147, 152, 155, 157, 164], "believ": [16, 39, 124], "belong": [20, 91, 130], "below": [0, 2, 5, 6, 7, 9, 10, 11, 14, 15, 16, 17, 18, 19, 21, 26, 29, 30, 31, 32, 33, 35, 39, 40, 75, 84, 91, 96, 98, 100, 105, 106, 107, 108, 110, 120, 121, 124, 127, 130, 131, 143, 151, 152], "bench": [2, 10, 14, 16, 19, 21, 26, 29, 30, 31, 32, 33, 39, 40, 52, 64, 92, 97, 124, 128, 129, 151, 152, 160], "benchmark": [9, 10, 12, 13, 14, 16, 17, 18, 20, 21, 22, 25, 27, 38, 62, 77, 78, 81, 84, 89, 92, 97, 98, 102, 123, 127, 128, 129, 131, 151, 152, 157, 158, 160, 162], "benchmark_2nod": 27, "benchmark_serv": [26, 29, 30, 31, 32, 33, 152], "benefici": [10, 12, 15, 17, 20, 39, 84, 91, 124, 130, 131], "benefit": [5, 7, 8, 10, 12, 14, 15, 16, 17, 19, 20, 23, 40, 60, 65, 87, 91, 107, 109, 111, 130, 136, 152], "berkelei": 24, "bert": [23, 93, 133, 136, 141, 142, 149, 152], "bert_attent": 133, "bert_attention_plugin": [23, 136], "bert_context_fmha_fp32_acc": [23, 136], "bertattent": 134, "bertattentionplugin": 133, "bertbas": 135, "bertforquestionansw": 135, "bertforsequenceclassif": [135, 142, 149], "bertmodel": 135, "besid": [20, 27, 154], "best": [8, 10, 11, 13, 14, 15, 16, 17, 19, 21, 38, 39, 76, 97, 105, 117, 123, 124, 126, 128, 129, 130, 147, 152, 156, 160], "best_of": [95, 147, 152], "best_path": 138, "best_path_len": 138, "best_path_length": 138, "best_perf_practice_on_deepseek": [13, 152], "bestofn": 11, "bestpathindic": 1, "bestpathlength": 1, "beta": [27, 97, 133, 145, 147, 150], "beta_fast": 133, "beta_slow": 133, "better": [0, 6, 8, 10, 12, 13, 14, 15, 16, 17, 19, 20, 21, 23, 33, 37, 40, 60, 61, 62, 63, 65, 75, 84, 86, 87, 90, 91, 94, 96, 102, 105, 106, 109, 111, 118, 119, 126, 127, 131, 132, 136, 147, 152, 166], "between": [0, 8, 11, 12, 13, 14, 15, 16, 17, 19, 20, 22, 26, 29, 30, 31, 32, 33, 34, 38, 40, 42, 70, 75, 80, 83, 84, 86, 88, 92, 102, 105, 106, 108, 109, 112, 113, 117, 118, 119, 123, 126, 131, 132, 133, 134, 139, 143, 147, 148, 152, 153, 164, 165], "beyond": [1, 4, 11, 19, 89, 91, 127], "bf16": [1, 2, 12, 13, 15, 18, 33, 75, 82, 93, 105, 111, 118, 119, 127, 131, 142, 149, 152, 163], "bfloat16": [23, 39, 75, 105, 117, 124, 125, 136, 141, 142, 147, 152], "bhuvanesh09": 152, "bi": [75, 105], "bia": [0, 15, 75, 103, 116, 117, 133, 134, 135, 147, 152], "bias": [116, 133], "bidirect": [133, 134], "bidirectionalglm": 133, "bielik": 142, "big": [19, 60], "bigcod": [82, 163], "bigger": [33, 109], "biggest": [73, 109], "billion": 2, "bin": [2, 9, 21, 26, 29, 30, 31, 32, 33, 38, 41, 42, 43, 45, 47, 48, 61, 62, 63, 83, 116, 117, 118, 123, 143, 152, 164], "binari": [38, 83, 113, 117, 123, 133, 164], "bind": [1, 9, 10, 11, 16, 58, 132, 138, 139, 147, 152, 154, 169, 170], "bindcapacityschedul": 170, "bindf": 36, "bit": [0, 1, 4, 16, 64, 75, 105, 133, 136, 141], "bitmask": 152, "bl": [17, 113], "black": 107, "blackwel": [2, 8, 12, 14, 16, 19, 20, 28, 40, 93, 96, 97, 101, 126, 127, 142, 150, 152], "blank": 86, "blip": [141, 152], "blip2": [93, 141, 142, 152], "blob": 13, "block": [0, 1, 10, 11, 12, 15, 16, 17, 20, 23, 44, 50, 58, 59, 75, 84, 86, 87, 91, 93, 96, 102, 105, 106, 109, 117, 130, 133, 136, 138, 139, 146, 147, 150, 152, 160, 169], "block_controlnet_hidden_st": 135, "block_id": 58, "block_num": 133, "block_po": 58, "block_siz": [58, 133, 134, 138], "block_sparse_block_s": 133, "block_sparse_homo_head_pattern": 133, "block_sparse_num_local_block": 133, "block_sparse_param": 134, "block_sparse_vertical_strid": 133, "blockhash": 0, "blockidx": 1, "blockkei": 0, "blockptr": 1, "blocksiz": 0, "blockspars": 133, "blocksparseattnparam": 134, "blockwis": [58, 152], "blog": [2, 3, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 40, 84, 90, 92, 152, 166], "bloodeagle40234": 152, "bloom": [106, 118, 141, 142, 152], "bloom_dict": 118, "bloomforcausallm": 135, "bloommodel": 135, "bm": 1, "bmm": [117, 152], "bo": [29, 31], "board": [81, 131, 162], "bodi": 117, "bold": 12, "book": 64, "bool": [0, 1, 58, 60, 75, 83, 107, 114, 116, 133, 134, 135, 136, 138, 147, 155, 164], "boolean": [1, 103, 110, 133, 135, 136], "boost": [2, 13, 15, 16, 18, 20, 60, 127, 130, 131], "boost_factor": 60, "boost_val": 60, "border": [29, 30, 31, 32, 33], "born": [115, 117, 143], "borough": [30, 33], "borrow": [39, 44, 124], "bos_token": 60, "bos_token_ad": 60, "bos_token_id": [60, 138], "boston": 30, "bot": 35, "both": [0, 2, 4, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 26, 29, 30, 31, 32, 33, 35, 39, 40, 49, 66, 75, 78, 84, 87, 88, 91, 92, 96, 98, 101, 102, 104, 105, 107, 108, 110, 113, 117, 118, 124, 126, 130, 132, 133, 134, 136, 139, 141, 147, 152, 154, 155, 158, 165], "bother": 58, "bottleneck": [2, 7, 8, 16, 20, 37, 87, 91, 104, 111, 126, 130], "bottom": [8, 10, 20, 121], "bound": [0, 5, 8, 12, 13, 14, 15, 20, 21, 37, 90, 106, 115, 117, 124, 133, 138, 139, 147, 152], "boundari": [15, 16, 20, 97, 106, 117, 133, 135, 137, 139, 147], "box": [2, 11, 35, 107], "bpru": 152, "br": 21, "brace": 147, "bracket": 20, "brahma": [39, 124], "branch": [3, 6, 12, 16, 18, 29, 30, 31, 32, 113, 147], "breadth": 113, "break": [13, 16, 20, 33, 34, 39, 58, 113, 124, 131, 147, 150, 152, 170], "breakdown": [20, 38, 39, 40, 123, 124, 125], "breakthrough": 150, "breviti": [2, 20], "bridg": [10, 16], "brief": [21, 35, 75, 135, 138, 155], "briefli": [11, 42, 70], "brife": 0, "bright": [60, 96], "brilliant": 12, "bring": [7, 12, 13, 14, 15, 16, 82, 148, 152, 153, 163], "broad": [21, 146], "broadcast": [13, 103, 133], "broadcast_help": 133, "broader": [81, 105, 146, 150, 152, 162], "broadli": [15, 75, 93], "broken": [10, 91, 130, 152], "bronx": [30, 33], "brooklyn": [30, 33], "brought": 16, "bsz": 134, "bu": 98, "bubbl": [8, 10], "budget": [11, 67, 91, 97, 114, 130, 147], "buffalo": 33, "buffer": [0, 1, 10, 16, 23, 24, 27, 29, 30, 31, 32, 33, 84, 87, 96, 102, 103, 108, 109, 133, 136, 147, 152, 169], "buffer_0": 1, "buffer_1": 1, "buffer_2": 1, "buffer_alloc": 138, "buffercast": 1, "buffercastornul": 1, "bufferdatatyp": 1, "buffermanag": 139, "buffermanagertest": 1, "bufferptr": 1, "bufferrang": 1, "buffers": 1, "bufferview": 0, "bug": [15, 35, 152], "build": [11, 12, 20, 21, 29, 30, 31, 32, 33, 36, 38, 39, 64, 67, 76, 77, 80, 84, 91, 100, 102, 103, 105, 106, 107, 109, 110, 112, 113, 114, 115, 117, 123, 126, 127, 128, 130, 132, 135, 136, 139, 143, 147, 150, 152, 156, 157, 161], "build_and_run_ad": [77, 81, 157, 162], "build_config": [23, 44, 64, 119, 127, 130, 131, 135, 144, 147], "build_connector_meta": 58, "build_dir": 98, "build_engin": 117, "build_flags_multiple_profil": 131, "build_serialized_network": 117, "build_sqsh": 98, "build_wheel": [2, 38, 98, 112, 123], "buildcacheconfig": [97, 147], "buildconfig": [44, 97, 114, 119, 127, 130, 131, 144, 147, 152], "builder": [114, 117, 119, 152], "builder_force_num_profil": 152, "builder_opt": 152, "built": [10, 11, 15, 17, 19, 23, 24, 26, 29, 30, 31, 32, 33, 36, 37, 40, 77, 79, 81, 84, 97, 98, 100, 101, 103, 106, 109, 117, 119, 124, 125, 131, 132, 133, 136, 139, 143, 146, 150, 152, 157, 159, 162], "bulid": 91, "bulk": 16, "bump": 1, "bumptaskinprogress": 1, "burden": 126, "busi": [0, 10, 17, 37, 60, 90], "button": 152, "buvnswrn": 152, "bw": [8, 152], "by_alia": 147, "by_nam": 147, "bypass": [17, 36, 84], "byt5": [142, 152], "byte": [0, 1, 12, 86, 111, 138, 147], "bytearrai": 147, "bytestostr": 1, "c": [0, 1, 2, 10, 14, 15, 17, 18, 20, 26, 27, 30, 33, 38, 44, 61, 62, 63, 84, 85, 91, 95, 105, 107, 113, 117, 120, 121, 123, 130, 133, 135, 138, 147, 149, 152, 154, 168, 169, 170], "c0": 8, "c1": 8, "c2c": [16, 20], "c440e2a3e7e14cd699295afc3739bf42": 21, "c4dep4_g1dep4": 17, "c5bf51b5cab94e10ba5da5266d12ee59": 30, "cach": [0, 1, 7, 8, 12, 13, 14, 15, 18, 19, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 36, 37, 39, 40, 44, 51, 52, 65, 67, 76, 77, 80, 81, 82, 85, 87, 89, 92, 97, 102, 103, 106, 110, 117, 119, 122, 124, 125, 130, 133, 136, 138, 140, 141, 145, 147, 148, 149, 150, 152, 153, 154, 155, 156, 157, 160, 162, 163, 170], "cache_block_id": 58, "cache_fold": 58, "cache_indir": 138, "cache_indir_t": 133, "cache_indirect": [75, 105, 133, 134, 138, 143], "cache_root": 147, "cache_salt": [86, 147], "cache_transceiver_config": [84, 147], "cached_properti": 147, "cachehitr": [0, 27], "cacheindirect": 1, "cachelevel": 0, "cachelevelupd": 0, "caches": 0, "cachesaltid": 0, "cachesaltidtyp": [0, 1], "cachest": 0, "cachetransceiv": 0, "cachetransceiverconfig": [0, 84, 97, 147], "cachetyp": 169, "cachevalu": 1, "calcul": [0, 3, 4, 6, 8, 12, 14, 15, 16, 17, 19, 20, 26, 29, 31, 32, 39, 86, 124, 132, 133, 138, 139, 147, 152], "calculate_speculative_resourc": 147, "calculatespeculativeresourc": 0, "calculatespeculativeresourcetupl": 0, "calib_batch": [127, 135, 147], "calib_batch_s": [127, 135, 147], "calib_config": [127, 147], "calib_dataset": [135, 137, 147], "calib_max_seq_length": [127, 135, 137, 147], "calib_s": [124, 137], "calibconfig": [97, 127, 147], "calibr": [7, 15, 16, 21, 23, 118, 127, 136, 147, 152], "california": 58, "call": [0, 1, 10, 11, 12, 14, 15, 16, 17, 20, 34, 37, 38, 44, 59, 60, 75, 83, 84, 87, 96, 103, 104, 105, 106, 107, 112, 117, 118, 119, 123, 125, 127, 133, 135, 136, 137, 138, 139, 147, 148, 152, 153, 154, 155, 169], "callabl": [118, 135, 147], "callback": [83, 103, 147, 164], "callstack": 10, "campaign": 60, "can": [0, 1, 2, 3, 4, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 35, 36, 37, 39, 40, 44, 46, 49, 50, 51, 58, 60, 61, 62, 63, 64, 68, 75, 77, 78, 80, 81, 83, 84, 86, 87, 89, 91, 92, 93, 94, 95, 96, 97, 98, 99, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 117, 118, 119, 120, 121, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 135, 136, 138, 139, 141, 142, 143, 145, 147, 148, 149, 151, 152, 153, 154, 155, 157, 158, 160, 161, 162, 164, 167, 169, 170], "canaccessp": 1, "canada": [30, 33], "canadian": 30, "cancel": [0, 103, 124, 147, 152], "cancelrequest": [0, 103], "candid": [0, 13, 14, 19, 95, 96, 106, 111, 113, 117, 147], "canenqueu": 0, "canenqueuerequest": 0, "cannot": [1, 8, 10, 11, 12, 13, 15, 16, 17, 20, 21, 83, 91, 92, 96, 106, 117, 118, 130, 131, 132, 133, 139, 143, 147, 152, 164, 170], "cap": [22, 125], "capabl": [3, 11, 13, 16, 17, 38, 53, 78, 97, 98, 123, 127, 147, 152, 158], "capac": [0, 1, 3, 5, 7, 16, 19, 20, 147, 152, 170], "capacitor_schedul": 170, "capacity_scheduler_polici": [132, 147], "capacityschedul": [154, 169, 170], "capacityschedulerpolici": [0, 97, 132, 147, 152], "capit": [27, 30, 33, 49, 50, 51, 54, 55, 56, 57, 65, 68, 88, 101, 125, 132, 144, 146, 147, 151, 165], "caption": 134, "captur": [14, 15, 16, 37, 75, 82, 96, 97, 147, 155, 163], "capturableguideddecod": 10, "capture_num_token": 147, "car": 30, "card": [64, 77, 157], "care": [11, 16, 17, 33, 78, 84, 158], "carefulli": [2, 12, 16, 19], "carlo": 11, "case": [0, 1, 2, 4, 7, 10, 11, 12, 13, 14, 15, 16, 17, 20, 23, 24, 26, 30, 33, 36, 39, 40, 44, 75, 84, 91, 96, 102, 105, 106, 108, 109, 110, 113, 124, 125, 127, 128, 129, 131, 133, 141, 146, 147, 152], "casefold": 147, "caseless": 147, "cast": [10, 15, 133], "cast_to_dtyp": 133, "castsiz": 1, "casual": 33, "cat": [2, 9, 14, 16, 20, 21, 26, 27, 29, 30, 31, 32, 33, 62, 78, 92, 158], "catalog": [26, 29, 30, 31, 32, 98, 99], "catch": 20, "categor": [15, 93, 113, 133], "categori": [34, 136], "categorical_sampl": 133, "caus": [8, 10, 16, 20, 23, 29, 84, 102, 103, 118, 119, 131, 143, 146, 147, 152], "causal": [14, 75, 133, 134, 155], "cautiou": 119, "caveat": [86, 127], "cd": [2, 14, 18, 33, 77, 81, 93, 94, 98, 115, 116, 124, 143, 157, 162, 167], "cdot": [26, 29, 30, 31, 32], "ceil": [1, 135], "ceil_mod": [133, 134], "ceildiv": 1, "center": [4, 5, 17, 21, 84, 147], "centr": 30, "central": [33, 83, 136, 164], "certain": [8, 16, 17, 37, 86, 92, 107, 116, 133], "certainti": 11, "certainty_threshold": 11, "cg": 135, "cga": 152, "chain": [10, 11, 14, 60], "challeng": [8, 13, 16, 17, 20, 84, 87, 92, 97], "chanc": [16, 23, 109, 132, 136], "chang": [2, 3, 5, 6, 10, 12, 14, 15, 16, 35, 39, 68, 75, 81, 84, 86, 87, 92, 94, 97, 98, 102, 105, 106, 108, 109, 110, 118, 119, 124, 131, 133, 135, 136, 138, 139, 143, 144, 145, 146, 147, 162, 169], "channel": [16, 21, 23, 133, 136, 141, 152], "chapter": 11, "char": [0, 1, 147], "charact": [35, 147], "characterist": [8, 17, 84], "charg": [75, 106, 117, 155], "chart": [4, 19], "chat": [5, 17, 18, 19, 21, 24, 26, 30, 33, 37, 43, 47, 49, 50, 51, 53, 54, 55, 56, 57, 60, 64, 65, 66, 71, 73, 74, 77, 78, 81, 82, 84, 89, 97, 101, 113, 144, 146, 151, 152, 157, 158, 160, 162, 163], "chat_complet": 30, "chat_templ": 27, "chat_template_kwarg": [24, 73], "chatbot": 64, "chatbot_lora_dir": 64, "chatcmpl": [18, 21, 30, 33, 151], "chatglm": [133, 141, 142, 152], "chatglm2": [142, 152], "chatglm3": [135, 142, 152], "chatglm_vers": 135, "chatglmconfig": 135, "chatglmforcausallm": 135, "chatglmgenerationsess": 138, "chatglmmodel": 135, "chatgpt": 21, "cheap": 96, "check": [10, 11, 12, 16, 18, 21, 29, 30, 31, 32, 33, 34, 36, 37, 40, 49, 58, 59, 84, 90, 97, 98, 101, 102, 103, 126, 127, 130, 131, 133, 138, 139, 143, 147, 148, 151, 152, 153, 166], "check_accuraci": [24, 116], "check_config": 135, "check_eagle_choic": 147, "check_gpt_mem_usag": 139, "checkbeamsearchdiversityr": 0, "checkbeamwidth": 0, "checkbeamwidtharrai": 0, "checkearlystop": 0, "checklengthpenalti": 0, "checkminp": 0, "checkmintoken": 0, "checknorepeatngrams": 0, "checknumreturnsequ": 0, "checkpoint": [2, 12, 13, 14, 15, 18, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 39, 49, 54, 77, 81, 82, 88, 93, 96, 97, 101, 115, 118, 119, 124, 125, 127, 136, 137, 138, 141, 143, 144, 146, 147, 148, 150, 151, 152, 153, 157, 162, 163, 165], "checkpoint_dir": [23, 83, 110, 114, 115, 116, 117, 119, 124, 143, 164], "checkpoint_format": 147, "checkpoint_load": [83, 147, 164], "checkposteriorvalu": 0, "checkremotedesc": 0, "checkrepetitionpenalti": 0, "checktemperatur": 0, "checktopk": 0, "checktopp": 0, "checktoppdecai": 0, "checktoppmin": 0, "checktoppresetid": 0, "chef": 143, "chen": 11, "china": 73, "chip": 58, "chmod": [29, 30, 31, 32, 33, 120], "choic": [0, 7, 14, 16, 18, 19, 21, 23, 24, 29, 30, 31, 32, 33, 65, 66, 67, 68, 73, 75, 113, 124, 126, 133, 138, 151, 155], "choos": [13, 15, 16, 17, 29, 30, 31, 32, 77, 80, 117, 119, 127, 133, 152, 157, 161], "chose": [16, 20], "chosen": [15, 95, 139, 168, 170], "chri": 58, "chrome": [38, 123], "chrono": 0, "chunk": [0, 11, 15, 22, 23, 27, 58, 85, 89, 97, 108, 122, 131, 133, 136, 138, 139, 140, 147, 149, 150, 152], "chunk_dim": 134, "chunk_length": 152, "chunk_scan": 133, "chunk_siz": [11, 133, 135], "chunkedcontextnexttoken": 1, "chunkedcontextnexttokenshost": 1, "ci": [1, 8, 36, 97, 152], "circular": [75, 87, 105], "citi": [30, 33, 73, 88, 151, 165], "ckpt": [39, 124], "ckpt_dir": [117, 119, 135], "ckpt_llama_3": 117, "ckpt_sourc": 147, "cl": [115, 119, 147], "claim": [1, 118], "claimpag": 1, "claimpageswithevict": 1, "clamp": [147, 152], "clamp_val": 147, "clara": [21, 58], "clarifi": 33, "class": [0, 1, 7, 10, 11, 23, 26, 29, 30, 31, 32, 33, 37, 44, 58, 60, 75, 77, 78, 80, 83, 86, 94, 95, 96, 97, 98, 105, 106, 107, 108, 114, 115, 117, 118, 119, 126, 127, 131, 133, 134, 135, 137, 138, 143, 146, 147, 148, 152, 153, 154, 155, 157, 158, 161, 164, 170], "class_dropout_prob": 134, "class_label": 134, "classic": [16, 21, 117], "classifi": [34, 134, 135], "classmethod": [115, 119, 134, 135, 136, 138, 147], "classvar": 147, "clean": [2, 10, 16, 38, 83, 98, 123, 143, 164], "cleaner": 12, "cleanup": [58, 83, 164], "clear": [8, 16, 21, 33, 34, 130, 138], "clear_logprob_param": 147, "clearer": 24, "clearli": [8, 10, 11, 12, 16, 132], "clearvirtualmemoryalloc": 1, "cli": [2, 9, 21, 39, 44, 116, 124, 126, 127, 130, 131, 151, 152, 160], "click": [58, 66, 68, 82, 120, 121, 163], "client": [0, 17, 20, 21, 27, 29, 30, 31, 32, 33, 40, 74, 84, 86, 89, 97, 103, 113], "client_id": [60, 95, 147], "clientid": 0, "clip": 133, "clip_before_cast": 133, "clip_qkv": [134, 135], "clip_vision_model": 135, "clipvisiontransform": 135, "clock": 13, "clone": [2, 11, 93, 98, 110, 125, 143, 146, 167], "clone_input": 107, "close": [2, 10, 12, 16, 17, 20, 23, 75, 105, 119, 131, 139], "closer": 8, "closur": 133, "cloud": [4, 21, 46, 120, 121], "cls_token": 134, "cluster": [13, 16, 18, 20, 21, 22, 23, 27, 46, 92, 101, 106, 117, 147, 152], "cluster_info": 152, "cluster_kei": [23, 152], "cluster_s": [22, 27], "clusteruuid": [84, 102], "cmake": [98, 152], "cmpl": [29, 31, 32], "cn": 8, "cnn": 24, "cnn_dailymail": [135, 147], "co": [0, 2, 8, 9, 10, 12, 14, 15, 16, 42, 70, 110, 133, 134, 143, 146], "coast": [30, 33, 151], "code": [7, 10, 11, 13, 15, 16, 18, 20, 21, 22, 26, 27, 29, 30, 31, 32, 33, 34, 36, 37, 38, 44, 50, 61, 62, 63, 75, 81, 82, 86, 96, 97, 100, 105, 107, 108, 111, 112, 113, 117, 119, 123, 124, 133, 141, 142, 143, 146, 147, 148, 150, 152, 153, 160, 162, 163, 169, 170], "codebas": [34, 108, 148, 153], "codec": 147, "codegemma": [82, 163], "codellama": [82, 152, 163], "codepath": 152, "codeqwen": 152, "coderham": 152, "codestr": [82, 163], "cognit": 87, "cogvlm": [142, 152], "cogvlmattent": 134, "cogvlmconfig": 135, "cogvlmforcausallm": 135, "coher": [106, 152], "cohereconfig": 135, "cohereforcausallm": 135, "coincid": 12, "cold": 16, "collabor": [8, 10, 11, 13, 15, 16, 17, 20, 106, 133], "collect": [1, 8, 12, 13, 15, 17, 20, 34, 40, 84, 94, 107, 111, 113, 117, 133, 147, 148, 153], "collect_and_bia": 134, "collector": 16, "collis": 89, "color": [64, 91, 130], "column": [110, 133, 141], "columnlinear": [110, 115, 134], "com": [2, 9, 13, 18, 27, 29, 30, 31, 32, 35, 36, 60, 93, 98, 117, 119, 133, 143, 152, 167], "combin": [0, 5, 8, 11, 12, 13, 14, 15, 16, 17, 18, 23, 26, 29, 30, 31, 32, 36, 40, 61, 62, 63, 66, 75, 78, 86, 89, 92, 96, 97, 107, 113, 124, 127, 130, 134, 135, 136, 145, 147, 152, 155, 158, 170], "combinedtimesteplabelembed": 134, "combinedtimesteptextprojembed": 134, "come": [4, 10, 16, 17, 19, 21, 84, 91, 94, 106, 110, 125, 126, 130, 132, 139, 143, 147], "comm": 147, "comma": [133, 138], "command": [2, 9, 17, 18, 21, 22, 24, 26, 27, 29, 30, 31, 32, 33, 35, 36, 38, 39, 58, 61, 62, 63, 64, 66, 68, 76, 78, 88, 89, 98, 99, 109, 110, 112, 115, 116, 117, 119, 120, 121, 123, 124, 125, 131, 136, 139, 143, 146, 151, 152, 156, 158, 165, 167], "commandr": 152, "comment": [35, 98, 152], "commerci": [31, 32], "commit": [2, 12, 15, 18, 20, 35, 36, 152], "commmod": 0, "common": [0, 2, 12, 15, 16, 19, 20, 22, 26, 33, 36, 49, 75, 77, 91, 97, 105, 108, 109, 113, 133, 139, 146, 147, 157, 169], "commonli": [8, 11, 13, 27, 107, 152], "commstat": 0, "commtyp": 0, "commun": [0, 8, 10, 11, 12, 15, 17, 21, 23, 31, 32, 81, 84, 97, 102, 106, 111, 117, 127, 133, 136, 142, 147, 152, 162], "communicationmod": 0, "communicationtyp": 0, "compact": [12, 147], "compani": [18, 58, 60, 84], "compar": [1, 2, 4, 5, 7, 10, 11, 12, 14, 15, 16, 17, 19, 20, 21, 24, 75, 89, 92, 118, 127, 130, 131, 132, 133, 147, 155], "comparison": [4, 13, 14, 39, 106, 124, 147], "compat": [9, 12, 14, 16, 17, 18, 20, 21, 22, 26, 27, 29, 30, 31, 32, 33, 82, 83, 84, 96, 98, 113, 119, 131, 134, 142, 146, 148, 151, 152, 153, 160, 163, 164], "compati": 26, "compatibal": 26, "compbin": 110, "compel": 8, "compelet": 30, "compet": [10, 16, 20], "competit": 21, "compil": [10, 18, 21, 29, 30, 31, 32, 33, 38, 39, 76, 77, 78, 80, 81, 106, 111, 112, 123, 124, 133, 143, 147, 152, 156, 157, 158, 160, 161, 162], "compile_backend": [76, 80, 82, 156, 160, 163], "compile_model": [76, 78, 156, 158, 160, 161], "complementari": 8, "complet": [0, 1, 8, 10, 11, 12, 16, 18, 20, 21, 26, 29, 30, 31, 32, 33, 34, 37, 39, 40, 41, 42, 44, 58, 66, 69, 70, 74, 77, 82, 83, 84, 88, 90, 91, 92, 96, 97, 98, 103, 106, 108, 109, 113, 124, 125, 130, 131, 147, 151, 152, 157, 160, 163, 164, 165, 166, 169, 170], "complete_sent": 60, "completion_token": [18, 21, 29, 30, 31, 32, 33, 151], "completionoutput": [44, 97, 147], "complex": [8, 11, 13, 16, 60, 78, 97, 100, 107, 108, 113, 117, 158], "compli": 27, "complic": [10, 14, 15, 16, 96, 148, 153], "compon": [7, 8, 10, 11, 12, 13, 14, 15, 16, 22, 34, 37, 75, 93, 96, 97, 103, 105, 117, 141, 154], "compos": [0, 11, 16, 20, 33, 97, 106, 124], "comprehens": [2, 8, 22, 27, 34, 76, 151, 156], "compress": [3, 15, 83, 164], "compris": [7, 8, 17], "comput": [0, 1, 3, 4, 5, 7, 8, 12, 13, 14, 15, 16, 17, 18, 20, 21, 23, 37, 38, 39, 49, 54, 55, 57, 58, 59, 60, 75, 81, 84, 86, 87, 90, 91, 92, 93, 97, 98, 101, 104, 105, 106, 109, 113, 117, 123, 124, 126, 127, 132, 133, 136, 139, 143, 144, 146, 147, 148, 151, 152, 153, 154, 155, 162, 166, 169], "computation": 8, "compute_relative_bia": 134, "computecontextlogit": 1, "computecountandindicedevic": 12, "computecumsumdevic": 12, "computed_block": 58, "computed_posit": 58, "computegenerationlogit": 1, "computenumpackedmask": 1, "concat": [13, 115, 133], "concat_kvcach": 13, "concat_qkv": 20, "concaten": [12, 13, 75, 105, 110, 118, 133, 147, 148, 153], "concept": [11, 17, 39, 92, 117, 124, 128, 129, 147, 152, 169], "conceptu": 1, "concern": [16, 117, 139], "concert": 8, "concis": 21, "conclud": [16, 20], "conclus": [19, 97, 128, 129], "concret": [16, 96, 147, 148, 153], "concurr": [1, 2, 4, 10, 11, 13, 14, 15, 16, 17, 18, 19, 21, 22, 26, 29, 30, 31, 32, 33, 37, 39, 84, 87, 88, 89, 90, 102, 113, 124, 152, 165, 166], "concurrency_": [26, 29, 30, 31, 32, 33], "concurrency_list": [26, 29, 30, 31, 32, 33], "cond_proj_dim": 134, "conda": 152, "condit": [0, 1, 8, 11, 16, 17, 35, 39, 84, 103, 106, 107, 113, 124, 133, 134, 152], "condition": [36, 133], "conditioning_embed": 134, "conditioning_embedding_dim": 134, "conduct": [12, 17, 20, 39, 84, 102, 105, 124], "confid": 11, "config": [0, 1, 2, 3, 9, 14, 15, 18, 20, 21, 22, 24, 26, 27, 39, 45, 51, 67, 75, 78, 84, 86, 87, 96, 105, 109, 110, 114, 115, 118, 119, 124, 130, 134, 135, 136, 138, 143, 147, 148, 152, 153, 158, 169], "config_cl": 147, "config_class": 135, "config_dir": 135, "config_fil": [27, 135, 147], "config_load": [83, 164], "configdict": 147, "configur": [0, 1, 2, 5, 17, 18, 19, 20, 22, 23, 26, 27, 34, 35, 39, 40, 52, 53, 59, 64, 75, 77, 80, 81, 82, 83, 84, 90, 91, 92, 95, 96, 97, 98, 101, 104, 105, 108, 112, 113, 118, 124, 125, 128, 129, 130, 132, 135, 138, 139, 143, 147, 152, 155, 157, 161, 162, 163, 164, 166], "configuration_llama": [148, 153], "configuration_mymodel": [148, 153], "configuration_util": [148, 153], "configuratorptr": 1, "confirm": [12, 49, 59, 101, 144, 146, 151], "conflict": 12, "conform": [10, 95, 147], "confus": 33, "congest": 16, "conjunct": 130, "connect": [0, 10, 16, 18, 20, 21, 29, 30, 31, 32, 33, 84, 102, 111, 117, 125, 126, 128, 129], "connecticut": 30, "connectioninfo": 0, "connectioninfotyp": 0, "connectionmanag": 0, "connector": [52, 147], "connector_cach": 58, "connector_cache_dir": 58, "connector_cache_fold": 58, "connector_cache_folder_kei": 58, "connector_modul": 58, "connector_scheduler_class": 58, "connector_worker_class": 58, "connectremoteag": 0, "consecut": 106, "consequ": [7, 126, 131], "conserv": [0, 35, 132], "consid": [0, 1, 2, 7, 8, 11, 12, 16, 17, 18, 34, 40, 50, 51, 64, 66, 91, 95, 110, 113, 130, 133, 147, 148, 153, 170], "consider": [7, 12, 16, 17, 20, 44, 84, 119], "consist": [4, 8, 11, 13, 20, 34, 39, 75, 107, 119, 124, 125, 133, 141, 143, 147, 152, 155], "consol": 120, "consolid": [16, 113], "const": [0, 1, 103], "const_iter": 1, "constant": [1, 16, 19, 20, 75, 105, 133, 139], "constant_to_tensor_": 133, "constantli": [49, 54, 55, 57, 101, 144, 146, 151], "constants_to_tensors_": 133, "constantthreshold": 1, "constexpr": [0, 1], "constitut": [17, 34], "constpointercast": 1, "constrain": [7, 10, 59, 92, 106, 152], "constraint": [0, 7, 10, 12, 16, 17, 75, 91, 92, 95, 105, 106, 133, 150], "construct": [0, 1, 11, 17, 19, 75, 80, 95, 96, 103, 113, 117, 124, 133, 147, 152, 155, 161], "constructor": [0, 64, 75, 97, 114, 146, 147, 151, 155], "consult": [38, 98, 113, 123], "consum": [0, 15, 16, 20, 35, 107, 133, 147], "consumpt": [4, 14, 23, 75, 87, 105, 136], "contact": 133, "contain": [0, 1, 10, 11, 12, 13, 16, 18, 20, 22, 23, 27, 35, 39, 40, 59, 61, 62, 63, 75, 81, 82, 83, 84, 86, 91, 97, 100, 101, 103, 105, 106, 107, 108, 110, 111, 116, 117, 118, 119, 121, 124, 133, 135, 136, 138, 141, 142, 145, 147, 151, 152, 154, 155, 162, 163, 164], "container_id": [18, 151], "container_imag": [61, 62, 63], "container_img": 27, "container_path": [29, 30, 32], "content": [1, 9, 18, 21, 24, 27, 29, 30, 31, 32, 33, 36, 41, 42, 43, 53, 58, 69, 70, 73, 84, 97, 102, 110, 119, 120, 133, 139, 147, 151, 152, 160], "content_typ": 147, "context": [0, 7, 10, 11, 14, 15, 16, 17, 19, 20, 23, 30, 33, 67, 84, 95, 97, 102, 104, 109, 122, 124, 128, 129, 133, 136, 138, 139, 143, 147, 150, 152, 155, 168, 169, 170], "context_and_gener": 147, "context_chunking_polici": [132, 147], "context_extra": 84, "context_fmha": [23, 91, 110, 136], "context_fmha_fp32_acc": 152, "context_fmha_typ": [105, 136, 139], "context_init": 170, "context_len": [75, 138, 155], "context_length": [133, 134, 138, 143], "context_logit": [29, 31, 32, 138, 147], "context_mem_s": 138, "context_onli": 147, "context_parallel_s": 147, "context_phas": [75, 105], "context_pre_onli": 134, "context_request": 170, "context_serv": [17, 84], "contextchunkingpolici": [0, 97, 132, 147, 152], "contextfmha": 1, "contextfmhatyp": 136, "contextidx": 0, "contextlogit": 0, "contextmanag": 146, "contextparallel": [0, 1], "contextphaseparam": [0, 147], "contextpositionid": 1, "contextprefillposit": 0, "contextrequest": 1, "contigu": [12, 84, 102, 108, 126, 133, 152], "continu": [1, 5, 7, 11, 12, 17, 20, 23, 37, 58, 75, 81, 91, 95, 97, 98, 103, 105, 113, 127, 130, 138, 147, 151, 162, 170], "contract": [39, 124], "contrast": [75, 106, 113, 155], "contrib": [3, 152], "contribut": [8, 10, 11, 14, 15, 16, 39, 81, 94, 119, 124, 133, 152, 162], "contributor": [8, 13, 16, 17, 139], "control": [0, 8, 18, 19, 20, 21, 26, 29, 30, 31, 32, 33, 44, 52, 66, 75, 76, 84, 91, 95, 97, 102, 105, 106, 107, 112, 123, 124, 125, 132, 133, 134, 138, 141, 147, 152, 156], "contronl": 11, "conv": 133, "conv1d": [23, 133, 134, 136], "conv2d": [133, 134], "conv3d": [133, 134], "conv_bia": 133, "conv_kernel": 138, "conv_stat": 135, "conv_state_or_ptr": 133, "conv_transpose2d": 133, "conv_weight": 133, "conveni": [1, 11, 98, 115, 119], "convent": [8, 83, 119, 133, 164], "converg": 20, "convers": [1, 6, 7, 17, 20, 64, 84, 87, 118, 150, 152], "convert": [0, 1, 12, 16, 20, 27, 39, 60, 75, 81, 83, 93, 110, 114, 115, 116, 117, 118, 119, 124, 125, 127, 143, 147, 152, 155, 162, 164], "convert_and_load_weights_into_trtllm_llama": 119, "convert_checkpoint": [104, 110, 114, 115, 116, 117, 119, 125, 126, 143, 152], "convert_enable_dis": 136, "convert_hf_mpt_legaci": 152, "convert_load_format": 147, "convert_util": 152, "convert_weights_from_custom_training_checkpoint": 119, "convkernel": 1, "convolut": [0, 138], "convtranspose2d": 134, "cooper": 97, "coordin": [12, 16, 20, 30, 33, 37, 97, 113, 133], "copi": [0, 1, 10, 11, 16, 23, 58, 75, 84, 86, 102, 109, 113, 121, 127, 133, 136, 139, 147, 152, 155], "copy_": 58, "copy_on_partial_reus": [86, 147], "copyfrom": 1, "copyonpartialreus": 0, "copytask": 1, "copytaskmappag": 1, "copyto": 0, "copytocpu": 0, "copytogpu": 0, "copytomanag": 0, "copytopag": 1, "copytopin": 0, "copytopooledpin": 0, "core": [3, 4, 6, 8, 10, 12, 15, 20, 21, 24, 37, 40, 78, 81, 82, 86, 94, 97, 98, 106, 107, 110, 114, 117, 119, 124, 126, 136, 143, 147, 152, 154, 158, 162, 163], "corner": [15, 30], "coroutin": [55, 56, 147], "corpor": 58, "correct": [10, 11, 12, 14, 19, 20, 33, 75, 81, 94, 103, 105, 110, 113, 152, 162], "correctli": [10, 84, 102, 109, 133, 148, 152, 153], "correl": 19, "correspond": [0, 1, 8, 10, 12, 14, 16, 17, 19, 27, 38, 75, 92, 96, 99, 104, 105, 107, 108, 110, 113, 118, 119, 123, 131, 133, 134, 138, 141, 143, 147, 148, 152, 153], "correspondingli": 16, "corridor": 30, "corrupt": 16, "cost": [8, 13, 14, 15, 16, 19, 21, 39, 60, 93, 109, 117, 124, 126, 139, 152], "costli": 13, "cot": [97, 152], "could": [0, 10, 12, 16, 20, 21, 33, 49, 54, 55, 56, 57, 84, 96, 98, 101, 107, 108, 109, 116, 125, 139, 143, 144, 146, 147, 151, 152], "couldn": [91, 130], "count": [0, 1, 8, 16, 20, 40, 47, 48, 60, 92, 96, 106, 124, 146, 147], "count_include_pad": [133, 134], "countlocallay": 1, "countlowerranklay": 1, "coupl": 10, "cours": 113, "court": [49, 101, 144, 146, 151], "cover": [2, 8, 16, 21, 26, 29, 30, 31, 32, 33, 35, 127, 128, 129, 131], "coverag": [10, 16], "cp312": 98, "cp_config": 147, "cp_group": [133, 134], "cp_rank": [133, 134], "cp_size": [133, 134, 137, 152], "cp_split_plugin": 133, "cpp": [2, 15, 38, 39, 40, 62, 75, 98, 103, 105, 106, 112, 117, 123, 124, 125, 138, 143, 152], "cpp_e2e": 138, "cpp_llm_onli": 138, "cpp_onli": 98, "cpu": [0, 1, 13, 14, 16, 19, 21, 22, 23, 27, 37, 58, 59, 75, 86, 88, 89, 90, 97, 108, 109, 110, 114, 117, 133, 139, 143, 147, 152, 155, 165, 166], "cpu_tensor": 58, "cpumemusag": [0, 147], "craft": 12, "crash": 152, "crd": 46, "creat": [1, 8, 10, 11, 13, 18, 19, 20, 26, 27, 36, 39, 40, 44, 46, 49, 50, 54, 55, 56, 57, 60, 69, 70, 71, 72, 73, 75, 78, 84, 86, 88, 91, 92, 95, 96, 97, 98, 101, 102, 103, 107, 108, 109, 113, 114, 115, 117, 119, 120, 124, 125, 130, 131, 133, 134, 135, 136, 138, 139, 144, 146, 147, 148, 151, 152, 153, 154, 155, 158, 165, 170], "create_allreduce_plugin": 133, "create_attention_const_param": 134, "create_builder_config": 114, "create_cuda_graph_metadata": [75, 96, 155], "create_execution_context": 138, "create_fake_weight": 133, "create_from_prompt": 11, "create_network": 117, "create_pytorch_model_based_executor": [169, 170], "create_runtime_default": 135, "create_scaffolding_output": 11, "create_sinusoidal_posit": 133, "create_sinusoidal_positions_for_attention_plugin": 133, "create_sinusoidal_positions_for_cogvlm_attention_plugin": 133, "create_sinusoidal_positions_long_rop": 133, "create_sinusoidal_positions_long_rope_for_attention_plugin": 133, "create_sinusoidal_positions_yarn": 133, "createloramodul": 1, "creation": [1, 37, 76, 133, 139, 147, 156], "creativ": [66, 106], "creator": [1, 147], "creatorptr": 1, "credenti": 9, "criteria": [37, 90, 138, 166], "critic": [8, 12, 13, 16, 17, 37, 39, 124, 143], "crop": 134, "cropped_pos_emb": 134, "cross": [0, 10, 11, 13, 14, 16, 30, 110, 111, 133, 138, 147, 152], "cross_attent": [134, 138], "cross_attention_dim": 134, "cross_attention_mask": [134, 138], "cross_attention_mask_for_context": 138, "cross_attention_mask_for_gen": 138, "cross_attention_norm": 134, "cross_attention_norm_num_group": 134, "cross_attention_packed_mask": 134, "cross_attn_dens": [23, 110], "cross_attn_k": [23, 110], "cross_attn_q": [23, 110], "cross_attn_qkv": [23, 110], "cross_attn_v": [23, 110], "cross_kv": 133, "cross_kv_cache_block_offset": [134, 138], "cross_kv_cache_fract": [138, 147], "cross_kv_cache_gen": [134, 135], "cross_kv_length": 133, "cross_kv_reus": [134, 135], "crossattentionmask": 0, "crosskvcachefract": [0, 152], "crosskvcachestat": 0, "crucial": [7, 8, 10, 33, 59, 113, 117, 154], "csv": 22, "ct": 33, "cta": 20, "ctor": 133, "ctrl": 26, "ctx": [0, 2, 8, 17], "ctx1dep4": 17, "ctx_len": 8, "ctx_param": [17, 84], "ctx_request_id": 147, "ctxenginepath": 0, "ctxexecutorconfig": 0, "ctxreqrat": 17, "cu": [13, 117], "cu12": 152, "cu128": 101, "cuassert": 143, "cubin": 152, "cubla": [15, 21], "cublaslt": [23, 131, 136], "cublasltmatmul": 15, "cublasscaledmm": 15, "cuda": [0, 1, 2, 12, 14, 15, 16, 18, 20, 21, 22, 26, 29, 30, 31, 32, 33, 38, 39, 58, 60, 65, 67, 75, 82, 84, 85, 95, 96, 97, 98, 101, 102, 105, 111, 117, 123, 124, 135, 138, 139, 140, 143, 147, 149, 152, 155, 160, 163, 169], "cuda_arch": 98, "cuda_architectur": [2, 98, 112], "cuda_graph": 65, "cuda_graph_batch_s": [26, 29, 30, 31, 32, 33, 76, 147, 152, 156, 160], "cuda_graph_cache_s": 147, "cuda_graph_config": [2, 9, 14, 15, 21, 40, 58, 62, 65, 67, 95, 147, 152], "cuda_graph_inst": 143, "cuda_graph_mod": [138, 143, 147], "cuda_hom": 101, "cuda_launch_block": 143, "cuda_stream": 143, "cuda_stream_guard": 138, "cuda_stream_sync": 133, "cuda_visible_devic": 84, "cudadevicegetstreampriorityrang": 1, "cudaevent_t": 1, "cudaeventdisabletim": 1, "cudagraph": [10, 76, 81, 82, 152, 156, 162, 163], "cudagraphcaches": 0, "cudagraphconfig": [65, 97, 147], "cudagraphlaunch": [20, 143], "cudagraphmod": 0, "cudagraphrunn": 96, "cudagriddependencysynchron": 12, "cudahostregist": 20, "cudalaunchhostfunc": 10, "cudamalloc": [1, 20, 84, 102], "cudamallocasync": [1, 84, 102], "cudamallocmanag": 20, "cudamemadvis": 20, "cudamempool": 1, "cudamempoolptr": 1, "cudaprofilerapi": [38, 123], "cudart": 143, "cudastream": 0, "cudastream_t": 1, "cudastreamcreatewithflag": 1, "cudastreamnonblock": 1, "cudastreamptr": [0, 1], "cudatriggerprogrammaticlaunchcomplet": 12, "cudavirtualmemori": 1, "cudavirtualmemoryalloc": 1, "cudavirtualmemorychunk": 1, "cudavirtualmemorymanag": 1, "cudevic": 1, "cudeviceptr": 1, "cudnn": [21, 152], "cufil": 0, "cuh": 13, "cumemaccessdesc": 1, "cumemallocationprop": 1, "cumemcr": 1, "cumemgenericallocationhandl": 1, "cumemimportfromshareablehandl": [84, 102], "cumlogprob": [0, 1], "cumlogprobscba": 1, "cumsum": [133, 152], "cumsumgenerationlength": 1, "cumsumlastdim": 133, "cumsumlength": 1, "cumul": [0, 1, 8, 66, 133, 147], "cumulative_logprob": [44, 147], "cupi": 10, "curand": 152, "curl": [9, 18, 21, 27, 29, 30, 31, 32, 33, 74, 84, 89, 97, 151, 160], "currenc": [39, 124], "current": [0, 1, 2, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 21, 23, 24, 27, 36, 37, 39, 44, 53, 64, 75, 78, 81, 83, 84, 86, 87, 90, 91, 94, 95, 96, 98, 99, 102, 103, 105, 110, 113, 124, 127, 130, 131, 132, 133, 136, 138, 139, 145, 147, 152, 154, 155, 158, 162, 164, 169, 170], "current_image_tag": 36, "current_prompt": 11, "current_stream": 143, "currentexpandindic": 1, "curti": 58, "curv": [6, 10, 17, 20], "custom": [3, 10, 11, 13, 14, 16, 17, 22, 23, 36, 46, 49, 60, 66, 75, 78, 80, 81, 82, 84, 95, 96, 97, 98, 106, 117, 119, 127, 131, 133, 136, 138, 146, 147, 150, 152, 154, 155, 158, 161, 162, 163], "custom_all_reduc": 152, "custom_format": [83, 164], "custom_mask": 133, "custom_module_dir": 22, "customallreduc": 152, "customcheckpointload": [83, 164], "customconfigload": [83, 164], "customdataset": 152, "customized_key_dict": 118, "customized_preprocess": 118, "customizedmodulea": 118, "customizedmoduleb": 118, "customweightload": [83, 164], "customweightmapp": [83, 164], "custream": 1, "cut": 21, "cutedsl": 147, "cutlass": [15, 21, 26, 29, 30, 31, 32, 33, 112, 147, 152], "cutlass_kernel": 112, "cxx11": [98, 152], "cycl": [20, 59], "cyclic": [8, 87, 138, 152], "d": [1, 9, 18, 21, 29, 30, 31, 32, 33, 39, 41, 42, 43, 61, 62, 63, 64, 84, 110, 112, 120, 124, 133, 134, 143, 151, 152, 160], "d0": 13, "d04e592bb4f6aa9cfee91e2e20afa771667e1d4b": 124, "d1": 96, "d2": 96, "d2h": 10, "d2t": 10, "d3": 96, "d_": 14, "d_6": 14, "dai": [81, 150, 162], "dailymail": 24, "dangl": 107, "data": [0, 1, 3, 4, 5, 6, 7, 8, 11, 12, 13, 16, 17, 19, 20, 21, 22, 23, 24, 26, 27, 29, 31, 32, 36, 39, 40, 58, 60, 67, 70, 75, 84, 85, 86, 89, 93, 102, 105, 106, 108, 111, 117, 118, 124, 125, 133, 135, 142, 143, 147, 148, 149, 152, 153], "data_devic": 22, "data_path": 62, "data_ptr": 152, "data_typ": [114, 116], "databas": [83, 164], "dataclass": [34, 58], "datacontext": 0, "dataset": [10, 11, 13, 14, 15, 18, 20, 21, 24, 26, 29, 30, 31, 32, 33, 38, 42, 62, 70, 76, 88, 92, 97, 123, 127, 147, 152, 156, 165], "dataset_fil": 40, "dataset_path": [24, 88, 124, 165], "datatyp": [0, 1, 106, 117, 133, 138, 141, 143], "datatypetrait": 1, "date": 119, "datetim": 147, "db": 35, "dbrx": [141, 142, 152], "dbrxconfig": 135, "dbrxforcausallm": 135, "dconv": 133, "de": 1, "deactiv": 44, "dead": 152, "deal": [75, 105, 107, 143], "dealloc": [1, 37, 108, 170], "death": [49, 101, 144, 146, 151], "debug": [0, 16, 20, 22, 23, 24, 27, 59, 79, 82, 86, 98, 108, 136, 138, 139, 147, 152, 159, 163], "debug_buff": 143, "debug_mod": [138, 143], "debug_tensors_to_sav": 138, "debugconfig": 0, "debuginputtensor": 0, "debugoutputtensor": 0, "debugtensor": 0, "debugtensornam": 0, "debugtensorsmaxiter": 0, "debugtensorsperiter": 0, "dec": [23, 138, 152], "decai": [0, 106, 147], "decid": [16, 39, 50, 75, 93, 105, 116, 124, 128, 130, 141, 154, 170], "decilmforcausallm": [142, 149], "decim": 147, "decis": [8, 16, 20, 64, 133], "declar": [1, 106, 107, 119, 154, 169], "decltyp": [0, 1], "decod": [0, 1, 8, 12, 13, 15, 16, 17, 18, 24, 27, 37, 52, 59, 66, 73, 75, 81, 84, 85, 87, 89, 90, 97, 102, 105, 106, 115, 119, 124, 133, 135, 136, 138, 142, 144, 146, 147, 148, 149, 150, 152, 153, 162, 166, 168, 169], "decode_batch": 138, "decode_duration_m": [51, 86, 147], "decode_regular": 138, "decode_retention_polici": 86, "decode_retention_prior": [51, 147], "decode_stream": 138, "decode_words_list": 138, "decode_wrapp": [75, 155], "decodedurationm": 0, "decoder_batch": 1, "decoder_input_id": [135, 138], "decoder_language_adapter_rout": 138, "decoder_lay": [148, 153], "decoder_start_token_id": 23, "decoderbuff": 1, "decoderenginebuff": 0, "decoderetentionprior": 0, "decoderjsonconfigstr": 0, "decoderlay": [148, 153], "decoderlayerlist": 115, "decoderlookaheadbuff": 1, "decodermaskedmultiheadattent": [75, 105], "decodermodel": [0, 135, 148, 153], "decodermodelforcausallm": [115, 119, 135, 148, 153], "decodermodelpath": 0, "decoderst": 152, "decoderxqarunn": [75, 105], "decoding_config": 147, "decoding_typ": [2, 9, 14, 18, 29, 96, 147], "decodingbaseconfig": 147, "decodingconfig": [0, 1], "decodinginputptr": 1, "decodingit": 0, "decodinglayerworkspac": 1, "decodingmod": [0, 1, 152], "decodingoutputptr": 1, "decompos": [16, 75, 91, 105], "decomposit": [88, 165], "decor": [10, 12, 34, 147, 148, 153], "decoupl": [10, 11, 13, 16, 17, 84, 87, 92, 112, 139], "decreas": [2, 3, 4, 8, 10, 20, 79, 127, 159], "dedic": [8, 10, 13, 15, 16, 17, 20, 21, 37, 143], "deduc": [16, 23, 24, 27, 152], "deem": 11, "deep": [4, 5, 12, 18, 21, 38, 97, 117, 123, 133, 147, 152], "deepcopi": 11, "deepep": [16, 152], "deeper": [12, 14], "deepgemm": [2, 29, 147, 152], "deeplearn": [117, 133, 143], "deepli": 16, "deepseek": [8, 10, 11, 12, 16, 20, 27, 28, 38, 68, 74, 82, 92, 93, 94, 96, 97, 123, 142, 149, 150, 152, 163], "deepseek_model": 96, "deepseek_r1_output": 29, "deepseek_v1": 152, "deepseek_v2": 152, "deepseek_v3": [13, 152], "deepseekforcausallm": 135, "deepseekv1config": 135, "deepseekv2": 133, "deepseekv2attent": 134, "deepseekv2config": 135, "deepseekv2forcausallm": 135, "deepseekv3forcausallm": [142, 149], "deepseekv3routingimpl": 15, "deepspe": 116, "def": [10, 11, 34, 49, 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 64, 65, 66, 67, 68, 83, 95, 101, 107, 115, 117, 118, 119, 125, 127, 131, 132, 143, 144, 146, 147, 148, 151, 153, 164, 170], "default": [0, 1, 10, 11, 14, 15, 19, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 44, 50, 51, 59, 60, 64, 65, 66, 67, 68, 75, 76, 77, 79, 82, 83, 84, 86, 87, 90, 91, 92, 93, 94, 95, 96, 98, 99, 102, 103, 104, 105, 106, 109, 112, 116, 118, 119, 120, 123, 127, 128, 129, 130, 131, 132, 133, 135, 136, 138, 139, 141, 143, 147, 148, 152, 153, 155, 156, 157, 159, 163, 164, 166], "default_factori": 58, "default_net": 133, "default_plugin_config": 135, "default_record_cr": 147, "default_trtnet": 117, "defaultvalu": 1, "defer": [20, 133], "defin": [0, 1, 2, 5, 8, 11, 16, 22, 23, 34, 35, 36, 39, 59, 75, 78, 83, 84, 86, 91, 95, 96, 103, 105, 107, 113, 116, 117, 118, 119, 124, 131, 133, 134, 136, 141, 147, 148, 150, 152, 153, 155, 158, 164], "definit": [11, 13, 46, 75, 81, 83, 97, 103, 105, 108, 119, 133, 143, 150, 152, 162, 164], "defrag": 20, "deftruth": 152, "degrad": [0, 18, 23, 127], "degre": [16, 36, 40, 49, 54, 55, 57, 101, 127, 131, 144, 146, 151], "del": 58, "delai": [8, 16, 17, 20, 26, 29, 30, 31, 32, 40, 84, 147, 152], "deleg": [75, 133, 155], "delet": [0, 1, 16, 136, 143, 147], "delimit": 147, "deliv": [2, 3, 6, 8, 12, 13, 14, 19, 21, 26, 40, 92, 150], "delta": [0, 13, 14, 133, 134], "delta_bia": 133, "delta_softplu": 133, "delv": 15, "demand": [13, 15, 16, 17, 60, 84, 92], "demo": [11, 13, 42, 59, 66, 70, 81, 162], "demo_prompt": 66, "demollm": [76, 77, 82, 156, 157, 163], "demonstr": [4, 8, 10, 11, 12, 13, 16, 17, 20, 21, 37, 59, 65, 66, 67, 80, 89, 95, 103, 118, 125, 127, 130, 131, 146, 161], "demonstrate_beam_search": 66, "demonstrate_combined_sampl": 66, "demonstrate_greedy_decod": 66, "demonstrate_multiple_sequ": 66, "demonstrate_temperature_sampl": 66, "demonstrate_top_k_sampl": 66, "demonstrate_top_p_sampl": 66, "demonstrate_with_logprob": 66, "denois": 134, "denot": 113, "dens": [30, 40, 75, 104, 105, 110, 116, 118, 133, 152], "dense_4h_to_h": 118, "dense_bia": 134, "dense_h_to_4h": 118, "densiti": [7, 21], "dep": 98, "dep4": [17, 30], "dep8": [17, 30], "depend": [0, 5, 10, 16, 17, 18, 20, 24, 27, 30, 33, 40, 50, 75, 84, 87, 95, 98, 101, 103, 105, 106, 107, 112, 113, 116, 125, 127, 131, 133, 139, 143, 147, 152, 169], "deploi": [11, 12, 16, 20, 27, 46, 81, 82, 92, 97, 113, 116, 162, 163], "deplot": [142, 152], "deploy": [7, 8, 13, 16, 17, 18, 19, 20, 21, 46, 78, 82, 84, 92, 124, 127, 145, 146, 150, 151, 152, 158, 163], "deprec": [23, 112, 124, 147, 152], "deprecatedparseprotocol": 147, "deprecationwarn": [34, 124], "depriorit": 112, "depriv": 107, "depth": [82, 113, 147, 163], "dequ": [0, 1], "dequant": [12, 75, 105, 111, 133], "deregistermemori": 0, "deriv": [12, 19, 96, 117, 118, 133, 139, 154], "desc": [0, 1], "descend": 86, "descendli": 106, "describ": [0, 2, 6, 16, 17, 21, 35, 39, 40, 42, 66, 70, 75, 86, 96, 98, 101, 105, 106, 108, 109, 110, 113, 115, 117, 118, 121, 124, 131, 133, 141, 143, 155], "descript": [0, 1, 22, 26, 27, 29, 30, 31, 32, 33, 34, 39, 40, 59, 64, 65, 75, 76, 77, 82, 97, 106, 110, 124, 131, 133, 136, 147, 155, 156, 157, 163], "descriptor": 147, "deseri": [0, 20, 119], "deserializeadditionalmodeloutput": 0, "deserializeadditionaloutput": 0, "deserializeagentst": 0, "deserializeblockkei": 0, "deserializebool": 0, "deserializecachest": 0, "deserializecachetransceiverconfig": 0, "deserializecommst": 0, "deserializecontextphaseparam": 0, "deserializedatatransceiverst": 0, "deserializedebugconfig": 0, "deserializedecodingconfig": 0, "deserializedecodingmod": 0, "deserializedisservingrequeststat": 0, "deserializedynamicbatchconfig": 0, "deserializeeagleconfig": 0, "deserializeexecutorconfig": 0, "deserializeextendedruntimeperfknobconfig": 0, "deserializeexternaldrafttokensconfig": 0, "deserializeguideddecodingconfig": 0, "deserializeguideddecodingparam": 0, "deserializeinflightbatchingstat": 0, "deserializeiterationstat": 0, "deserializeiterationstatsvec": 0, "deserializekvcacheconfig": 0, "deserializekvcachecreateddata": 0, "deserializekvcacheev": 0, "deserializekvcacheeventdiff": 0, "deserializekvcacheremoveddata": 0, "deserializekvcacheretentionconfig": 0, "deserializekvcachestat": 0, "deserializekvcachestoredblockdata": 0, "deserializekvcachestoreddata": 0, "deserializekvcacheupdateddata": 0, "deserializelookaheaddecodingconfig": 0, "deserializeloraconfig": 0, "deserializemodeltyp": 0, "deserializemropeconfig": 0, "deserializemultimodalinput": 0, "deserializeorchestratorconfig": 0, "deserializeoutputconfig": 0, "deserializeparallelconfig": 0, "deserializepeftcacheconfig": 0, "deserializeprompttuningconfig": 0, "deserializerequest": 0, "deserializerequestperfmetr": 0, "deserializerequeststag": 0, "deserializerequeststat": 0, "deserializerequeststatsperiter": 0, "deserializerequeststatsperiterationvec": 0, "deserializerespons": 0, "deserializeresult": 0, "deserializesamplingconfig": 0, "deserializeschedulerconfig": 0, "deserializesocketst": 0, "deserializespecdecfastlogitsinfo": 0, "deserializespecdecodingstat": 0, "deserializespeculativedecodingconfig": 0, "deserializestaticbatchingstat": 0, "deserializestr": 0, "deserializetensor": 0, "deserializetimepoint": 0, "deserializetokenrangeretentionconfig": 0, "deserializeuniquetoken": 0, "design": [1, 2, 7, 8, 10, 11, 12, 13, 14, 15, 17, 18, 19, 20, 22, 37, 59, 75, 81, 82, 83, 84, 86, 92, 97, 111, 113, 117, 118, 119, 125, 146, 150, 151, 154, 155, 162, 163, 164, 169], "desir": [22, 36, 40, 75, 92, 95, 103, 133, 147, 155, 164], "desired_world_s": [80, 161], "despit": [8, 11], "destin": [61, 62, 63], "destroi": [1, 139], "destroyipcmemori": 1, "destruct": 1, "destructor": 1, "detach": 18, "detail": [0, 2, 8, 10, 11, 12, 13, 15, 16, 17, 20, 21, 23, 24, 27, 29, 33, 35, 36, 39, 40, 44, 46, 49, 53, 58, 75, 84, 86, 89, 92, 95, 103, 105, 111, 113, 115, 117, 124, 125, 127, 132, 133, 135, 139, 143, 147, 151, 152, 154, 155, 169], "detect": [0, 16, 20, 22, 24, 27, 36, 103, 133, 147, 152], "detect_format": 118, "detect_shard": [78, 158, 161], "determin": [0, 1, 11, 14, 16, 17, 20, 37, 75, 81, 82, 84, 86, 87, 91, 96, 105, 106, 110, 119, 126, 127, 132, 133, 135, 141, 147, 154, 162, 163, 169, 170], "determinenumpag": 1, "determinist": [66, 131, 147, 152], "detoken": [20, 21, 37, 147, 152, 154], "detokenizedgenerationresultbas": 147, "dev": [9, 16, 18, 21, 24, 29, 30, 31, 32, 33, 81, 97, 101, 152, 162], "dev_container_imag": 36, "dev_trtllm_imag": 98, "devcontain": 36, "devel": [98, 120, 121], "develop": [8, 11, 13, 14, 16, 17, 18, 20, 21, 24, 29, 30, 31, 32, 34, 35, 36, 49, 54, 55, 57, 58, 81, 82, 84, 89, 94, 98, 101, 115, 116, 117, 119, 120, 125, 133, 142, 144, 146, 148, 150, 151, 152, 153, 162, 163], "deviat": [16, 22, 40], "devic": [0, 1, 10, 16, 17, 20, 21, 22, 29, 30, 58, 60, 84, 102, 127, 133, 135, 137, 138, 143, 147], "device_cache_perc": [88, 165], "device_id": 138, "device_map": 137, "device_memory_size_v2": 139, "device_num_expert": 133, "device_request_typ": 135, "deviceallocationnvl": 1, "devicecach": 1, "devicecacheperc": 0, "deviceid": [0, 1, 102], "dgx": [2, 15, 21, 35, 40, 106, 117], "di": [14, 16, 17, 84], "diagnost": 1, "diagon": 133, "diagram": [11, 12, 15, 17, 84, 113], "dial": 50, "diamond": [13, 15, 24], "dict": [34, 78, 83, 115, 118, 119, 133, 135, 138, 147, 148, 152, 153, 158, 164, 169], "dict_kei": 143, "dictat": 130, "dictionari": [19, 78, 83, 116, 118, 134, 147, 158, 164], "did": 12, "didn": [91, 130], "differ": [0, 1, 2, 7, 8, 11, 12, 14, 15, 16, 17, 19, 20, 21, 22, 23, 26, 29, 30, 31, 32, 33, 39, 40, 42, 59, 66, 70, 75, 78, 82, 83, 84, 87, 88, 91, 92, 93, 95, 96, 98, 102, 104, 105, 106, 108, 109, 111, 115, 116, 117, 118, 119, 124, 125, 127, 130, 131, 133, 135, 136, 138, 139, 141, 147, 150, 151, 152, 155, 158, 163, 164, 165, 168], "differenti": 133, "difficult": [8, 10], "difficulti": [11, 96], "difftyp": 1, "diffus": [42, 70, 134, 152], "diffusersattent": 134, "difi": 11, "digit": [19, 147], "dilat": [133, 134], "dim": [0, 1, 133, 134, 135, 138, 143], "dim0": 133, "dim1": 133, "dim_head": 134, "dim_in": 134, "dim_out": 134, "dim_rang": 133, "dimems": 1, "dimens": [0, 1, 12, 15, 16, 26, 75, 77, 105, 106, 110, 133, 134, 135, 139, 143, 148, 152, 153, 157], "dimension": 133, "diminish": [11, 16], "dimrang": 133, "dimtype64": [0, 1], "dir": [9, 22, 26, 29, 30, 31, 32, 33, 39, 44, 98, 124], "direct": [0, 12, 17, 30, 78, 84, 101, 102, 111, 119, 143, 158], "directli": [0, 10, 11, 12, 14, 15, 16, 17, 20, 29, 30, 31, 32, 33, 34, 44, 75, 81, 83, 84, 86, 93, 96, 98, 102, 106, 107, 113, 117, 119, 121, 124, 131, 132, 133, 146, 147, 151, 152, 155, 162, 164, 170], "directori": [0, 9, 16, 20, 21, 22, 23, 29, 30, 31, 32, 39, 40, 61, 62, 63, 64, 81, 83, 98, 103, 115, 116, 117, 118, 119, 124, 125, 135, 138, 146, 147, 148, 152, 153, 162, 164], "dirnam": 58, "disabl": [0, 1, 10, 12, 16, 21, 22, 23, 24, 27, 36, 50, 59, 84, 90, 95, 96, 105, 106, 109, 114, 118, 124, 127, 131, 132, 133, 136, 138, 139, 147, 152, 160, 166], "disable_chunked_context": 22, "disable_finalize_fus": 147, "disable_forward_chunk": 135, "disable_kv_cach": 138, "disable_kv_cache_reus": 24, "disable_overlap_schedul": [15, 68, 84, 90, 95, 96, 147, 166], "disable_weight_only_quant_plugin": 135, "disable_xqa": 105, "disablelookahead": 1, "disablelookaheaddecod": 1, "disableseamlesslookaheaddecod": 1, "disadvantag": [119, 126], "disagg": [94, 101, 152], "disagg_config": [17, 84], "disagg_executor": 0, "disaggexecutororchestr": 0, "disaggreg": [0, 8, 10, 20, 46, 85, 94, 97, 147, 149, 150, 152, 160], "disaggregated_param": [21, 29, 30, 31, 32, 33, 147], "disaggregatedparam": [97, 147], "disaggserverbenchmark": 152, "disallow": [10, 147], "disappear": 10, "discard": [11, 59, 127, 147], "disclaim": [14, 125, 127, 130, 131], "disclosur": 152, "disconnect": 152, "discourag": [0, 60, 106, 147], "discov": [10, 20, 66, 101, 117], "discoveri": 36, "discrep": [17, 98, 148, 153], "discret": 86, "discuss": [11, 14, 16, 20, 105, 125, 127, 131, 132, 152], "disk": [83, 98, 103, 119, 164], "dispar": 8, "dispatch": [0, 11, 12, 13, 16, 17, 44, 84, 92, 96, 104, 119, 152], "displai": [8, 16, 147], "disrupt": 92, "disservingrequeststat": 0, "disservingstat": 0, "dist": [2, 38, 39, 40, 62, 123, 124, 125], "distanc": [20, 75, 105, 133], "distil": [11, 82, 152, 163], "distinct": [8, 13, 17, 59, 83, 108, 110, 113, 133, 164], "distinguish": [33, 109], "distribut": [1, 13, 16, 19, 22, 26, 52, 61, 75, 91, 92, 94, 104, 105, 106, 117, 124, 133, 138, 139, 144, 146, 150], "distserv": 102, "dit": [135, 152], "div": 133, "dive": [12, 14, 38, 97, 123], "diverg": [78, 158], "divers": [0, 8, 38, 106, 123], "diversity_penalti": 106, "divid": [11, 12, 14, 16, 86, 87, 118, 133, 152], "divup": 133, "dl": 7, "dlsym": 0, "do": [1, 2, 7, 10, 11, 12, 13, 14, 15, 16, 17, 20, 21, 22, 26, 29, 30, 31, 32, 33, 35, 44, 58, 75, 84, 93, 96, 97, 98, 102, 107, 118, 119, 125, 127, 131, 133, 143, 147, 148, 153, 155], "do_cross_attent": [133, 134], "do_layer_norm_befor": 116, "do_sampl": 106, "doactivationkernel": 20, "doc": [1, 2, 6, 13, 16, 21, 117, 121, 127, 131, 133, 143, 152], "docker": [2, 9, 26, 61, 62, 63, 97, 143, 152], "docker_run_arg": 2, "dockerfil": [98, 120], "docstr": 34, "document": [0, 4, 5, 7, 10, 12, 14, 16, 17, 22, 26, 29, 34, 38, 39, 40, 41, 42, 43, 44, 45, 47, 48, 53, 69, 70, 71, 72, 73, 75, 84, 87, 88, 98, 99, 102, 105, 106, 108, 109, 110, 113, 115, 116, 117, 119, 122, 123, 125, 132, 133, 139, 141, 143, 147, 154, 155, 165], "doe": [0, 2, 3, 10, 11, 15, 16, 19, 23, 35, 36, 39, 40, 58, 67, 72, 75, 76, 84, 91, 96, 99, 102, 105, 106, 110, 113, 119, 124, 131, 133, 138, 139, 142, 147, 148, 152, 153, 156, 170], "doesn": [1, 8, 13, 20, 29, 30, 31, 32, 37, 39, 44, 68, 75, 87, 105, 120, 124, 130, 131, 152, 160], "dog": 33, "dollar": [39, 124], "domain": [16, 20, 84, 102, 111], "domin": [8, 13, 16, 152], "don": [8, 9, 11, 15, 16, 58, 84, 92, 102, 113, 119, 120, 126, 131, 133, 138], "done": [1, 2, 15, 16, 17, 18, 26, 29, 30, 31, 32, 33, 50, 84, 91, 96, 109, 117, 124, 127, 130, 133, 136, 147, 148, 153], "dongjiyingdji": 152, "dora": [23, 133, 134, 136], "dora_plugin": [23, 110, 133, 136], "dot": [13, 19, 118, 133], "doubl": [0, 4, 11, 19, 33, 128, 129, 131, 143], "doubt": 11, "down": [0, 3, 10, 11, 14, 15, 16, 20, 33, 64, 103, 110, 126, 133, 138], "down_proj": 118, "downgrad": 152, "download": [21, 24, 29, 30, 31, 32, 33, 39, 61, 62, 63, 64, 68, 97, 98, 100, 101, 124, 125, 143, 146, 152], "downsampl": 12, "downscale_freq_shift": 134, "downsid": 131, "downstream": [89, 141], "dp": [0, 2, 3, 6, 8, 12, 13, 15, 17, 21, 30, 33, 147, 152], "dp4ep4": 21, "dp8": [13, 15], "dprank": 0, "dpsize": 0, "dpu": 21, "draft": [0, 1, 13, 14, 18, 19, 20, 23, 97, 135, 138, 147, 152], "draft_len": 135, "draft_model": 96, "draft_path": 138, "draft_target": [68, 147], "draft_target_model": 113, "draft_token": [135, 147], "draft_tokens_extern": [23, 135], "draftacceptancethreshold": 1, "draftbuff": 1, "drafter": [10, 96, 113, 147, 152], "draftindic": 1, "draftlen": 1, "draftlogit": 1, "draftlogitshost": 1, "draftoverhead": 0, "draftparticipantid": 0, "draftpath": 1, "draftpathshost": 1, "draftprob": 1, "draftrequestid": 0, "drafttarget": 96, "drafttargetdecodingconfig": [96, 97, 147], "drafttoken": [0, 1], "drafttokenid": 1, "drafttokenidshost": 1, "drafttokensextern": 1, "dram": [0, 117], "dramat": [8, 87], "drastic": [15, 37], "drat": 96, "draw": 19, "dreamgenx": 152, "drive": [60, 117, 124], "driven": [8, 16], "driver": [16, 20, 29, 30, 31, 32, 33, 37, 84, 102, 139, 152], "drop": [2, 8, 12, 14, 15, 84, 102, 127, 130, 132], "dropout": [134, 152], "dropout_prob": 134, "dry": [33, 147], "dry_run": [23, 147, 152], "dst": 1, "dstate": 133, "dstdesc": 0, "dsttype": 1, "dt_proj": 133, "dt_rank": 133, "dtype": [1, 10, 12, 26, 29, 31, 32, 39, 67, 86, 93, 107, 110, 114, 115, 116, 117, 119, 124, 125, 126, 133, 134, 135, 136, 137, 138, 143, 147, 149, 152, 169], "dual": 98, "duck": 147, "due": [0, 1, 2, 5, 8, 11, 12, 13, 15, 16, 18, 20, 21, 27, 29, 30, 31, 32, 33, 35, 36, 39, 59, 75, 91, 92, 95, 98, 112, 113, 119, 124, 125, 130, 132, 138, 152, 155, 168], "duke": 30, "dummi": [22, 88, 125, 147, 152, 165], "dump": [0, 16, 98, 103, 147], "dump_debug_buff": 138, "dumps_kwarg": 147, "duplic": [15, 20, 78, 152, 158], "duplicate_data": 133, "durat": [0, 12, 16, 20, 26, 29, 30, 31, 32, 86, 125], "duration_cast": 0, "duration_m": [86, 147], "durationm": 0, "dure": [0, 1, 6, 8, 10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 23, 35, 37, 38, 39, 75, 76, 77, 83, 84, 86, 88, 91, 95, 96, 98, 102, 105, 106, 107, 111, 112, 113, 114, 117, 123, 124, 131, 132, 136, 138, 139, 143, 147, 152, 155, 156, 157, 164, 165, 169], "dutch": 30, "dynam": [0, 11, 13, 14, 16, 17, 20, 23, 39, 78, 82, 84, 92, 94, 96, 102, 124, 133, 135, 136, 138, 139, 147, 150, 152, 158, 163, 170], "dynamic_batch_config": 147, "dynamic_batch_moving_average_window": 147, "dynamic_quant_bf16tonvfp4": 13, "dynamic_tree_max_topk": 147, "dynamicbatchconfig": [0, 97, 147], "dynamicbatchmovingaveragewindow": 0, "dynamicbatchsizeconfig": 0, "dynamicdecodelay": 1, "dynamicqu": 13, "dynamictreemaxtopk": 0, "dynamictreemaxtopkhost": 1, "dynamicyamlmixinforset": [78, 158], "dynamo": [97, 113, 146, 150], "dynamodeploymentgraph": 46, "dynasor": [97, 152], "dynasor_generation_control": 11, "dynasorgenerationcontrol": 11, "dynlibload": 0, "e": [0, 10, 11, 14, 15, 18, 20, 21, 24, 26, 27, 30, 34, 36, 37, 38, 39, 61, 62, 63, 75, 78, 83, 84, 90, 92, 98, 99, 102, 103, 105, 108, 109, 110, 111, 118, 120, 123, 124, 133, 136, 138, 141, 143, 146, 147, 148, 151, 152, 153, 158, 166], "e2": [15, 17, 19, 97, 152], "e2el": [26, 29, 30, 31, 32, 33], "e2m1": 12, "e4m3": [4, 12, 93, 111], "e5m2": 4, "e728f08114c042309efeae4df86a50ca": 29, "e752184d1181494c940579c007ab2c5f": 18, "each": [0, 1, 2, 8, 10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 35, 37, 39, 40, 44, 51, 58, 60, 61, 62, 63, 66, 75, 83, 84, 86, 87, 91, 92, 94, 96, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 113, 116, 117, 124, 125, 126, 130, 131, 132, 133, 134, 136, 138, 139, 141, 143, 147, 152, 154, 155, 164, 169, 170], "eager": [15, 37, 152], "eagl": [0, 1, 9, 10, 18, 23, 85, 95, 135, 138, 147, 149, 150, 152, 168], "eagle3": [10, 68, 97, 113, 147, 152], "eagle3_layers_to_captur": 147, "eagle3_one_model": [18, 68, 96, 147], "eagle_choic": [138, 147], "eagle_dynamic_tree_max_top_k": 138, "eagle_posterior_threshold": 138, "eagle_temperatur": 135, "eagle_use_dynamic_tre": 138, "eaglechoic": [0, 1], "eagleconfig": [0, 135], "eagledecodingconfig": [68, 96, 97, 147], "eagleforcausallm": 135, "eagleinput": 1, "eaglenetctxcontextlengthshost": 1, "eaglenetctxpastkeyvaluelengthshost": 1, "eaglenetctxrequesttypeshost": 1, "eaglenetgencontextlengthshost": 1, "eaglenetgenpastkeyvaluelengthshost": 1, "eaglenetgenrequesttypeshost": 1, "ealge2": 14, "earli": [1, 8, 11, 34, 138, 143, 152], "earlier": [0, 11, 116, 127, 143], "early_stop": [106, 138, 147, 152], "early_stop_criteria": 138, "earlystop": [0, 1, 106], "eas": [11, 16, 17, 40], "easi": [7, 11, 16, 59, 81, 83, 125, 146, 150, 162, 164], "easier": [2, 12, 14, 16, 20, 39, 117, 119, 124], "easili": [2, 10, 13, 16, 118, 133], "east": [30, 115, 117, 143], "eastern": 151, "ebnf": [0, 95, 103, 147], "echo": [18, 20, 27, 62, 63, 84, 120, 121], "econom": 21, "ecosystem": [21, 146, 150], "eddi": 152, "edg": [4, 21], "edit": [36, 40, 98, 113], "edu": 24, "eec": 24, "ef648e7489c040679d87ed12db5d3214": 151, "effect": [0, 8, 10, 11, 12, 13, 14, 15, 20, 21, 23, 29, 30, 31, 32, 33, 37, 59, 60, 66, 84, 86, 87, 91, 92, 102, 106, 111, 113, 127, 130, 131, 136, 147, 152], "effici": [8, 11, 12, 13, 14, 15, 16, 17, 20, 21, 23, 35, 37, 42, 49, 54, 55, 57, 70, 75, 81, 82, 84, 87, 88, 89, 91, 92, 101, 104, 105, 106, 109, 113, 117, 136, 139, 142, 144, 145, 146, 147, 150, 151, 154, 155, 162, 163, 165, 169], "effort": [8, 10, 14, 15, 16, 17, 20, 21, 30, 113, 116, 127, 152], "eg": 40, "egx": 21, "eight": [2, 3], "einop": 133, "einstein": 133, "einsum": 133, "einsum_eq": 133, "either": [0, 1, 11, 13, 15, 20, 24, 35, 83, 92, 103, 133, 139, 143, 146, 147, 152, 164], "elaps": [8, 26, 29, 30, 31, 32, 86], "element": [0, 1, 12, 16, 35, 75, 87, 91, 105, 106, 110, 111, 133, 134, 141, 147], "element_typ": 1, "elementwis": [107, 133], "elementwise_affin": 134, "elementwise_binari": 133, "elementwise_sub": 107, "elementwise_sum": 107, "elementwiseoper": [107, 133], "eleutherai": [29, 31, 32, 39, 124], "elicit": 11, "elif": [66, 68, 170], "elimin": [8, 10, 12, 13, 15, 23, 39, 84, 91, 102, 113, 124, 127, 130, 136, 150, 152], "ellipsi": 133, "els": [0, 11, 44, 58, 59, 60, 67, 68, 96, 117, 118, 119, 133, 143, 170], "emb": [70, 117, 134], "embed": [0, 14, 23, 89, 109, 115, 124, 133, 138, 147, 148, 152, 153, 155], "embed_dim": 134, "embed_posit": 134, "embed_positions_for_gpt_attent": 134, "embed_positions_for_gpt_attention_loc": 134, "embed_positions_loc": 134, "embed_token": [118, 148, 153], "embedding_bia": 147, "embedding_dim": 134, "embedding_multipli": 135, "embedding_parallel_mod": 147, "embedding_scal": 135, "embedding_sharding_dim": [116, 135], "embeddingbia": [0, 1], "embeddingt": [0, 1], "emerg": [7, 13, 16], "emit": 147, "emot": 66, "emphas": 11, "emphasi": 116, "empir": [8, 12, 16], "emploi": [8, 16, 17, 37, 83, 113, 136, 154, 170], "employe": 60, "empow": [8, 13], "empti": [0, 1, 44, 86, 113, 133, 147, 152, 170], "emptybuff": 1, "emptygenslot": 0, "emptytensor": 1, "emul": [133, 152], "en": 152, "enabl": [0, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 22, 23, 27, 29, 31, 32, 36, 37, 39, 40, 44, 56, 57, 59, 60, 65, 66, 75, 76, 77, 78, 82, 84, 87, 88, 89, 90, 91, 93, 95, 96, 97, 98, 102, 103, 105, 106, 107, 110, 111, 112, 113, 114, 117, 118, 121, 124, 125, 130, 132, 133, 134, 135, 136, 138, 141, 143, 147, 148, 149, 150, 151, 152, 153, 155, 156, 157, 158, 163, 165, 166, 169], "enable_allreduc": 133, "enable_attention_dp": [2, 16, 21, 26, 27, 29, 30, 31, 32, 33, 40, 62, 92, 147], "enable_autotun": [18, 147, 152], "enable_bal": [8, 30, 147], "enable_batch_size_tun": 147, "enable_block_reus": [9, 18, 27, 33, 50, 59, 65, 67, 86, 147], "enable_build_cach": [147, 152], "enable_chunked_context": [2, 22, 138, 152], "enable_chunked_prefil": [27, 87, 147, 152], "enable_context_fmha_fp32_acc": [138, 147], "enable_debug_output": [23, 143, 147], "enable_forward_chunk": 135, "enable_fp8": 111, "enable_fullgraph": 147, "enable_if_t": 1, "enable_inductor": 147, "enable_iter_perf_stat": [27, 147], "enable_iter_req_stat": 147, "enable_kv_cache_reus": 109, "enable_layerwise_nvtx_mark": 147, "enable_lm_head_tp_in_adp": 147, "enable_lora": 147, "enable_max_num_tokens_tun": [147, 152], "enable_min_lat": [18, 147], "enable_mixed_sampl": 152, "enable_multi_devic": 152, "enable_offload": 59, "enable_overlap_schedul": 27, "enable_pad": [2, 15, 21, 26, 29, 30, 31, 32, 33, 40, 65, 147], "enable_paged_kv_cach": 136, "enable_partial_reus": [86, 147], "enable_piecewise_cuda_graph": 147, "enable_prompt_adapt": [147, 152], "enable_qkv": 134, "enable_think": 73, "enable_tqdm": 147, "enable_trt_overlap": 152, "enable_trtllm_sampl": 95, "enable_ucx": 152, "enable_userbuff": 147, "enable_xqa": 152, "enableattentiondp": [0, 1], "enablebatchsizetun": 0, "enableblockreus": [0, 109], "enablechunkedcontext": 0, "enablecontextfmhafp32acc": 0, "enabled_with_fp32_acc": 105, "enablelookaheaddecod": 1, "enablemaxnumtokenstun": 0, "enablepartialreus": 0, "enableseamlesslookaheaddecod": [0, 1], "enabletrtoverlap": 0, "enc": [23, 138, 152], "enc_dec": 106, "encapsul": [75, 105, 106, 117, 133], "encdecmodelrunn": 138, "encod": [0, 4, 11, 12, 13, 23, 27, 37, 60, 75, 89, 105, 106, 133, 136, 138, 140, 141, 142, 147, 152], "encode_base64_content_from_url": 70, "encoded_vocab": [0, 103], "encodedvocab": [0, 103], "encoder_hidden_st": [134, 135], "encoder_input_featur": 138, "encoder_input_id": 138, "encoder_input_len_rang": 152, "encoder_input_length": [133, 134, 138], "encoder_language_adapter_rout": 138, "encoder_max_input_length": [134, 138], "encoder_output": [134, 135, 138], "encoder_output_length": 138, "encoder_run": 138, "encoderenginebuff": 0, "encoderhiddens": 1, "encoderinputfeatur": 0, "encoderinputtokenid": 0, "encoderjsonconfigstr": 0, "encoderlen": 0, "encodermodel": [0, 135], "encodermodelpath": 0, "encoderoutput": 0, "encoderoutputlength": 0, "encount": [2, 10, 18, 20, 21, 29, 30, 31, 32, 33, 40, 84, 94, 101, 102, 118, 143, 147], "encourag": [0, 16, 26, 60, 106, 119, 147], "end": [0, 1, 8, 11, 14, 19, 21, 22, 23, 33, 37, 39, 60, 75, 77, 91, 97, 98, 105, 106, 117, 124, 127, 131, 132, 133, 136, 147, 152, 157, 169], "end_dim": 133, "end_id": [138, 147, 152], "end_thinking_phase_token": 147, "end_token": [0, 147], "endeavor": [13, 16, 17], "endid": [0, 1], "endpoint": [9, 18, 21, 47, 48, 84, 89, 147, 151, 152], "endswith": [118, 147], "enforc": [125, 133, 147], "engag": 11, "engin": [0, 1, 6, 8, 11, 12, 13, 14, 15, 16, 17, 20, 21, 22, 23, 24, 27, 39, 40, 44, 50, 51, 64, 76, 77, 84, 85, 86, 91, 92, 102, 103, 105, 106, 107, 110, 113, 114, 119, 126, 127, 130, 131, 132, 133, 135, 136, 138, 139, 143, 147, 149, 152, 156, 157, 160], "engine_buff": 138, "engine_dir": [22, 114, 115, 116, 117, 119, 124, 125, 138, 143], "engine_inspector": 138, "engine_llama_3": 117, "engine_nam": 138, "engine_output": 23, "engineaddr": 1, "enginebuff": [0, 1], "enginefilenam": 1, "engineinput": 1, "engineoutput": 1, "enginepath": 1, "engines": 1, "english": [19, 30, 33], "enhanc": [2, 8, 11, 13, 14, 15, 16, 17, 22, 37, 60, 75, 81, 82, 84, 89, 104, 106, 113, 132, 139, 145, 147, 155, 162, 163], "enjoi": [49, 54, 55, 57, 101, 121, 144, 146, 151], "enough": [2, 14, 21, 59, 75, 86, 105, 109, 130, 139, 152, 154, 170], "enqueu": [0, 103, 117, 138, 139, 152], "enqueuecontext": 0, "enqueuegener": 0, "enqueuerequest": [0, 103], "enroot": 98, "ensembl": 17, "ensur": [8, 9, 10, 12, 14, 16, 18, 20, 22, 29, 30, 31, 32, 33, 34, 37, 38, 39, 78, 84, 95, 98, 102, 103, 104, 107, 112, 119, 124, 130, 147, 148, 153, 158, 169], "enter": [11, 40, 91, 98, 107, 120, 130, 169], "enterpris": [21, 53], "entir": [0, 3, 8, 11, 13, 16, 37, 39, 40, 78, 92, 96, 103, 110, 117, 124, 133, 139, 147, 158, 169], "entri": [0, 1, 8, 11, 20, 22, 24, 34, 35, 37, 39, 57, 81, 98, 110, 124, 133, 152, 162], "entrypoint": [27, 120, 125, 146], "enum": [0, 1, 11, 136, 147], "enumer": [0, 1, 50, 51, 56, 60, 65, 66, 67, 96], "env": [30, 33, 36, 41, 42, 43, 45, 47, 48, 79, 84, 124, 159], "envelop": 16, "environ": [2, 8, 9, 12, 13, 16, 17, 20, 29, 30, 31, 32, 38, 39, 42, 58, 61, 62, 63, 70, 72, 75, 81, 97, 98, 101, 106, 111, 113, 123, 124, 125, 127, 130, 131, 143, 145, 146, 152, 155, 162], "environment": 118, "eo": [22, 26, 29, 30, 31, 32, 33, 60, 106, 147], "eof": [2, 9, 14, 16, 21, 26, 27, 29, 30, 31, 32, 33, 62, 92], "eos_id": [16, 22], "eos_token": 60, "eos_token_id": [60, 103, 138], "ep": [2, 8, 12, 13, 14, 17, 20, 21, 22, 27, 29, 39, 40, 97, 104, 124, 133, 134, 152], "ep16": 20, "ep2": 13, "ep2tp4": 13, "ep32": [12, 16, 20], "ep4": [16, 20], "ep4tp2": 13, "ep8": [15, 16, 20], "ep8tp8": 13, "ep_load_balanc": [16, 92], "ep_siz": [9, 16, 18, 21, 24, 26, 27, 40, 45], "epd": 149, "eplb": [8, 29, 152], "epsilon": [0, 133], "eq": 133, "equal": [0, 1, 8, 12, 15, 16, 23, 26, 29, 30, 31, 32, 44, 84, 92, 95, 103, 104, 126, 133, 134, 139, 147], "equal_progress": [132, 147], "equat": [6, 133], "equilibr": 8, "equival": [11, 13, 15, 100, 127, 133, 148, 153], "equvili": 23, "era": 150, "erenup": 152, "eri": 33, "err": [61, 62, 63], "error": [0, 1, 15, 18, 21, 22, 23, 24, 27, 29, 30, 31, 32, 33, 36, 40, 79, 84, 98, 101, 102, 103, 110, 119, 125, 130, 138, 139, 147, 152, 159], "errormsg": 0, "especi": [10, 11, 12, 14, 16, 17, 19, 20, 23, 49, 54, 55, 57, 84, 101, 102, 107, 126, 130, 136, 144, 146, 151, 169], "essenti": [11, 16, 37, 39, 98, 113, 124], "establish": [12, 15, 16, 17, 84, 102], "estim": [16, 124, 152, 170], "et": 3, "etc": [0, 1, 10, 16, 21, 30, 33, 34, 37, 38, 39, 83, 92, 113, 123, 124, 127, 131, 138, 139, 143, 146, 147, 148, 153], "ethnzhng": 152, "etp": 92, "euo": [30, 33], "eval": [10, 29, 30, 31, 32, 53, 97, 151], "evalu": [2, 4, 5, 8, 11, 12, 15, 17, 24, 33, 89, 97, 111, 151, 152], "even": [0, 7, 10, 11, 13, 16, 17, 19, 20, 21, 23, 27, 33, 36, 75, 84, 91, 93, 105, 106, 117, 119, 125, 130, 133, 136, 138, 139, 147], "evenli": [13, 92, 104], "event": [0, 1, 10, 147, 152], "event_buffer_max_s": 147, "eventbuffermaxs": 0, "eventid": 0, "eventptr": 1, "eventu": 112, "ever": [0, 96, 131], "everi": [0, 8, 10, 11, 12, 13, 15, 16, 17, 20, 21, 29, 35, 39, 60, 92, 96, 103, 118, 124, 125, 126, 133, 138], "everydai": 33, "everyon": [10, 11, 14], "everyth": [96, 117], "evict": [0, 1, 11, 14, 22, 39, 59, 86, 91, 108, 109, 110, 124, 125, 130, 152], "evidenc": 8, "evolv": [13, 105, 119, 169], "ewr": 30, "ex": [62, 63], "exact": [2, 75, 105, 139], "exact_match": [29, 31, 32], "exactli": [10, 35, 96], "exam": [11, 13], "examin": [16, 113], "exampl": [0, 3, 5, 7, 9, 10, 12, 14, 16, 17, 20, 21, 26, 27, 29, 30, 31, 32, 33, 34, 35, 36, 39, 40, 44, 50, 51, 53, 58, 61, 66, 67, 73, 75, 78, 80, 81, 83, 84, 86, 87, 88, 91, 92, 93, 94, 95, 96, 97, 98, 99, 102, 105, 106, 107, 109, 112, 113, 114, 115, 119, 125, 126, 127, 128, 129, 130, 131, 132, 133, 138, 139, 141, 142, 143, 144, 145, 147, 148, 149, 151, 152, 153, 155, 158, 160, 161, 162, 164, 165, 167, 168, 170], "example_cuda_graph_config": 65, "example_kv_cache_config": 65, "exaon": [93, 118, 142, 149, 152], "exaone4forcausallm": [142, 149], "exc": 56, "exce": [0, 8, 19, 132, 133, 147], "exceed": [0, 139], "excel": [11, 12, 19, 20], "except": [0, 1, 10, 11, 13, 14, 16, 20, 23, 60, 73, 75, 91, 92, 96, 103, 105, 106, 119, 126, 133, 136, 143, 147, 152], "excess": [16, 75, 105], "exchang": [97, 147], "excit": [49, 54, 55, 56, 57, 101, 144, 146, 151], "excl": [26, 29, 30, 31, 32], "exclud": [1, 10, 127, 133, 147, 152], "exclude_default": 147, "exclude_input_from_output": 147, "exclude_modul": [116, 147, 152], "exclude_non": 147, "exclude_unset": 147, "excludeinputfromoutput": 0, "exclus": [1, 10, 78, 96, 106, 141, 152, 158], "exec": [29, 31, 32, 38, 123, 151], "execut": [0, 8, 10, 11, 12, 13, 15, 16, 17, 20, 21, 26, 29, 30, 31, 32, 35, 36, 37, 38, 39, 76, 81, 84, 90, 91, 92, 103, 106, 110, 113, 117, 119, 123, 124, 130, 132, 133, 138, 139, 147, 150, 154, 156, 160, 162, 170], "executeloopbackrequest": 0, "executor": [1, 10, 17, 44, 64, 84, 88, 89, 96, 102, 109, 113, 114, 124, 132, 138, 139, 147, 152, 154, 165], "executor_config": 169, "executorconfig": [0, 84, 103, 114, 147], "executorexamplefastlogit": 152, "exhaust": [0, 17, 77, 157], "exhibit": [8, 19], "exist": [1, 10, 11, 13, 15, 16, 19, 20, 23, 29, 30, 31, 32, 36, 37, 39, 58, 72, 75, 80, 83, 88, 98, 106, 109, 110, 113, 118, 119, 124, 138, 147, 150, 152, 155, 161, 164, 165], "exist_ok": 58, "exit": [11, 16, 20, 27, 40, 138], "exp": 133, "expand": [0, 5, 7, 11, 12, 14, 20, 81, 82, 97, 133, 138, 147, 152, 162, 163], "expand_dim": 133, "expand_dims_lik": 133, "expand_mask": 133, "expand_shap": 133, "expanded_idx_to_permuted_idx": 133, "expandinputrow": 12, "expandinputrowskernel": 20, "expandtab": 147, "expans": 133, "expect": [0, 5, 9, 10, 14, 16, 17, 18, 20, 23, 29, 30, 31, 32, 33, 39, 44, 59, 61, 62, 63, 75, 76, 86, 96, 105, 106, 111, 115, 117, 118, 119, 124, 125, 128, 129, 133, 136, 143, 147, 152, 156, 160], "expend": 11, "expens": [17, 84, 103, 113, 126, 127, 132], "experi": [6, 7, 10, 12, 13, 15, 16, 17, 19, 20, 21, 29, 30, 31, 32, 33, 37, 38, 59, 60, 77, 97, 112, 113, 123, 143, 146, 150, 152, 157], "experiment": [8, 14, 27, 34, 61, 62, 63, 106, 152, 162], "experiment_config": [78, 158], "experimentconfig": [77, 78, 157, 158], "expert": [2, 8, 21, 22, 24, 26, 27, 30, 31, 32, 33, 40, 57, 81, 84, 97, 102, 110, 131, 147, 150, 152, 162], "expert_scale_factor": 133, "expert_statist": 16, "expert_statistic_eplb": 16, "expert_statistic_iter_rang": 16, "expert_statistic_path": 16, "expertid": 16, "expertis": [8, 13, 15, 16, 17, 20], "expir": [0, 86], "explain": [15, 19, 33, 34, 35, 75, 84, 91, 102, 106, 117, 130, 133, 139, 141, 154, 155], "explan": [2, 15, 21, 29, 30, 31, 32, 34, 35, 131, 138, 139], "explicit": [0, 1, 16, 27, 113, 133, 152], "explicit_draft_token": [23, 113, 135], "explicitdrafttoken": [0, 1], "explicitdrafttokensdtyp": 1, "explicitdrafttokensinput": 1, "explicitdrafttokensmodul": 1, "expliciteosstop": 0, "explicitli": [1, 12, 15, 16, 23, 29, 30, 44, 78, 83, 102, 107, 113, 117, 118, 147, 152, 158, 164], "explor": [11, 12, 13, 15, 16, 20, 95, 97, 113, 150, 151], "expon": 4, "exponenti": [17, 113], "export": [2, 9, 13, 14, 16, 20, 23, 39, 47, 48, 61, 62, 63, 78, 81, 82, 98, 116, 119, 124, 137, 138, 143, 146, 152, 158, 162, 163], "export_fmt": [93, 167], "expos": [0, 10, 11, 18, 20, 21, 34, 78, 83, 98, 106, 117, 121, 127, 151, 152, 158, 160, 164], "express": [0, 95, 103, 133, 147], "extend": [0, 11, 13, 14, 15, 16, 19, 20, 94, 95, 103, 109, 117, 131, 133, 146, 147, 150, 152], "extend_ctx": 96, "extended_runtime_perf_knob_config": [147, 152], "extendedruntimeperfknobconfig": [0, 97, 147], "extens": [17, 20, 35, 39, 83, 116, 124, 152, 164], "extent": 20, "extern": [0, 10, 11, 107, 108, 118, 138, 139, 151, 152], "external_checkpoint_dir": 118, "external_kei": 118, "external_weight": 118, "externaldrafttoken": 0, "externaldrafttokensconfig": [0, 1], "externaldrafttokensinput": 1, "externalstream": 60, "extra": [0, 2, 10, 11, 12, 13, 14, 19, 20, 23, 26, 27, 35, 37, 39, 40, 45, 75, 77, 78, 84, 90, 96, 101, 102, 105, 109, 113, 116, 124, 126, 127, 136, 138, 147, 152, 157, 158, 165, 166], "extra_arg": 62, "extra_bodi": [72, 88, 165], "extra_encoder_opt": 27, "extra_id": 109, "extra_llm_api_fil": [26, 29, 30, 31, 32, 33], "extra_llm_api_opt": [2, 9, 14, 16, 18, 21, 22, 24, 27, 29, 30, 31, 32, 33, 39, 40, 45, 62, 73, 76, 84, 88, 96, 124, 156, 160, 165], "extra_llm_api_options_eplb": 16, "extra_llm_config": 27, "extra_resource_manag": 147, "extra_token": 134, "extract": [0, 10, 11, 16, 22, 29, 31, 32, 38, 81, 96, 98, 103, 123, 128, 129, 133, 138, 147, 162], "extrapol": 133, "extrem": [8, 13, 16, 20, 91, 96, 117, 127, 130, 131], "f": [0, 18, 30, 38, 49, 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 64, 65, 66, 67, 68, 72, 73, 75, 101, 105, 106, 120, 123, 125, 132, 133, 143, 144, 146, 147, 151], "fabric": [84, 102, 152], "face": [16, 24, 29, 30, 31, 32, 33, 39, 44, 81, 82, 93, 94, 103, 110, 114, 119, 124, 135, 147, 151, 152, 162, 163], "facilit": [16, 17, 96, 107, 113, 151], "fact": [30, 33, 39, 96, 124, 131], "factor": [7, 12, 15, 16, 20, 60, 126, 127, 133, 134, 139, 141, 152], "factori": [77, 80, 119, 138, 147, 152, 157, 161], "factual": 106, "fail": [0, 8, 16, 18, 20, 21, 29, 30, 31, 32, 33, 36, 73, 84, 102, 138, 139, 143, 147, 152, 170], "fail_fast_on_attention_window_too_larg": [27, 138, 147], "failfastonattentionwindowtoolarg": 0, "failur": [16, 84, 102, 118, 147, 152], "fairli": 117, "fairseq": [142, 152], "fake": [109, 152], "fakebuff": 1, "falcon": [7, 39, 116, 124, 141, 142, 152], "falconconfig": 135, "falconforcausallm": 135, "falconmodel": 135, "fall": [12, 37, 40, 111, 152], "fallback": [96, 118, 147, 152], "fals": [0, 1, 9, 11, 13, 15, 18, 19, 21, 23, 26, 27, 29, 30, 31, 32, 33, 50, 53, 58, 60, 62, 67, 73, 76, 77, 80, 86, 92, 96, 103, 105, 106, 107, 109, 116, 133, 134, 135, 136, 137, 138, 147, 152, 156, 157, 160, 161], "false_output_valu": 133, "false_valu": 133, "famili": [16, 35, 105, 118, 142, 152], "familiar": [76, 106, 117, 125, 126, 128, 129, 146, 156], "famou": [30, 33, 106], "faq": 97, "far": [0, 11, 14, 103], "fast": [0, 9, 10, 16, 18, 19, 20, 21, 59, 75, 81, 105, 108, 113, 124, 126, 138, 147, 152, 162], "fast_build": [23, 147, 152], "fastapi": 152, "fastapi_serv": 152, "faster": [4, 5, 12, 14, 15, 20, 21, 23, 40, 59, 75, 96, 105, 119, 125, 133], "fastest": 19, "fastlogit": 0, "fault": [16, 152], "favor": [8, 152], "favorit": 64, "fc": [116, 117, 118, 143], "fc2": [12, 147], "fc_gate": 134, "fc_gate_dora": 134, "fc_gate_lora": 134, "fc_gate_plugin": 134, "fd": 0, "featur": [0, 7, 10, 12, 13, 14, 15, 16, 17, 19, 20, 21, 22, 23, 29, 30, 31, 32, 35, 36, 39, 40, 61, 62, 63, 82, 84, 86, 87, 91, 95, 98, 100, 102, 103, 105, 107, 108, 110, 111, 113, 116, 117, 119, 124, 127, 130, 131, 132, 133, 136, 138, 142, 146, 147, 148, 153, 155, 163, 168], "feature_dim": 138, "feb": 11, "februari": 15, "fed": [40, 89, 135], "feed": [104, 133], "feedback": [16, 94, 152], "feel": [33, 64], "fetch": [0, 10, 14, 27, 37, 154], "few": [7, 14, 15, 16, 33, 84, 91, 96, 101, 102, 109, 117, 119, 130], "fewer": [3, 8, 19, 75, 91, 105, 113, 147, 155], "fewshot": 24, "fewshot_as_multiturn": 24, "ffn": [13, 104], "ffn_hidden_s": 134, "fhma": 152, "field": [0, 11, 12, 27, 29, 39, 44, 58, 75, 78, 84, 96, 102, 106, 111, 116, 119, 121, 124, 127, 135, 136, 138, 141, 147, 152, 155, 158], "field_nam": 147, "fieldinfo": 147, "fifo": [16, 20], "figur": [8, 10, 11, 12, 13, 14, 16, 17, 19, 20, 84, 87], "file": [0, 2, 9, 14, 16, 18, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 34, 35, 36, 38, 39, 40, 46, 47, 48, 58, 73, 76, 77, 83, 88, 92, 96, 103, 104, 105, 107, 109, 116, 117, 118, 119, 123, 124, 138, 147, 148, 152, 153, 156, 157, 160, 164, 165], "file_path": 58, "file_prefix": 147, "filedesc": 0, "filenam": [0, 22, 26, 29, 30, 31, 32, 33, 98], "filepath": 1, "filesystem": [0, 1], "fill": [1, 10, 11, 49, 75, 86, 101, 118, 121, 133, 144, 146, 147, 151, 155], "fill_attention_const_params_for_long_rop": 134, "fill_attention_const_params_for_rop": 134, "fill_attention_param": 134, "fill_none_tensor_list": 134, "fill_valu": 133, "fillchar": 147, "fillemptyfieldsfromruntimedefault": 0, "filloper": 133, "filltaskstensor": 1, "filter": [19, 29, 31, 32, 83, 96, 164], "filter_medusa_logit": 138, "filter_weight": [83, 164], "final": [0, 1, 8, 10, 11, 12, 13, 14, 16, 17, 20, 21, 23, 26, 29, 30, 31, 32, 37, 44, 110, 120, 133, 147, 152, 170], "final_logit_softcap": 135, "final_output_id": 138, "finalize_decod": 138, "finalizemoerout": 12, "finalizemoeroutingkernel": 20, "find": [2, 8, 10, 11, 12, 15, 16, 17, 95, 97, 127, 133, 143, 147, 152], "find_best_medusa_path": 138, "fine": [2, 15, 16, 88, 92, 113, 124, 131, 134, 147, 150, 165], "finer": [29, 30, 31, 32, 33, 107], "finetun": 13, "finish": [0, 1, 10, 12, 14, 16, 20, 37, 44, 103, 106, 108, 119, 124, 138, 147, 154, 169], "finish_reason": [18, 21, 29, 30, 31, 32, 33, 147, 151, 152], "finished_gen_req_id": 58, "finishedst": 1, "finishedsum": 1, "finishreason": [0, 1, 152], "first": [0, 1, 5, 7, 8, 10, 11, 12, 14, 15, 16, 17, 18, 20, 21, 23, 27, 33, 35, 39, 40, 51, 58, 59, 66, 75, 78, 83, 84, 86, 89, 91, 94, 97, 98, 102, 103, 105, 106, 107, 109, 110, 113, 120, 124, 125, 127, 130, 131, 132, 133, 139, 143, 146, 147, 148, 152, 153, 155, 158, 164, 169, 170], "first_come_first_serv": [132, 147], "first_gen_token": 147, "first_lay": 138, "firstgentoken": 0, "firstit": 0, "firstli": [12, 15, 16, 91, 120, 130, 139], "firstscheduledtim": 0, "firsttokentim": 0, "fit": [0, 1, 3, 4, 12, 27, 92, 94, 105, 126, 127, 138, 147, 170], "fitting_request": 170, "five": [19, 30], "fix": [10, 11, 14, 15, 17, 19, 20, 39, 84, 86, 87, 102, 108, 110, 113, 124, 139], "fjosw": 152, "flag": [0, 1, 6, 9, 12, 16, 20, 21, 22, 24, 27, 29, 30, 31, 32, 33, 39, 44, 76, 77, 91, 94, 103, 105, 110, 119, 124, 127, 128, 130, 132, 133, 139, 152, 156, 157], "flags_siz": 1, "flan": [141, 142], "flash": [75, 105, 117], "flashattent": [75, 105, 117], "flashinf": [29, 31, 32, 75, 76, 77, 78, 80, 82, 152, 155, 156, 157, 158, 160, 161, 163], "flashinferattent": [75, 155], "flashmla": [14, 152], "flatten": [1, 6, 16, 110, 133, 134], "flattenedinouts": 1, "flattenn": 1, "flavor": 96, "flayer": 107, "flayerinfomemo": 107, "flexibl": [10, 13, 16, 21, 29, 31, 32, 44, 78, 83, 96, 98, 113, 119, 146, 150, 158, 164], "flexibli": [11, 20], "flight": [1, 8, 39, 81, 84, 89, 97, 124, 130, 132, 139, 150, 152, 162], "flip": 133, "flip_sin_to_co": 134, "float": [0, 1, 4, 60, 67, 93, 106, 114, 116, 117, 132, 133, 134, 135, 138, 141, 147], "float16": [23, 107, 110, 114, 115, 116, 119, 126, 133, 135, 136, 143, 147], "float2": 133, "float32": [0, 23, 116, 133, 134, 135, 136, 147], "floattensor": [148, 153], "floattyp": [0, 1], "floor_div": 133, "floordiv": 133, "flop": [12, 15], "flow": [9, 11, 12, 13, 15, 17, 96, 107, 118, 119, 125, 126, 127, 130, 131, 152, 154, 170], "fluctuat": [8, 29, 30, 31, 32, 33, 84, 102], "flush": 12, "fly": [75, 105, 133, 141], "fmha": [0, 23, 91, 133, 136, 138, 139, 147, 152], "fmt_dim": 1, "focu": [7, 10, 11, 12, 13, 16, 38, 60, 94, 107, 123, 152], "focus": [11, 20, 21, 33, 34, 66, 94, 113, 124, 127, 128, 129, 152], "fold": 139, "folder": [0, 36, 58, 77, 103, 106, 119, 125, 141, 142, 152, 157], "folder_trt_llm": 117, "follow": [1, 2, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 21, 23, 24, 26, 27, 29, 30, 31, 32, 33, 34, 35, 36, 37, 39, 40, 44, 46, 50, 55, 56, 61, 62, 63, 75, 77, 78, 79, 80, 81, 82, 83, 84, 89, 92, 93, 95, 96, 98, 101, 102, 103, 106, 107, 110, 112, 113, 115, 116, 117, 118, 119, 121, 124, 125, 126, 127, 128, 129, 130, 131, 133, 136, 141, 142, 146, 147, 148, 149, 151, 152, 153, 155, 157, 158, 159, 161, 162, 163, 164, 167, 168, 169], "footprint": [3, 15, 75, 93, 105, 139], "for_each_rank": 135, "forbid": 147, "forc": [0, 11, 13, 16, 17, 36, 39, 59, 75, 84, 105, 124, 147, 152], "force_drop_id": 134, "force_dynamic_quant": 147, "force_multi_block_mod": 124, "force_nccl_all_reduce_strategi": 152, "force_num_profil": 147, "force_words_id": 106, "forcefulli": 10, "forecast": 113, "forev": 96, "forget": 33, "fork": [38, 123], "form": [0, 11, 12, 17, 20, 75, 92, 95, 96, 103, 105, 113, 133, 147], "formal": 152, "format": [0, 4, 7, 12, 14, 15, 18, 22, 27, 29, 30, 31, 32, 33, 48, 73, 75, 82, 89, 95, 97, 98, 103, 111, 116, 118, 119, 125, 127, 138, 139, 143, 147, 150, 152, 155, 163], "format_map": 147, "former": [7, 117], "formula": [12, 15, 17, 133], "forth": 16, "forthcom": [18, 21], "fortun": 10, "forum": 152, "forward": [0, 1, 10, 11, 14, 16, 17, 20, 37, 58, 75, 84, 96, 104, 107, 113, 115, 117, 132, 133, 134, 135, 143, 148, 152, 153, 154, 155, 169, 170], "forward_loop": 124, "forward_with_cfg": 135, "forward_without_cfg": 135, "forwardasync": 1, "forwarddispatch": 1, "forwardref": 147, "forwardsync": 1, "found": [2, 4, 10, 11, 16, 20, 36, 50, 51, 58, 60, 75, 84, 98, 102, 103, 104, 105, 106, 107, 113, 117, 124, 125, 127, 131, 141, 147, 170], "foundat": [10, 11, 12, 14, 20], "four": [12, 13, 14, 51, 59, 83, 103, 107, 113, 116, 134, 164], "fourth": [86, 103], "fp": [141, 152], "fp16": [3, 4, 7, 12, 21, 23, 39, 75, 82, 93, 105, 110, 111, 114, 116, 118, 124, 127, 131, 133, 136, 142, 143, 152, 163], "fp32": [0, 13, 15, 23, 75, 82, 93, 105, 133, 136, 138, 142, 143, 147, 152, 163], "fp4": [2, 14, 15, 16, 20, 23, 29, 31, 32, 68, 93, 136, 142, 146, 152], "fp4_gemm": 112, "fp8": [3, 5, 6, 7, 13, 14, 15, 16, 18, 19, 21, 22, 23, 26, 29, 31, 32, 39, 49, 54, 82, 88, 95, 97, 101, 111, 119, 124, 128, 131, 133, 136, 139, 142, 144, 146, 147, 149, 151, 152, 155, 163, 165, 167, 168], "fp8_block_scal": 147, "fp8_blockscale_gemm": 152, "fp8_inputs_overrid": 133, "fp8_kv_cach": [75, 105, 141], "fp8_pb_wo": 152, "fp8_per_channel_per_token": 147, "fp8_qdq": 141, "fp8_rowwise_gemm_plugin": [23, 136], "fp_valu": [75, 105], "fpa_intb": 152, "frac": [8, 17, 26, 29, 30, 31, 32], "fraction": [0, 17, 24, 27, 29, 30, 31, 32, 33, 76, 80, 86, 133, 134, 138, 147, 156, 160], "fragment": 29, "framework": [8, 21, 81, 83, 94, 97, 113, 115, 116, 119, 133, 152, 162, 164], "franc": [49, 50, 51, 54, 55, 56, 57, 65, 68, 88, 101, 115, 117, 125, 132, 143, 144, 146, 151, 165], "free": [0, 1, 10, 11, 15, 16, 20, 24, 27, 29, 30, 31, 32, 33, 50, 60, 76, 86, 88, 91, 96, 108, 110, 117, 118, 130, 134, 135, 138, 139, 147, 152, 156, 160, 165, 169], "free_gpu_memory_fract": [9, 33, 44, 50, 65, 67, 68, 86, 132, 147, 152], "free_hostfunc_user_data": 10, "free_mem_ratio": [76, 80, 156, 160, 161], "free_resourc": [96, 154, 169], "freed": [8, 86, 96, 124], "freedom": 119, "freegpumemoryfract": [0, 139, 152], "freenumblock": [0, 27], "freez": 15, "french": [88, 165], "freq": 133, "frequenc": [39, 124, 134], "frequency_penalti": [138, 147, 152], "frequencypenalti": [0, 1, 106], "frequent": [8, 10, 11, 20, 34, 92, 109, 143, 147], "friend": [0, 1, 39, 124], "friendli": [16, 133], "from": [0, 1, 2, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 35, 36, 37, 39, 40, 44, 49, 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 75, 78, 80, 82, 83, 84, 86, 87, 88, 90, 91, 92, 93, 94, 95, 96, 97, 99, 100, 101, 102, 103, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 124, 125, 126, 127, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 143, 144, 145, 147, 148, 150, 151, 152, 153, 154, 155, 158, 161, 163, 164, 165, 166, 167, 168, 169, 170], "from_argu": [135, 136], "from_attribut": 147, "from_checkpoint": [119, 135], "from_config": 135, "from_dict": [135, 147], "from_dir": 138, "from_engin": 138, "from_hugging_fac": [115, 118, 119, 135], "from_jax": 119, "from_json_fil": [135, 147], "from_kera": 119, "from_kwarg": 147, "from_meta_ckpt": [119, 135], "from_model_config_cpp": 138, "from_nemo": [119, 135], "from_orm": 147, "from_pretrain": [11, 135], "from_prun": 135, "from_pybind": 147, "from_serialized_engin": 138, "from_str": 133, "fromfil": 117, "front": [12, 147], "frontier": [8, 150], "fruit": 15, "fu": 11, "full": [0, 4, 5, 12, 14, 15, 16, 17, 24, 26, 27, 29, 30, 31, 32, 33, 35, 38, 39, 58, 60, 75, 78, 86, 92, 104, 105, 106, 109, 110, 113, 123, 124, 126, 133, 138, 139, 143, 147, 150, 152, 158], "full_stop_token": 60, "fulli": [15, 35, 49, 152], "fun": [30, 33], "funcnam": 0, "function": [0, 1, 10, 11, 13, 14, 16, 20, 27, 33, 37, 38, 65, 75, 78, 81, 83, 94, 95, 96, 103, 105, 114, 115, 117, 119, 123, 131, 135, 136, 138, 139, 141, 142, 143, 146, 147, 150, 152, 158, 162, 164, 169, 170], "functiont": 0, "functool": 147, "fundament": [8, 20], "further": [3, 7, 10, 11, 14, 15, 16, 17, 18, 23, 39, 75, 84, 89, 92, 96, 103, 104, 105, 113, 117, 124, 127, 131, 136, 155], "furthermor": [13, 16, 17, 84, 113, 127], "fuse": [10, 13, 15, 20, 21, 23, 75, 82, 89, 105, 113, 117, 131, 133, 136, 148, 152, 153, 155, 163], "fuse_a": [13, 15], "fuse_fp4_qu": [23, 136], "fuse_qkv_project": 135, "fuseattentionwithbiaspass": 107, "fused_a": 92, "fused_gate_up_dora": 134, "fused_gate_up_lora": 134, "fused_mo": 147, "fusedgatedmlp": [133, 134], "fusedmo": 152, "fusevalu": 1, "fusion": [15, 23, 75, 81, 82, 97, 107, 130, 136, 139, 141, 147, 152, 155, 162, 163], "fusion_op": 133, "futur": [7, 10, 12, 16, 23, 33, 39, 49, 50, 51, 53, 54, 55, 56, 57, 60, 65, 66, 68, 75, 84, 86, 91, 96, 97, 98, 101, 102, 105, 106, 108, 112, 113, 118, 119, 124, 125, 132, 133, 139, 144, 145, 146, 147, 151, 152], "fuyu": [142, 152], "fw": [83, 164], "g": [10, 11, 14, 15, 20, 21, 24, 26, 30, 34, 36, 37, 39, 61, 62, 63, 78, 83, 90, 91, 92, 99, 103, 108, 111, 118, 124, 130, 138, 147, 148, 151, 153, 158, 166], "g00": 8, "g01": 8, "g0m": 8, "g1": [91, 130], "g10": 8, "g11": 8, "g1m": 8, "g2": [91, 130], "gain": [8, 12, 16, 19, 37, 126, 130], "game": 18, "gamma": 133, "gap": [8, 10, 11, 17, 19, 20], "garbag": [20, 34, 147], "garbage_collection_gen0_threshold": [34, 147], "gate": [23, 110, 118, 125, 133, 136, 152], "gate_a": 133, "gate_a_bia": 133, "gate_bia": 133, "gate_proj": 118, "gate_up_proj": 152, "gate_x": 133, "gate_x_bia": 133, "gatedmlp": [133, 134], "gather": [0, 1, 23, 55, 56, 133, 138, 147], "gather_all_token_logit": [23, 152], "gather_context_logit": [23, 135, 138, 147], "gather_dim": [117, 133], "gather_generation_logit": [23, 135, 138, 147], "gather_last_token_logit": 133, "gather_nd": 133, "gather_output": 134, "gathercontext": [0, 152], "gatheredid": 1, "gatherel": 133, "gathergenerationlogit": 0, "gathermod": 133, "gathertre": 1, "gatherv2": 133, "gb": [5, 15, 59, 84, 98, 102, 124, 147], "gb200": [8, 12, 15, 17, 20, 21, 29, 30, 40, 84, 92, 97, 102, 142, 150, 152], "gc": 20, "gcc": [98, 152], "gd": 0, "geforc": 152, "gegelu": 133, "gegelu_limit": 134, "geglu": 133, "gelu": [133, 135], "gelu_pytorch_tanh": 152, "gelu_tanh": 134, "gemm": [10, 15, 16, 20, 23, 81, 82, 92, 93, 107, 130, 133, 136, 139, 152, 162, 163], "gemm_allreduc": 133, "gemm_allreduce_plugin": [23, 136, 138], "gemm_fc1": 13, "gemm_k": 12, "gemm_plugin": [23, 110, 114, 116, 117, 124, 127, 131, 134, 136], "gemm_q": 12, "gemm_qkv": 12, "gemm_swiglu": 133, "gemm_swiglu_plugin": [23, 127, 136], "gemm_v": 12, "gemma": [82, 93, 119, 140, 141, 142, 149, 152, 163], "gemma2": 142, "gemma2_added_field": 135, "gemma2_config": 135, "gemma3": [83, 152, 164], "gemma3_added_field": 135, "gemma3_config": 135, "gemma3_weight_mapp": [83, 164], "gemma3forcausallm": [83, 142, 149, 164], "gemma3forconditionalgener": [142, 149], "gemma3hfweightmapp": [83, 164], "gemma_added_field": 135, "gemma_config_kwarg": 135, "gemmaconfig": 135, "gemmaforcausallm": 135, "gen": [8, 10, 17, 147, 152], "gen2dep4": 17, "gen4": 17, "gen8": 17, "gen_extra": 84, "gen_kwarg": [29, 31, 32], "genai": [7, 74, 97], "genattent": 13, "genenginepath": 0, "gener": [0, 1, 2, 3, 4, 6, 8, 10, 11, 12, 13, 14, 15, 17, 19, 20, 21, 22, 23, 24, 26, 29, 30, 31, 32, 33, 34, 36, 37, 38, 39, 40, 49, 50, 51, 52, 58, 59, 65, 66, 67, 68, 76, 77, 81, 82, 86, 88, 91, 93, 94, 96, 97, 99, 101, 103, 106, 109, 113, 116, 117, 118, 119, 123, 124, 125, 126, 128, 129, 130, 131, 132, 133, 135, 138, 139, 142, 143, 144, 147, 148, 150, 151, 152, 153, 154, 155, 156, 157, 162, 163, 165, 167, 168, 169, 170], "generate_alibi_bias": 133, "generate_alibi_slop": 133, "generate_async": [11, 44, 55, 56, 147, 152], "generate_eplb_config": 16, "generate_logn_sc": 133, "generate_tllm_weight": 118, "generate_with_stream": 34, "generated_text": [50, 51, 64, 125, 132], "generatejsonschema": 147, "generation_complet": 170, "generation_control": 11, "generation_dir": 11, "generation_in_progress": 170, "generation_kwarg": 11, "generation_kwargs_list": 11, "generation_logit": [138, 147], "generation_onli": 147, "generation_phas": [75, 105], "generation_request": 170, "generation_serv": [17, 84], "generation_task": 11, "generation_to_complet": 170, "generation_with_dynasor_cot": 11, "generationexecutor": [84, 102, 152], "generationlength": 1, "generationlengthsdevic": 1, "generationlengthshost": 1, "generationlengthshostcopi": 1, "generationlogit": 0, "generationmixin": 135, "generationoutput": 34, "generationresult": [11, 147], "generationresultbas": 147, "generationsequ": 138, "generationsess": [105, 138, 139], "generationstep": 1, "generationtask": 11, "genericprompttuningparam": 1, "genert": 102, "genexecutorconfig": 0, "genidx": 0, "genlengthlogitsprocessor": 60, "genlenthlogitsprocesor": 60, "genreqr": 17, "genrequest": 1, "geograph": [30, 33], "geographi": 33, "get": [0, 1, 6, 11, 14, 15, 16, 18, 20, 21, 27, 29, 30, 31, 32, 38, 44, 58, 66, 72, 75, 78, 87, 96, 98, 101, 103, 105, 107, 110, 114, 118, 120, 121, 123, 125, 127, 133, 138, 143, 147, 150, 151, 152, 158, 167, 170], "get_1d_sincos_pos_embed_from_grid": 134, "get_2d_sincos_pos_emb": 134, "get_2d_sincos_pos_embed_from_grid": 134, "get_audio_featur": 138, "get_batch_cache_indic": 169, "get_batch_idx": 138, "get_block_offset": 138, "get_buff": 169, "get_build_config_default": 147, "get_comm": 147, "get_config_group": 135, "get_context_phase_param": 147, "get_default_config_load": [83, 164], "get_default_weight_load": [83, 164], "get_executor_config": 147, "get_finish": 58, "get_first_past_key_valu": 134, "get_hf_config": 135, "get_initialized_weight_mapp": [83, 164], "get_input": 107, "get_kv_cache_ev": 147, "get_kv_cache_events_async": 147, "get_max_resource_count": [169, 170], "get_needed_resource_to_complet": [169, 170], "get_next_medusa_token": 138, "get_num_free_block": 169, "get_num_heads_kv": 138, "get_num_new_matched_token": 58, "get_output": [107, 117], "get_par": [107, 133], "get_pybind_enum_field": 147, "get_pybind_variable_field": 147, "get_pytorch_backend_config": 147, "get_request_typ": 147, "get_rope_index": 138, "get_runtime_s": 147, "get_seq_idx": 138, "get_shap": 118, "get_slic": 118, "get_stat": [147, 152], "get_stats_async": 147, "get_timestep_embed": 134, "get_token": 58, "get_us": [107, 133], "get_visual_featur": 138, "get_vocab": [0, 103], "get_weight": 134, "get_zcopi": [84, 102], "getacceptancethreshold": 0, "getacceptedlengthscumsum": 1, "getacceptedpackedpath": 1, "getadditionalmodeloutput": 0, "getadditionaloutputnam": 0, "getaddr": 0, "getaddress": 1, "getagentst": 0, "getallnewtoken": 1, "getallottedtimem": 0, "getattentionconfig": 0, "getattentiondpeventsgatherperiodm": 0, "getattr": [11, 60], "getbackend": 0, "getbackendagentdesc": 0, "getbackendtyp": 0, "getbadword": 0, "getbatchingtyp": 0, "getbatchsizet": 0, "getbeamsearchbuff": 1, "getbeamsearchdiversityr": 0, "getbeamwidth": 0, "getbeamwidtharrai": 0, "getbuffermanag": 1, "getcacheindirectioninput": 1, "getcacheindirectionoutput": 1, "getcachesaltid": 0, "getcachest": 0, "getcachetransceiverconfig": 0, "getcapac": 1, "getcapacityschedulerpolici": 0, "getclientid": 0, "getcommptr": 1, "getcommst": 0, "getcommunicationmod": 0, "getcommunicationtyp": 0, "getconf": 20, "getconfig": 0, "getconnect": 0, "getconnectioninfo": 0, "getcontextchunkingpolici": 0, "getcontextexecutor": 0, "getcontextfmha": 1, "getcontextparallel": 1, "getcontextparallelgroup": 1, "getcontextparallelrank": 1, "getcontextphaseparam": 0, "getcopyonpartialreus": 0, "getcpu": 1, "getcpudiff": 1, "getcrossattentionmask": 0, "getcrosskvcachefract": 0, "getcudagraphcaches": 0, "getcudagraphmod": 0, "getcumlogprob": 1, "getdata": 0, "getdatatyp": [0, 1], "getdatatypenam": 1, "getdebugconfig": 0, "getdebuginputtensor": 0, "getdebugoutputtensor": 0, "getdebugtensornam": 0, "getdebugtensorsmaxiter": 0, "getdecodedurationm": 0, "getdecoderetentionprior": 0, "getdecoderstream": 1, "getdecodingconfig": 0, "getdecodingmod": 0, "getdefaultbatchslot": 1, "getdefaulteaglechoic": 1, "getdesc": 0, "getdevic": 1, "getdevicecacheperc": 0, "getdeviceid": 0, "getdeviceof": 1, "getdimens": 1, "getdirectori": 0, "getdrafttoken": 0, "getdstdesc": 0, "getdynamicbatchconfig": 0, "getdynamicbatchmovingaveragewindow": 0, "getdynamictreemaxtopk": 0, "geteaglebuff": 1, "geteaglechoic": 0, "geteagleconfig": 0, "getearlystop": 0, "getembeddingbia": 0, "getembeddingt": 0, "getenablebatchsizetun": 0, "getenableblockreus": 0, "getenablechunkedcontext": 0, "getenablecontextfmhafp32acc": 0, "getenablemaxnumtokenstun": 0, "getenablepartialreus": 0, "getenabletrtoverlap": 0, "getencodedvocab": 0, "getencoderhiddens": 1, "getencoderinputfeatur": 0, "getencoderinputtokenid": 0, "getencoderoutputlength": 0, "getendid": 0, "geterrormsg": 0, "geteventbuffermaxs": 0, "getexecutionconfig": 1, "getexplicitdrafttokensbuff": 1, "getextendedruntimeperfknobconfig": 0, "getexternaldrafttokensconfig": 0, "getfailfastonattentionwindowtoolarg": 0, "getfastlogit": 0, "getfd": 0, "getfinishedsum": 1, "getfinishreason": 1, "getfirstgentoken": 0, "getfirstlocallay": 1, "getfreegpumemoryfract": 0, "getfrequencypenalti": 0, "getfunctionpoint": 0, "getgatheredid": 1, "getgathergenerationlogit": 0, "getgemmallreducedtyp": 1, "getgenerationstep": 1, "getgenexecutor": 0, "getgpu": 1, "getgpudiff": 1, "getgpuspergroup": 1, "getgpuspernod": 1, "getgpuweightsperc": [0, 114], "getguid": 0, "getguideddecodingconfig": 0, "getguideddecodingparam": 0, "getguidetyp": 0, "gethandl": 0, "gethiddens": 1, "gethostcaches": 0, "gethostmemori": 1, "getid": 1, "getinittozero": 1, "getinputtokenextraid": 0, "getinputtokenid": 0, "getinst": [0, 1], "getipcunicastpoint": 1, "getisorchestr": 0, "getiterstatsmaxiter": 0, "getjointdecodinginput": 1, "getjointdecodingoutput": 1, "getkvcacheconfig": 0, "getkvcacheconfigref": 0, "getkvcacheeventmanag": 0, "getkvcacheretentionconfig": 0, "getkvcachetyp": 1, "getkvdatatyp": 1, "getlanguageadapteruid": 0, "getlastrank": 1, "getlatestdebugtensor": 0, "getlatestev": 0, "getlatestiterationstat": [0, 103], "getlatestrequeststat": 0, "getlayertyp": 1, "getlen": 0, "getlengthpenalti": 0, "getlevel": 1, "getlocalagentdesc": 0, "getlocalrank": 1, "getlogit": 0, "getlogitsdtyp": 1, "getlogitspostprocessor": 0, "getlogitspostprocessorconfig": 0, "getlogitspostprocessornam": 0, "getlogprob": 1, "getlookaheadbuff": 1, "getlookaheadconfig": 0, "getlookaheaddecodingconfig": 0, "getlookaheaddecodingmaxnumrequest": 0, "getloraconfig": 0, "getloramodul": 1, "getloraprefetchdir": 0, "getmanagedweightsmapopt": 1, "getmanageweightstyp": 1, "getmaxadapters": 0, "getmaxattentionwindowvec": 0, "getmaxbatchs": [0, 1], "getmaxbeamwidth": [0, 1], "getmaxdecodingdecodertoken": 1, "getmaxdecodingdrafttoken": 1, "getmaxdecodingenginetoken": 1, "getmaxdecodingtoken": 1, "getmaxdraftpathlen": 1, "getmaxencoderlen": 1, "getmaxgputotalbyt": 0, "getmaxinputlen": 1, "getmaxlorarank": 1, "getmaxnonleafnodesperlay": 1, "getmaxnumpath": 1, "getmaxnumsequ": 1, "getmaxnumtoken": [0, 1], "getmaxpagesperblock": 1, "getmaxpagesperblockdevic": 0, "getmaxpagesperblockhost": 0, "getmaxpathlen": 1, "getmaxpositionembed": 1, "getmaxpromptembeddingtables": 1, "getmaxqueues": 0, "getmaxseqidlemicrosecond": 0, "getmaxsequencelen": 1, "getmaxsequencelength": 1, "getmaxtoken": 0, "getmaxtokensinbuff": 0, "getmedusachoic": [0, 1], "getmemorytyp": [0, 1], "getmemorytypenam": 1, "getminp": 0, "getmintoken": 0, "getmlphiddens": 1, "getmodelconfig": [0, 1], "getmodelconfigmut": 1, "getmodelnam": 1, "getmodelvari": 1, "getmpist": 0, "getmropeconfig": 0, "getmropepositiondelta": 0, "getmroperotarycossin": 0, "getmultiblockmod": 0, "getmulticastpoint": 1, "getmultimodalembed": 0, "getmultimodalhash": 0, "getmultimodalinput": 0, "getmultimodallength": 0, "getmultimodalposit": 0, "getnam": [0, 1], "getnbattentionlay": 1, "getnbhead": 1, "getnbkvhead": 1, "getnblay": 1, "getnbrnnlay": 1, "getnextdrafttoken": 1, "getnextdrafttokenslength": 1, "getngrams": 0, "getnoderank": 1, "getnoderankof": 1, "getnorepeatngrams": 0, "getnormalizelogprob": 0, "getnotifiedsyncmessag": 0, "getnumcopystream": [0, 1], "getnumdecodingenginetoken": 1, "getnumdevicemodulelay": 0, "getnumensurework": 0, "getnumhostmodulelay": 0, "getnumkvheadsforgivenlay": 1, "getnumkvheadsperlay": 1, "getnumkvheadsperlayerlocalrang": 1, "getnumlanguag": 1, "getnumnod": 0, "getnumpackedmask": 1, "getnumpag": 1, "getnumputwork": 0, "getnumresponsesreadi": 0, "getnumreturnbeam": [0, 1], "getnumreturnsequ": 0, "getnumtransformerlay": 1, "getonboardblock": 0, "getop": 0, "getoptimaladapters": 0, "getoptprofilessplitpoint": 1, "getorchestratorconfig": 0, "getorchleadercomm": 0, "getoutputconfig": 0, "getpadid": 0, "getpagedcontextfmha": 1, "getpageptr": 1, "getpagewidth": 1, "getparallelconfig": 0, "getparentid": 1, "getparticipantid": 0, "getpath": 1, "getpathopt": 1, "getpeftcacheconfig": 0, "getperblockretentionprioritydur": 0, "getpin": 1, "getpinneddiff": 1, "getpinnedpool": 1, "getpinnedpooldiff": 1, "getpipelineparallel": 1, "getpipelineparallelgroup": 1, "getpipelineparallelrank": 1, "getpositionid": 0, "getposteriorthreshold": 0, "getppreducescatt": 1, "getprecis": 1, "getpresencepenalti": 0, "getprevdrafttokenslength": 1, "getprior": 0, "getprocessorbatch": 0, "getprocessormap": 0, "getprompttableoffload": 0, "getprompttuningconfig": 0, "getquantmod": 1, "getrank": 1, "getrecvpollperiodm": 0, "getremotenam": 0, "getrepetitionpenalti": 0, "getrepl": 0, "getreqid": 0, "getrequestid": 0, "getrequeststatsmaxiter": 0, "getrequesttyp": 0, "getresult": [0, 103], "getreturnallgeneratedtoken": 0, "getrnnconfig": 1, "getrotaryembeddingdim": 1, "getruntimedefault": 1, "getruntimetyp": 0, "getsamplingconfig": [0, 1], "getschedulerconfig": 0, "getschedulerconfigref": 0, "getse": 0, "getsecondaryoffloadminprior": 0, "getselfidx": 0, "getsequencelength": 1, "getserializedst": 0, "getshap": [0, 1], "getsinktokenlength": 0, "getsiz": [0, 1], "getsizeinbit": 1, "getsizeinbyt": [0, 1], "getsizeperhead": 1, "getskipcrossattnblock": 0, "getslotsperpag": 1, "getsocketst": 0, "getspawnprocess": 0, "getspecdecconfig": 0, "getspeculativedecodingmod": 1, "getspeculativedecodingmodul": 1, "getspeculativedecodingmoduleptr": 1, "getsrcdesc": 0, "getstat": 0, "getstatu": 1, "getstoptokenid": 0, "getstopword": 0, "getstream": [0, 1], "getsyncmessag": 0, "gettag": 0, "gettaskid": 0, "gettemperatur": 0, "gettensorparallel": 1, "gettensorparallelgroup": 1, "gettensorparallelrank": 1, "getter": 106, "gettoken": 0, "gettokenizerstr": 0, "gettokenrangeretentionconfig": 0, "gettokensperblock": 1, "gettopk": 0, "gettopp": 0, "gettoppdecai": 0, "gettoppmin": 0, "gettoppresetid": 0, "gettotalnumpag": 1, "gettransfermod": 0, "gettyp": [0, 1], "getunderlyingdecod": 1, "getunicastpoint": 1, "getusegpudirectstorag": 0, "getuseuvm": 0, "getuvm": 1, "getuvmdiff": 1, "getverificationsets": 0, "getvers": 1, "getvirtualmemoryalloc": 1, "getvirtualmemorymanag": 1, "getvocabs": 1, "getvocabsizepad": 1, "getweight": 0, "getwindows": 0, "getworkerexecutablepath": 0, "getworlds": 1, "gh200": [35, 150, 152], "ghz": 53, "gib": [109, 139], "gid": [0, 36], "gigabyt": 5, "gij": 8, "gil": 20, "git": [2, 9, 18, 36, 93, 98, 110, 143, 146, 167], "github": [2, 11, 13, 16, 18, 22, 29, 30, 31, 32, 60, 81, 93, 97, 98, 99, 119, 152, 162, 167], "give": [9, 12, 14, 15, 19, 20, 21, 33, 73, 78, 95, 103, 127, 130, 135, 158, 168], "given": [0, 1, 2, 5, 8, 10, 16, 20, 30, 35, 38, 86, 91, 93, 96, 103, 106, 110, 118, 119, 123, 128, 129, 130, 133, 134, 135, 137, 138, 139, 141, 147, 152, 169], "givyboi": 64, "glm": [133, 142, 152], "glm4": 152, "glob": 35, "global": [0, 8, 12, 13, 15, 19, 20, 105, 108, 117, 152], "global_max_input_length": 138, "global_max_output_length": 138, "globalrequestid": 0, "glossari": [3, 6], "gm": [78, 143, 158], "gn0": 8, "gn1": 8, "gnm": 8, "gnu": 98, "go": [10, 11, 33, 75, 91, 105, 106, 126, 152], "goal": [16, 21, 132], "goe": [14, 39, 124, 146], "golden": 10, "good": [2, 11, 15, 16, 20, 39, 103, 117, 124, 126, 130, 131, 147], "googl": [82, 142, 149, 163], "got": [0, 1, 11, 33, 49, 53, 54, 55, 56, 57, 60, 64, 101, 124, 143, 144, 147, 151], "gp": 33, "gpqa": [13, 15, 24, 30], "gpt": [1, 4, 7, 23, 28, 35, 39, 75, 93, 97, 105, 113, 117, 124, 133, 136, 139, 141, 142, 143, 149, 152], "gpt2": [135, 143], "gpt3": 5, "gpt_attent": [6, 105, 107, 133, 152], "gpt_attention_plugin": [23, 110, 117, 124, 134, 136, 138, 143, 152], "gpt_attention_plugin_remove_pad": 107, "gpt_ib_ptun": 35, "gpt_oss": 30, "gpt_oss_output": 30, "gpt_variant": [135, 152], "gptattent": 107, "gptattentionpluginremovepaddingrewritepass": 107, "gptconfig": 135, "gptdecod": 106, "gptdecoderbatch": 152, "gptdecoderptr": 1, "gptforcausallm": 135, "gptj": 135, "gptjconfig": 135, "gptjforcausallm": 135, "gptjmodel": 135, "gptlmheadmodel": 143, "gptmanag": 152, "gptmanagerbenchmark": [98, 109, 152], "gptmodel": 135, "gptmodelconfig": 152, "gptneoxforcausallm": 135, "gptneoxmodel": 135, "gptq": [7, 93, 142, 152], "gptsession": 152, "gptsessionbenchmark": 152, "gpu": [0, 1, 4, 5, 6, 7, 8, 9, 12, 14, 17, 18, 19, 20, 21, 23, 24, 26, 27, 29, 30, 31, 32, 33, 35, 37, 40, 44, 50, 58, 59, 61, 62, 63, 65, 75, 76, 77, 81, 84, 86, 87, 88, 89, 90, 91, 92, 93, 97, 98, 101, 102, 103, 104, 105, 106, 108, 109, 110, 111, 113, 116, 119, 125, 126, 127, 131, 133, 135, 136, 138, 142, 143, 146, 147, 152, 154, 155, 156, 157, 160, 162, 165, 166], "gpu_typ": 35, "gpu_weights_perc": [114, 138], "gpudirect": 0, "gpumemusag": [0, 27], "gpus_per_nod": [23, 24, 27, 147], "gpuspernod": [1, 106], "gpusync": 1, "gpuweightsperc": [0, 114], "gqa": [3, 6, 23, 75, 92, 97, 105, 108, 133, 136, 152, 155], "grace": [16, 20, 109, 142, 147], "grade": [82, 163], "gradient": 4, "gradual": [112, 119], "grain": [16, 92, 107], "gram": [96, 97, 113, 152], "grammar": [0, 95, 97, 103, 147], "granit": [82, 142, 152, 163], "granular": [20, 76, 156], "graph": [0, 2, 14, 15, 16, 19, 20, 26, 29, 30, 31, 32, 33, 38, 39, 46, 65, 67, 75, 77, 78, 81, 82, 84, 85, 95, 96, 97, 102, 117, 123, 124, 133, 138, 139, 140, 143, 147, 149, 152, 155, 157, 158, 160, 162, 163, 169], "graph_rewrit": 107, "graphic": [18, 58], "gratitud": 14, "gre": 27, "great": [3, 8, 10, 12, 16, 18, 21, 30, 84], "greater": [0, 6, 7, 8, 13, 16, 23, 75, 84, 86, 102, 105, 133, 147], "greatli": [10, 12, 14, 20, 109, 119, 127, 131], "greedi": [0, 9, 37, 66, 95, 96, 106, 147, 154], "greedy_sampl": 147, "greedysampl": 0, "greedysamplinghost": 1, "greener": 11, "grid": [117, 127, 130, 133, 134], "grid_search_engin": 125, "grid_siz": 134, "grok": [142, 152], "groovi": 35, "gross": 29, "ground": [38, 123], "group": [0, 3, 15, 16, 20, 29, 30, 31, 32, 33, 86, 97, 103, 104, 106, 108, 117, 133, 134, 136, 141, 147, 152, 155], "group_cl": 135, "group_norm": 133, "group_rms_norm": 152, "group_siz": [116, 133, 147], "groupedrmsnorm": 13, "groupgemm": [15, 16], "groupnorm": [133, 134], "groupwis": 136, "grow": [1, 12, 17, 19, 75, 91, 113, 130], "gsm8k": [15, 29, 31, 32], "gt": 133, "gtc": [2, 13], "guarante": [0, 8, 10, 12, 16, 20, 39, 40, 81, 94, 97, 106, 109, 119, 124, 125, 127, 132, 162], "guaranteed_no_evict": [0, 22, 39, 124, 132, 147], "guaranteednoevictschedul": 170, "guard": 125, "guardian": [82, 163], "guid": [0, 2, 7, 9, 18, 21, 29, 30, 31, 32, 33, 38, 40, 46, 52, 75, 76, 81, 85, 89, 93, 101, 117, 123, 125, 126, 127, 129, 131, 133, 143, 146, 147, 149, 150, 152, 155, 156, 162], "guidanc": [8, 20, 75, 95, 113, 131, 134, 135], "guided_decod": [53, 95, 147], "guided_decoding_backend": [53, 73, 95, 147], "guided_decoding_param": 95, "guideddecod": 10, "guideddecodingbackend": 0, "guideddecodingconfig": [0, 103], "guideddecodingparam": [0, 53, 95, 97, 103, 147], "guidedrequest": 10, "guidelin": [81, 126, 162], "guidetyp": [0, 103], "gw": 107, "h": [9, 11, 14, 18, 20, 21, 23, 29, 30, 31, 32, 33, 41, 42, 43, 75, 77, 84, 103, 105, 113, 118, 125, 133, 135, 151, 152, 157, 160], "h0": 14, "h1": 133, "h100": [7, 18, 23, 29, 35, 40, 97, 119, 125, 127, 128, 129, 130, 150, 152], "h20": 23, "h200": [4, 10, 23, 29, 40, 97, 150, 152], "ha": [0, 1, 2, 3, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 29, 30, 31, 32, 33, 37, 38, 39, 40, 60, 75, 83, 84, 86, 91, 94, 96, 97, 98, 99, 103, 105, 109, 110, 111, 116, 117, 118, 119, 121, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 138, 139, 141, 143, 147, 151, 152, 154, 164, 169, 170], "habitu": 35, "had": [2, 12, 15, 91, 119, 127, 130], "half": [0, 1, 15, 20, 117, 125, 133], "halv": [4, 133], "hand": [12, 35, 109, 113, 126], "handl": [0, 1, 3, 8, 10, 11, 12, 13, 17, 20, 21, 37, 83, 84, 87, 89, 91, 96, 102, 104, 108, 118, 119, 125, 127, 130, 131, 132, 133, 134, 147, 148, 150, 153, 154, 164], "handle_per_step": 138, "handler": 11, "hang": [0, 10, 16, 84, 102, 143, 152], "hao": 11, "happen": [1, 10, 11, 16, 86, 101, 103, 106, 109, 117, 139, 143], "happi": 138, "har": [15, 29, 31, 32], "hard": [11, 75, 82, 105, 147, 163], "harder": 106, "hardwar": [7, 12, 15, 18, 19, 21, 26, 28, 35, 37, 44, 97, 98, 108, 152], "has_affin": 133, "has_bia": 133, "has_config_group": 135, "has_position_embed": 138, "has_scal": 133, "has_token_type_embed": 138, "has_zero_point": [116, 147], "hascontextawaitthread": 0, "hasdraftlogit": 1, "haserror": [0, 103], "hasgenawaitthread": 0, "hash": [0, 58, 86, 89, 147], "hash_valu": 58, "hashed_token": 58, "hasresult": 0, "hasrnnconfig": 1, "hasspeculativedecodingmodul": 1, "hattizai": 152, "have": [0, 1, 2, 3, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 29, 30, 31, 32, 37, 38, 39, 40, 51, 58, 61, 62, 63, 64, 68, 75, 78, 82, 83, 84, 86, 87, 91, 93, 96, 101, 102, 103, 104, 105, 106, 109, 110, 112, 113, 116, 117, 118, 119, 120, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 136, 138, 139, 142, 143, 146, 147, 148, 151, 152, 153, 158, 163, 164], "hbm": 97, "hbm3e": [5, 12], "hcxvisionforcausallm": [142, 149], "head": [1, 3, 14, 15, 19, 23, 77, 82, 84, 86, 97, 106, 108, 113, 117, 124, 133, 134, 136, 147, 152, 155, 157, 163], "head_dim": [75, 155, 169], "head_siz": [75, 105, 133, 135, 138, 152], "header": 102, "headquart": 58, "headsiz": 133, "headsperlay": 1, "health": [18, 21, 27, 29, 30, 31, 32, 33, 64, 97], "healthi": 9, "heard": 33, "heat": 106, "heavi": [20, 35, 131], "heavier": 126, "heavili": [8, 12, 16, 20], "height": [8, 26, 48, 134, 138], "held": [10, 20, 86], "hello": [37, 49, 50, 51, 54, 55, 56, 57, 61, 64, 65, 88, 93, 95, 101, 125, 132, 144, 146, 151, 165, 167, 168], "help": [10, 12, 13, 14, 16, 17, 20, 23, 24, 29, 30, 31, 32, 33, 38, 39, 40, 41, 42, 53, 59, 60, 64, 65, 66, 67, 68, 69, 70, 73, 75, 77, 84, 87, 91, 98, 102, 103, 105, 107, 117, 123, 124, 125, 128, 129, 130, 131, 132, 133, 136, 146, 147, 151, 152, 154, 157, 160], "helper": [1, 35, 133], "henc": [10, 20, 148, 153], "hendryck": 24, "here": [2, 4, 5, 10, 11, 14, 15, 16, 24, 26, 29, 30, 31, 32, 33, 34, 44, 49, 53, 60, 75, 83, 86, 93, 95, 98, 101, 103, 107, 110, 114, 115, 116, 117, 118, 119, 121, 123, 125, 126, 127, 130, 131, 133, 138, 139, 141, 143, 145, 147, 151, 155, 164, 169, 170], "hesit": 11, "heterogen": [8, 84, 102], "heurist": [15, 22, 36, 39, 75, 97, 105, 124, 133, 147, 152], "hf": [21, 22, 23, 24, 27, 39, 40, 49, 54, 55, 56, 57, 61, 62, 63, 72, 77, 81, 82, 83, 93, 101, 106, 110, 114, 118, 124, 125, 138, 142, 143, 144, 146, 147, 151, 157, 160, 162, 163, 164, 167], "hf_config_or_dir": 135, "hf_gemma3": [83, 164], "hf_home": 36, "hf_lora_convert": 110, "hf_model": [124, 135], "hf_model_card_or_dir": [80, 161], "hf_model_dir": [83, 114, 115, 116, 119, 135, 164], "hf_model_nam": 124, "hf_model_or_dir": 135, "hf_quant_config": [39, 124], "hf_token": [9, 39, 124], "hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx": 9, "hfcheckpointload": [83, 164], "hfconfigordir": 135, "hgx": 5, "hi": 110, "hidden": [0, 11, 12, 13, 14, 20, 21, 75, 78, 92, 96, 103, 104, 105, 106, 110, 113, 133, 134, 147, 152, 158], "hidden_act": [116, 134, 135], "hidden_dim": [0, 75, 105, 133], "hidden_dim_per_head": [75, 91, 105, 133], "hidden_dtyp": 134, "hidden_s": [0, 12, 75, 78, 107, 116, 118, 133, 134, 135, 138, 148, 153, 155, 158], "hidden_size_in": 110, "hidden_size_out": 110, "hidden_size_per_head": 133, "hidden_st": [12, 96, 115, 133, 134, 135, 138, 143, 148, 153], "hidden_states_for_emb": 135, "hiddens": [0, 1, 106], "hiddenst": 12, "hide": [10, 13, 15, 37], "hierarch": 116, "hierarchi": [119, 133], "high": [3, 7, 8, 10, 11, 13, 14, 15, 17, 18, 19, 20, 29, 30, 31, 32, 34, 51, 58, 80, 82, 84, 91, 92, 93, 97, 103, 113, 115, 117, 119, 124, 132, 133, 139, 146, 150, 152, 161, 163], "higher": [0, 1, 3, 4, 6, 8, 10, 12, 15, 16, 17, 19, 20, 21, 40, 66, 75, 78, 86, 87, 91, 95, 105, 106, 109, 110, 113, 118, 132, 139, 148, 152, 153, 158], "highest": [4, 5, 8, 21, 34, 51, 78, 106, 107, 147, 158], "highli": [11, 15, 16, 19, 20, 21, 37, 38, 113, 117, 123, 127], "highlight": [4, 7, 91, 97, 127, 130], "highwai": 30, "hin": 14, "hinderlit": 29, "hint": [20, 34, 133], "histor": [20, 92], "histori": 15, "hit": [0, 15, 19, 37, 40, 58, 59, 130, 131, 147, 152], "hk": 113, "hmac": 147, "hmm": [11, 33], "ho": 110, "hoc": [119, 138], "hold": [0, 1, 10, 11, 16, 59, 83, 86, 92, 103, 104, 107, 108, 109, 110, 113, 126, 134, 139, 147, 154, 164], "home": [2, 36, 39, 67, 124], "home_dir": 36, "homework": 33, "homo_head_pattern": 134, "homogen": [84, 102], "honor": 30, "hood": 146, "hope": [8, 11, 12, 13, 16, 17, 33], "hopefulli": 10, "hopper": [2, 3, 4, 7, 9, 12, 14, 15, 16, 21, 23, 28, 75, 93, 97, 98, 105, 109, 127, 136, 142, 150, 152], "horizont": [15, 23, 136], "host": [1, 9, 15, 17, 18, 19, 21, 26, 27, 29, 30, 31, 32, 33, 37, 45, 59, 63, 84, 98, 101, 110, 121, 131, 133, 147, 151, 152], "host_cache_s": [59, 86, 88, 147, 165], "host_context_length": [133, 134, 135, 138, 143], "host_context_progress": [133, 134, 143], "host_cross_kv_cache_block_offset": [134, 138], "host_cross_kv_cache_pool_map": 134, "host_cross_kv_cache_pool_point": 134, "host_kv_cache_block_offset": [133, 134, 138, 143], "host_kv_cache_block_point": 143, "host_kv_cache_pool_map": [133, 134, 143], "host_kv_cache_pool_point": [133, 134, 143], "host_max_attention_window_s": [133, 134, 143], "host_past_key_value_length": [133, 134, 143], "host_path": [29, 30, 32], "host_request_typ": [133, 134, 135, 143], "host_runtime_perf_knob": [133, 134, 143], "host_sink_token_length": [133, 134, 143], "hostcaches": [0, 109], "hostfunc": 10, "hostmemori": 1, "hostnam": [17, 27, 84], "hot": [16, 20, 33, 92], "hottest": 16, "hour": 125, "hous": [16, 126], "how": [0, 8, 12, 16, 17, 19, 20, 21, 23, 26, 29, 30, 31, 32, 33, 34, 35, 38, 39, 49, 59, 61, 67, 75, 77, 80, 84, 88, 89, 91, 93, 97, 102, 103, 113, 115, 117, 118, 119, 121, 123, 125, 127, 128, 129, 131, 133, 136, 139, 141, 143, 145, 147, 151, 154, 155, 157, 160, 161, 165], "howev": [2, 3, 8, 10, 11, 12, 13, 14, 15, 16, 20, 21, 39, 75, 78, 87, 91, 96, 103, 105, 113, 119, 124, 126, 127, 130, 131, 132, 139, 148, 152, 153, 154, 158], "hpc": 4, "html": [1, 22, 29, 30, 31, 32, 117, 133, 143], "http": [0, 1, 2, 9, 11, 12, 13, 18, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 35, 36, 41, 42, 43, 60, 69, 70, 71, 72, 73, 84, 88, 90, 93, 98, 101, 104, 110, 117, 119, 133, 136, 141, 143, 146, 151, 152, 160, 165, 166, 167], "http_code": [9, 18, 21, 29, 30, 31, 32, 33], "huang": 58, "hub": [18, 21, 24, 29, 30, 31, 32, 39, 64, 93, 124, 147, 152, 167], "hudson": [30, 33], "hug": [24, 29, 30, 31, 32, 33, 39, 44, 81, 82, 93, 94, 103, 110, 114, 119, 124, 135, 147, 151, 152, 162, 163], "huge": [20, 152], "hugepag": 20, "hugepages": 20, "hugepages_fre": 20, "hugepages_rsvd": 20, "hugepages_surp": 20, "hugepages_tot": 20, "hugetlb": 20, "huggingfac": [0, 2, 9, 12, 21, 22, 29, 30, 31, 32, 36, 39, 40, 42, 64, 70, 81, 88, 96, 110, 115, 116, 118, 119, 124, 125, 142, 143, 146, 148, 149, 150, 151, 152, 153, 162, 165], "huggingface_exampl": [93, 167], "huggingface_hub": [9, 64], "huggingface_model_card": [93, 167], "human": [13, 37, 39, 94, 124], "hundr": 16, "hurt": [10, 15, 16, 91, 131], "hw": [13, 15, 16], "hybrid": [92, 104, 152], "hyper": 116, "hyperclova": 140, "hyperclovax": [142, 152], "hypothes": 95, "hypothesi": 113, "i": [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 34, 35, 36, 37, 39, 40, 41, 43, 44, 49, 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 71, 75, 76, 77, 78, 79, 81, 82, 83, 84, 87, 88, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 113, 114, 115, 116, 117, 118, 119, 121, 124, 125, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 138, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 162, 163, 164, 165, 167, 168, 169, 170], "ia3": [75, 105], "iactivationlay": 117, "ib": [35, 84, 102], "ibm": [82, 163], "ibrahimamin1": 152, "ibufferptr": 1, "iconstantlay": 133, "icudaengin": [138, 139], "id": [0, 1, 10, 14, 16, 18, 20, 21, 22, 24, 29, 30, 31, 32, 33, 39, 40, 44, 56, 60, 75, 86, 92, 96, 103, 109, 124, 133, 134, 138, 146, 147, 151, 155, 169], "idea": [10, 12, 15, 16, 110, 131], "ideal": [8, 16, 17, 96, 107, 127, 130, 152], "idempot": 147, "ident": [15, 20, 23, 95, 98, 103, 109, 133, 136, 168], "identifi": [0, 11, 16, 17, 37, 39, 106, 110, 113, 117, 124, 130, 133, 147, 152], "identity_plugin": 136, "idl": [0, 20, 37, 90, 166], "idtyp": [0, 103], "idx": [67, 138], "ieee": 141, "ieinsumlay": 133, "ielementwiselay": 133, "iexecutioncontext": [138, 139], "ifb": [8, 17, 84, 97, 113, 140, 152], "ifilllay": 133, "igatherlay": 133, "ignor": [9, 23, 26, 29, 30, 31, 32, 33, 36, 39, 78, 124, 133, 136, 138, 147, 158], "ignore_eo": [147, 152], "igptdecod": 1, "ihostmemori": [1, 117, 138], "ii": [30, 105, 133], "ij": 133, "ijk": 133, "ijl": 133, "ik": 133, "ikl": 133, "ilay": [107, 117], "illeg": 152, "illustr": [8, 11, 13, 14, 16, 17, 26, 29, 37, 84, 87, 90, 91, 107, 113], "ilogg": 1, "ilooplay": 133, "imag": [9, 21, 22, 26, 27, 29, 30, 31, 32, 39, 42, 48, 61, 62, 63, 70, 89, 97, 100, 101, 124, 134, 138, 142, 149, 152], "image64": 70, "image_base64": 27, "image_data_format": 22, "image_grid_thw": 138, "image_nam": 33, "image_patches_indic": 138, "image_path": 138, "image_s": 135, "image_tag": [33, 99], "image_token_index": 138, "image_url": [27, 42, 70], "imagin": 11, "imatrixmultiplylay": 133, "imb": 16, "imbal": [8, 16, 20, 21, 92, 130], "imbalanc": 16, "immedi": [8, 11, 20, 33, 37, 75, 81, 105, 113, 125, 143, 162], "immut": 1, "impact": [3, 7, 8, 12, 13, 14, 15, 16, 20, 27, 37, 64, 91, 111, 113, 126, 127, 130, 131, 132], "imped": [7, 16], "impl": [0, 170], "implement": [3, 10, 12, 15, 17, 18, 21, 22, 27, 58, 60, 77, 81, 82, 83, 84, 91, 92, 95, 96, 97, 102, 103, 105, 106, 108, 112, 113, 116, 117, 119, 133, 134, 135, 141, 142, 143, 147, 148, 152, 153, 154, 157, 162, 163, 164, 169, 170], "impli": [10, 12], "implic": [8, 10], "implicit": [1, 10, 105, 113, 133], "implicitli": 1, "import": [1, 3, 7, 10, 11, 14, 15, 18, 20, 22, 33, 34, 35, 37, 44, 49, 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 80, 83, 86, 87, 88, 91, 93, 95, 96, 101, 111, 113, 118, 119, 125, 127, 130, 131, 132, 142, 144, 145, 146, 148, 151, 152, 153, 154, 161, 164, 165, 167, 168, 169], "importantli": [16, 36], "impos": 7, "imposs": 10, "improv": [3, 4, 5, 6, 7, 8, 10, 12, 13, 14, 15, 16, 17, 19, 20, 21, 23, 26, 29, 30, 31, 32, 33, 37, 39, 40, 49, 54, 55, 57, 60, 65, 75, 89, 90, 91, 94, 101, 105, 109, 111, 117, 124, 125, 127, 128, 129, 130, 131, 136, 144, 145, 146, 147, 151, 152, 155, 166], "in_channel": 134, "in_featur": [116, 117, 134], "in_hidden_s": 133, "in_len": 107, "in_point": 133, "in_progress": 138, "incex": 147, "includ": [0, 1, 3, 4, 6, 8, 10, 11, 12, 14, 15, 16, 17, 20, 21, 22, 23, 24, 27, 29, 30, 31, 32, 33, 34, 35, 37, 39, 40, 44, 53, 75, 78, 81, 82, 84, 89, 91, 96, 98, 103, 105, 106, 109, 110, 112, 113, 116, 117, 118, 124, 127, 131, 133, 141, 143, 144, 146, 147, 150, 152, 154, 155, 158, 162, 163, 169, 170], "include_stop_str_in_output": 147, "inclus": [84, 102, 133], "incom": [8, 37], "incompat": [23, 81, 145, 152, 162], "inconsist": 11, "incorpor": [0, 13, 16, 81, 89, 152, 162], "incorrect": [12, 109, 113, 152], "increas": [0, 2, 4, 5, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 35, 37, 38, 39, 60, 75, 84, 86, 87, 91, 97, 105, 109, 113, 117, 123, 124, 125, 127, 131, 132, 133, 136, 152, 170], "increasingli": [8, 11, 20], "increment": [16, 98, 152], "incur": [13, 17, 37, 84, 102, 117], "inde": 139, "indent": 147, "independ": [0, 1, 12, 17, 84, 92, 102, 103, 113, 133], "index": [0, 1, 13, 18, 20, 21, 29, 30, 31, 32, 33, 35, 44, 75, 97, 101, 103, 108, 113, 118, 133, 138, 147, 151, 152, 155], "index_select": 133, "indic": [0, 1, 8, 10, 11, 12, 17, 20, 21, 75, 86, 103, 105, 106, 113, 116, 132, 133, 134, 138, 139, 147, 169], "indim": 1, "indimfirst": 1, "indirect": 1, "individu": [8, 13, 16, 17, 35, 152, 164], "indivis": 152, "inductor": 147, "industri": [21, 39, 124], "ineffect": 10, "ineffici": [11, 12, 13, 75, 91, 105], "inetworkdefinit": [107, 117, 133], "inevit": [10, 117], "infeas": 103, "infer": [0, 2, 3, 4, 5, 8, 10, 12, 13, 15, 20, 23, 29, 30, 31, 32, 33, 37, 38, 40, 42, 46, 70, 77, 78, 81, 82, 84, 86, 87, 88, 89, 92, 93, 94, 96, 97, 102, 106, 110, 113, 117, 119, 123, 125, 126, 127, 128, 129, 131, 132, 133, 138, 141, 143, 144, 146, 147, 150, 152, 154, 157, 158, 162, 163, 165], "infer_shap": 138, "inferencemax": 12, "inferenceoptim": [78, 158], "inferencerequest": 152, "infin": [10, 121], "infiniband": [84, 102], "infinit": [39, 40, 117, 124], "inflat": 13, "inflight": [0, 27, 75, 91, 105, 110, 113, 122, 124, 128, 129, 130, 133, 147, 152, 155, 170], "inflight_request_id": 170, "inflightbatch": 0, "inflightbatchingstat": 0, "influenc": [13, 131], "info": [0, 16, 22, 23, 24, 26, 27, 30, 58, 79, 124, 139, 143, 152, 159], "inform": [0, 1, 3, 6, 10, 13, 14, 16, 17, 20, 22, 27, 29, 30, 31, 32, 39, 40, 66, 73, 75, 80, 81, 83, 84, 86, 89, 98, 101, 102, 103, 105, 106, 108, 113, 116, 117, 124, 125, 142, 143, 152, 161, 162, 164], "infrastructur": [8, 11, 83, 164], "infti": [8, 106], "ingest": [78, 158], "inher": [8, 16, 92], "inherit": [11, 78, 95, 118, 119, 133, 147, 148, 153, 154, 158, 169, 170], "init": [1, 2, 15, 18, 98, 147, 152], "init_audio_encod": 138, "init_backend": 147, "init_build_config": 147, "init_calib_config": 147, "init_image_encod": 138, "init_llm": 138, "init_model_and_config": [83, 164], "init_processor": 138, "init_token": 138, "init_with_new_llm": 11, "initi": [1, 8, 9, 10, 11, 12, 13, 16, 17, 18, 19, 21, 22, 29, 30, 31, 32, 33, 36, 37, 39, 75, 76, 77, 78, 83, 84, 86, 88, 91, 94, 96, 102, 113, 118, 124, 127, 130, 131, 138, 139, 143, 147, 148, 152, 153, 155, 156, 157, 158, 164, 165, 170], "initial_global_assign": 16, "initial_prompt": 11, "initial_prompt_token_num": 11, "initialis": [136, 147], "initializecommand": 36, "initializer_list": [0, 1], "initmemorypool": 139, "inittozero": 1, "inject": 96, "inlin": [0, 1], "inner": [78, 133, 158], "inner_layernorm": [134, 135], "innov": [12, 15, 16, 18], "inp": [133, 147], "inpaint": [42, 70], "inplac": [10, 95], "inprogress": 1, "input": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 13, 14, 15, 16, 17, 19, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 34, 37, 38, 39, 40, 44, 47, 48, 62, 70, 84, 86, 87, 89, 91, 92, 96, 97, 103, 106, 107, 109, 110, 111, 113, 117, 118, 123, 124, 125, 126, 128, 129, 131, 132, 133, 134, 135, 136, 138, 139, 143, 147, 148, 152, 153, 154, 155, 170], "input_1": 133, "input_1_": 133, "input_audio": 138, "input_context": 67, "input_featur": 135, "input_fil": [67, 152], "input_id": [13, 39, 109, 115, 124, 133, 135, 138, 143, 148, 153], "input_imag": 138, "input_layernorm": [115, 116, 118, 148, 153], "input_length": [133, 134, 135, 138], "input_list": 133, "input_n": 133, "input_n_": 133, "input_queri": 67, "input_sequence_len": 19, "input_str": 11, "input_text": [115, 117, 138], "input_timing_cach": [23, 147], "input_token_extra_id": 138, "inputbuff": 1, "inputdesc": 117, "inputdtyp": 1, "inputgentokenshost": 1, "inputpack": [1, 106], "inputs_emb": [148, 153], "inputtokenextraid": 0, "inputtokenid": 0, "ins": 147, "insert": [10, 12, 16, 81, 82, 96, 107, 117, 124, 133, 147, 162, 163], "insert_cached_attent": [76, 78, 156, 158, 160, 161], "insert_cached_mla_attent": 161, "insertinputtensor": 1, "insid": [1, 2, 9, 12, 14, 15, 20, 29, 30, 31, 32, 33, 75, 98, 113, 118, 119, 133, 139, 151, 155], "insight": [8, 13, 16, 17], "insiz": 1, "inspect": [21, 23, 38, 78, 123, 139, 158], "inspir": [10, 12, 14], "instabl": 8, "instal": [9, 18, 21, 24, 29, 30, 31, 32, 33, 46, 61, 62, 63, 81, 94, 97, 98, 119, 120, 125, 146, 148, 151, 152, 153, 162], "instanc": [0, 8, 10, 11, 13, 16, 17, 20, 21, 29, 30, 31, 32, 33, 44, 58, 60, 75, 84, 86, 95, 96, 102, 103, 106, 107, 108, 113, 117, 136, 138, 139, 146, 147, 151, 152, 155], "instance_idx": 143, "instanti": [125, 132, 147, 169], "instead": [2, 3, 8, 11, 12, 16, 20, 21, 22, 34, 35, 44, 59, 75, 81, 86, 87, 88, 94, 96, 98, 101, 107, 109, 113, 117, 119, 131, 132, 133, 139, 147, 152, 162, 165], "instruct": [2, 15, 17, 18, 20, 21, 24, 27, 29, 30, 31, 32, 39, 40, 42, 48, 49, 54, 67, 68, 70, 78, 82, 89, 93, 95, 98, 99, 101, 113, 124, 125, 126, 131, 132, 142, 144, 146, 148, 149, 151, 152, 153, 158, 160, 163, 167, 168], "instrument": 15, "int": [0, 1, 34, 56, 58, 60, 67, 75, 84, 87, 95, 106, 115, 116, 117, 119, 130, 133, 134, 135, 136, 138, 147, 148, 153, 155, 169, 170], "int32": [1, 10, 23, 105, 133, 136, 143], "int32_t": [0, 1, 133], "int4": [7, 16, 21, 23, 44, 118, 119, 142, 152], "int4_weight": 141, "int64": [1, 10, 106, 133, 143], "int64_t": [0, 1], "int8": [1, 7, 16, 21, 22, 23, 93, 116, 118, 119, 127, 133, 139, 142, 147, 152], "int8_kv_cach": [75, 105, 141, 152], "int8_t": [0, 1], "int8_weight": 141, "int8awq": 127, "int_clip": 133, "integ": [22, 39, 73, 75, 86, 87, 91, 105, 124, 133, 136, 141, 147, 152], "integr": [11, 16, 17, 20, 21, 22, 37, 75, 76, 80, 81, 84, 89, 94, 97, 113, 146, 150, 152, 154, 155, 156, 160, 161, 162, 169, 170], "intellig": [18, 66, 92, 150], "intend": [24, 29, 30, 31, 32, 33, 36, 98, 139], "intens": [8, 10, 15, 16], "intent": 125, "intention": [9, 119, 147], "intenum": [96, 133], "inter": [16, 21, 39, 84, 102, 125, 126, 127, 130, 131, 143, 152], "inter_layernorm": 135, "inter_s": 118, "interact": [16, 17, 21, 37, 84, 98, 103, 113, 143, 151], "interchang": [10, 108], "interconect": 126, "interconnect": [106, 125, 126, 127, 130, 131], "interest": [12, 16, 39, 124], "interestingli": 20, "interf": 20, "interfac": [1, 11, 20, 34, 58, 80, 83, 96, 117, 119, 125, 138, 146, 148, 152, 153, 154, 161, 164], "interfer": [16, 17, 84, 143], "interleav": [15, 75, 91, 105, 117], "intermedi": [10, 11, 15, 22, 75, 105, 117, 143], "intermediate_s": [116, 135], "intermitt": 10, "intern": [1, 2, 13, 15, 27, 30, 36, 37, 58, 75, 83, 103, 105, 108, 112, 119, 125, 128, 129, 139, 143, 152, 164, 169], "internal_cutlass_kernel": 112, "internal_error": [22, 23, 24, 27, 79, 159], "internlm": [141, 142, 152], "internlm2": [141, 142, 152], "internvl2": 152, "interpol": 133, "interpolation_scal": 134, "interpret": [10, 20, 35, 98, 103, 130, 147], "interrupt": 20, "intersect": 102, "interst": 30, "intertwin": 131, "interv": [8, 26, 27, 29, 30, 31, 32, 147], "interven": 10, "intflag": [135, 137], "intpsplitdim": 1, "intra": 126, "introduc": [2, 4, 8, 10, 11, 12, 13, 14, 16, 17, 19, 20, 37, 75, 83, 84, 87, 88, 90, 92, 94, 119, 121, 141, 152, 164, 165, 166], "introduct": [20, 21, 97, 128, 129, 151, 152], "inttensor": [138, 148, 153], "intuit": [15, 128, 129], "inv": 133, "inv_freq": 133, "invalid": [1, 16, 20, 67, 68, 84, 102, 143, 152], "invalid_st": 1, "invalidateremoteag": 0, "inventori": [39, 124], "invers": [75, 105], "invest": [39, 124], "investig": [2, 10, 20, 152], "invoc": 152, "invok": [0, 10, 11, 16, 20, 35, 37, 76, 80, 96, 103, 107, 143, 156, 161, 170], "invokequant": 117, "involv": [0, 1, 7, 14, 15, 17, 20, 26, 75, 84, 89, 113, 117, 134, 154, 155, 169], "io": [9, 11, 18, 21, 22, 26, 29, 30, 31, 32, 75, 105, 120, 121, 139, 151, 152], "ip": [0, 152], "ipc": [9, 18, 21, 26, 29, 30, 31, 32, 98, 146, 147, 151, 152], "ipc_uc_handl": 1, "ipc_uc_ptr": 1, "ipc_uc_va": 1, "ipcmemori": 1, "ipcnvl": 1, "ipcnvlsalloc": 1, "ipcnvlsfre": 1, "ipcnvlshandl": 1, "ipcnvlssupport": 1, "ipluginv3lay": 133, "ireducelay": 133, "irrelev": 10, "irrespect": [0, 106, 147], "is_alibi": 133, "is_caus": 134, "is_comm_sess": 147, "is_const_v": 1, "is_context_fmha_en": 136, "is_cuda_graph": [75, 155], "is_cutlass_min_lat": 133, "is_def": 133, "is_dora": 110, "is_dynam": 133, "is_enc_dec": 138, "is_expert": 134, "is_fin": 147, "is_gated_activ": 133, "is_gemma_2": 135, "is_gemma_3": 135, "is_keep_al": [19, 68, 96, 147], "is_loc": 134, "is_medusa_mod": 138, "is_mla_en": 133, "is_mla_enabled_flag": 133, "is_module_excluded_from_quant": 147, "is_mrop": 133, "is_network_input": 133, "is_orchestrator_mod": 138, "is_public_pool": [19, 68, 96, 147], "is_qkv": 134, "is_redrafter_mod": 138, "is_rop": 133, "is_trt_wrapp": 133, "is_use_oldest": [19, 68, 96, 147], "is_valid": [133, 134], "is_valid_cross_attn": 134, "isagentst": 0, "isalnum": 147, "isalpha": 147, "isascii": 147, "isauto": 0, "isbeamsearch": 0, "iscomplet": 0, "iscontextparallel": 1, "iscontinuouskvcach": 1, "iscrossattent": 1, "isdecim": 147, "isdigit": 147, "isdon": 1, "isdora": 1, "isdrafttokensextern": 1, "iseagl": [0, 1], "iselectlay": 133, "isexplicitdrafttoken": [0, 1], "isexternaldrafttoken": 0, "isfin": [0, 103], "isfirstcontextparallelrank": 1, "isfirstpipelineparallelrank": 1, "isfirsttensorparallelrank": 1, "isgreedysampl": 0, "ishufflelay": 133, "isidentifi": 147, "iskeyword": 147, "iskvcacheen": 1, "isl": [0, 3, 4, 5, 6, 8, 12, 13, 14, 15, 16, 20, 21, 24, 26, 29, 30, 31, 32, 33, 39, 40, 78, 84, 97, 124, 131, 158], "isl8192": 17, "island": [30, 33], "islastpipelineparallelrank": 1, "isleg": 0, "islicelay": 133, "isload": 1, "islookahead": 0, "islookaheaddecod": 1, "islow": 147, "ismedusa": [0, 1], "ismpist": 0, "ismultimod": 1, "isn": [16, 33, 58, 143], "isnon": 1, "isnumer": 147, "isoffload": 0, "isoftmaxlay": 133, "isol": 21, "isorchestr": 0, "ispagedkvcach": 1, "isparticip": [0, 152], "ispipelineparallel": 1, "ispoint": 1, "isprint": 147, "isrnnbas": 1, "issequencefin": [0, 103], "issocketst": 0, "isspac": 147, "issu": [8, 10, 11, 12, 14, 16, 18, 21, 29, 30, 31, 32, 33, 36, 39, 40, 64, 75, 84, 86, 94, 98, 101, 102, 105, 117, 119, 124, 125, 133, 143], "istensorparallel": 1, "isthreadsaf": 0, "istitl": 147, "istopk": 0, "istopkandtopp": 0, "istopkortopp": 0, "istopp": 0, "istransformerbas": 1, "istream": [0, 1], "isunsign": 1, "isupp": 147, "isusebantoken": 0, "isusebanword": 0, "isuseexpliciteosstop": 0, "isusefrequencypenalti": 0, "isusemaxlengthstop": 0, "isuseminlength": 0, "isuseminp": 0, "isusenorepeatngrams": 0, "isuseoccurrencepenalti": 0, "isusepenalti": 0, "isusepresencepenalti": 0, "isuserepetitionpenalti": 0, "isusestopcriteria": 0, "isusestopword": 0, "isusetemperatur": 0, "isusevariablebeamwidthsearch": 0, "iswhisp": 1, "ite": 138, "item": [0, 11, 15, 103, 138, 147], "itensor": [0, 133], "itensorbind": 1, "itensorptr": 1, "iter": [0, 1, 8, 10, 11, 13, 14, 16, 19, 20, 21, 22, 27, 34, 37, 59, 75, 87, 91, 96, 103, 105, 113, 118, 124, 125, 130, 131, 132, 138, 147, 152], "iter_i": 8, "iter_stat": 152, "iter_stats_max_iter": [59, 147], "iteration_log": 22, "iterationresult": 147, "iterationstat": 0, "iterationtyp": 0, "iterlatencym": [0, 27], "iterlatencymillisec": 152, "iterstat": 0, "iterstatsmaxiter": 0, "iterstatsvec": 0, "ith": 133, "itl": [16, 33, 39, 127, 131, 152], "its": [0, 1, 3, 5, 11, 12, 13, 14, 16, 17, 19, 21, 35, 37, 39, 53, 59, 75, 84, 86, 91, 92, 94, 96, 103, 105, 106, 107, 108, 114, 116, 117, 118, 119, 124, 126, 128, 129, 130, 131, 133, 138, 139, 146, 147, 154, 155, 170], "itself": [10, 15, 16, 78, 103, 138, 147, 152, 158], "itsuji": [39, 124], "iunarylay": 133, "j": [4, 7, 8, 11, 14, 39, 61, 62, 63, 75, 105, 106, 124, 133, 141, 142, 152], "jacobi": 113, "jai": 152, "jame": 30, "jamesthez": 152, "janpetrov": 152, "japanes": [39, 110, 124], "jargon": 33, "jax": [21, 116, 119], "jenkin": [36, 97], "jensen": 58, "jersei": [30, 33], "jetson": 21, "jfk": 30, "ji": 133, "jit": [2, 152], "jj": 133, "jk": 133, "jl749": 152, "job": [35, 62, 63, 117], "john": [29, 30], "join": [11, 17, 58, 83, 84, 147, 164], "joint": 15, "joint_attention_kwarg": 135, "joint_attn_forward": 134, "joke": 66, "journei": [13, 151], "jpeg": 27, "jpg": [39, 124], "json": [0, 1, 9, 10, 16, 18, 21, 22, 24, 26, 29, 30, 31, 32, 33, 36, 38, 39, 41, 42, 43, 47, 48, 53, 67, 74, 84, 95, 97, 103, 116, 123, 124, 136, 147, 151, 152, 160], "json_data": 147, "json_object": 147, "json_schema": 147, "json_schema_extra": 147, "jsonconfigstr": 0, "jsondecodeerror": 73, "jsonl": [22, 39, 67, 124], "jsonseri": 0, "judgement": 16, "juli": 12, "just": [0, 1, 11, 14, 15, 16, 19, 29, 30, 31, 32, 33, 39, 60, 61, 62, 63, 64, 89, 96, 98, 113, 124, 125, 132, 138, 139, 152], "justic": [49, 64, 101, 144, 146, 151], "justifi": 147, "k": [1, 13, 14, 15, 19, 37, 66, 75, 92, 95, 96, 97, 104, 105, 106, 110, 113, 133, 141, 143, 147, 148, 152, 153, 155, 168], "k8": 97, "k_b_proj_tran": 133, "k_dim": 133, "k_nope": 12, "k_nope_s": 12, "k_proj": [39, 83, 88, 118, 124, 148, 153, 164, 165], "kattent": 1, "kattn_dens": 1, "kattn_k": 1, "kattn_q": 1, "kattn_qkv": 1, "kattn_v": 1, "kauto": 0, "kb": 20, "kbatchedpostprocessornam": [0, 103], "kbeamsearch": 0, "kbf16": 0, "kblk": 0, "kbool": [0, 1], "kbyte_typ": 1, "kc_cache_retention_config": 152, "kcancel": 0, "kchatglm": 1, "kcontext": 1, "kcontext_in_progress": 0, "kcontinu": 1, "kcpu": [0, 1], "kcpu_pin": 0, "kcpu_pinnedpool": 0, "kcross_attn_dens": 1, "kcross_attn_k": 1, "kcross_attn_q": 1, "kcross_attn_qkv": 1, "kcross_attn_v": 1, "kdatatyp": 1, "kdecoder_onli": [0, 114], "kdefault": 0, "kdefault_num_tokens_per_block": 1, "kdefaultbatchsizet": 0, "kdefaultdynamicbatchmovingaveragewindow": 0, "kdefaultgpumemfract": 0, "kdefaultgpuspernod": 1, "kdefaultiterstatsmaxiter": 0, "kdefaultlookaheaddecodingngram": 0, "kdefaultlookaheaddecodingverificationset": 0, "kdefaultlookaheaddecodingwindow": 0, "kdefaultmaxadapters": 0, "kdefaultmaxpagesperblockdevic": 0, "kdefaultmaxpagesperblockhost": 0, "kdefaultmaxseqidlemicrosecond": 0, "kdefaultoptimaladapters": 0, "kdefaultprior": 0, "kdefaultrequeststatsmaxiter": 0, "kdefaultretentionprior": 0, "kdisabl": 1, "kdrafttokensextern": 1, "kdram": 0, "kdynamicpostprocessornameprefix": 0, "keagl": [0, 1], "kebnf_grammar": [0, 103], "keep": [0, 12, 13, 15, 16, 19, 21, 33, 35, 40, 75, 91, 105, 106, 112, 119, 132, 133, 147, 152], "keepdim": 133, "keepend": 147, "kei": [0, 3, 7, 8, 9, 11, 12, 14, 15, 16, 19, 20, 21, 22, 33, 37, 39, 40, 59, 75, 77, 83, 86, 89, 91, 96, 97, 103, 109, 117, 124, 130, 135, 138, 143, 147, 154, 155, 157, 164, 169], "kenabl": 1, "kencdec": 1, "kencoder_decod": 0, "kencoder_in_progress": 0, "kencoder_onli": 0, "kend_id": 0, "kennedi": 30, "kept": [16, 19, 75, 105, 119, 133, 147], "kequal_progress": 0, "kera": 119, "kernel": [1, 3, 14, 15, 17, 23, 29, 31, 32, 37, 38, 65, 67, 75, 77, 82, 91, 93, 96, 97, 105, 109, 112, 117, 123, 127, 131, 133, 136, 138, 139, 143, 147, 150, 152, 157, 163], "kernel_s": [67, 133, 134, 147], "kexplicitdrafttoken": [0, 1], "kexternaldrafttoken": 0, "key_length": [133, 134], "keyvaluecacheparam": [134, 135], "keyword": [78, 118, 133, 139, 147, 152, 158], "kfile": 0, "kfirst_come_first_serv": 0, "kfloat": [1, 117], "kfp16": 0, "kfp32": [0, 147], "kfp8": 0, "kgener": 1, "kgeneration_complet": 0, "kgeneration_in_progress": 0, "kglm": 1, "kgpt": 1, "kgpu": [0, 1], "kguaranteed_no_evict": 0, "khalf": 1, "kick": 35, "kill": 18, "kind": [10, 11, 13, 16, 104, 105, 107, 170], "kinflight": 0, "king": 30, "kint32": [0, 1], "kint64": [0, 1], "kint8": [0, 1], "kinvalid": 1, "kispoint": 1, "kisunsign": 1, "kj": 133, "kjson": [0, 103], "kjson_schema": [0, 103], "kleader": 0, "klength": 0, "klinear": 1, "kllguidanc": 0, "klookahead": 0, "klookaheaddecod": 1, "km": 30, "kmamba": 1, "kmax_util": 0, "kmaxretentionprior": 0, "kmedusa": [0, 1], "kminretentionprior": 0, "kmla": 0, "kmlp_4h_to_h": 1, "kmlp_gate": 1, "kmlp_gate_up": 1, "kmlp_h_to_4h": 1, "kmlp_router": 1, "kmoe_4h_to_h": 1, "kmoe_gat": 1, "kmoe_h_to_4h": 1, "kmoe_rout": 1, "kmpi": 0, "knegativeinfin": 1, "knob": [0, 26, 132, 133, 147, 152], "knone": 1, "knoop": 1, "knot_finish": 0, "know": [11, 30, 33, 38, 51, 106, 123, 132, 133], "knowledg": [19, 33], "known": [11, 14, 16, 29, 33, 35, 75, 91, 92, 101, 105, 112, 113, 117, 133, 142, 151], "knumflag": 0, "kobj": 0, "kopt_profiles_split_point": 1, "korchestr": 0, "korea": 27, "kosmo": [142, 152], "kpage": 1, "kpin": 1, "kpinnedpool": 1, "kqueu": 0, "kread": 0, "krecurr": 1, "krecurrentgemma": 1, "kregex": [0, 103], "kstatic": 0, "kstatic_batch": 0, "kstop_word": 0, "kstructural_tag": 0, "ktimed_out": 0, "ktopk": 0, "ktopktopp": 0, "ktopp": 0, "ktrtpointertyp": 1, "kubernet": [17, 46, 84], "kuint8": [0, 1], "kunderlyingtyp": 1, "kunish": 110, "kunknown": 0, "kunsign": 1, "kusebantoken": 0, "kusebanword": 0, "kuseexpliciteosstop": 0, "kusefrequencypenalti": 0, "kusemaxlengthstop": 0, "kuseminlength": 0, "kuseminp": 0, "kusenorepeatngrams": 0, "kuseoccurrencepenalti": 0, "kusepenalti": 0, "kusepresencepenalti": 0, "kuserepetitionpenalti": 0, "kusestandardstopcriteria": 0, "kusestopword": 0, "kusetemperatur": 0, "kusevariablebeamwidthsearch": 0, "kuvm": [0, 1], "kv": [0, 1, 3, 7, 8, 12, 14, 15, 18, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 37, 39, 40, 44, 51, 52, 65, 67, 76, 77, 81, 82, 85, 87, 89, 92, 97, 102, 103, 110, 117, 119, 122, 124, 125, 130, 133, 136, 138, 140, 145, 147, 148, 149, 150, 152, 153, 154, 155, 156, 157, 160, 162, 163, 170], "kv_b_proj": 133, "kv_cach": [0, 65], "kv_cache_block_offset": [133, 134, 138, 143], "kv_cache_block_point": 143, "kv_cache_config": [9, 27, 33, 39, 44, 50, 58, 59, 65, 67, 68, 87, 93, 132, 147, 169], "kv_cache_connector": 58, "kv_cache_dtyp": [2, 26, 67, 124, 127, 137, 169], "kv_cache_enable_block_reus": [138, 152], "kv_cache_fract": 67, "kv_cache_free_gpu_mem_fract": [2, 16, 21, 22, 40, 132], "kv_cache_free_gpu_memory_fract": [17, 18, 21, 24, 26, 27, 45, 138, 152], "kv_cache_host_memory_byt": 109, "kv_cache_host_s": 59, "kv_cache_manag": [0, 75, 152, 154, 155, 169, 170], "kv_cache_max_token": 59, "kv_cache_page_s": 59, "kv_cache_param": [75, 134, 135, 155], "kv_cache_quant_algo": [39, 88, 116, 124, 127, 147, 165], "kv_cache_quant_mod": [105, 133], "kv_cache_retention_config": [51, 147], "kv_cache_reus": 27, "kv_cache_scaling_factor": [75, 105, 116], "kv_cache_tensor": 58, "kv_cache_typ": [23, 117, 138, 147, 152], "kv_connector_config": [58, 147], "kv_dtype": 135, "kv_head": 134, "kv_host_cache_byt": 109, "kv_lora_rank": [133, 134], "kv_orig_quant_scal": 133, "kv_quant_orig_scal": 133, "kvalue_status_load": 1, "kvalue_status_miss": 1, "kvalue_status_process": 1, "kvcach": [0, 11, 13, 84, 102, 138, 152], "kvcacheblock": 108, "kvcacheblockpool": 108, "kvcacheconfig": [0, 44, 50, 59, 65, 67, 68, 86, 87, 93, 97, 105, 109, 132, 139, 147, 152], "kvcacheconnectorconfig": [58, 147], "kvcacheconnectorschedul": 58, "kvcacheconnectorwork": 58, "kvcachecreateddata": [0, 147], "kvcacheev": 0, "kvcacheeventdata": 0, "kvcacheeventdiff": 0, "kvcacheeventmanag": 0, "kvcachehitr": 0, "kvcachehitrateperrequest": 0, "kvcacheindex": 1, "kvcachemanag": [0, 37, 75, 91, 105, 109, 138, 155, 169], "kvcachemetr": 0, "kvcacheparam": [75, 155], "kvcacheremoveddata": [0, 147], "kvcacheretentionconfig": [0, 51, 97, 147], "kvcaches": 0, "kvcachestat": [0, 27], "kvcachestoredblockdata": 0, "kvcachestoreddata": [0, 147], "kvcachetransferend": 0, "kvcachetransferm": 0, "kvcachetransfermod": [0, 147], "kvcachetransferstart": 0, "kvcachetyp": [1, 138, 147], "kvcachetypefromstr": 1, "kvcacheupdateddata": [0, 147], "kvfactor": 0, "kvheadnum": 133, "kvram": 0, "kwarg": [11, 24, 34, 77, 78, 80, 83, 118, 119, 133, 134, 135, 138, 147, 148, 152, 153, 157, 158, 161, 164], "kwd": 147, "kwrite": 0, "kxgrammar": 0, "l": [27, 39, 61, 62, 63, 113, 124, 142, 149], "l0_a100": 35, "l0_mergerequest": 35, "l0_sanity_check": 35, "l0_test": 35, "l2": [23, 152], "l20": 23, "l304": 13, "l345": 13, "l4": [21, 23], "l40": [21, 23, 150], "l440": 13, "l506": 13, "l546": 13, "l823": 13, "lab": [11, 39, 124], "label": [107, 133, 134, 135, 147], "labelembed": 134, "lack": [0, 1], "laguardia": 30, "lai": 14, "lake": [30, 33], "lambda": [0, 103], "lamportinitializeal": 1, "landmark": [30, 33], "langchain": 11, "languag": [0, 3, 8, 12, 13, 16, 17, 19, 37, 38, 81, 88, 90, 93, 96, 106, 113, 117, 123, 133, 141, 142, 149, 150, 152, 154, 162, 165, 166], "language_adapt": [138, 152], "language_adapter_config": 138, "language_adapter_rout": [135, 138], "language_adapter_uid": 138, "language_model": 118, "languageadapterconfig": 138, "languageadapteruid": 0, "laps": 51, "laptop": 21, "larg": [0, 2, 3, 7, 8, 12, 13, 15, 17, 18, 20, 23, 27, 37, 38, 39, 42, 59, 70, 88, 89, 90, 91, 92, 96, 97, 105, 109, 111, 113, 117, 119, 123, 124, 126, 127, 130, 131, 133, 136, 138, 139, 142, 143, 147, 150, 152, 154, 165, 166], "larger": [0, 4, 5, 7, 15, 17, 19, 20, 21, 33, 37, 40, 59, 75, 105, 106, 109, 113, 114, 124, 133, 138, 139, 147, 152], "largest": [3, 4, 5, 30, 33, 96, 106, 133], "last": [0, 1, 10, 12, 13, 14, 16, 18, 19, 26, 29, 30, 31, 32, 59, 60, 75, 87, 91, 103, 105, 110, 111, 113, 115, 130, 132, 133, 135, 147], "last_lay": 138, "last_process_for_ub": 133, "last_token_id": [133, 135, 143], "last_token_ids_for_logit": 135, "last_tokens_id": 133, "lastdraftindic": 1, "lastdraftlen": 1, "lastdraftpath": 1, "lastdrafttoken": 1, "lastgenerationlength": 1, "lastit": 0, "lastli": 21, "lastpositionidsbas": 1, "lasttokentim": 0, "late": [11, 64], "latenc": [0, 4, 5, 7, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 23, 37, 39, 40, 75, 76, 84, 91, 96, 97, 105, 109, 113, 130, 131, 132, 133, 136, 147, 150, 152, 156], "latent": [12, 15, 77, 134, 135, 157], "later": [0, 1, 5, 18, 29, 30, 31, 32, 33, 78, 86, 106, 110, 113, 117, 119, 127, 131, 138, 139, 143, 145, 158], "latest": [0, 2, 15, 18, 29, 30, 31, 32, 89, 93, 97, 98, 117, 120, 152], "latitud": 33, "latter": [7, 30, 36, 91, 103, 152], "launch": [10, 15, 16, 17, 20, 27, 37, 61, 62, 63, 65, 89, 90, 96, 97, 98, 101, 109, 117, 125, 143, 145, 146, 152, 160, 166], "launch_hostfunc": 10, "launch_llama_3": 117, "layer": [0, 1, 12, 14, 16, 17, 20, 21, 23, 75, 78, 84, 86, 87, 91, 92, 96, 97, 102, 104, 105, 106, 107, 108, 110, 113, 115, 116, 117, 118, 126, 133, 136, 138, 139, 141, 143, 147, 148, 152, 153, 155, 158, 169], "layer1": 110, "layer_id": 96, "layer_idx": [58, 75, 110, 115, 133, 138, 148, 153, 155], "layer_names_onli": [23, 147], "layer_norm": [133, 134], "layer_quant_mod": 147, "layer_typ": 138, "layer_updates_per_it": [16, 29], "layerid": [1, 110], "layeridx": 1, "layernorm": [23, 115, 131, 133, 134, 136, 152], "layernorm_quantization_plugin": 136, "layernorm_shar": 134, "layernorm_typ": 134, "layernormpositiontyp": 133, "layernormtyp": [133, 134], "layertyp": [1, 107], "layerwis": 147, "layout": [9, 75, 91, 130, 152], "lead": [8, 11, 12, 16, 17, 20, 21, 23, 29, 31, 39, 40, 64, 84, 91, 98, 102, 107, 109, 113, 117, 124, 125, 127, 130, 131, 136, 147, 152], "leader": [0, 18, 138], "leaf": 86, "leak": [10, 152], "learn": [4, 5, 7, 16, 18, 19, 21, 49, 54, 55, 57, 94, 101, 127, 133, 144, 146, 151], "learned_absolut": [116, 133, 134, 135], "least": [0, 16, 19, 27, 64, 75, 86, 91, 103, 105, 119, 130, 138, 147], "leav": [17, 84, 86, 91, 130, 131, 132], "left": [11, 17, 19, 20, 40, 78, 130, 132, 133, 147, 158], "legaci": [11, 118, 132, 136, 152], "legend": 8, "lego": 11, "len": [0, 1, 11, 26, 29, 30, 31, 32, 33, 39, 58, 60, 67, 77, 124, 133, 138, 147, 157, 170], "length": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 17, 19, 21, 22, 23, 24, 27, 29, 30, 31, 32, 33, 39, 40, 51, 60, 67, 75, 77, 84, 86, 87, 91, 96, 105, 109, 124, 125, 128, 129, 131, 132, 133, 138, 139, 143, 147, 152, 155, 157, 169], "length_penalti": [106, 138, 147], "lengthlengthpenalti": 106, "lengthpenalti": [0, 1, 106], "leq": 8, "less": [0, 4, 10, 14, 16, 19, 40, 75, 86, 87, 95, 103, 105, 106, 117, 133, 147], "lesser": 86, "let": [10, 11, 12, 13, 16, 21, 30, 39, 44, 91, 107, 115, 116, 118, 120, 124, 130, 133], "letter": 133, "level": [0, 1, 8, 10, 11, 12, 14, 15, 17, 20, 21, 22, 23, 24, 27, 34, 38, 39, 75, 78, 80, 81, 91, 95, 96, 97, 103, 105, 108, 110, 112, 115, 116, 118, 119, 123, 124, 139, 146, 147, 148, 150, 152, 153, 158, 161, 162], "leverag": [3, 8, 10, 12, 13, 14, 16, 17, 18, 19, 21, 29, 30, 31, 32, 33, 81, 82, 89, 113, 127, 150, 162, 163, 164], "lf": [2, 18, 98, 110, 146], "lfz941": 152, "lga": 30, "lgai": [142, 149], "lh": 1, "lib": [119, 124], "liberti": [30, 33], "libnam": 0, "libnvinfer_plugin_tensorrt_llm": 98, "libopenmpi": [81, 101, 162], "librari": [16, 17, 20, 21, 22, 31, 75, 81, 82, 84, 98, 112, 117, 143, 146, 150, 152, 155, 162, 163], "libtensorrt_llm": 98, "libzmq3": 101, "licens": [100, 101, 146], "life": 64, "lifecycl": [46, 94, 108], "lifetim": 20, "light": [82, 163], "lightweight": [10, 16, 19, 24, 81, 82, 105, 162, 163], "like": [0, 7, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 26, 29, 30, 31, 32, 33, 35, 36, 37, 39, 44, 49, 53, 54, 55, 56, 57, 59, 60, 61, 62, 63, 64, 66, 75, 78, 81, 82, 83, 84, 86, 91, 92, 93, 96, 99, 101, 102, 103, 105, 106, 107, 109, 113, 116, 117, 119, 124, 125, 126, 127, 130, 131, 132, 133, 136, 139, 143, 144, 145, 146, 147, 148, 151, 152, 153, 154, 158, 162, 163, 164, 169], "likelihood": [60, 104, 109, 113], "likewis": 83, "limit": [0, 7, 8, 10, 11, 13, 14, 15, 16, 17, 18, 20, 21, 22, 27, 44, 59, 75, 84, 87, 91, 96, 97, 101, 102, 103, 105, 106, 107, 117, 119, 125, 130, 132, 133, 136, 138, 139, 142, 146, 147, 155], "lin": 3, "line": [2, 7, 17, 22, 26, 29, 30, 31, 32, 33, 35, 36, 39, 50, 60, 64, 67, 78, 86, 98, 109, 124, 125, 127, 131, 139, 147, 151, 152, 158, 169, 170], "linear": [1, 12, 15, 75, 81, 110, 113, 115, 116, 117, 133, 139, 141, 148, 152, 153, 155, 162], "linearactiv": 134, "linearapproximategelu": 134, "linearbas": 134, "lineargeglu": 134, "lineargelu": 134, "linearli": 139, "linearswiglu": 134, "link": [2, 13, 19, 20, 21, 35, 109, 120, 121, 152], "linspac": 133, "lint": 147, "linux": [20, 29, 30, 31, 32, 33, 36, 100, 142, 152], "linux_x86_64": 98, "list": [0, 1, 10, 14, 16, 26, 29, 30, 31, 32, 33, 34, 35, 36, 39, 44, 51, 58, 60, 75, 76, 77, 78, 82, 83, 86, 87, 95, 96, 97, 98, 103, 105, 106, 107, 116, 117, 118, 122, 124, 125, 133, 134, 135, 138, 142, 143, 147, 152, 155, 156, 157, 158, 163, 164, 169, 170], "list_siz": 134, "listen": 9, "liter": [33, 136, 147], "littl": [14, 16, 131], "live": [35, 139], "livecodebench": 13, "ljust": 147, "lkm2835": 152, "ll": [7, 14, 21, 33, 96], "ll128": [12, 16, 20], "llama": [4, 5, 7, 10, 18, 19, 23, 24, 26, 31, 32, 39, 49, 54, 67, 68, 72, 76, 78, 81, 82, 83, 93, 94, 95, 96, 97, 101, 106, 110, 113, 114, 118, 119, 125, 126, 128, 129, 130, 132, 135, 136, 140, 141, 142, 144, 145, 146, 148, 149, 150, 151, 152, 153, 156, 158, 160, 162, 163, 164, 167, 168], "llama2": [3, 4, 75, 97, 110, 152], "llama3": [26, 28, 68, 97, 133, 152], "llama4": [16, 28, 87, 92, 97, 147, 152], "llama4_output": 32, "llama4forconditionalgener": [142, 149], "llama_13b": 5, "llama_70b": 5, "llama_7b": [110, 114], "llama_7b_with_lora_qkv": 110, "llama_model_path": 44, "llamaconfig": [135, 148, 153], "llamaforcausallm": [118, 119, 135, 142, 149], "llamamodel": 135, "llava": [93, 118, 140, 141, 142, 152], "llava_dict": 118, "llavallamamodel": [142, 149], "llavanextforconditionalgener": [142, 149], "llavanextvisionconfig": 135, "llavanextvisionwrapp": 135, "llguidanc": [0, 10, 53, 73, 95, 147, 152], "llm": [0, 3, 6, 8, 13, 15, 22, 23, 24, 26, 27, 28, 35, 36, 37, 38, 40, 41, 42, 43, 45, 46, 47, 48, 50, 51, 53, 54, 55, 56, 58, 59, 60, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 75, 76, 80, 82, 83, 84, 86, 87, 88, 91, 94, 95, 96, 99, 100, 101, 102, 103, 105, 106, 107, 108, 109, 110, 111, 115, 117, 122, 123, 126, 127, 128, 129, 131, 132, 133, 135, 136, 137, 138, 141, 143, 145, 147, 148, 153, 154, 155, 156, 161, 163, 164, 165, 167, 168, 169, 170], "llm_advanc": 65, "llm_arg": [34, 40, 58, 78, 80, 88, 147, 158, 161, 165], "llm_engine_dir": 138, "llm_id": 147, "llm_inference_async_rai": 94, "llm_inference_distribut": 146, "llm_inference_distributed_rai": 94, "llm_kv_cache_offload": 59, "llm_mgmn_": 152, "llm_models_root": 72, "llm_option": 40, "llm_ptq": [93, 167], "llm_sampl": 66, "llm_sparse_attent": 67, "llm_util": 147, "llm_worker": 11, "llmapi": [16, 26, 27, 34, 40, 44, 50, 51, 53, 58, 59, 61, 62, 63, 65, 67, 68, 88, 93, 95, 96, 103, 127, 147, 152, 165], "llmarg": [34, 40, 78, 97, 111, 147, 152, 158], "llmrequest": [1, 58, 152, 169, 170], "llmrequestptr": 1, "llmrequestst": 170, "lm": [29, 31, 32, 113, 147], "lm_eval": [24, 29, 31, 32], "lm_head": [10, 115, 118, 124, 152], "lmm": [39, 106, 124], "lmsy": [90, 166], "ln_emb": 118, "ln_f": [115, 118], "lo": 146, "load": [0, 1, 6, 9, 12, 13, 15, 17, 18, 21, 22, 23, 29, 30, 31, 32, 33, 37, 40, 58, 59, 67, 73, 76, 77, 78, 81, 84, 88, 97, 110, 115, 116, 117, 119, 124, 125, 131, 132, 135, 137, 138, 139, 146, 147, 150, 151, 152, 156, 157, 158, 160, 162, 165], "load_balanc": [16, 29, 147], "load_base64_imag": 27, "load_config": [83, 164], "load_format": 147, "load_model_on_cpu": 135, "load_tensor": 118, "load_test_audio": 138, "load_test_data": 138, "load_weight": [83, 148, 153, 164], "loaded_weight": 134, "loader": [97, 147, 152], "loadformat": 147, "loadinprogress": 1, "loadremoteag": 0, "loadweight": 1, "local": [1, 2, 9, 13, 16, 21, 23, 24, 29, 31, 32, 33, 36, 39, 40, 49, 54, 55, 56, 57, 61, 62, 63, 68, 81, 92, 98, 99, 101, 116, 117, 124, 127, 136, 144, 147, 151, 152, 160, 162, 169], "local_build": 36, "local_in_featur": 134, "local_layer_idx": 134, "local_model": [61, 62, 63], "local_model_path": 21, "local_out_featur": 134, "local_path": 31, "local_path_to_model": 146, "local_us": [2, 33, 98], "localcr": 1, "localhost": [9, 17, 18, 21, 26, 27, 29, 30, 31, 32, 33, 41, 42, 43, 45, 47, 48, 69, 70, 71, 72, 73, 84, 88, 151, 160, 165], "localinadapters": 1, "localindim": 1, "localinouts": 1, "localins": 1, "localn": 147, "localoutadapters": 1, "localoutdim": 1, "localouts": 1, "localreduct": 13, "localscaless": 1, "localtotals": 1, "locat": [15, 16, 20, 29, 30, 31, 32, 33, 35, 40, 75, 98, 106, 107, 117, 124, 133, 143, 151, 155], "locate_accepted_draft_token": 138, "lock": [10, 16, 20, 39, 124], "lockstep": 0, "log": [0, 1, 9, 20, 21, 22, 23, 24, 26, 27, 39, 59, 61, 62, 63, 66, 75, 81, 95, 96, 105, 108, 120, 124, 133, 136, 139, 147, 152, 162, 168], "log_ctx_0": 84, "log_ctx_1": 84, "log_field_chang": 136, "log_gen_0": 84, "log_level": [22, 23, 24, 27], "log_sampl": [29, 31, 32], "log_softmax": 133, "logger": [58, 79, 159], "logic": [10, 11, 17, 20, 24, 36, 37, 60, 83, 90, 95, 96, 103, 108, 118, 119, 134, 135, 147, 148, 152, 153, 154, 164, 170], "login": [9, 120], "logit": [0, 1, 10, 12, 13, 14, 37, 39, 52, 85, 96, 97, 106, 113, 124, 133, 138, 143, 147, 149, 152, 168], "logits_dtyp": [23, 116, 135], "logits_processor": [60, 95, 138, 147], "logits_processor_map": 138, "logits_processor_nam": 138, "logitspostprocessor": 0, "logitspostprocessorbatch": [0, 103], "logitspostprocessorconfig": [0, 103, 152], "logitspostprocessormap": 0, "logitspostprocessornam": 0, "logitsprocessor": [60, 95, 138, 147, 152], "logitsprocessorlist": 138, "logitsvec": 1, "logn": [133, 152], "logn_scal": 133, "logprob": [0, 1, 18, 21, 29, 30, 31, 32, 33, 44, 66, 147, 151, 152], "logprobs_diff": 147, "logprobscba": 1, "logprobstil": 1, "london": 143, "long": [7, 8, 10, 12, 16, 17, 23, 29, 30, 31, 32, 33, 38, 75, 84, 86, 88, 91, 92, 97, 105, 123, 124, 125, 126, 127, 130, 131, 136, 139, 150, 152, 165], "long_factor": 133, "long_mscal": [133, 134], "long_rop": 133, "long_rope_embed_posit": 134, "long_rope_embed_positions_for_gpt_attent": 134, "long_rope_rotary_cos_sin": 133, "long_rope_rotary_inv_freq": [133, 134], "longer": [0, 8, 10, 13, 15, 16, 18, 20, 21, 29, 30, 31, 32, 33, 40, 87, 91, 106, 109, 130, 133, 147, 170], "longest": [14, 91, 130, 133], "longitud": 33, "longrop": 133, "longtensor": [60, 138], "look": [0, 6, 10, 11, 12, 16, 39, 58, 84, 96, 98, 103, 119, 124, 152], "lookahead": [0, 1, 147, 152], "lookahead_config": [138, 147], "lookahead_decod": [23, 135], "lookaheadalgoconfig": 1, "lookaheadconfig": 0, "lookaheaddecod": 1, "lookaheaddecodingbuff": 1, "lookaheaddecodingconfig": [0, 1, 97, 147], "lookaheadinput": 1, "lookaheadoutput": 1, "lookaheadprompt": 1, "lookaheadruntimebuff": 1, "lookup": [19, 35, 96, 133, 134, 147, 152], "lookup_plugin": 133, "lookuperror": 147, "loop": [0, 10, 11, 19, 20, 37, 96, 103, 106, 117, 118, 132, 147], "lopuhin": 152, "lora": [0, 1, 22, 52, 74, 81, 83, 85, 97, 103, 122, 133, 134, 135, 136, 138, 147, 150, 152, 162, 164], "lora_0": [39, 124], "lora_adapt": [88, 165], "lora_ckpt_sourc": [23, 88, 138, 147, 165], "lora_config": [39, 64, 88, 124, 135, 147, 152, 165], "lora_dir": [23, 39, 64, 88, 110, 124, 138, 165], "lora_help": [64, 147, 165], "lora_hidden_st": 134, "lora_int_id": [39, 72, 88, 124, 147, 165], "lora_layer_param": 134, "lora_manag": [88, 138, 152], "lora_nam": [39, 72, 88, 124, 147, 165], "lora_param": 135, "lora_path": [39, 72, 88, 124, 147, 165], "lora_plugin": [23, 110, 133, 136, 138], "lora_rank": [110, 133], "lora_req1": [88, 165], "lora_req2": [88, 165], "lora_request": [39, 64, 72, 88, 124, 147, 165], "lora_runtime_param": 134, "lora_target_modul": [23, 39, 88, 110, 124, 135, 138, 165], "lora_task_uid": 138, "lora_uid": 138, "lora_weights_point": 133, "loracachefullexcept": 1, "loracachepagemanag": 1, "loraconfig": [0, 64, 88, 110, 135, 147, 152, 165], "loraexpectedexcept": 1, "loraid": 0, "loramanag": 138, "loramodulenam": 1, "loraparam": 135, "loraprefetchdir": 0, "lorarequest": [64, 88, 97, 147, 165], "loraruntimeparam": 134, "lorataskidtyp": [0, 1], "loraweight": 110, "loss": [7, 8, 21, 127], "lost": 152, "lot": [14, 75, 91, 105, 109, 117], "loudspeak": 5, "love": 18, "lovelac": [93, 142, 150, 152], "low": [2, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 19, 20, 23, 29, 30, 31, 32, 39, 75, 81, 95, 96, 97, 105, 115, 119, 133, 136, 147, 150, 152, 162], "low_lat": 21, "low_latency_benchmark": 21, "low_latency_gemm": [112, 133], "low_latency_gemm_plugin": [23, 124, 127, 134, 136], "low_latency_gemm_swiglu": 133, "low_latency_gemm_swiglu_plugin": [23, 127, 136], "low_rank": 133, "lower": [0, 1, 2, 6, 7, 8, 15, 17, 21, 29, 30, 31, 32, 33, 34, 40, 66, 78, 84, 86, 91, 93, 96, 97, 102, 106, 107, 109, 110, 127, 133, 139, 147, 158], "lowercas": 147, "lowest": [21, 51, 78, 86, 147, 158], "lowprecis": [111, 133, 147], "lowprecisiondata": 12, "loyalti": 60, "lpddr5x": 16, "lru": [1, 86, 109, 133], "lstrip": 147, "lt": 133, "lunch": 16, "luotuo": [72, 110], "m": [0, 2, 4, 8, 13, 16, 17, 20, 22, 26, 27, 29, 30, 31, 32, 33, 34, 39, 40, 47, 48, 53, 64, 88, 124, 125, 127, 130, 131, 133, 139, 141, 165], "ma": [30, 33], "macceptancethreshold": 0, "machin": [2, 7, 18, 109, 152], "macro": 112, "madditionalmodeloutput": 0, "maddr": 0, "maddress": 1, "made": [12, 20, 86, 99, 152, 170], "madv_hugepag": 20, "madvis": 20, "magentnam": 0, "magic": 16, "magpi": 19, "mahmoudashraf97": 152, "mai": [0, 1, 2, 8, 10, 11, 12, 13, 14, 16, 17, 18, 20, 21, 23, 26, 29, 30, 31, 32, 33, 34, 36, 38, 39, 40, 50, 61, 62, 63, 75, 78, 81, 84, 91, 96, 98, 102, 103, 105, 106, 109, 110, 111, 112, 113, 116, 117, 118, 119, 120, 123, 124, 125, 131, 132, 133, 134, 136, 139, 143, 146, 147, 148, 149, 152, 153, 154, 155, 158, 162, 169], "main": [3, 6, 10, 11, 12, 13, 14, 16, 18, 19, 20, 22, 29, 30, 31, 32, 38, 42, 44, 49, 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 64, 65, 66, 67, 68, 70, 83, 96, 101, 103, 106, 108, 123, 125, 127, 131, 132, 133, 139, 143, 144, 146, 147, 148, 151, 153, 164], "mainland": 30, "mainli": [14, 16, 147], "mainstream": [17, 84], "maintain": [3, 4, 7, 8, 12, 16, 39, 81, 83, 92, 95, 96, 110, 112, 124, 127, 141, 152, 162, 164], "mainten": [8, 37], "major": [13, 20, 30, 34, 119, 139, 152], "majority_vote_control": 11, "majorityvot": 11, "majorityvotecontrol": 11, "make": [1, 2, 7, 8, 11, 12, 13, 14, 16, 18, 19, 20, 21, 29, 30, 31, 32, 33, 39, 58, 59, 64, 75, 78, 84, 91, 94, 96, 97, 98, 101, 102, 105, 107, 110, 112, 117, 119, 120, 121, 124, 125, 132, 133, 143, 146, 147, 150, 152, 158], "make_causal_mask": 134, "make_env": 36, "makedir": 58, "makeloopbackag": 0, "makeshap": 1, "maketran": 147, "maketransferag": 0, "malachowski": 58, "malici": 86, "mallotedtim": 0, "mallreducecommptr": 1, "mamba": [23, 133, 136, 141, 142, 147, 152], "mamba1": 133, "mamba2": [133, 152], "mamba_conv1d": 133, "mamba_conv1d_plugin": [23, 136, 138], "mamba_ssm_cache_dtyp": 147, "mamba_vers": 133, "mambaconfig": 135, "mambaforcausallm": 135, "manag": [0, 1, 10, 11, 15, 16, 17, 21, 23, 37, 44, 46, 59, 60, 75, 76, 82, 83, 84, 91, 94, 96, 97, 101, 102, 105, 113, 117, 125, 132, 136, 138, 139, 145, 146, 150, 151, 152, 154, 155, 156, 163, 164], "manage_weight": 136, "managedweight": 0, "managedweightsmap": 1, "manageweightstyp": 1, "manageweighttyp": 1, "mandatori": [1, 34, 103, 116], "manhattan": [30, 33], "mani": [0, 10, 11, 14, 15, 16, 18, 19, 21, 23, 26, 29, 30, 31, 32, 40, 75, 84, 86, 87, 91, 96, 102, 105, 108, 109, 113, 117, 119, 121, 127, 130, 132, 133, 136, 142, 143, 147], "manipul": 107, "manner": [12, 16, 20, 107], "mantissa": 4, "manual": [10, 15, 16, 35, 44, 78, 81, 93, 101, 138, 143, 146, 147, 158, 162], "manufactur": [39, 124], "map": [0, 1, 8, 10, 11, 13, 16, 17, 18, 20, 21, 29, 30, 31, 32, 34, 35, 39, 40, 75, 83, 84, 92, 96, 103, 105, 107, 111, 115, 116, 117, 118, 119, 133, 134, 135, 136, 138, 147, 152, 164, 169], "map_loc": 58, "map_weight": [83, 164], "mappingintstrani": 147, "mappingnamespac": 147, "mard1no": 152, "margin": [10, 39, 124, 130], "mark": [1, 12, 19, 34, 35, 84, 91, 96, 107, 130, 133, 143, 149], "mark_as_remov": 107, "mark_output": [103, 133], "markalldon": 1, "markdon": 1, "markdown": 33, "marker": [11, 20, 35, 147], "market": 60, "marks101": 152, "marktaskdon": 1, "mask": [0, 1, 13, 14, 75, 87, 105, 113, 133, 134, 135, 138, 155], "mask_typ": 133, "masked_scatt": 133, "masked_scatter_": 133, "masked_select": [133, 152], "massachusett": 30, "massiv": [2, 21], "master": [126, 127, 128, 129], "mat2": 133, "match": [0, 14, 17, 19, 20, 24, 29, 30, 31, 32, 33, 34, 35, 37, 39, 58, 76, 82, 86, 95, 96, 104, 107, 113, 124, 133, 134, 138, 143, 147, 152, 156, 160, 163], "match_and_rewrit": 107, "materi": [1, 103], "materializewithtag": 1, "math": [11, 12, 13, 15, 142, 149], "math500": 11, "matichon": 152, "matmul": [23, 105, 117, 127, 133, 136, 141], "matric": [88, 104, 165], "matrix": [6, 12, 15, 21, 39, 75, 92, 97, 105, 117, 124, 126, 133, 145, 155], "mattentionconfig": 0, "mattentiondpeventsgatherperiodm": 0, "mattentionlayernumperpp": 0, "mattentiontyp": 0, "matter": [21, 109], "maverick": [82, 96, 97, 113, 150, 163], "max": [0, 1, 3, 4, 5, 8, 9, 10, 15, 16, 19, 22, 26, 29, 30, 31, 32, 33, 39, 59, 75, 77, 78, 87, 91, 110, 125, 127, 128, 131, 133, 138, 139, 143, 147, 152, 155, 157, 158], "max_": 8, "max_all_reduce_block": 1, "max_attention_window": [86, 87, 132, 147, 152], "max_attention_window_s": [105, 132, 133, 138], "max_attn_valu": 134, "max_batch_s": [2, 9, 10, 11, 14, 18, 19, 21, 22, 23, 24, 26, 27, 39, 44, 45, 59, 65, 67, 75, 76, 78, 80, 105, 110, 114, 116, 117, 119, 124, 127, 130, 131, 133, 135, 138, 139, 143, 144, 147, 152, 156, 158, 160, 161, 169], "max_beam_width": [23, 24, 27, 44, 75, 91, 95, 103, 105, 133, 135, 138, 139, 147], "max_block": [133, 170], "max_blocks_per_seq": 138, "max_blocks_per_sequ": 133, "max_boost_slid": [39, 124], "max_cache_storage_gb": 147, "max_concurr": 147, "max_context_length": [133, 134, 138, 139], "max_cpu_lora": [64, 88, 165], "max_decoder_input_len": 135, "max_decoder_seq_len": 23, "max_dist": [105, 133, 134], "max_draft_len": [9, 18, 19, 23, 68, 96, 135, 137, 147], "max_draft_token": 138, "max_encoder_input_len": [23, 135, 147], "max_gen_token": 135, "max_gpu_total_byt": 147, "max_input_len": [22, 23, 39, 91, 110, 114, 116, 117, 124, 135, 138, 139, 147], "max_input_length": [24, 133, 134, 135, 138], "max_kv_seqlen": 133, "max_lora": [64, 88, 165], "max_lora_rank": [23, 39, 64, 88, 110, 124, 165], "max_low_rank": 133, "max_matching_ngram_s": [19, 68, 96, 147], "max_medusa_token": 138, "max_multimodal_len": 23, "max_new_token": [67, 138, 139], "max_ngram_s": 147, "max_non_leaves_per_lay": 147, "max_num_draft_token": 10, "max_num_request": [75, 155, 169, 170], "max_num_sequ": 152, "max_num_stream": 147, "max_num_token": [2, 9, 11, 21, 22, 23, 24, 26, 27, 44, 45, 67, 75, 87, 92, 96, 124, 127, 130, 131, 135, 139, 144, 147, 152, 155], "max_output_len": [91, 117, 138, 143, 152], "max_output_length": 24, "max_period": 134, "max_position_embed": [91, 116, 133, 134, 135], "max_position_embedding_len": 133, "max_power_limit": [39, 124], "max_prompt_adapter_token": 147, "max_prompt_embedding_table_s": [23, 138, 147, 152], "max_record": 147, "max_retri": [29, 31, 32], "max_seq_len": [9, 18, 22, 23, 24, 26, 27, 59, 65, 67, 78, 80, 110, 114, 116, 117, 124, 132, 133, 134, 135, 138, 139, 147, 152, 158, 161, 169], "max_seqlen": [75, 91, 105, 133], "max_seqlen_for_logn_sc": 134, "max_sequence_length": [75, 105, 138], "max_split_size_mb": 29, "max_throughput": 21, "max_throughput_benchmark": 21, "max_token": [9, 11, 18, 21, 29, 30, 31, 32, 33, 41, 42, 43, 50, 51, 53, 58, 59, 60, 65, 66, 67, 68, 69, 70, 71, 72, 78, 84, 86, 88, 132, 147, 151, 158, 160, 165], "max_token_count": 60, "max_tokens_in_buff": [84, 147], "max_tokens_in_paged_kv_cach": [132, 138, 152], "max_total_draft_token": 147, "max_util": [0, 22, 132, 147], "max_verification_set_s": 147, "max_window_s": 147, "maxaccepteddrafttokensperstep": 1, "maxacceptedtoken": 1, "maxadapters": 0, "maxattentionwindow": 1, "maxattentionwindowvec": [0, 1], "maxbadwordslen": 1, "maxbatchs": [0, 1, 106], "maxbatchsizeruntim": 0, "maxbatchsizeruntimeupperbound": 0, "maxbatchsizestat": 0, "maxbatchsizetunerrecommend": 0, "maxbeamwidth": [0, 1, 103, 152], "maxdecoderstep": 1, "maxdecodingdrafttoken": 1, "maxdecodingtoken": [0, 1], "maxdraftpathlen": [0, 1], "maxdrafttoken": [0, 1], "maxencoderlen": 1, "maxgenerationlength": 1, "maxgenlengthdevic": 1, "maxgenlengthhost": 1, "maxgentoken": 1, "maxgputotalbyt": 0, "maxim": [0, 3, 5, 8, 12, 13, 15, 18, 21, 37, 39, 90, 124, 132, 166], "maximum": [0, 1, 2, 5, 8, 11, 16, 19, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 39, 40, 67, 75, 77, 84, 86, 87, 91, 96, 102, 103, 105, 106, 124, 127, 133, 134, 138, 139, 143, 147, 150, 152, 157, 169], "maxinputlen": [1, 106], "maxinputlength": 1, "maxlength": 1, "maxlengthstop": 0, "maxlorarank": 1, "maxmedusahead": 1, "maxnewtoken": [1, 152], "maxnonleafnodesperlay": 1, "maxnumactiverequest": 0, "maxnumblock": [0, 27], "maxnumpath": 1, "maxnumsequ": [1, 152], "maxnumtoken": [0, 1], "maxnumtokensruntim": 0, "maxnumtokensstat": 0, "maxnumtokenstunerrecommend": 0, "maxoutputlength": 103, "maxpagesperblock": 1, "maxpagesperblockdevic": 0, "maxpagesperblockhost": 0, "maxpathdraftlen": 1, "maxpathlen": [0, 1], "maxpositionembed": [0, 1], "maxpromptembeddingtables": 1, "maxqueues": 0, "maxseqidlemicrosecond": 0, "maxseqlen": 1, "maxsequencelen": [1, 106], "maxsequencelength": 1, "maxsplit": 147, "maxstopwordslen": 1, "maxtoken": [0, 139, 152], "maxtokensinbuff": 0, "maxtokensperenginestep": 1, "maxtokensperstep": 1, "mayb": 33, "maybe_capture_hidden_st": 96, "maybe_to_pybind": 147, "mb": [20, 139, 147], "mbackedstorag": 1, "mbackend": 0, "mbackendagentdesc": 0, "mbackendtyp": 0, "mbackground": 1, "mbackstream": 1, "mbacktyp": 1, "mbadhandl": 1, "mbart": [142, 152], "mbatchingtyp": 0, "mbatchsizet": 0, "mbeamsearchbuff": 1, "mbeamsearchdiversityr": 0, "mbeamwidth": 0, "mbeamwidtharrai": 0, "mbind": 20, "mbindoffset": 1, "mbp": 53, "mbuffer": 1, "mbuffermanag": 1, "mc_handl": 1, "mc_ptr": 1, "mc_va": 1, "mcachemap": 1, "mcachemutex": 1, "mcachepagemanag": 1, "mcachest": 0, "mcachetransceiverconfig": 0, "mcapacityschedulerpolici": 0, "mcommmod": 0, "mcommptr": 1, "mcommstat": 0, "mcommtyp": 0, "mcomputecontextlogit": 1, "mcomputegenerationlogit": 1, "mconfig": [0, 1], "mconfigur": 1, "mconnectioninfo": 0, "mcontextchunkingpolici": 0, "mcontextfmha": 1, "mcontextparallel": [0, 1], "mcopyonpartialreus": 0, "mcp": [11, 152], "mcpu": 1, "mcpudiff": 1, "mcreator": 1, "mcrosskvcachefract": 0, "mct": 11, "mcudagraphcaches": 0, "mcudagraphmod": 0, "mcumlogprobstmp": 1, "md": [13, 81, 96, 113, 115, 133, 147, 152, 154, 162], "mdatatyp": [0, 1], "mdebugconfig": 0, "mdebuginputtensor": 0, "mdebugoutputtensor": 0, "mdebugtensornam": 0, "mdebugtensorsmaxiter": 0, "mdecod": 1, "mdecodedurationm": 0, "mdecoderetentionprior": 0, "mdecoderstream": 1, "mdecodingconfig": 0, "mdecodinglayerworkspac": 1, "mdecodingmod": [0, 1], "mdefaulteaglechoic": 1, "mdefaultmedusachoic": 1, "mdefaultposteriorthreshold": 1, "mdesc": [0, 1], "mdevic": 1, "mdevicebuffermanag": 1, "mdevicecacheperc": 0, "mdeviceid": [0, 1], "mdirectori": 0, "mdllmutex": 0, "mdogreedysampl": 1, "mdonetask": 1, "mdprank": 0, "mdpsize": 0, "mdrafttoken": 0, "mdstdesc": 0, "mdynamicbatchconfig": 0, "mdynamicbatchmovingaveragewindow": 0, "mdynamicdecodelay": 1, "mdynamictreemaxtopk": 0, "me": [9, 30, 33, 42, 58, 64, 66, 70, 73, 151, 160], "meaglechoic": 0, "meagleconfig": 0, "mean": [1, 2, 4, 5, 10, 11, 12, 14, 15, 16, 17, 19, 22, 24, 26, 29, 30, 31, 32, 33, 38, 39, 40, 47, 48, 50, 62, 64, 75, 86, 91, 96, 104, 105, 106, 109, 113, 116, 118, 119, 123, 124, 125, 126, 132, 133, 136, 138, 139, 147], "meaning": [1, 2, 15, 127, 131], "meant": [33, 128, 129, 136, 147], "meantim": 152, "meanwhil": [10, 20], "mearlystop": 0, "measur": [0, 3, 5, 6, 7, 8, 12, 14, 15, 16, 19, 20, 21, 39, 76, 84, 102, 124, 125, 152, 156], "mechan": [12, 16, 17, 20, 36, 84, 86, 103, 117, 147, 169, 170], "media": [39, 124, 152], "media_path": [39, 124], "median": [11, 26, 29, 30, 31, 32], "medic": 21, "medium": [7, 30, 82, 143, 152, 163], "medusa": [0, 1, 22, 23, 133, 135, 138, 147, 152], "medusa_choic": [22, 113, 124, 138, 147], "medusa_decode_and_verifi": 138, "medusa_hidden_act": 137, "medusa_logit": 138, "medusa_model_dir": 137, "medusa_output_token": 138, "medusa_path": 138, "medusa_position_offset": 138, "medusa_temperatur": [113, 138], "medusa_topk": 138, "medusa_tree_id": 138, "medusachoic": [0, 1], "medusaconfig": 135, "medusacurtokensperstep": 1, "medusadecodingconfig": [97, 147], "medusaforcausallm": 135, "medusainput": 1, "medusalogit": 1, "medusapath": 1, "medusatargettokensperstep": 1, "medusatreeid": 1, "meet": [7, 16, 17, 30, 33, 133], "megan": 29, "memavail": 20, "membeddingt": 0, "member": [0, 1, 11, 20, 106, 107, 114, 117, 133], "memcpi": 20, "memfre": 20, "meminfo": 20, "memlock": [9, 18, 21, 26, 98, 143, 151], "memori": [0, 1, 3, 4, 6, 7, 10, 12, 13, 14, 15, 16, 17, 18, 19, 21, 22, 23, 24, 27, 29, 30, 31, 32, 33, 39, 40, 44, 50, 75, 76, 80, 83, 84, 87, 88, 91, 92, 93, 96, 97, 102, 104, 105, 106, 108, 110, 117, 118, 119, 124, 125, 130, 131, 133, 136, 138, 143, 147, 150, 152, 155, 156, 160, 164, 165, 169], "memorydesc": 0, "memorypoolfre": [1, 139], "memorypoolreserv": [1, 139], "memorypooltrimto": 1, "memorypoolus": 1, "memorytyp": [0, 1], "memorytypestr": 1, "memset": [1, 12], "memsetconfigur": 1, "memtot": 20, "memtyp": 1, "memusagechang": 139, "menableattentiondp": [0, 1], "menablebatchsizetun": 0, "menableblockreus": 0, "menablechunkedcontext": 0, "menablecontextfmhafp32acc": 0, "menablemaxnumtokenstun": 0, "menablepartialreus": 0, "menabletrtoverlap": 0, "mencodedvocab": 0, "mencoderhiddens": 1, "mengineaddr": 1, "menginebuff": 1, "menginepath": 1, "mengines": 1, "mental": 64, "mental_health_lora_dir": 64, "mention": [2, 11, 12, 20, 21, 29, 30, 31, 32, 33, 44, 106, 119, 127], "mentri": 1, "mentryit": 1, "menu": [120, 121], "merg": [12, 13, 16, 30, 36, 133, 152], "merlin": 21, "meshgrid": 133, "meshgrid2d": 133, "messag": [9, 13, 18, 20, 21, 24, 30, 33, 40, 41, 42, 69, 70, 73, 75, 84, 102, 111, 133, 139, 147, 151, 152, 160], "met": [0, 1, 103, 113], "meta": [24, 31, 32, 39, 40, 68, 76, 78, 82, 119, 124, 125, 132, 142, 146, 147, 149, 156, 158, 160, 163], "meta_ckpt_dir": 135, "metadata": [17, 20, 27, 39, 58, 75, 84, 96, 108, 124, 148, 153, 155], "metadata_server_config_fil": 27, "metal": [145, 152], "meth": 146, "method": [0, 1, 3, 8, 10, 11, 14, 15, 16, 17, 20, 36, 39, 44, 59, 60, 75, 81, 83, 84, 87, 93, 95, 96, 97, 101, 103, 105, 106, 112, 113, 114, 116, 117, 119, 124, 136, 138, 141, 143, 147, 148, 152, 153, 154, 162, 164, 169, 170], "metric": [0, 8, 15, 16, 17, 20, 24, 33, 38, 39, 40, 76, 84, 123, 124, 125, 127, 130, 131, 147, 152, 156], "metrics_log_interv": 27, "metro": 30, "metropoli": [21, 30], "metropolitan": 33, "mevent": 1, "meventbuffermaxs": 0, "mexecutionconfig": 1, "mextendedruntimeperfknobconfig": 0, "mfailfastonattentionwindowtoolarg": 0, "mfastlogit": 0, "mfirstgentoken": 0, "mfirsttim": 1, "mflagptr": 1, "mfreegpumemoryfract": 0, "mfreepageid": 1, "mfrequencypenalti": 0, "mfuntowicz": 152, "mgathergenerationlogit": 0, "mgemmallreducedtyp": 1, "mgmn": [16, 152], "mgpu": 1, "mgpudiff": 1, "mgpuspernod": 1, "mgpuweightsperc": 0, "mgreedysampl": 0, "mguid": 0, "mguideddecodingconfig": 0, "mguidetyp": 0, "mh": 113, "mh1": 113, "mha": [3, 15, 23, 75, 81, 82, 86, 91, 105, 108, 133, 136, 138, 155, 162, 163], "mhandl": 1, "mhandler": 0, "mhiddens": 1, "mhostcaches": 0, "mi": [30, 141], "mib": 139, "micro": [0, 139], "microbatchid": 0, "microbatchschedul": [154, 170], "microsecond": 0, "microsoft": [78, 82, 116, 142, 149, 158, 163], "mid": [19, 34, 152], "middl": [11, 38, 123], "mig": [20, 21], "might": [0, 2, 7, 10, 16, 23, 33, 39, 86, 87, 98, 103, 117, 119, 124, 125, 126, 131, 138, 139, 143, 147, 152, 169], "migrat": [34, 119, 136, 152], "million": [39, 124], "millisecond": [0, 86, 147], "millisecondstyp": 0, "mimpl": 0, "min": [0, 1, 4, 13, 14, 15, 22, 39, 106, 124, 125, 131, 133, 143, 147], "min_lat": 133, "min_length": [106, 138], "min_p": [0, 106, 138, 147], "min_token": 147, "mind": [7, 132, 146], "mindim": 1, "mindimfirst": 1, "mini": [82, 152, 163], "minim": [8, 12, 13, 16, 17, 20, 21, 37, 83, 84, 89, 91, 92, 130, 152, 164], "minimum": [0, 8, 18, 21, 22, 39, 40, 75, 105, 106, 124, 127, 133, 138, 139, 147], "minitron": [82, 142, 149, 152, 163], "minittozero": 1, "minlat": [111, 147], "minlength": [1, 106, 152], "minnormedscorescba": 1, "minor": [37, 152], "minp": [0, 1, 106], "minprogresstask": 1, "minputpack": 1, "minputtokenextraid": 0, "mintoken": [0, 152], "mintpsplitdim": 1, "minut": [0, 7, 18, 21, 125], "mip": 0, "mipcmemoryhandl": 1, "mirco": 0, "mirror": 147, "mirror_pybind_enum": 147, "mirror_pybind_field": 147, "mish": 134, "mismatch": [119, 143], "misorchestr": 0, "mispagefre": 1, "miss": [0, 39, 107, 124, 152], "missedblock": [0, 27], "missedblocksperrequest": 0, "mission": [8, 13, 16, 17], "mistral": [39, 82, 93, 104, 124, 127, 131, 140, 141, 142, 149, 152, 163], "mistral3": [142, 152], "mistral3forconditionalgener": [142, 149], "mistralai": [39, 82, 124, 142, 149, 163], "mistralforcausallm": [142, 149], "misus": 152, "miterstatsmaxiter": 0, "mitig": [8, 13, 16, 119], "mix": [15, 21, 33, 84, 102, 126, 131, 152], "mixed_precis": 147, "mixed_sampl": 152, "mixer": 152, "mixtral": [39, 82, 93, 104, 110, 124, 127, 131, 141, 142, 149, 152, 163], "mixtralforcausallm": [142, 149], "mixtur": [15, 16, 26, 29, 30, 31, 32, 33, 131, 152], "mjointdecodinginput": 1, "mjointdecodingoutput": 1, "mkdir": [9, 29, 30, 31, 32, 120], "mkvcacheconfig": 0, "mkvcachetyp": 1, "mkvfactor": 0, "ml": [133, 152], "mla": [8, 12, 13, 14, 77, 92, 97, 133, 149, 152, 157], "mlayertyp": 1, "mlen": 0, "mlengthpenalti": 0, "mllama": [142, 152], "mllamaconfig": 135, "mllamaforcausallm": 135, "mllamaforconditionalgener": [142, 149], "mlogit": 0, "mlogitsdtyp": 1, "mlogitspostprocessorconfig": 0, "mlookaheaddecodingconfig": 0, "mlookaheaddecodingmaxnumrequest": 0, "mlop": 21, "mloramodul": 1, "mloraprefetchdir": 0, "mlp": [23, 110, 115, 117, 118, 133, 136, 143, 148, 152, 153], "mlp_4h_to_h": [23, 110], "mlp_bia": 135, "mlp_gate": [23, 110], "mlp_gate_up": [23, 110], "mlp_h_to_4h": [23, 110], "mlp_output": 143, "mlp_router": [23, 110], "mlperf": [21, 97], "mlphiddens": 1, "mlptype": 133, "mm": [20, 152], "mm_data": [39, 124], "mm_embed": [59, 147], "mm_embedding_handl": [30, 33, 147], "mm_embedding_offload": 138, "mm_encoder_onli": 147, "mma": [15, 133], "mmanag": 1, "mmanagedweightsmap": 1, "mmanageweightstyp": 1, "mmap": 20, "mmaxadapters": 0, "mmaxattentionwindow": 0, "mmaxattentionwindowvec": 0, "mmaxbatchs": [0, 1], "mmaxbeamwidth": [0, 1], "mmaxdecodingdecodertoken": 1, "mmaxdecodingdrafttoken": 1, "mmaxdecodingenginetoken": 1, "mmaxdraftpathlen": 1, "mmaxencoderlen": 1, "mmaxgputotalbyt": 0, "mmaxinputlen": 1, "mmaxlorarank": 1, "mmaxnonleafnodesperlay": 1, "mmaxnumpackedmask": 1, "mmaxnumpath": 1, "mmaxnumsequ": 1, "mmaxnumtoken": [0, 1], "mmaxpagesperblock": 1, "mmaxpagesperblockdevic": 0, "mmaxpagesperblockhost": 0, "mmaxpositionembed": 1, "mmaxpromptembeddingtables": 1, "mmaxqueues": 0, "mmaxseqidlemicrosecond": 0, "mmaxsequencelen": 1, "mmaxsequencelength": 1, "mmaxtoken": 0, "mmaxtokensinbuff": 0, "mmedusachoic": 0, "mmemori": 1, "mmemorytyp": 1, "mmha": [133, 152], "mminp": 0, "mmintoken": 0, "mmlphiddens": 1, "mmlu": [7, 13, 152], "mmlu_llmapi": 152, "mmmu": [39, 124], "mmode": 1, "mmodelconfig": [0, 1], "mmodelnam": 1, "mmodelvari": 1, "mmoduleidtomodul": 1, "mmprojector": 152, "mmropepositiondelta": 0, "mmroperotarycossin": 0, "mmultiblockmod": 0, "mmulticast": 1, "mmultimodalhash": 0, "mmultimodallength": 0, "mmultimodalposit": 0, "mmutex": 1, "mname": [0, 1], "mnbattentionlay": 1, "mnbhead": 1, "mnbkvheadsperlay": 0, "mnblayer": 1, "mnbrnnlayer": 1, "mngramsiz": 0, "mnnvl": [16, 92, 111, 133, 147, 152], "mnorepeatngrams": 0, "mnormalizelogprob": 0, "mnumcopystream": [0, 1], "mnumdecodingenginetoken": 1, "mnumdevicemodulelay": 0, "mnumensurework": 0, "mnumhostmodulelay": 0, "mnumkvheadsperattentionlay": 1, "mnumkvheadspercrossattentionlay": 1, "mnumlanguag": 1, "mnumnod": 0, "mnumputwork": 0, "mnumreturnbeam": 0, "mnumreturnsequ": 0, "mnumsm": 1, "mnumtransformerslay": 1, "mobil": 58, "modal": [22, 89, 93, 141, 149], "mode": [0, 1, 10, 15, 18, 20, 21, 22, 23, 24, 27, 36, 37, 53, 61, 62, 63, 75, 84, 88, 94, 96, 104, 105, 107, 117, 118, 132, 133, 134, 138, 139, 141, 147, 148, 152, 153, 165], "mode_t": 0, "model": [0, 1, 3, 4, 5, 6, 7, 11, 12, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 26, 27, 34, 37, 38, 41, 42, 43, 44, 49, 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 75, 76, 77, 78, 80, 83, 84, 85, 86, 87, 88, 90, 91, 94, 95, 101, 102, 103, 104, 105, 108, 109, 110, 111, 114, 116, 119, 123, 125, 128, 132, 133, 134, 136, 137, 138, 139, 140, 141, 144, 145, 147, 151, 155, 156, 157, 158, 160, 161, 164, 165, 166, 167, 168, 169, 170], "model_architectur": 147, "model_arg": [29, 31, 32], "model_cl": 134, "model_cls_fil": 23, "model_cls_nam": 23, "model_computed_field": 147, "model_config": [23, 83, 138, 147, 148, 153, 164], "model_config_cpp": 138, "model_construct": 147, "model_copi": 147, "model_dir": [11, 14, 61, 89, 110, 114, 115, 116, 117, 118, 119, 124, 126, 135, 137, 143], "model_dump": 147, "model_dump_json": 147, "model_engin": [154, 169], "model_extra": 147, "model_factori": [80, 161], "model_field": 147, "model_fields_set": 147, "model_format": 147, "model_json_schema": 147, "model_kwarg": [78, 80, 158, 161], "model_nam": [16, 40, 62, 138], "model_parametrized_nam": 147, "model_path": [16, 21, 22, 26, 29, 31, 32, 38, 39, 62, 66, 67, 88, 114, 123, 124, 165], "model_post_init": [136, 147], "model_qu": 124, "model_rebuild": 147, "model_valid": 147, "model_validate_json": 147, "model_validate_str": 147, "model_weights_load": [118, 152], "modelconfig": [0, 83, 106, 138, 148, 152, 153, 164], "modelconfigpython": 138, "modelengin": [37, 96, 154, 169], "modelidtomodel": 1, "modeling_deepseekv3": [13, 15], "modeling_gemma3": [83, 164], "modeling_llama": [148, 153], "modeling_mymodel": [148, 153], "modeling_opt": [148, 153], "modeling_util": [83, 88, 147, 148, 153, 164, 165], "modelmodel_dump": 147, "modelmodel_dump_json": 147, "modelnam": 1, "modelopt": [29, 31, 32, 39, 40, 116, 119, 124, 137, 152], "modelpath": 0, "modelrunn": [116, 138, 152], "modelrunnercpp": [138, 152], "modelrunnermixin": 138, "models_as_dict": 147, "modeltyp": [0, 114], "modelvari": 1, "modelweightsformat": 118, "modelweightsload": [118, 152], "moder": [8, 17, 66, 84], "modern": 138, "modif": [12, 30, 107, 117, 147], "modifi": [10, 11, 12, 20, 39, 78, 88, 95, 96, 97, 98, 103, 107, 124, 127, 131, 132, 143, 150, 152, 158, 165], "modul": [0, 1, 8, 11, 12, 13, 16, 17, 21, 22, 23, 75, 83, 84, 96, 97, 98, 105, 106, 115, 116, 117, 118, 131, 133, 134, 135, 137, 138, 143, 147, 148, 152, 153, 164], "modular": [11, 16, 83, 146, 150, 164], "modularli": [17, 84], "module1": 13, "module10": 13, "module11": 13, "module12": 13, "module13": 13, "module2": 13, "module3": 13, "module4": 13, "module5": 13, "module6": 13, "module7": 13, "module8": 13, "module9": 13, "module_id": 110, "module_nam": [83, 164], "module_names_breakdown": [83, 164], "module_weight": [83, 164], "moduleid": [1, 110], "moduleidtomodel": 1, "modulelist": [148, 153], "moduletyp": 1, "modulo": 133, "moe": [8, 12, 13, 14, 16, 23, 26, 31, 32, 33, 40, 57, 97, 110, 118, 131, 133, 135, 136, 147, 152], "moe_4h_to_h": [23, 110], "moe_backend": [21, 29], "moe_cluster_parallel_s": 147, "moe_config": [2, 9, 14, 16, 21, 147], "moe_ep_s": 104, "moe_expert_parallel_s": [57, 92, 147], "moe_finalize_allreduce_residual_rms_norm": 133, "moe_gat": [23, 110], "moe_gemm": 112, "moe_h_to_4h": [23, 110], "moe_load_balanc": 16, "moe_plugin": [23, 136], "moe_rout": [23, 110], "moe_shared_": 16, "moe_tensor_parallel_s": [57, 92, 147], "moe_tp_siz": 104, "moeallreduceparam": 133, "moecomputeroutekernel": 12, "moeconfig": [97, 135, 147], "moeloadbalancerconfig": 147, "moetopk": 152, "moment": 103, "monboardblock": 0, "mondemand": 1, "monitor": [17, 23, 84, 92, 108, 147], "monitor_memori": [23, 147], "monolith": [75, 91, 105], "monost": 0, "mont": 11, "month": [10, 39, 124], "monthli": [29, 30, 31, 32], "mop": 0, "mopenipc": 1, "moptimaladapters": 0, "morchestratorconfig": 0, "morchleadercomm": 0, "more": [0, 1, 3, 4, 5, 7, 8, 10, 11, 13, 14, 15, 16, 17, 19, 20, 21, 22, 23, 24, 27, 29, 30, 31, 32, 33, 34, 35, 36, 39, 40, 44, 46, 49, 60, 66, 75, 76, 77, 80, 81, 84, 86, 87, 91, 92, 95, 97, 98, 102, 103, 104, 105, 106, 107, 108, 113, 115, 116, 117, 124, 125, 127, 130, 131, 132, 133, 136, 139, 143, 144, 146, 147, 148, 151, 152, 153, 155, 156, 157, 161, 162, 168, 170], "most": [0, 1, 3, 4, 5, 7, 8, 10, 11, 13, 15, 17, 20, 21, 33, 34, 35, 36, 38, 49, 54, 55, 57, 59, 66, 76, 84, 86, 87, 91, 95, 100, 101, 106, 108, 113, 117, 119, 123, 128, 129, 131, 132, 133, 139, 143, 144, 146, 147, 150, 151, 152, 156, 168], "mostli": [16, 96, 136], "motiv": 97, "mount": [9, 21, 27, 29, 30, 31, 32, 61, 62, 63, 97, 98], "mount_dest": [61, 62, 63], "mount_dir": [61, 62, 63], "mountain": 30, "moutdim": 1, "moutdimfirst": 1, "mouth": 33, "moutputbeamhypothes": 1, "mouttpsplitdim": 1, "move": [0, 1, 16, 34, 96, 108, 119, 133, 143, 147, 152], "movement": [108, 117], "movi": 33, "mownsev": 1, "mownsstream": 1, "mp3": 27, "mp4": [27, 42, 70], "mpageblock": 1, "mpagedcontextfmha": 1, "mpagedst": 1, "mpagemanagerconfig": 1, "mpages": 1, "mpagesmutex": 1, "mpagewidth": 1, "mparallelconfig": 0, "mparticipantid": 0, "mpeftcacheconfig": 0, "mpi": [0, 1, 17, 23, 27, 38, 61, 62, 63, 84, 94, 101, 106, 117, 119, 123, 124, 125, 133, 143, 147, 152], "mpi4pi": [125, 143, 146, 152], "mpi_barri": 119, "mpi_comm_world": 106, "mpi_group_barri": 1, "mpi_sess": 147, "mpicomm": 0, "mpicommsess": [97, 147], "mpin": 1, "mpinneddiff": 1, "mpinnedpool": 1, "mpinnedpooldiff": 1, "mpipelineparallel": [0, 1], "mpirun": [21, 116, 117, 125, 143, 146, 152], "mpisess": 147, "mpistat": 0, "mpitopologi": [134, 135], "mpointer": 1, "mpool": 1, "mport": 0, "mposteriorthreshold": 0, "mppreducescatt": 1, "mprecis": 1, "mpresencepenalti": 0, "mprocessorbatch": 0, "mprocessormap": 0, "mprompttableoffload": 0, "mprop": 1, "mpt": [7, 141, 142, 152], "mptforcausallm": 135, "mptmodel": 135, "mptrexpertcount": 152, "mqa": [3, 6, 13, 23, 75, 92, 97, 105, 108, 133, 136, 152, 155], "mquantmod": 1, "mrank": [0, 1], "mrecvpollperiodm": 0, "mremotenam": 0, "mrepetitionpenalti": 0, "mreplic": 0, "mreqid": 0, "mrequeststatsmaxiter": 0, "mrnnconfig": 1, "mrope": [0, 133], "mrope_param": [134, 138], "mrope_position_delta": [133, 134, 138], "mrope_rotary_cos_sin": [133, 134], "mrope_rotary_cos_sin_s": 135, "mropeconfig": 0, "mropeparam": [134, 138], "mropepositiondelta": 0, "mroperoratysinco": 0, "mrotaryembeddingdim": 1, "mruntimedefault": 1, "mruntimestream": 1, "msamplingconfig": 1, "mscale": 133, "mscale_all_dim": 133, "mschedulerconfig": 0, "msecondaryofflineminprior": [0, 147], "msecondaryoffloadminprior": 0, "mseed": 0, "mselfidx": 0, "msg": [0, 1, 13, 147], "msinktokenlength": 0, "msize": 1, "msizeperhead": [0, 1], "mskipcrossattnblock": 1, "msl": 1, "mslotsperpag": 1, "mspawnprocess": 0, "mspeculativedecodingconfig": 0, "mspeculativedecodingmod": 1, "mspeculativedecodingmodul": 1, "msrcdesc": 0, "mstate": [0, 1], "mstoptokenid": 0, "mstream": 1, "msyncmessag": 0, "mt": 19, "mt5": 142, "mtag": [0, 1], "mtaskid": 0, "mtemperatur": 0, "mtensor": 0, "mtensorparallel": [0, 1], "mtoken": 0, "mtokenizerstr": 0, "mtokenrangeretentionconfig": 0, "mtokensperblock": [0, 1], "mtopk": 0, "mtopp": 0, "mtoppdecai": 0, "mtoppmin": 0, "mtoppresetid": 0, "mtotalnumpag": 1, "mtp": [2, 10, 16, 17, 29, 68, 85, 95, 97, 147, 149, 150, 152, 168], "mtp3": 17, "mtp3_autoregress": 13, "mtp3_top1": 13, "mtp3_top10": 13, "mtp3_top15": 13, "mtp3_vanilla": 13, "mtp_eagle_one_model": 147, "mtpdecodingconfig": [68, 96, 97, 147], "mtprank": 1, "mtransfermod": 0, "mtrimpool": 1, "mtype": [0, 1], "much": [10, 11, 12, 14, 16, 20, 38, 40, 91, 109, 117, 123, 130, 139], "mul": 133, "multi": [0, 3, 10, 11, 12, 14, 15, 16, 21, 23, 26, 42, 61, 62, 63, 70, 76, 77, 81, 82, 86, 87, 92, 93, 94, 97, 98, 102, 103, 104, 106, 109, 110, 113, 116, 119, 125, 133, 135, 136, 139, 141, 147, 150, 152, 155, 156, 157, 160, 162, 163], "multi_block_mod": [105, 138, 147, 152], "multi_gpu": 67, "multi_round": [26, 29, 30, 31, 32, 33], "multiblockmod": 0, "multicast": 1, "multicastconfigur": 1, "multidimension": 133, "multihead": [3, 117], "multiheadlatentattent": 161, "multimap": 1, "multimod": [0, 22, 23, 39, 74, 97, 124, 138, 142, 144, 146, 147, 152], "multimodal_embedding_handl": 147, "multimodal_hash": 147, "multimodalembed": 0, "multimodalencod": [97, 147], "multimodalhash": 0, "multimodalinput": 0, "multimodallength": 0, "multimodalmodelrunn": 138, "multimodalposit": 0, "multinod": 126, "multinomi": 106, "multipl": [0, 1, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 23, 24, 29, 30, 34, 35, 52, 59, 66, 75, 78, 82, 84, 86, 87, 88, 91, 92, 95, 100, 102, 103, 104, 105, 106, 107, 108, 109, 113, 117, 118, 125, 126, 127, 130, 133, 134, 136, 138, 143, 147, 150, 152, 155, 158, 163, 165], "multiple_profil": [23, 124, 127, 131, 136, 152], "multipli": [11, 15, 21, 75, 105, 118, 133], "multiply_and_lora": 134, "multiply_collect": 134, "multiprocessor": 117, "multithread": 0, "multiturn": 24, "munsign": 1, "musecrossattent": 1, "musedynamictre": 0, "musegemmallreduceplugin": 1, "musegptattentionplugin": 1, "musegpudirectstorag": 0, "museloraplugin": 1, "musemambaconv1dplugin": 1, "musemrop": 1, "musepositionembed": 1, "museshapeinfer": 1, "musetokentypeembed": 1, "museuvm": 0, "must": [0, 1, 10, 16, 17, 20, 21, 23, 29, 30, 31, 32, 34, 51, 53, 59, 75, 83, 84, 86, 91, 92, 95, 96, 101, 102, 103, 104, 105, 106, 109, 110, 113, 117, 121, 127, 133, 134, 136, 138, 141, 143, 147, 164], "mutabl": [0, 1], "mutablepageptr": 1, "mutex": [0, 1, 20], "mutlictaskvmod": 152, "mutual": [20, 106, 141], "muvm": 1, "muvmdiff": 1, "mvalu": 1, "mverificationsets": 0, "mversion": 1, "mvocabs": 1, "mvocabsizepad": 1, "mweight": 0, "mwindows": 0, "mworkerexecutablepath": 0, "mworldconfig": 1, "mxfp4": [30, 93, 152], "mxfp8": [30, 93, 152], "mxfp8xmxfp4": 152, "my": [1, 37, 39, 49, 50, 51, 54, 55, 56, 57, 65, 84, 88, 93, 95, 101, 102, 124, 144, 146, 151, 165, 167, 168], "my_config": [78, 158], "my_faster_on": 44, "my_llm_task": 146, "my_model": 115, "my_profile_export": [47, 48], "my_test": 35, "myattent": [148, 153], "myconfig": [148, 153], "mycustomlogitsprocessor": 95, "mycustomweightload": [83, 164], "mydecoderlay": [115, 148, 153], "mydraft": 96, "mymodel": [115, 148, 153], "mymodelforcausallm": [115, 148, 153], "n": [1, 8, 9, 11, 14, 15, 18, 20, 21, 26, 27, 29, 30, 31, 32, 33, 37, 39, 49, 53, 54, 55, 56, 57, 59, 60, 61, 62, 63, 64, 65, 66, 75, 76, 84, 87, 90, 91, 95, 96, 97, 101, 102, 105, 110, 113, 116, 117, 124, 125, 130, 133, 134, 135, 136, 139, 140, 141, 143, 144, 146, 147, 149, 151, 152, 156, 166], "n1": [18, 33, 60, 65], "n2": [18, 33, 60], "n3": [33, 60], "n4": [33, 60], "n_worker": 147, "na": [39, 124, 152], "nah": 33, "naiv": [75, 82, 131, 163], "naivepatternrewriter_replaceaddwithsub": 107, "nalso": [21, 33], "name": [0, 1, 9, 10, 12, 22, 24, 26, 27, 29, 30, 31, 32, 33, 34, 37, 39, 40, 49, 50, 51, 54, 55, 56, 57, 58, 62, 65, 66, 68, 73, 83, 92, 93, 95, 96, 97, 101, 103, 106, 107, 110, 112, 116, 117, 120, 124, 125, 133, 135, 136, 137, 138, 143, 144, 146, 147, 148, 151, 152, 153, 160, 164, 167, 168], "named_network_output": 143, "named_paramet": 118, "namedtupl": 147, "namespac": [0, 1, 135, 136, 146, 147], "nano": [82, 163], "nanobind": 152, "nanoflow": [90, 166], "nation": [39, 124], "nationwid": [39, 124], "nativ": [4, 15, 16, 21, 22, 82, 119, 146, 148, 150, 152, 153, 163], "native_quant_flow": 135, "nativegenerationcontrol": 11, "natur": [8, 15, 16, 19, 42, 70, 119, 125, 147], "naur": [0, 95, 103, 147], "naver": 142, "nbattentionlay": [0, 1], "nbdim": 1, "nbhead": 1, "nbkvhead": [0, 1], "nbkvheadperlay": 0, "nblayer": 1, "nbook": 29, "nbrnnlayer": 1, "ncache_transceiver_config": 84, "nccl": [13, 16, 23, 84, 102, 111, 117, 133, 136, 143, 147, 152], "nccl_graph_mixing_support": [84, 102], "nccl_p2p_level": 152, "nccl_plugin": [23, 136], "nccl_symmetr": [133, 147], "ncclplugin": 117, "ncclrecv": [16, 133], "ncclsend": [16, 133], "ncoordin": 33, "ncuda_graph_config": 18, "nd": [39, 124, 133], "ndarrai": [133, 134, 138], "ndim": 133, "nearest": [15, 26, 29, 30, 31, 32, 33, 37, 133, 147], "nearli": [4, 15, 20, 97, 107], "necess": 113, "necessari": [1, 10, 11, 13, 15, 16, 36, 37, 64, 86, 92, 104, 113, 127, 133, 147, 152, 169], "necessarili": [1, 84, 102, 117, 139], "necessit": [8, 16], "need": [1, 2, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 24, 27, 29, 30, 31, 32, 33, 34, 35, 39, 40, 44, 53, 57, 58, 59, 61, 62, 63, 64, 75, 83, 84, 86, 87, 91, 94, 95, 96, 98, 101, 102, 103, 105, 106, 107, 109, 113, 114, 115, 116, 117, 118, 119, 120, 124, 125, 126, 127, 130, 131, 132, 133, 135, 136, 138, 139, 143, 146, 147, 148, 150, 152, 153, 154, 155, 164, 169, 170], "needed_block": 170, "needs_kv_cache_rewind": 96, "needsdecoderprologu": 1, "needskvcacherewind": 1, "neg": [1, 10, 60, 132, 133, 147], "neglig": [7, 12, 109, 130], "neighbor": 33, "neighborhood": [30, 33], "neither": [103, 133, 139, 147], "nemo": [23, 82, 94, 116, 125, 138, 141, 142, 150, 152, 163], "nemo_ckpt_dir": 135, "nemo_prompt_convert": 138, "nemotron": [82, 142, 149, 152, 163], "nemotron_na": 152, "nemotronforcausallm": [142, 149], "nemotronh_nano_vl_v2": 149, "nemotronna": [142, 149, 152], "nemotronnasforcausallm": [142, 149], "nenable_attention_dp": 18, "nenable_min_lat": 18, "neox": [75, 105, 106, 141, 142, 152], "nest": [78, 107, 158], "net": [19, 109, 143, 147], "net_guard": 107, "network": [15, 16, 21, 23, 40, 53, 75, 95, 97, 103, 104, 105, 107, 111, 117, 119, 133, 139, 141, 143, 147, 152], "neural": [104, 107, 117, 152], "neva": [142, 152], "never": [1, 10, 20, 39, 51, 86, 107, 124, 132, 147], "nevertheless": 16, "new": [0, 1, 4, 5, 8, 10, 11, 12, 13, 14, 16, 19, 20, 29, 30, 31, 32, 33, 37, 40, 41, 43, 49, 54, 55, 56, 57, 58, 60, 67, 69, 71, 82, 83, 86, 91, 92, 94, 96, 97, 98, 101, 103, 105, 106, 107, 109, 110, 113, 114, 119, 120, 130, 131, 133, 138, 144, 145, 146, 147, 150, 151, 152, 154, 160, 163, 164, 169], "new_block_id": 58, "new_decoder_architectur": [116, 135], "new_generated_id": 138, "new_input": 107, "new_line_token": 60, "new_method": 34, "new_nam": [83, 164], "new_out": 107, "new_request": 58, "new_shap": 133, "new_tensor": 133, "new_token": [58, 138], "new_workflow": 152, "newactiverequestsqueuelatencym": 0, "newark": 30, "newer": [142, 152], "newest": [5, 19, 96, 147], "newli": [0, 10, 11, 14, 16, 20, 91, 130, 147], "newlin": 35, "newsiz": 1, "newtoken": 1, "newtokensstep": 1, "newtokensvec": 1, "newvalu": 0, "next": [1, 4, 10, 11, 12, 14, 16, 26, 28, 29, 30, 31, 32, 37, 86, 90, 91, 96, 97, 98, 110, 113, 117, 119, 126, 127, 130, 131, 132, 138, 139, 140, 142, 149, 150, 152, 166], "next_logit": 138, "next_medusa_input_id": 138, "next_medusa_logit": 138, "next_step_buff": 138, "next_step_tensor": 138, "nextdraftindic": 1, "nextdraftlen": 1, "nextdraftpath": 1, "nextdraftprob": 1, "nextdrafttoken": 1, "nextdrafttokenslen": 1, "nextflattoken": 1, "nextgenerationlength": 1, "nextn": 14, "nextpositionoffset": 1, "nfinal": 33, "nfirst": 33, "ngc": [9, 21, 29, 30, 31, 32, 36, 98, 100, 101, 145, 152], "ngoanpv": 152, "ngram": [0, 10, 68, 106, 135, 147, 150, 152], "ngramdecodingconfig": [19, 68, 96, 97, 147], "ngramsiz": 0, "ngroup": 133, "nhead": 133, "nhere": 53, "ni": [33, 53, 141], "nic": [16, 84, 102], "nice": 16, "nif": 30, "nixl": [0, 17, 84, 147, 152], "nj": [33, 56], "njane": [49, 54, 55, 56, 57, 101, 144, 146, 151], "njason": 64, "nkv_cache_config": 18, "nlet": [21, 33], "nmh": 138, "nmt": [138, 142, 152], "nn": [83, 133, 148, 153, 164], "no_kv_cache_reus": 152, "no_quant": 147, "no_repeat_ngram_s": [106, 138, 147], "no_schedule_after_st": 170, "no_schedule_until_st": 170, "no_skip_tokenizer_init": 22, "no_weights_load": 22, "noauxtckernel": 13, "node": [0, 12, 14, 15, 16, 17, 18, 20, 23, 24, 26, 38, 61, 62, 63, 84, 92, 94, 97, 98, 102, 106, 111, 123, 125, 126, 133, 136, 138, 141, 143, 147, 150, 152], "noexcept": [0, 1], "noh": 33, "nomin": [49, 56, 101, 144, 146, 151], "non": [0, 7, 10, 11, 12, 13, 14, 15, 16, 20, 23, 29, 30, 31, 32, 33, 39, 40, 59, 75, 77, 84, 86, 89, 102, 105, 108, 114, 117, 119, 133, 136, 143, 147, 152, 157], "non_block": [58, 60], "non_gated_vers": 133, "none": [1, 11, 19, 23, 24, 26, 27, 29, 31, 32, 37, 39, 44, 51, 58, 60, 64, 66, 67, 68, 75, 83, 86, 87, 90, 91, 95, 106, 107, 115, 118, 119, 124, 125, 130, 133, 134, 135, 136, 137, 138, 143, 147, 148, 152, 153, 155, 164], "nonetyp": [138, 147], "nontrivi": 96, "nonzero": 133, "nope": 12, "nor": [16, 139, 147], "norepeatngrams": [0, 1, 106], "norm": [2, 15, 38, 39, 40, 62, 118, 123, 124, 125, 133, 136, 147, 148, 152, 153], "norm_before_bmm1": [134, 135], "norm_elementwise_affin": 134, "norm_ep": 134, "norm_epsilon": [116, 135], "norm_factor": [75, 105], "norm_num_group": 134, "norm_pre_residual_weight": 133, "norm_quant_fus": [23, 136], "norm_typ": 134, "norm_weight": 133, "normal": [0, 7, 10, 11, 13, 14, 15, 16, 19, 20, 22, 39, 106, 109, 110, 114, 124, 133, 139, 147, 152], "normalize_log_prob": 147, "normalize_weight": 110, "normalized_shap": [133, 134], "normalizelogprob": [0, 1], "normedscorescba": 1, "north": [30, 115, 117, 143], "northeast": [30, 33], "northeastern": [29, 30, 31, 32, 33, 151], "not_op": 133, "notabl": 7, "notat": [14, 147], "note": [1, 2, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 21, 23, 26, 29, 30, 31, 32, 33, 35, 39, 44, 58, 61, 62, 63, 75, 78, 83, 84, 87, 91, 95, 98, 100, 102, 107, 109, 110, 111, 112, 113, 117, 121, 124, 127, 130, 132, 133, 136, 138, 139, 141, 142, 143, 144, 145, 146, 147, 148, 149, 153, 158, 164, 169], "noth": [20, 96], "notic": 64, "notifysyncmessag": 0, "notimplementederror": [11, 119], "nougat": [93, 141, 142, 152], "novel": 8, "now": [3, 11, 12, 13, 14, 16, 19, 20, 29, 30, 31, 32, 33, 39, 40, 67, 81, 91, 106, 112, 113, 116, 118, 124, 130, 136, 139, 147, 152, 162], "np": 133, "nprioriti": 33, "npy": 138, "npytorch_backend_config": 27, "nsight": [12, 97], "nspeculative_config": 18, "nsy": [12, 38, 123], "ntask": [27, 61, 62, 63, 117], "nthat": 33, "nthere": 136, "nucleu": 66, "null": [1, 9, 18, 21, 29, 30, 31, 32, 33, 39, 62, 76, 116, 124, 136, 151, 156], "nullopt": [0, 1], "nullptr": [0, 1], "num": [0, 1, 2, 22, 26, 29, 30, 31, 32, 33, 38, 39, 40, 62, 78, 91, 123, 124, 125, 127, 128, 131, 147, 158], "num_accepted_token": 96, "num_attention_head": [116, 133, 134, 135], "num_aud_token": 138, "num_beam": [106, 138], "num_beam_group": 106, "num_block": [138, 169], "num_bucket": [133, 134], "num_capture_lay": 147, "num_channel": [134, 135], "num_class": 134, "num_computed_block": 58, "num_computed_token": 58, "num_concurr": [29, 31, 32], "num_context": [75, 155], "num_ctx_token": [75, 155], "num_draft_token": [0, 133, 138], "num_draft_tokens_alloc": 96, "num_eagle_lay": 147, "num_embed": 134, "num_experts_per_tok": 104, "num_fewshot": [24, 29, 31, 32], "num_gener": [75, 155], "num_gpu": [21, 30, 33], "num_group": [133, 134], "num_head": [12, 75, 91, 92, 105, 118, 133, 138, 155], "num_hidden_lay": [78, 80, 116, 135, 148, 153, 158, 161, 169], "num_imag": 138, "num_img_token": 138, "num_inst": [17, 84], "num_key_value_head": [116, 135, 169], "num_kv_head": [75, 108, 133, 134, 138, 155, 169], "num_kv_heads_origin": 133, "num_kv_heads_per_cross_attn_lay": 138, "num_kv_heads_per_lay": 138, "num_lay": [133, 134, 138, 169], "num_ln_in_parallel_attn": 135, "num_local_block": 134, "num_local_expert": 104, "num_lora_module_lay": 110, "num_lora_modules_lay": 110, "num_medusa_head": [135, 137, 138, 147], "num_medusa_lay": [135, 137], "num_multimodal_token": 0, "num_nextn_predict_lay": [2, 14, 29, 68, 96, 147], "num_nextn_predict_layers_from_model_config": 147, "num_orig_po": 133, "num_po": 133, "num_postprocess_work": [27, 30, 33, 147], "num_profil": 135, "num_prompt": [26, 29, 30, 31, 32, 33], "num_q_head": 13, "num_request": [2, 14, 21, 22, 39, 40, 88, 124, 165], "num_return_sequ": [138, 152], "num_sampl": [24, 38, 67, 123], "num_slot": [16, 29], "num_task": 134, "num_token": [8, 13, 75, 105, 133, 155], "num_tokens_per_block": [133, 169], "num_tokens_per_task": 134, "num_video": 138, "numa": [16, 111], "numa_alloc_onnod": 20, "numacceptedtoken": 0, "numactiverequest": [0, 27], "numactl": [16, 20], "numattentionhead": 1, "numavailablepag": 1, "numbeamscba": 1, "number": [0, 1, 2, 6, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 35, 39, 40, 59, 61, 62, 63, 75, 76, 77, 84, 86, 87, 91, 93, 95, 96, 102, 103, 104, 105, 106, 108, 113, 117, 124, 125, 126, 127, 130, 131, 132, 133, 134, 138, 139, 141, 143, 147, 148, 152, 153, 155, 156, 157, 169], "numblockspercachelevel": 0, "numcompletedrequest": 0, "numcontextrequest": [0, 1], "numcopystream": [0, 1], "numctxgpu": 17, "numctxsequ": 1, "numctxtoken": 0, "numdevicemodulelay": 0, "numdrafttoken": [0, 1], "numdrafttokenshost": 1, "numeaglelay": 1, "numel": 138, "numensurework": 0, "numer": [13, 20, 34, 37, 106, 111, 124, 142, 147], "numexpert": 1, "numgeneratedtoken": 0, "numgengpu": 17, "numgenrequest": 0, "numgensequ": 1, "numgentoken": [0, 1], "numhead": 106, "numhostmodulelay": 0, "numkvattentionhead": 1, "numkvhead": 106, "numlanguag": 1, "numlay": 106, "nummissedblock": 0, "numnewactiverequest": 0, "numnewallocatedblock": 0, "numnewtokenscumsum": 152, "numnod": [0, 152], "numpag": 1, "numpausedrequest": 0, "numpi": [110, 133, 138], "numputwork": 0, "numqueuedrequest": [0, 152], "numrequestswithdrafttoken": 0, "numreturnbeam": 0, "numreturnsequ": [0, 1, 103], "numreusedblock": 0, "numscheduledrequest": 0, "numsequ": 1, "numslot": 1, "numtoken": 1, "numtotalallocatedblock": 0, "numtransformerslay": 1, "nvbugspro": 35, "nvcc": 2, "nvcr": [9, 21, 26, 29, 30, 31, 32, 151, 152], "nvfp4": [12, 13, 16, 22, 23, 29, 31, 32, 39, 82, 93, 124, 136, 147, 152, 163, 167], "nvidia": [2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 14, 16, 17, 18, 20, 21, 22, 23, 26, 27, 29, 30, 31, 32, 33, 35, 36, 39, 40, 41, 42, 43, 45, 47, 48, 49, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 75, 82, 84, 91, 92, 93, 94, 95, 96, 97, 98, 99, 101, 102, 116, 117, 119, 120, 124, 125, 131, 133, 136, 139, 142, 143, 144, 146, 149, 150, 152, 163, 167, 168], "nvila": [89, 142, 152], "nvinfer1": [0, 1], "nvl": [1, 23, 152], "nvl36": 126, "nvl72": [12, 15, 20, 21, 29, 40, 126, 142], "nvlink": [12, 16, 17, 21, 84, 92, 102, 106, 111, 125, 126, 128, 129, 152], "nvswitch": [13, 21, 117], "nvtx": [20, 147], "nwait": 33, "nwe": 21, "ny": 33, "nyc": [30, 33], "nyeah": 33, "nyou": 53, "n\u7b54\u6848": 72, "o": [0, 1, 6, 9, 10, 13, 15, 18, 20, 21, 29, 30, 31, 32, 33, 38, 58, 61, 62, 63, 72, 107, 110, 119, 123, 143], "o_proj": [12, 118], "oai": [42, 70], "oauthtoken": 9, "obei": 143, "obj": 147, "obj0": 147, "obj1": 147, "object": [0, 1, 8, 18, 20, 21, 29, 30, 31, 32, 33, 34, 44, 51, 53, 60, 73, 78, 80, 83, 86, 95, 96, 103, 109, 115, 117, 118, 119, 133, 134, 135, 136, 138, 139, 147, 151, 152, 154, 158, 161, 164], "observ": [8, 12, 15, 17, 19, 20, 40, 59, 84, 96, 102], "obtain": [1, 12, 17, 40, 96, 99, 133], "obtain_answ": 11, "obviou": [2, 15], "occas": 143, "occasion": 152, "occup": [12, 37, 75, 90, 105, 139, 166], "occupi": [7, 10, 15, 16, 30, 139], "occur": [8, 17, 84, 96, 106, 109, 147, 152, 169, 170], "occurr": 147, "ocean": [30, 33, 66], "off": [2, 10, 15, 17, 35, 37, 38, 81, 96, 109, 112, 123, 127, 130, 131, 139, 152, 162], "offer": [7, 8, 10, 11, 13, 17, 19, 20, 36, 75, 93, 117, 155], "offic": 53, "officenetsecur": 53, "offici": [2, 12, 14, 30, 39, 75, 82, 105, 124, 163], "offlin": [5, 8, 15, 24, 40, 49, 89, 92, 97, 115, 124, 152], "offload": [0, 1, 16, 23, 52, 108, 114, 147, 152], "offloadconfigur": 1, "offloading_dis": 59, "offloading_en": 59, "offset": [1, 10, 133, 138, 141, 152], "offsetdim": 1, "ofitensor": 0, "often": [0, 3, 7, 11, 12, 13, 16, 17, 33, 37, 84, 103, 108, 113, 126, 127, 133, 147], "oh": 11, "ok": 143, "okai": [11, 33], "old": [14, 107, 110, 143, 147], "older": [98, 109, 119, 142], "oldest": [19, 96, 110, 147], "oldvalu": 0, "omegaconf": [78, 158], "omit": [1, 9, 12, 95, 99, 103, 119, 133], "ompi": [101, 143], "ompi_mca_btl_tcp_if_includ": 146, "ompi_mca_oob_tcp_if_includ": 146, "onboard": [0, 82, 109, 139, 147, 163], "onboard_block": 147, "onboardblock": 0, "onc": [0, 10, 11, 14, 16, 17, 18, 19, 21, 26, 27, 35, 58, 66, 75, 84, 91, 96, 98, 101, 103, 105, 106, 107, 117, 127, 133, 139, 146, 147, 160], "ondemand": 1, "one": [0, 1, 3, 8, 9, 10, 11, 12, 13, 14, 15, 17, 19, 20, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 36, 37, 39, 64, 75, 81, 82, 83, 84, 86, 90, 91, 96, 102, 103, 104, 105, 106, 107, 108, 109, 111, 113, 116, 117, 118, 119, 120, 124, 125, 126, 127, 131, 132, 133, 134, 136, 138, 139, 143, 147, 148, 151, 152, 153, 162, 163, 166, 170], "ones": [0, 10, 20, 78, 110, 147, 151, 158], "oneshot": [13, 111, 133, 147], "oneshotallreduc": 13, "oneshotar": 13, "onevis": [142, 152], "ongo": [8, 10, 16, 21, 29, 30, 31, 32, 37, 119], "onli": [0, 1, 2, 7, 8, 9, 10, 11, 12, 14, 15, 16, 17, 19, 20, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 34, 35, 39, 40, 44, 58, 59, 60, 66, 75, 76, 77, 78, 83, 84, 86, 87, 91, 92, 94, 96, 97, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 112, 113, 114, 115, 117, 118, 119, 124, 125, 126, 127, 130, 131, 132, 133, 134, 136, 138, 139, 142, 147, 149, 152, 154, 156, 157, 158, 164, 170], "onlin": [5, 11, 24, 26, 29, 49, 92, 97, 152], "only_cross_attent": 134, "onnx": [21, 23, 133], "onnx__gathernd": 133, "ontario": 30, "onto": 106, "oom": [1, 2, 3, 6, 15, 29, 30, 31, 32, 33, 139], "ootb": [15, 152], "op": [0, 1, 9, 15, 21, 75, 84, 94, 102, 107, 133, 147, 152], "op_and": 133, "op_or": 133, "op_xor": 133, "opaqu": 107, "opaque_st": 147, "open": [3, 11, 13, 15, 16, 20, 38, 67, 94, 100, 101, 106, 112, 123, 143, 150, 151, 152], "openai": [9, 11, 17, 18, 20, 26, 27, 29, 30, 31, 32, 33, 74, 84, 88, 97, 151, 152, 160, 165], "openaiapi": 11, "openelm": [82, 163], "openipc": 1, "openmpi": 152, "opensora": 152, "opensourc": 152, "openssh": 120, "oper": [0, 1, 8, 10, 12, 13, 15, 16, 17, 20, 23, 26, 29, 30, 31, 32, 33, 37, 39, 65, 75, 81, 82, 83, 103, 105, 106, 107, 111, 113, 116, 117, 118, 124, 126, 127, 131, 133, 136, 139, 142, 147, 152, 154, 155, 162, 163, 164, 169], "opportun": [11, 39, 124], "opposit": [10, 60], "opt": [7, 15, 76, 78, 81, 82, 93, 103, 116, 120, 133, 141, 142, 143, 152, 156, 158, 160, 162, 163], "opt_batch_s": [135, 147], "opt_num_token": [23, 135, 147], "optforcausallm": [116, 135], "optim": [1, 3, 4, 5, 6, 7, 9, 11, 16, 18, 21, 22, 23, 26, 29, 30, 31, 32, 33, 39, 40, 49, 54, 60, 65, 78, 81, 82, 83, 86, 87, 90, 91, 92, 93, 94, 96, 97, 98, 101, 103, 106, 107, 108, 111, 112, 113, 117, 119, 124, 126, 127, 128, 129, 133, 136, 139, 142, 143, 144, 145, 146, 147, 151, 152, 154, 155, 158, 160, 162, 163, 164, 166, 167, 169], "optimaladapters": [0, 1], "option": [0, 1, 4, 9, 14, 20, 22, 23, 24, 26, 27, 34, 35, 36, 38, 39, 40, 44, 50, 51, 60, 62, 64, 66, 68, 75, 77, 78, 83, 84, 86, 91, 93, 95, 96, 99, 100, 101, 103, 106, 107, 108, 111, 112, 113, 115, 119, 123, 124, 125, 126, 128, 129, 130, 133, 136, 138, 139, 143, 146, 147, 148, 151, 152, 153, 155, 157, 158, 160, 164, 165, 168, 169], "optionalbufferptr": 1, "optionaltensorptr": 1, "optmodel": 135, "optvec": 1, "orchestr": [0, 16, 17, 37, 83, 84, 97, 113, 143, 147, 152, 164], "orchestrator_typ": [94, 147], "orchestratorconfig": 0, "orchleadercomm": 0, "order": [0, 1, 3, 10, 12, 36, 39, 40, 78, 79, 84, 86, 100, 102, 105, 108, 118, 124, 127, 132, 133, 134, 139, 147, 158, 159], "ordin": 147, "org": [0, 1, 23, 29, 30, 31, 32, 90, 101, 104, 110, 133, 136, 141, 166], "organ": [35, 108, 169], "orient": [8, 15, 16, 17], "origin": [0, 10, 12, 14, 15, 16, 19, 20, 24, 30, 60, 75, 81, 88, 91, 96, 105, 107, 110, 111, 133, 147, 148, 152, 153, 162, 165], "original_batch": 19, "original_max_po": 133, "original_max_position_embed": [133, 134], "originaltemperatur": 1, "orin": 21, "oserror": 152, "osl": [3, 4, 5, 6, 8, 12, 13, 14, 15, 16, 20, 21, 24, 26, 29, 30, 31, 32, 33, 39, 40, 97, 124, 131], "osl256": 17, "oss": [28, 93, 97, 112, 149], "ostream": [0, 1], "other": [0, 1, 3, 8, 10, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 33, 34, 35, 39, 40, 44, 51, 61, 62, 63, 75, 83, 84, 86, 87, 91, 92, 94, 96, 98, 102, 103, 104, 105, 106, 109, 111, 112, 113, 117, 118, 119, 125, 126, 127, 130, 131, 132, 133, 136, 138, 139, 143, 146, 147, 149, 152, 155, 164, 170], "other_audio_input": 138, "other_decoder_input": 138, "other_vision_input": 138, "othercach": 1, "otherwis": [0, 1, 10, 11, 20, 33, 39, 44, 58, 59, 75, 84, 96, 102, 103, 105, 106, 124, 133, 138, 143, 147, 155], "our": [2, 7, 8, 10, 11, 12, 13, 14, 15, 16, 19, 20, 39, 40, 49, 53, 54, 55, 57, 58, 79, 81, 89, 91, 101, 124, 127, 130, 131, 133, 143, 144, 146, 148, 151, 152, 153, 159, 162], "out": [0, 1, 3, 4, 5, 6, 10, 12, 13, 14, 15, 16, 18, 19, 21, 29, 30, 31, 32, 33, 35, 36, 38, 40, 49, 61, 62, 63, 84, 89, 91, 98, 102, 110, 119, 123, 127, 130, 131, 133, 139, 146, 151, 152], "out_bia": 134, "out_channel": 134, "out_context_dim": 134, "out_dim": 134, "out_featur": [116, 117, 134], "out_hidden_s": 133, "out_of_tree_exampl": [148, 153], "out_point": 133, "out_tp": [3, 6], "outcom": [20, 59], "outdim": 1, "outdimfirst": 1, "outer": [78, 133, 158], "outermost": 11, "outgrow": 19, "outlin": [38, 123], "outperform": 17, "output": [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 20, 21, 22, 23, 24, 27, 33, 34, 37, 38, 39, 40, 44, 47, 48, 49, 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 62, 64, 65, 66, 67, 68, 75, 84, 88, 91, 95, 101, 105, 106, 107, 109, 110, 113, 117, 123, 125, 126, 127, 128, 129, 131, 132, 133, 134, 136, 138, 143, 144, 146, 147, 151, 152, 154, 155, 165, 170], "output_a": 59, "output_b": 59, "output_bench": 26, "output_ctx0": 17, "output_ctx1": 17, "output_cum_log_prob": 138, "output_dim": 134, "output_dir": [23, 110, 114, 115, 116, 117, 119, 124, 126, 135, 137, 143], "output_directori": 147, "output_dtyp": [133, 134], "output_gen0": 17, "output_gen1": 17, "output_generation_logit": 138, "output_id": 138, "output_json": 22, "output_log_prob": 138, "output_multiplier_scal": 135, "output_pad": [133, 134], "output_path": [16, 29, 31, 32], "output_s": 134, "output_seqlen": [3, 6], "output_sequence_length": 138, "output_str": 11, "output_timing_cach": [23, 147], "output_token": [39, 124], "outputbuff": 1, "outputconfig": [0, 44, 103, 152], "outputidscba": 1, "outputlen": 0, "outputlogprob": 1, "outputtokenid": [0, 103], "outsid": [75, 84, 102, 113, 119, 151, 155], "outsiz": 1, "outstand": [10, 11, 12, 14, 20], "outtpsplitdim": 1, "outweigh": 126, "over": [0, 1, 2, 4, 5, 7, 8, 10, 11, 12, 13, 15, 17, 18, 19, 21, 29, 30, 31, 32, 33, 38, 66, 76, 91, 109, 113, 118, 121, 123, 124, 126, 130, 131, 133, 152, 156, 160], "overal": [2, 8, 10, 11, 12, 14, 15, 16, 17, 26, 37, 75, 84, 91, 103, 105, 109, 111, 113, 126, 127, 130, 131, 132, 148, 153], "overcom": [13, 75, 105, 117], "overflow": 1, "overhead": [0, 8, 10, 12, 13, 14, 15, 17, 19, 21, 37, 65, 75, 84, 90, 92, 102, 103, 117, 126, 147, 152, 155, 166], "overiew": 124, "overkil": 33, "overlap": [0, 2, 10, 13, 14, 15, 16, 78, 81, 85, 89, 95, 96, 97, 102, 113, 145, 147, 149, 152, 158, 162, 170], "overlap_schedul": 68, "overload": [0, 1], "overrid": [1, 11, 44, 78, 97, 118, 119, 133, 138, 147, 152, 158, 160], "overridden": [36, 98, 147], "override_field": 135, "overshadow": 126, "oversubscrib": [21, 125, 146], "overus": 35, "overview": [2, 7, 8, 16, 38, 39, 81, 97, 98, 102, 103, 108, 123, 124, 125, 145, 154, 160, 162], "overwhelm": [33, 64], "overwrit": [22, 24, 26, 27, 75, 78, 105, 158], "own": [0, 1, 2, 10, 11, 14, 16, 18, 19, 21, 39, 44, 78, 81, 82, 83, 94, 96, 98, 109, 113, 116, 117, 118, 119, 148, 153, 158, 162, 163], "ownership": 0, "ownsev": 1, "ownsstream": 1, "p": [0, 9, 18, 21, 26, 29, 30, 31, 32, 37, 60, 61, 62, 63, 66, 95, 96, 106, 113, 120, 135, 138, 147, 151, 152, 168], "p2p": [16, 133], "p50": [39, 40, 124], "p90": [39, 40, 124, 125], "p95": [39, 40, 124, 125], "p99": [26, 29, 30, 31, 32, 39, 40, 124, 125], "p_max": 0, "p_x": 0, "pa": 33, "pack": [0, 1, 10, 23, 86, 91, 106, 132, 133, 135, 136, 139, 148, 153], "packag": [24, 39, 81, 98, 101, 103, 124, 125, 143, 152, 162], "packed_length": 135, "packedinput": 1, "packedmask": 1, "packedmaskhost": 1, "packedmaskhostcopi": 1, "packedmasksdevic": 1, "packedpositionid": 1, "packet": 12, "pad": [0, 1, 10, 15, 23, 24, 26, 27, 29, 30, 31, 32, 33, 37, 75, 78, 91, 96, 106, 107, 110, 133, 134, 136, 138, 139, 147, 152, 158], "pad_id": [138, 147], "pad_lda": 134, "pad_ldc": 134, "pad_token_id": 138, "padding_2d": 133, "padding_back": 133, "padding_bottom": 133, "padding_en": [26, 152], "padding_front": 133, "padding_left": 133, "padding_mod": 134, "padding_right": 133, "padding_sid": 11, "padding_top": 133, "padid": 0, "page": [1, 5, 23, 35, 39, 77, 80, 81, 96, 97, 102, 106, 109, 117, 124, 125, 127, 133, 136, 139, 147, 150, 152, 155, 157, 160, 161, 162], "page_s": [20, 147], "pagealign": 1, "paged_context_fmha": [127, 152], "paged_kv_cach": [23, 110, 124, 136, 138], "paged_st": [23, 136, 138], "pagedcontextfmha": 1, "pagedkvcach": 106, "pagedst": 1, "pageid": 1, "pageidx": 1, "pagemanagerconfig": 1, "pageptr": 1, "pagewidth": 1, "paid": 16, "pair": [0, 1, 3, 14, 19, 86, 127, 131, 133, 147], "panel": 11, "panoram": 33, "paper": [4, 10, 14, 15, 17, 75, 84, 92, 96, 102, 110, 113, 141, 155], "par": [130, 131], "parallel": [0, 1, 2, 3, 5, 6, 8, 10, 11, 14, 17, 18, 19, 21, 22, 24, 27, 30, 31, 32, 33, 40, 57, 75, 81, 84, 85, 87, 94, 97, 102, 103, 105, 106, 113, 116, 117, 127, 128, 129, 133, 134, 135, 136, 139, 146, 147, 148, 149, 150, 152, 153, 162, 170], "parallel_attent": [116, 135], "parallel_config": [92, 147], "parallelconfig": [0, 152], "parallelprocess": 11, "param": [0, 1, 49, 54, 55, 56, 57, 101, 118, 133, 134, 138, 144, 146, 147, 151], "paramet": [0, 1, 2, 8, 9, 11, 15, 17, 18, 19, 21, 22, 23, 24, 26, 27, 34, 35, 36, 39, 60, 61, 66, 75, 76, 78, 83, 84, 86, 87, 88, 91, 95, 103, 104, 105, 108, 109, 110, 113, 114, 116, 117, 118, 119, 124, 126, 127, 128, 129, 132, 133, 134, 135, 136, 138, 139, 147, 152, 155, 156, 158, 164, 165, 168], "parametr": [138, 147], "params_imply_greedy_decod": 147, "parent": [0, 1, 39, 118, 119, 124, 147], "parenthash": 0, "parentid": 1, "pareto": [10, 17, 20], "pari": [49, 54, 55, 56, 57, 101, 144, 146, 151], "park": 33, "pars": [1, 22, 78, 83, 147, 158, 164], "parse_arg": [59, 64, 65, 67], "parse_argu": 67, "parse_fil": 147, "parse_obj": 147, "parse_raw": 147, "parser": [27, 59, 64, 65, 67, 74, 78, 83, 97, 147, 158, 164], "part": [1, 10, 11, 15, 19, 30, 33, 35, 40, 86, 96, 97, 98, 103, 104, 107, 117, 118, 119, 130, 131, 132, 133, 138, 139, 146, 147, 152], "part2": 152, "parti": [10, 100, 101, 152], "partial": [0, 8, 10, 13, 35, 37, 58, 104, 109, 117, 126, 138, 147], "particip": [0, 20, 133, 152], "participantid": [0, 102], "particular": [0, 10, 30, 36, 86, 96, 103, 128, 129, 130, 131], "particularli": [8, 10, 12, 13, 15, 16, 17, 20, 37, 78, 84, 98, 131, 158, 169], "partit": [21, 61, 62, 63, 75, 92, 105, 110, 117, 147], "pass": [0, 1, 10, 16, 20, 21, 22, 24, 29, 30, 31, 32, 33, 34, 35, 37, 38, 39, 44, 58, 60, 64, 75, 77, 83, 92, 95, 96, 103, 105, 107, 109, 110, 113, 117, 118, 123, 124, 125, 127, 130, 131, 133, 134, 135, 136, 138, 139, 146, 147, 148, 152, 153, 154, 155, 157, 164, 167, 170], "password": 9, "past": [0, 14, 16, 17, 75, 87, 91, 105], "past_key_valu": [133, 134], "past_key_value_length": 134, "past_key_values_length": 134, "past_kv_length": 138, "past_sequence_length": 138, "patch": [134, 138], "patch_siz": [134, 135], "path": [0, 1, 2, 9, 14, 16, 18, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 34, 36, 38, 39, 40, 44, 49, 54, 55, 56, 57, 58, 61, 62, 63, 64, 66, 68, 72, 75, 77, 83, 88, 93, 96, 98, 101, 103, 105, 112, 113, 116, 118, 123, 124, 125, 127, 133, 138, 144, 146, 147, 151, 152, 157, 160, 164, 165], "path_to_llama_from_hf": 154, "pathlib": [58, 72, 147], "pathlik": 135, "pathorn": 152, "pathsoffset": 1, "pattern": [8, 10, 13, 15, 16, 19, 20, 35, 59, 76, 82, 92, 95, 104, 133, 147, 152, 156, 163], "patternanalyz": 107, "patternrewrit": 107, "paus": [0, 16, 132, 170], "paused_request": 170, "payload": 12, "pcie": [16, 23, 111], "pd": [16, 152], "pdf": [0, 104, 110], "pdl": [13, 20, 21, 152], "peak": [0, 2, 3, 4, 8, 13, 40], "peer": 16, "peft": 147, "peft_cache_config": [44, 88, 147, 152, 165], "peftcacheconfig": [0, 88, 147, 165], "peftcachemanag": [0, 152], "penal": [0, 106, 147], "penalti": [8, 95, 152, 168], "penalty_alpha": 106, "pend": [58, 170], "pending_load": 58, "pending_request": 170, "pennsylvania": [30, 33], "peopl": [11, 12, 24, 33], "pep": 10, "per": [0, 1, 2, 3, 5, 6, 8, 10, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 27, 39, 40, 59, 61, 62, 63, 75, 84, 86, 87, 91, 93, 102, 103, 105, 106, 108, 111, 113, 117, 119, 124, 125, 126, 127, 133, 134, 136, 139, 141, 147, 152], "per_channel": 141, "per_group": 141, "per_token": 141, "per_token_scal": 133, "perceiv": 4, "percent": [0, 114], "percentag": [22, 39, 40, 110, 114, 124, 125], "percentil": [26, 29, 30, 31, 32, 33, 39, 124, 152], "perf": [0, 2, 15, 17, 21, 40, 74, 97, 133, 147, 152], "perf_best_practic": 152, "perf_metrics_max_request": 147, "perfect": [8, 16, 17], "perfectli": [11, 16], "perform": [0, 1, 3, 5, 6, 11, 14, 15, 23, 26, 27, 37, 39, 40, 44, 58, 65, 75, 81, 82, 83, 84, 86, 87, 89, 91, 92, 94, 96, 97, 98, 102, 103, 105, 106, 107, 110, 112, 117, 118, 119, 124, 126, 129, 130, 132, 133, 136, 138, 142, 146, 147, 148, 151, 152, 153, 155, 160, 162, 163, 164, 169], "performantli": 3, "perhap": 86, "period": [0, 8, 16, 86, 147], "permiss": 152, "permut": 133, "perplex": [82, 163], "persimmon": 152, "persist": [7, 8, 20, 58], "persistentkvcacheconnectorlead": 58, "persistentkvcacheconnectormetadata": 58, "persistentkvcacheconnectorwork": 58, "person": [60, 64, 120], "perspect": [8, 10, 20, 92], "pertain": [78, 158], "phase": [0, 3, 6, 8, 10, 11, 13, 14, 15, 16, 17, 20, 23, 84, 87, 91, 96, 102, 107, 113, 124, 128, 129, 130, 131, 132, 133, 136, 139, 147, 150, 152, 155, 169], "phi": [78, 82, 93, 133, 140, 141, 142, 149, 152, 158, 163], "phi3config": 135, "phi3forcausallm": [135, 142, 149], "phi3model": 135, "phi4mmforcausallm": [142, 149], "phiconfig": 135, "phiforcausallm": 135, "phimodel": 135, "physic": [20, 133, 139], "pick": 130, "pickl": 152, "pictur": [30, 33], "pie": 19, "piec": [1, 16, 91, 130], "piecewis": [147, 152], "pil": 22, "pillar": 12, "pin": [0, 1, 109], "ping": 152, "pinnedmemusag": 0, "pinnedpool": 1, "pip": [2, 9, 24, 29, 30, 31, 32, 33, 81, 94, 98, 100, 152, 162], "pip3": [81, 101, 162], "pipefail": [30, 33], "pipelin": [0, 1, 3, 6, 21, 22, 23, 24, 27, 37, 40, 57, 78, 90, 97, 103, 106, 117, 124, 128, 129, 136, 139, 147, 150, 152, 158, 170], "pipeline_parallel_s": [57, 126, 127, 147], "pipelineparallel": [0, 1, 106], "pipelineparallelismrank": 1, "pitfal": [109, 119], "pixart": 134, "pixartalphatextproject": 134, "pixel": 89, "pixel_valu": 135, "pkl5": 152, "pl": [39, 101, 124], "place": [1, 10, 16, 20, 23, 30, 86, 101, 104, 133, 136, 148, 152, 153], "placemen": 16, "placement": [13, 16, 92], "plai": [21, 91, 130], "plain": 33, "plan": [13, 16, 17, 20, 33, 81, 98, 103, 105, 147, 152, 162], "planner": 152, "plateau": 91, "platform": [8, 16, 21, 32, 39, 46, 49, 54, 55, 57, 98, 101, 120, 121, 124, 144, 145, 146, 151, 152], "pleas": [2, 3, 5, 6, 7, 13, 15, 16, 17, 18, 21, 27, 29, 30, 31, 32, 39, 40, 44, 46, 53, 75, 78, 84, 86, 87, 89, 94, 98, 99, 102, 105, 107, 111, 113, 115, 121, 124, 126, 128, 129, 133, 143, 144, 145, 146, 147, 152, 158, 170], "plot": [17, 19], "plu": [12, 16, 17, 82, 111, 138, 163], "plug": 21, "plugin": [75, 83, 98, 105, 106, 107, 114, 116, 130, 133, 135, 139, 141, 143, 147, 152, 164], "plugin_config": [127, 131, 133, 135, 147], "plugin_namespac": 107, "plugin_typ": 107, "plugin_v2": 107, "plugin_v2_gemm_0": 143, "pluginconfig": [136, 147], "pluginfield": 152, "pluginv2build": 143, "pm": [2, 13, 39, 124], "pmi": 143, "pmi2_init": 143, "pmix": [27, 61, 62, 63, 117, 143], "png": [27, 42, 48, 70], "po": 134, "point": [1, 4, 7, 11, 16, 17, 20, 21, 24, 33, 37, 40, 53, 57, 75, 81, 91, 93, 105, 117, 126, 132, 133, 141, 143, 147, 151, 152, 162], "pointer": [0, 1, 10, 16, 106, 118, 133, 138, 152], "pointerelementtyp": 1, "pointermemorymap": 1, "polar": [142, 149], "polic": 33, "polici": [0, 1, 16, 19, 21, 22, 39, 51, 92, 124, 125, 139, 147], "poll": [0, 27], "polyhedr": 117, "pong": 152, "pool": [0, 1, 15, 16, 17, 19, 20, 75, 84, 86, 91, 96, 105, 133, 138, 147, 152, 169, 170], "pooled_project": [134, 135], "pooled_projection_dim": 134, "pooledpin": 0, "poor": 8, "popd": 143, "popfirstgentoken": 0, "popul": [1, 29, 30, 31, 32, 73, 75, 105, 117, 133, 147], "popular": [7, 11, 14, 105, 116, 119, 121, 146, 150], "port": [0, 9, 17, 18, 21, 27, 29, 30, 31, 32, 33, 45, 84, 121, 151], "portfolio": 5, "portion": [92, 104, 126, 133, 139], "pos_emb_typ": 133, "pos_embd_param": [75, 155], "pos_embed_max_s": 134, "pos_embed_typ": 134, "pose": 131, "posit": [0, 1, 12, 13, 14, 59, 113, 124, 133, 134, 138, 147, 152, 155], "position_embed": [133, 134], "position_embedding_typ": [75, 105, 116, 133, 134, 135], "position_encoding_2d": 135, "position_id": [75, 135, 138, 143, 148, 153, 155], "positionalembeddingparam": [75, 155], "positionembeddingtyp": [75, 105, 133, 134, 135], "positionid": [0, 1], "positionidsbas": 1, "positionidsdevic": 1, "positionidshost": 1, "positionidshostcopi": 1, "positionoffset": 1, "positionoffsetsdevic": 1, "positionoffsetshost": 1, "positionoffsetshostcopi": 1, "posix": 0, "posix_debug_fallback": 0, "possibl": [2, 10, 11, 12, 14, 15, 16, 17, 20, 23, 29, 30, 31, 32, 33, 36, 38, 40, 44, 75, 84, 91, 95, 98, 101, 102, 103, 105, 106, 109, 113, 117, 123, 124, 127, 130, 132, 133, 139, 143, 147, 152, 154], "possibli": [1, 108, 133], "post": [0, 4, 7, 8, 9, 10, 12, 13, 14, 15, 16, 17, 20, 37, 38, 40, 84, 85, 116, 123, 133, 147, 149, 151, 152], "post_act_fn": 134, "post_attention_layernorm": [118, 148, 153], "post_input_id": 138, "post_layernorm": [115, 116, 118, 133, 143], "post_merg": 35, "post_pad": 133, "post_proc": 152, "post_prompt": 138, "post_strid": 133, "posterior_threshold": 147, "posterioralpha": 1, "posterioralphahost": 1, "posteriorthreshold": [0, 1], "posteriorthresholdhost": 1, "postproc_param": 147, "postproc_work": 147, "postprocess": [27, 134, 147, 152], "postprocess_tokenizer_dir": 147, "postprocessor": [0, 147], "postprocparam": 147, "postprocwork": 147, "potenti": [0, 1, 8, 10, 11, 12, 15, 16, 19, 20, 23, 38, 39, 86, 108, 113, 123, 124, 127, 136, 148, 153], "pow": 133, "power": [5, 7, 12, 13, 15, 16, 20, 21, 60, 78, 86, 109, 117, 130, 152, 158], "pp": [0, 3, 6, 17, 22, 27, 39, 40, 84, 102, 106, 110, 124, 125, 133, 136, 152], "pp1": 40, "pp2": [17, 40, 84, 124], "pp4": 40, "pp8": 40, "pp_communicate_final_output_id": 138, "pp_communicate_new_token": 138, "pp_reduce_scatt": [23, 131, 136], "pp_size": [22, 24, 27, 33, 40, 45, 116, 117, 124, 126, 137, 152], "ppreducescatt": 1, "pq": 147, "pr": [10, 12, 13, 16, 20, 30], "practic": [4, 5, 13, 15, 16, 17, 76, 96, 97, 105, 108, 117, 139, 152, 156], "practition": [29, 30, 31, 32], "pre": [0, 1, 16, 17, 20, 21, 24, 26, 35, 36, 37, 39, 75, 84, 92, 95, 96, 98, 100, 101, 103, 105, 116, 124, 133, 139, 147, 150, 151, 152, 155], "pre_input_id": 138, "pre_layernorm": 133, "pre_merg": 35, "pre_onli": 134, "pre_pad": 133, "pre_prompt": 138, "pre_quant_scal": [116, 147], "pre_strid": 133, "prebuilt": 98, "preced": [117, 133], "precis": [1, 2, 3, 7, 11, 16, 20, 21, 22, 23, 29, 33, 39, 93, 97, 106, 118, 124, 127, 131, 136, 139, 142, 147, 150, 152], "precompute_relative_attention_bia": 135, "precomputed_relative_attent": 134, "predefin": [75, 113, 148, 153, 155], "predict": [1, 8, 10, 13, 14, 16, 37, 105, 113, 152], "predictor": 113, "predictsdrafttoken": 1, "prefer": [7, 20, 98, 112, 160], "prefer_managed_weight": 134, "prefer_plugin": 133, "preferenti": 86, "prefetch": 13, "prefil": [0, 10, 12, 15, 16, 17, 22, 27, 58, 84, 85, 96, 97, 128, 129, 140, 147, 149, 150, 152], "prefix": [11, 14, 19, 26, 29, 30, 31, 32, 33, 35, 86, 96, 103, 113, 116, 125, 133, 143, 146, 147], "prefix_token_ad": 60, "preliminari": [3, 5, 6, 16], "preload": 118, "prem": 21, "premis": [12, 14], "prepar": [0, 12, 13, 14, 16, 24, 29, 30, 31, 32, 37, 38, 62, 75, 84, 91, 95, 96, 102, 123, 130, 133, 135, 141, 152, 155, 168], "prepare_dataset": [2, 38, 39, 40, 62, 97, 123, 124, 125], "prepare_draft_token": 96, "prepare_input": [135, 139], "prepare_position_ids_for_cogvlm": 138, "prepare_recurrent_input": 135, "prepare_resourc": [96, 154, 169], "prepend": 143, "preprocess": [22, 83, 89, 118, 138, 141, 152, 164], "preprocess_weights_hook": 135, "preprocessed_dataset": 22, "preprocessor": [17, 39, 124], "preqrequisit": 101, "prequant_scaling_factor": 116, "prerequisit": [97, 101], "presenc": [106, 117], "presence_penalti": [138, 147, 152], "presencepenalti": [0, 1, 106], "present": [0, 8, 9, 10, 16, 17, 20, 26, 39, 40, 84, 87, 91, 96, 124, 130, 131, 141, 147, 152], "preserv": 127, "presid": [49, 50, 51, 101, 125, 132, 144, 146, 151], "press": 26, "pressur": [16, 20], "pretrained_config": [83, 148, 153, 164], "pretrained_model_name_or_path": 135, "pretrainedconfig": [115, 119, 135, 136, 147, 148, 153], "pretrainedmodel": [119, 135, 139], "pretrainedtoken": 60, "pretrainedtokenizerbas": 147, "prevdrafttokenslen": 1, "prevent": [8, 10, 11, 13, 15, 21, 22, 29, 30, 31, 32, 33, 86, 87, 146, 152], "preview": 152, "previou": [1, 2, 4, 9, 11, 12, 14, 16, 19, 20, 37, 39, 83, 86, 90, 99, 103, 104, 112, 113, 119, 124, 125, 126, 127, 130, 131, 132, 152], "previous": [1, 3, 19, 20, 37, 86, 91, 96, 112, 127, 130, 132, 152], "previous_batch": [37, 90], "previous_tensors_devic": [37, 90], "prevscor": 1, "price": [39, 124], "priem": 58, "primari": [0, 1, 7, 12, 16, 20, 30, 34, 83, 86, 108, 139, 152, 164, 170], "primarili": [2, 12, 20, 75, 155], "primary_kernel": 12, "primit": [15, 16, 20, 117], "principl": [8, 16], "print": [1, 10, 11, 22, 35, 40, 44, 49, 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 75, 101, 105, 124, 125, 132, 139, 143, 144, 146, 147, 151], "print_iter_log": [2, 21, 26, 62, 147], "printabl": 147, "prior": [23, 33, 98, 101, 103], "priorit": [7, 8, 37, 86, 91, 94, 130, 132], "prioriti": [0, 1, 78, 86, 108, 109, 118, 147, 158], "prioritytyp": 0, "priorityupd": 0, "privat": [0, 1, 18, 19, 106, 136, 147], "privileg": 107, "prm": [142, 149], "prmreward": 11, "pro": [13, 19, 40, 150, 152], "prob": [133, 152, 168], "probabilist": 134, "probabl": [0, 1, 10, 13, 14, 21, 33, 66, 95, 106, 109, 113, 133, 138, 147, 152], "probe_answ": 11, "probe_respons": 11, "probe_suffix": 11, "probe_suffix_token_num": 11, "probe_task": 11, "probe_text": 11, "probil": 1, "problem": [2, 8, 10, 11, 15, 20, 75, 92, 94, 105, 143, 152], "problemat": [10, 96], "proc": [20, 118], "proccessed_weight": 118, "proccessed_zero": 118, "proce": [11, 17, 20, 84, 98, 147], "procedur": [2, 96], "proceed": [8, 10, 117], "process": [0, 1, 2, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 22, 23, 24, 26, 29, 30, 31, 32, 33, 36, 37, 38, 39, 40, 53, 57, 58, 59, 60, 61, 62, 63, 75, 81, 82, 83, 84, 87, 89, 90, 91, 92, 95, 102, 103, 105, 106, 108, 111, 113, 116, 117, 119, 123, 124, 125, 126, 130, 131, 132, 133, 136, 138, 143, 146, 147, 148, 150, 151, 152, 153, 154, 155, 162, 163, 164, 170], "process_input": 138, "process_logits_including_draft": 138, "processor": [0, 17, 52, 75, 85, 89, 97, 105, 135, 138, 147, 149, 152], "processorbatch": 0, "processormap": 0, "prod": 133, "produc": [0, 1, 10, 11, 12, 21, 39, 44, 91, 95, 96, 103, 107, 117, 124, 125, 127, 130, 131, 133, 152], "product": [5, 8, 9, 16, 17, 19, 20, 21, 24, 58, 60, 75, 76, 78, 82, 84, 91, 92, 104, 105, 113, 117, 130, 131, 132, 133, 155, 156, 158, 163], "profession": 18, "profil": [12, 20, 23, 47, 48, 97, 127, 130, 133, 136, 138, 139, 143, 147, 152], "profiling_verbos": [23, 147], "profit": [39, 113, 124], "program": [10, 11, 20, 49, 54, 55, 57, 58, 101, 119, 132, 143, 144, 146, 151], "progress": [1, 8, 11, 13, 39, 81, 124, 133, 147, 162], "proj": [116, 118, 143], "project": [10, 12, 15, 20, 75, 81, 94, 98, 100, 101, 105, 110, 133, 134, 148, 153, 162, 169], "projector_hidden_act": 135, "prologu": [61, 62, 63], "promin": 113, "promis": [12, 14, 17, 113, 119], "promot": 152, "prompt": [0, 2, 10, 11, 17, 19, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 34, 37, 39, 43, 44, 49, 50, 51, 53, 54, 55, 56, 57, 59, 60, 61, 64, 65, 66, 67, 68, 71, 72, 75, 77, 78, 84, 86, 88, 91, 95, 96, 101, 103, 106, 109, 113, 115, 124, 125, 130, 131, 132, 134, 138, 144, 146, 147, 151, 152, 155, 157, 158, 165, 168], "prompt_a": 59, "prompt_adapter_request": [147, 152], "prompt_b": 59, "prompt_budget": [67, 147], "prompt_embedding_t": [134, 135, 138], "prompt_embedding_table_s": 135, "prompt_id": 60, "prompt_len": [75, 155], "prompt_logprob": [147, 152], "prompt_lookup": 152, "prompt_lookup_num_token": 106, "prompt_tabl": 138, "prompt_task": [135, 138], "prompt_token": [18, 21, 29, 30, 31, 32, 33, 151], "prompt_token_id": [21, 29, 30, 31, 32, 33, 44, 147], "prompt_vocab_s": [135, 138], "promptadapterrequest": 147, "promptinput": [147, 152], "promptlen": 0, "promptli": [10, 17, 84], "prompttableoffload": 0, "prompttuningconfig": 0, "prompttuningembed": 134, "prompttuningen": 1, "pronounc": [8, 12, 16, 20, 113], "proof": 169, "prop": 1, "propag": [1, 109, 152], "propel": 8, "proper": [20, 22, 34, 39, 78, 102, 124, 158], "properli": [10, 16, 18, 20, 21, 29, 30, 31, 32, 33, 34, 101, 118, 130, 132], "properti": [19, 36, 50, 53, 73, 95, 103, 133, 135, 136, 138, 147], "proport": [75, 105], "proportion": 19, "propos": [0, 8, 11, 13, 19, 96, 97], "proposer_task": 11, "proprietari": [83, 164], "protect": [1, 34, 57, 146, 152], "proto": [12, 147], "protocol": [0, 10, 12, 17, 20, 27, 53, 84], "protopackeddata": 12, "prototyp": [20, 34, 97, 147, 162], "prototype_control": 11, "proud": [8, 13, 16, 17], "prove": [15, 113], "prover": [82, 163], "provid": [0, 1, 2, 3, 4, 7, 8, 10, 11, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 44, 50, 51, 53, 60, 64, 75, 76, 78, 80, 81, 82, 83, 84, 86, 87, 88, 91, 97, 98, 102, 103, 104, 105, 106, 107, 108, 109, 110, 112, 113, 115, 116, 119, 120, 123, 124, 125, 126, 127, 128, 129, 130, 131, 133, 138, 139, 142, 143, 146, 147, 148, 150, 151, 152, 153, 154, 155, 156, 158, 161, 162, 163, 164, 165], "provinc": 30, "proxy_dispatch_result_thread": 124, "prune": [10, 107, 113, 133], "pseudo": [75, 98, 105, 133, 141], "pt": [22, 58], "pth": [83, 118, 152, 164], "ptq": [7, 127, 152], "ptr": 1, "ptr_idx": 118, "ptrdiff_t": 1, "ptune": [35, 152], "ptuning_setup": 138, "ptuning_setup_fuyu": 138, "ptuning_setup_llava_next": 138, "ptuning_setup_phi3": 138, "ptuning_setup_pixtr": 138, "ptuningconfig": 0, "public": [0, 1, 7, 34, 121, 152], "publish": [2, 3, 6, 12, 29, 30, 31, 32, 39, 40, 99, 124, 152], "pull": [2, 9, 18, 21, 35, 98, 152], "pull_pipe_addr": 147, "punctuat": 147, "puneeshkhanna": 152, "purchas": [39, 124], "pure": [21, 82, 138], "purpos": [1, 8, 15, 16, 17, 26, 34, 75, 91, 96, 98, 105, 108, 125, 127, 130, 131, 136], "pursu": [49, 54, 55, 57, 101, 144, 146, 151], "push": [15, 16, 97, 120], "push_pipe_addr": 147, "pushd": 143, "put": [1, 10, 13, 20, 30, 33, 61, 62, 63, 91, 101, 116, 130, 146], "put_zcopi": [84, 102], "pwd": [2, 98], "py": [2, 13, 14, 15, 16, 26, 29, 30, 31, 32, 33, 34, 36, 38, 39, 40, 50, 51, 58, 59, 61, 62, 66, 67, 77, 81, 89, 94, 96, 97, 98, 103, 104, 105, 107, 110, 112, 113, 114, 115, 116, 117, 118, 119, 123, 124, 125, 126, 127, 133, 136, 138, 143, 146, 148, 152, 153, 154, 157, 162, 169, 170], "py3": 152, "py_draft_token": 96, "py_executor_cr": 170, "py_rewind_len": 96, "pybind": [147, 152], "pybind_class": 147, "pybind_equ": 147, "pybind_inst": 147, "pybindmirror": 147, "pydant": [34, 78, 136, 147, 152, 158], "pydantic_cor": 147, "pydanticserializationerror": 147, "pydanticundefin": 147, "pyexecutor": [34, 37, 58, 90, 96, 152, 169, 170], "pynvml": 152, "pypi": [98, 100, 152], "pytest": [34, 35], "python": [1, 2, 11, 14, 15, 17, 20, 22, 26, 27, 29, 30, 31, 32, 33, 34, 35, 37, 38, 39, 40, 44, 55, 56, 59, 66, 67, 77, 78, 81, 82, 88, 94, 101, 105, 106, 107, 110, 113, 115, 117, 119, 123, 124, 125, 126, 141, 146, 147, 148, 150, 151, 152, 153, 154, 157, 158, 162, 163, 165, 169, 170], "python3": [2, 29, 30, 31, 32, 33, 38, 39, 61, 62, 89, 98, 110, 112, 114, 116, 123, 124, 143], "python_bind": 2, "python_e2": 138, "python_plugin": 152, "pythonpath": [2, 62, 63], "pytorch": [2, 10, 11, 14, 15, 16, 21, 22, 24, 27, 29, 30, 31, 32, 35, 37, 40, 50, 51, 52, 58, 66, 67, 75, 76, 82, 83, 84, 88, 89, 93, 95, 96, 97, 98, 101, 107, 113, 116, 133, 146, 147, 148, 152, 154, 155, 156, 163, 164, 165, 167, 168, 169, 170], "pytorch_cuda_alloc_conf": 29, "pytorch_extra_arg": 62, "pytorch_model": 143, "pytorch_model_registri": 169, "pytorchconfig": [75, 147, 152, 155], "pytorchmodelengin": [154, 169], "pzzzzz5142": 152, "q": [3, 9, 13, 15, 22, 39, 75, 84, 102, 105, 106, 110, 124, 133, 143, 148, 153, 155], "q_b_proj": 133, "q_dim": 133, "q_lora_rank": [133, 134], "q_proj": [39, 83, 88, 118, 124, 148, 153, 164, 165], "q_scale": [75, 105, 133, 134, 135], "qa": [24, 29, 30, 31, 32, 113], "qformat": [124, 137], "qgmma": 152, "qingquansong": 152, "qk_layernorm": [134, 135], "qk_nope_head_dim": [133, 134], "qk_norm": 134, "qk_rope_head_dim": [133, 134], "qkv": [12, 107, 110, 116, 118, 133, 143, 152, 155], "qkv_bia": [133, 152], "qkv_dim": 133, "qkv_proj": [83, 148, 153, 164], "qo_indptr": [75, 155], "qpi": 111, "qserv": [136, 152], "qserve_gemm_plugin": 136, "quadrat": [75, 105, 139], "qualifi": 35, "qualiti": [14, 16, 21, 95, 127, 131], "qualnam": [133, 135, 137, 147], "quant": [12, 39, 93, 119, 124, 133, 136, 147, 152, 167], "quant_algo": [39, 44, 88, 116, 118, 119, 124, 127, 135, 147, 165], "quant_config": [44, 75, 88, 119, 127, 135, 147, 155, 165], "quant_medusa_head": 137, "quant_mod": [119, 134, 135, 138, 147], "quantalgo": [44, 88, 97, 127, 135, 137, 147, 165], "quantconfig": [44, 75, 88, 97, 119, 127, 135, 147, 152, 155, 165], "quanticonfig": 119, "quantifi": [8, 75], "quantiz": [2, 3, 4, 13, 15, 20, 21, 22, 23, 29, 31, 32, 40, 49, 54, 75, 81, 82, 97, 101, 105, 106, 111, 117, 118, 122, 125, 128, 129, 133, 134, 135, 136, 138, 142, 144, 145, 146, 147, 148, 150, 151, 152, 153, 155, 162, 163], "quantizaton": 124, "quantize_and_export": 137, "quantize_kwarg": 135, "quantize_lm_head": [137, 152], "quantize_per_token_plugin": 136, "quantize_tensor_plugin": 136, "quantized_valu": [75, 105], "quantizedkernel": 117, "quantizetensorplugin": 117, "quantmod": [1, 75, 105, 106, 133, 134, 135, 137, 138, 147], "quantmodewrapp": [133, 147], "quebec": 30, "queen": [30, 33], "queri": [3, 15, 17, 18, 21, 27, 29, 30, 31, 32, 33, 39, 65, 77, 84, 86, 97, 103, 106, 108, 113, 117, 124, 133, 139, 152, 155, 157, 169], "query_dim": 134, "query_key_valu": 118, "query_length": 134, "query_pre_attn_scalar": 135, "question": [11, 16, 19, 30, 39, 64, 124, 139, 143], "queu": [0, 40, 91, 130], "queue": [0, 10, 27, 35, 37, 147, 154], "quick": [28, 50, 51, 81, 105, 124, 125, 150, 155, 162], "quick_gelu": 133, "quicker": 126, "quickli": [20, 89, 119, 151], "quickstart": [50, 51, 125, 146], "quickstart_advanc": [14, 61], "quickstart_multimod": 89, "quit": [26, 107], "quot": [35, 152], "qweight": 118, "qwen": [11, 27, 33, 39, 48, 59, 82, 89, 93, 118, 124, 133, 135, 141, 142, 149, 152, 163], "qwen1": [142, 152], "qwen2": [26, 27, 39, 42, 48, 70, 82, 89, 110, 124, 140, 142, 149, 152, 163], "qwen2_5_vlforconditionalgener": [142, 149], "qwen2_audio_7b_instruct": 35, "qwen2audio": 152, "qwen2forcausallm": [142, 149], "qwen2forprocessrewardmodel": [142, 149], "qwen2forrewardmodel": [142, 149], "qwen2forsequenceclassif": 152, "qwen2vl": 152, "qwen2vlforconditionalgener": [142, 149], "qwen3": [16, 28, 59, 82, 92, 97, 142, 149, 152, 163], "qwen3_output": 33, "qwen3forcausallm": [142, 149], "qwen3mo": [142, 149], "qwen3moeforcausallm": [142, 149], "qwen3next": 149, "qwen3nextforcausallm": 149, "qwenforcausallm": [118, 135], "qwenforcausallmgenerationsess": 138, "qwenvl": 152, "qwq": [82, 142, 149, 163], "qychen": 110, "qzero": 118, "r": [1, 24, 27, 49, 50, 51, 53, 54, 55, 56, 57, 59, 60, 64, 67, 94, 101, 110, 125, 132, 133, 143, 144, 146, 147, 151, 152], "r1": [10, 11, 12, 16, 20, 27, 28, 68, 74, 82, 92, 93, 96, 97, 150, 152, 163], "r1_in_tensorrt": [13, 152], "race": 152, "radix": [86, 169], "rai": [97, 147], "rais": [11, 12, 60, 67, 68, 119, 125, 143, 147, 152], "raise_error": 147, "ram": 59, "rand": [22, 39, 124, 133], "rand_data": 133, "rand_data_valid": 135, "randint": 10, "random": [0, 19, 22, 24, 26, 29, 30, 31, 32, 33, 40, 47, 48, 66, 106, 133, 147, 152], "random_imag": 26, "random_se": [24, 135, 138, 147], "randomdatasampl": 1, "randomdatavalid": 1, "randomli": [39, 40, 124], "randomse": [1, 106, 152], "randomseedtyp": 0, "rang": [0, 1, 10, 11, 12, 17, 18, 22, 38, 39, 58, 60, 81, 86, 106, 109, 113, 123, 124, 131, 133, 135, 139, 141, 142, 143, 146, 147, 148, 150, 153, 162], "rank": [0, 1, 2, 8, 12, 15, 16, 20, 21, 23, 37, 39, 81, 84, 92, 97, 102, 103, 104, 106, 110, 119, 124, 133, 135, 138, 139, 143, 147, 152, 162], "rank0": 116, "rank1": 116, "rapid": [12, 40, 113], "rapidli": 18, "rate": [0, 2, 10, 12, 13, 14, 15, 17, 22, 26, 29, 30, 31, 32, 37, 39, 40, 47, 48, 59, 96, 124, 125, 152], "rather": [15, 16, 18, 75, 101, 105, 107, 113], "ratio": [15, 16, 17, 19, 86, 147], "ration": [17, 84], "rational": 15, "raw": [21, 27, 37, 83, 89, 164], "raw_audio": 138, "raw_imag": 138, "ray_orchestr": 94, "rayexecutor": 94, "raygpuwork": 94, "rc": [29, 30, 31, 32], "rcn": [29, 30, 31, 32], "rdma": [17, 84, 102], "re": [2, 7, 11, 16, 18, 29, 30, 31, 32, 33, 75, 91, 96, 147, 150, 152, 155], "reach": [0, 11, 17, 20, 37, 75, 91, 105, 116, 124, 127, 132, 146, 147], "reachabl": 36, "react": 16, "read": [0, 2, 10, 12, 13, 14, 16, 20, 23, 39, 64, 75, 87, 102, 103, 105, 113, 115, 117, 118, 124, 147, 152], "read_config_from_the_custom_training_checkpoint": 119, "read_input": 67, "readabl": [37, 39, 124], "reader": 133, "readi": [0, 8, 9, 10, 12, 18, 21, 29, 30, 31, 32, 33, 37, 81, 89, 98, 160, 162], "readm": [94, 113, 125, 144, 146, 152], "real": [2, 8, 13, 16, 19, 20, 21, 22, 87, 92, 96, 98, 107, 112, 125, 127, 130, 131, 133, 143], "realist": [91, 160], "realiti": 130, "realiz": [109, 113], "realli": 33, "rearrang": 133, "reason": [0, 8, 10, 11, 13, 14, 16, 18, 27, 30, 33, 35, 37, 39, 74, 75, 82, 91, 96, 97, 105, 106, 117, 119, 124, 126, 130, 131, 133, 143, 147, 163], "reasoning_cont": [21, 30, 33], "reasoning_pars": [27, 45, 147], "reassess": 11, "rebalanc": 16, "rebuild": [36, 91, 131, 133, 143, 147], "receiv": [0, 1, 12, 16, 17, 20, 26, 29, 30, 31, 32, 33, 84, 92, 102, 103, 104, 111, 113, 127, 133, 147, 152], "recent": [1, 4, 8, 11, 12, 13, 36, 75, 86, 105, 112], "recept": [17, 84], "recip": [13, 15, 93, 97, 141, 147], "reclaim": 0, "recogn": [13, 16, 17, 39, 113, 124, 148, 153], "recommend": [2, 4, 7, 10, 15, 16, 17, 21, 24, 26, 27, 29, 30, 31, 32, 33, 34, 39, 60, 75, 84, 91, 98, 102, 105, 106, 113, 115, 118, 124, 128, 129, 130, 132, 143, 147, 148, 152, 153, 155], "recomput": 59, "recompute_scale_factor": 133, "reconfigur": [101, 103], "reconnect": 94, "reconstruct": [75, 105, 133], "record": [1, 2, 10, 13, 14, 16, 39, 107, 147, 150], "record_cr": 147, "record_stat": 147, "recored": 0, "recov": [59, 147], "rectangular": 30, "recur": 59, "recurr": [10, 113], "recurrentgemma": [141, 142, 152], "recurrentgemmaforcausallm": 135, "recurs": [2, 18, 78, 98, 146, 158], "recv": [0, 16, 117, 133], "recvconnect": 0, "recvpollperiodm": 0, "recycl": [75, 91, 105, 169], "redesign": [12, 152], "redirect": [107, 147], "redistribut": [16, 20, 92], "redownload": [21, 29, 30, 31, 32], "redraft": [133, 135, 138, 152], "redrafter_draft_len_per_beam": 138, "redrafter_num_beam": 138, "redrafterforllamalm": 135, "redrafterforqwenlm": 135, "redraftermixin": 135, "reduc": [2, 3, 6, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 23, 29, 30, 31, 32, 33, 35, 37, 38, 39, 40, 60, 65, 75, 84, 86, 87, 90, 91, 92, 93, 96, 98, 102, 103, 104, 105, 109, 111, 113, 117, 123, 124, 125, 126, 130, 132, 133, 136, 139, 143, 146, 147, 150, 152, 155, 166], "reduce_fus": [23, 124, 127, 131, 136, 152], "reduce_scatt": [20, 133], "reduceoper": 133, "reducescatt": [20, 23, 131, 136, 152], "reduct": [8, 11, 13, 111, 113, 132, 133, 152], "redund": [8, 13, 16, 86, 113, 152], "redux": 152, "ref": 67, "ref_templ": 147, "refactor": [2, 12, 96, 119, 152], "refer": [0, 1, 2, 9, 12, 16, 17, 18, 20, 21, 26, 27, 29, 30, 31, 32, 33, 34, 39, 40, 41, 42, 43, 44, 45, 47, 48, 49, 60, 67, 69, 70, 71, 72, 73, 75, 77, 78, 81, 82, 84, 89, 94, 98, 99, 102, 103, 105, 106, 107, 108, 110, 113, 117, 119, 121, 124, 125, 126, 127, 128, 129, 131, 133, 142, 144, 146, 148, 150, 151, 152, 153, 155, 157, 158, 162], "referenc": 127, "reference_wrapp": [0, 103], "references_commit": 34, "refin": [20, 152], "refit": [23, 117, 147, 152], "refit_engin": 117, "reflect": [8, 20, 84, 102, 130], "refresh": [2, 124], "regard": [81, 94, 133, 162], "regardless": [11, 20, 83, 143, 164], "regex": [24, 95, 103, 147], "region": [11, 30, 38, 123], "regist": [16, 20, 58, 83, 120, 143, 147, 148, 152, 153, 164], "register_auto_model": [148, 153], "register_checkpoint_load": [83, 164], "register_checkpoint_weight_load": [83, 164], "register_config_load": [83, 164], "register_error": 147, "register_kv_cach": 58, "register_mapp": [83, 164], "register_network_output": 143, "registerdesc": 0, "registermemori": 0, "regress": [29, 31, 96, 105, 106, 117, 152], "regular": [0, 13, 95, 103, 105, 133, 147], "regularli": 11, "reimplement": 12, "reinforc": [94, 128, 129], "reject": [0, 14, 96, 97], "rel": [3, 16, 75, 91, 109, 130, 132, 133, 152], "rel_attn_t": 134, "relat": [12, 19, 30, 38, 81, 84, 99, 102, 104, 108, 118, 123, 133, 136, 139, 143, 145, 146, 148, 152, 153, 162, 169], "relationship": [8, 139], "relative_attent": [133, 134], "relative_attention_bia": 133, "relax": [8, 16, 91, 96, 97, 105], "relaxed_delta": [13, 14, 68, 96, 147], "relaxed_topk": [13, 14, 68, 96, 147], "releas": [1, 3, 6, 7, 9, 10, 16, 17, 20, 21, 26, 29, 30, 31, 32, 34, 75, 83, 84, 86, 97, 98, 100, 101, 105, 106, 108, 119, 133, 135, 139, 141, 142, 147, 151, 164], "release_build": [18, 33, 98], "release_run": [33, 98], "releasepag": 1, "releasest": 0, "releasewithtag": 1, "relev": [33, 36, 83, 98, 106, 169], "reli": [16, 17, 20, 38, 82, 87, 102, 105, 107, 119, 123, 141, 146, 163], "reliabl": [81, 94, 162], "reload": [16, 103], "relu": [116, 117, 133, 143], "remain": [0, 1, 8, 12, 13, 16, 20, 34, 35, 40, 78, 86, 91, 94, 98, 107, 109, 113, 114, 125, 127, 130, 131, 133, 139, 147, 152, 158], "remaind": [58, 86, 127], "remaining_chunk": 58, "remaining_token": 58, "remark": [13, 14, 20], "remateri": 1, "remedi": 10, "rememb": 16, "remind": [105, 155], "remot": [1, 12, 16, 22, 147], "remotenam": 0, "remov": [0, 1, 2, 10, 12, 14, 17, 21, 23, 24, 27, 34, 35, 75, 84, 86, 91, 96, 98, 105, 106, 107, 108, 117, 118, 127, 133, 136, 139, 147, 148, 152, 153], "remove_const_t": 1, "remove_cv_t": 0, "remove_duplicated_kv_head": 135, "remove_input_pad": [23, 91, 105, 110, 133, 134, 136, 138], "remove_pointer_t": 1, "remove_reference_t": 1, "remove_sequ": 169, "removeprefix": 147, "removesuffix": 147, "renam": 152, "rendezv": [84, 102], "reopen": 36, "reorder": [133, 134], "reorder_kv_cache_for_beam_search": 138, "rep": [38, 123], "repeat": [0, 14, 15, 65, 86, 87, 105, 133, 147], "repeat_interleav": 133, "repeatedli": 113, "repetit": [0, 19, 60, 106, 133, 147], "repetition_penalti": [106, 138, 147, 152], "repetitionpenalti": [0, 1, 106], "replac": [1, 2, 9, 11, 15, 18, 20, 29, 31, 32, 39, 59, 91, 92, 107, 117, 118, 119, 124, 125, 127, 132, 133, 139, 147, 148, 153], "replace_add_with_sub": 107, "replace_all_uses_with": [107, 133], "replace_input_with": 107, "replace_output_uses_with": 107, "replace_outputs_uses_with": 107, "replai": [10, 16], "replic": [0, 13, 16, 103, 133], "replica": 92, "replit": [141, 142, 152], "repo": [9, 21, 119, 125, 143, 152], "repo_id": 64, "report": [14, 15, 16, 18, 22, 38, 39, 40, 108, 123, 124, 139, 152], "report_json": [21, 22], "report_load_statist": 16, "reportpluginerror": 143, "repositori": [2, 9, 24, 36, 96, 98, 113, 120, 146, 151], "repr": 147, "repres": [0, 1, 2, 3, 7, 8, 11, 12, 13, 16, 19, 20, 51, 53, 64, 91, 92, 108, 112, 113, 124, 130, 133, 138, 147, 170], "represent": [12, 37, 83, 107, 117, 147, 164], "reproduc": [10, 20, 22, 39, 97, 124, 152], "req": [2, 26, 29, 30, 31, 32, 39, 40, 58, 96, 124, 125, 127, 130, 131], "req_id": [60, 95], "req_stat": 170, "reqbeamwidth": 1, "reqid": 0, "reqpromptlength": 1, "request": [0, 1, 2, 4, 6, 8, 10, 11, 14, 15, 16, 17, 19, 20, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 35, 37, 38, 39, 40, 47, 48, 51, 58, 59, 62, 75, 76, 84, 87, 88, 89, 92, 96, 97, 102, 105, 106, 109, 110, 117, 123, 124, 125, 127, 130, 131, 132, 133, 139, 147, 150, 151, 152, 154, 155, 156, 160, 165, 169, 170], "request_finish": 58, "request_id": [44, 58, 75, 147, 155], "request_json": 22, "request_perf_metr": [147, 152], "request_stats_max_iter": 147, "request_timeout": 27, "request_typ": 147, "request_type_context_and_gener": 0, "request_type_context_onli": 0, "request_type_generation_onli": 0, "requesterror": [97, 147], "requestid": [0, 102, 103], "requestidtyp": 0, "requestlist": 170, "requestoutput": [44, 97, 147, 152], "requestperfmetr": [0, 147], "requestschedul": 170, "requeststag": 0, "requeststat": 0, "requeststatsmaxiter": 0, "requeststatsperit": 0, "requeststatsperiter": 0, "requeststatsvec": 0, "requesttoken": 103, "requesttyp": [0, 1, 147], "requesttypesdevic": 1, "requestvector": 1, "requir": [0, 2, 3, 7, 8, 9, 10, 11, 12, 13, 15, 16, 17, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 34, 35, 36, 39, 40, 53, 64, 73, 75, 76, 81, 83, 84, 86, 87, 91, 94, 96, 98, 101, 102, 105, 106, 109, 110, 113, 117, 118, 119, 124, 125, 126, 127, 131, 133, 134, 136, 139, 142, 143, 146, 147, 152, 156, 162, 164, 169], "require_ln_f": 135, "requiresattentionmask": 1, "rerun": [21, 29, 30, 31, 32, 131], "rescale_output_factor": 134, "research": [14, 16, 31, 32, 49, 54, 55, 57, 101, 105, 121, 141, 144, 146, 151], "reserv": [0, 1, 20, 24, 27, 29, 30, 31, 32, 33, 132, 138, 139, 147, 170], "reserved_block": 170, "reset": [0, 1, 39, 106, 124, 138, 147, 152], "resetspeculativedecodingmodul": 1, "reshap": [1, 133], "reshapebuff": 1, "reshapecacheindirectionbuff": 1, "reshapespeculativedecodingbuff": 1, "resid": [16, 92, 110], "residu": [8, 96, 133, 143], "residual_connect": 134, "residual_mlp": 135, "residual_multipli": 135, "residual_rms_norm": 133, "residual_rms_norm_out_quant_fp8": 133, "residual_rms_norm_out_quant_nvfp4": 133, "residual_rms_norm_quant_fp8": 133, "residual_rms_norm_quant_nvfp4": 133, "residual_rms_prepost_norm": 133, "residualadd": [23, 131, 136, 152], "resiz": 1, "resize_kv_cach": [76, 156, 160, 161], "resolv": [10, 17, 20, 42, 70, 78, 84, 143, 147, 158], "resourc": [0, 10, 13, 15, 17, 20, 21, 31, 35, 37, 46, 75, 83, 84, 91, 96, 102, 105, 119, 150, 151, 154, 164, 169, 170], "resource_manag": [96, 147], "resourcemanag": 96, "respect": [10, 17, 18, 21, 36, 44, 132, 133, 138, 139, 141, 147, 148, 153, 170], "respond": 20, "respons": [0, 8, 10, 11, 12, 17, 20, 21, 26, 27, 29, 30, 31, 32, 33, 37, 39, 44, 66, 68, 69, 70, 71, 72, 73, 83, 84, 88, 90, 95, 102, 108, 124, 133, 147, 154, 164, 165, 166], "response_format": 73, "response_json": 73, "responsepostprocesswork": 147, "responsewithid": 0, "responsewrapp": 147, "rest": [1, 17, 75, 84, 96, 105, 127, 147], "restart": 0, "restor": 1, "restoremod": 1, "restrict": [0, 35, 98, 103, 106, 133, 147, 168], "result": [0, 1, 3, 4, 5, 7, 10, 11, 12, 14, 15, 16, 17, 19, 21, 22, 23, 26, 27, 29, 30, 31, 32, 33, 35, 37, 39, 44, 67, 75, 84, 87, 90, 92, 95, 98, 104, 105, 111, 113, 117, 124, 126, 127, 128, 129, 130, 131, 133, 134, 136, 147, 148, 152, 153, 155, 168, 170], "result_dir": [26, 29, 30, 31, 32, 33], "result_handl": 147, "ret": 67, "retail": [39, 124], "retain": [3, 5, 14, 21, 96], "retent": [0, 51, 147], "retentionprior": 0, "retentionpriorityanddur": 0, "rethink": [97, 113], "rethrown": 1, "retri": 35, "retriev": [1, 11, 17, 27, 37, 40, 84, 118, 133, 147], "retrievebadhandl": 1, "return": [0, 1, 9, 11, 12, 17, 18, 20, 21, 29, 30, 31, 32, 33, 34, 37, 44, 58, 59, 60, 66, 67, 83, 84, 91, 95, 96, 103, 107, 110, 113, 115, 117, 118, 119, 124, 130, 133, 134, 135, 138, 139, 143, 147, 152, 164, 169, 170], "return_all_generated_token": 138, "return_context_logit": 147, "return_dict": 138, "return_encoder_output": [138, 147], "return_generation_logit": 147, "return_perf_metr": 147, "returnallgeneratedtoken": [0, 103], "returncontextlogit": 0, "returnencoderoutput": 0, "returngenerationlogit": 0, "returnlogprob": 0, "returnperfmetr": 0, "reus": [0, 10, 11, 14, 20, 23, 24, 35, 36, 50, 51, 59, 67, 84, 85, 89, 96, 97, 102, 103, 108, 122, 133, 136, 138, 139, 140, 147, 148, 149, 150, 152, 153, 160, 169], "reusabl": [11, 16, 86, 108, 109], "reusedblock": [0, 27], "reusedblocksperrequest": 0, "reveal": [8, 11, 13, 15, 20], "revers": [1, 12, 133], "revert": [86, 133], "review": [12, 16, 39, 76, 100, 101, 124, 156], "revis": 147, "revisit": 97, "reward": 11, "reward_control": 11, "reward_kwarg": 11, "rewind": [14, 152], "rewrit": [81, 133, 148, 152, 153, 162], "rewritepatternmanag": 107, "rewrt": 143, "rf": 143, "rfind": [58, 147], "rg_lru": 133, "rgc": [39, 124], "rh": [0, 1], "rich": 116, "right": [11, 17, 19, 20, 21, 127, 133, 143, 147], "rigor": [39, 124], "rindex": 147, "risk": [10, 78, 84, 102, 117, 127, 132, 158], "river": [30, 33], "rjust": 147, "rl": 94, "rlhf": 94, "rm": [9, 21, 26, 29, 30, 31, 32, 98, 133, 142, 143, 148, 149, 151, 153], "rms_norm": [13, 133, 148, 153], "rmsnorm": [13, 110, 133, 134, 135, 136, 148, 152, 153], "rmsnorm_quantization_plugin": 136, "rndv": [84, 102], "rnn": [23, 136, 152], "rnn_conv_dim_s": 138, "rnn_head_siz": 138, "rnn_hidden_s": 138, "rnn_state": 135, "rnnconfig": 1, "rnnconvdims": 1, "rnnheadsiz": 1, "rnnhiddens": 1, "ro": [2, 36], "roadmap": 97, "roberta": [142, 152], "robertaforquestionansw": 135, "robertaforsequenceclassif": 135, "robertamodel": 135, "robin": 17, "robot": [21, 66], "robust": [13, 16, 82, 152, 163], "rock": 133, "rocket": 147, "rocketkv": 67, "rocketsparseattentionconfig": [67, 97, 147], "roi": 60, "role": [9, 17, 18, 21, 24, 27, 30, 33, 41, 42, 53, 69, 70, 73, 84, 91, 117, 130, 151, 160], "roll": [1, 10, 33], "rollback": 10, "rooflin": 15, "room": 20, "root": [2, 20, 21, 29, 30, 31, 32, 98, 116, 120, 125, 133, 146, 147, 152], "root_lay": 107, "rootless": 36, "rope": [12, 13, 15, 133, 138, 147, 152, 155], "rope_gpt_neox": [75, 105, 133, 135], "rope_gptj": [75, 105, 133], "rope_local_base_freq": 135, "rope_scaling_config": 133, "rope_scaling_long_factor": 134, "rope_scaling_long_mscal": 134, "rope_scaling_short_factor": 134, "rope_scaling_short_mscal": 134, "ropeembeddingutil": 133, "rotari": [0, 13, 133, 138, 147, 148, 153, 155], "rotary_bas": 135, "rotary_cos_sin": 133, "rotary_dim": 135, "rotary_embed": [148, 153], "rotary_embedding_bas": [133, 134], "rotary_embedding_base_loc": 134, "rotary_embedding_beta_fast": 134, "rotary_embedding_beta_slow": 134, "rotary_embedding_dim": [75, 105, 133, 135], "rotary_embedding_long_m_scal": 133, "rotary_embedding_max_posit": 133, "rotary_embedding_mscal": 134, "rotary_embedding_mscale_all_dim": 134, "rotary_embedding_origin_max_posit": 134, "rotary_embedding_original_max_posit": 133, "rotary_embedding_percentag": 134, "rotary_embedding_sc": 134, "rotary_embedding_scal": 133, "rotary_embedding_scale_typ": 133, "rotary_embedding_short_m_scal": 133, "rotary_inv_freq": [133, 134], "rotary_inv_freq_loc": 134, "rotary_pct": 135, "rotary_sc": [134, 135], "rotaryembed": [148, 153], "rotaryembeddingdim": [0, 1], "rotaryscalingtyp": 133, "rotat": 152, "rotate_every_two": 133, "rotate_half": 133, "roug": 24, "rouge_path": 24, "roughli": [12, 19, 30, 33], "round": [11, 12, 17, 19, 59, 133, 147], "round_robin": 17, "round_trip": 147, "roundtrip": 12, "rout": [12, 15, 16, 17, 20, 84, 92, 102, 152], "router": [15, 16, 17, 84, 92, 104, 110, 152], "router_gemm": 13, "routin": [16, 96, 107], "routingkerneltopk": 13, "row": [10, 91, 110, 130, 133, 141, 152], "rowlinear": [110, 134], "rowwis": [93, 136, 147, 152], "rpartit": 147, "rpc": 147, "rr": 152, "rslora": 152, "rsp": 147, "rsplit": 147, "rst": 103, "rstrip": 147, "rt": 21, "rtx": [40, 150, 152], "rubric": 133, "rule": [78, 105, 126, 143, 158], "run": [0, 1, 3, 7, 10, 11, 12, 13, 15, 17, 18, 19, 20, 22, 23, 24, 25, 27, 35, 36, 50, 52, 55, 56, 58, 59, 65, 66, 75, 81, 82, 84, 90, 91, 92, 94, 96, 97, 98, 100, 101, 102, 103, 105, 106, 109, 112, 113, 115, 116, 117, 120, 121, 126, 127, 130, 131, 132, 133, 138, 139, 141, 143, 147, 148, 152, 153, 154, 155, 162, 163, 166, 169], "run_all_demonstr": 66, "run_cmd": 98, "run_dtm_ngram": 113, "run_eagle3": 68, "run_mtp": 68, "run_ngram": 68, "run_rocketkv": 67, "run_sqsh": 98, "run_task": 11, "runner": [0, 116, 138], "runningleon": 152, "runpod": 120, "runtim": [0, 13, 14, 16, 17, 19, 20, 21, 22, 23, 27, 34, 38, 39, 52, 60, 64, 75, 76, 77, 78, 81, 83, 91, 97, 103, 105, 113, 114, 123, 124, 125, 128, 130, 133, 134, 135, 143, 146, 147, 148, 150, 152, 153, 155, 156, 157, 158, 160, 162, 164, 170], "runtime_config": 44, "runtime_default": 135, "runtime_error": 1, "runtime_rank": 138, "runtimedefault": [0, 135], "runtimedefaultsin": 135, "runtimeerror": [143, 147], "runtimetensor": 138, "rw": [9, 21, 29, 30, 31, 32], "s0": [75, 91, 105], "s1": [75, 91, 105], "s2": [75, 91, 105], "sacrif": [12, 13], "sad": 138, "saeyoonoh": 152, "safe": [1, 15, 20, 107, 131], "safer": 133, "safetensor": [83, 116, 118, 143, 152, 164], "sage_attn": 133, "sage_attn_k_block_s": 133, "sage_attn_k_quant_s": 133, "sage_attn_q_block_s": 133, "sage_attn_q_quant_s": 133, "sage_attn_v_block_s": 133, "sage_attn_v_quant_s": 133, "sageattent": 133, "sai": [16, 33, 38, 91, 123, 125, 130], "said": 127, "sake": [20, 91, 130], "sale": [39, 60, 124], "salloc": 98, "salt": [0, 147], "same": [0, 1, 4, 8, 10, 11, 12, 14, 15, 16, 17, 20, 21, 22, 23, 26, 29, 30, 31, 32, 33, 35, 39, 40, 51, 61, 62, 63, 66, 75, 84, 86, 87, 92, 94, 96, 97, 98, 102, 103, 105, 106, 107, 108, 109, 110, 111, 113, 114, 117, 119, 124, 127, 131, 132, 133, 134, 136, 138, 139, 146, 147, 152], "sampl": [0, 1, 2, 10, 13, 14, 19, 22, 24, 29, 30, 31, 32, 37, 38, 39, 40, 49, 52, 54, 55, 56, 57, 60, 64, 67, 75, 90, 97, 101, 103, 105, 117, 122, 123, 124, 133, 134, 138, 144, 145, 146, 147, 150, 151, 152, 166], "sample_num": 11, "sample_proj_bia": 134, "sample_st": [37, 90], "sample_weight_strip": 152, "samplemod": 133, "sampler": [22, 30, 37, 85, 96, 147, 149, 152], "sampler_opt": 22, "sampler_typ": [9, 147], "samplertyp": 147, "sampling_config": 138, "sampling_param": [11, 44, 49, 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 65, 66, 67, 88, 95, 101, 125, 132, 144, 146, 147, 151, 152, 165, 168], "samplingconfig": [0, 44, 103, 106, 138, 152], "samplingparam": [44, 49, 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 65, 66, 67, 68, 88, 95, 97, 101, 125, 132, 144, 146, 147, 151, 152, 165, 168], "sandbox": 98, "saniti": [101, 126, 127, 131], "santa": 58, "santacod": [141, 142], "satfinit": 141, "satisfi": [11, 12, 16, 17, 106, 118, 152], "satur": [16, 20, 21, 91], "save": [2, 8, 10, 12, 14, 15, 16, 21, 23, 26, 29, 30, 31, 32, 33, 38, 58, 75, 86, 96, 105, 109, 113, 119, 120, 123, 124, 127, 131, 132, 139, 147, 152], "save_checkpoint": [119, 135], "save_config": [119, 135], "save_hidden_st": 135, "save_kv_lay": 58, "savehiddenstatesdecodingconfig": [97, 147], "savest": 147, "saw": 127, "sbatch": [61, 62, 63, 117], "sbsa": [101, 145, 152], "scaffold": [97, 148, 152, 153], "scaffolding_llm": 152, "scaffoldingllm": 152, "scalabl": [8, 16, 20, 21], "scalar": [106, 111, 133], "scalartyp": 152, "scale": [0, 11, 15, 17, 21, 23, 51, 84, 92, 93, 96, 97, 106, 110, 118, 127, 133, 134, 136, 141, 147, 152], "scale_d0": 133, "scale_d1": 133, "scale_factor": 133, "scale_output": 133, "scale_qk": 134, "scale_typ": 133, "scalia": [49, 101, 144, 146, 151], "scaling_factor": 133, "scaling_long_factor": 133, "scaling_short_factor": 133, "scalingfactor": 12, "scalingvecpoint": 1, "scan": 19, "scanreducetempstorag": 1, "scanreducetempstoragebyt": 1, "scantempstorag": 1, "scantempstoragebyt": 1, "scarc": 35, "scatter": [16, 107, 133, 136], "scatter_nd": 133, "scenario": [2, 5, 7, 8, 10, 12, 13, 15, 16, 17, 20, 21, 22, 23, 40, 59, 75, 87, 89, 91, 92, 96, 105, 111, 113, 116, 121, 124, 125, 127, 130, 131, 136, 151, 152], "scene": 11, "scfg": 138, "schedul": [0, 2, 10, 11, 14, 15, 16, 19, 22, 23, 24, 27, 29, 30, 31, 32, 33, 34, 39, 81, 82, 84, 85, 95, 96, 97, 102, 103, 109, 110, 124, 125, 131, 139, 145, 147, 149, 152, 162, 163], "schedule_request": 170, "scheduled_batch": [37, 90], "scheduled_request": 170, "scheduledrequest": [10, 96], "scheduler_config": [132, 147], "scheduler_output": 58, "scheduler_polici": 22, "schedulerconfig": [0, 97, 132, 147, 152], "scheduleroutput": 58, "schedulerpolici": 152, "scheduling_param": 147, "schedulingparam": 147, "schema": [0, 10, 39, 53, 74, 95, 97, 103, 124, 136, 147, 152], "schema_gener": 147, "schema_json": 147, "scheme": [0, 86, 147, 152], "scicod": 13, "scienc": [49, 54, 55, 57, 58, 101, 144, 146, 151], "scope": [11, 14, 152], "score": [15, 86, 106], "scout": [19, 28, 82, 97, 142, 149, 152, 163], "scratch": [16, 36, 59, 67, 124, 125, 126, 131], "script": [2, 16, 17, 20, 22, 26, 29, 30, 31, 32, 33, 35, 36, 38, 39, 40, 59, 61, 62, 63, 78, 81, 84, 89, 92, 93, 98, 110, 112, 113, 115, 117, 119, 120, 123, 124, 125, 136, 141, 143, 146, 148, 152, 153, 158, 162, 167], "sd3": 134, "sd35adalayernormzerox": 134, "sd3patchemb": 134, "sd3transformer2dmodel": 135, "sd3transformer2dmodelconfig": 135, "sdxl": 152, "se": 150, "seamless": [82, 97, 145, 150, 152, 163], "seamlessli": [80, 97, 146, 150, 161], "search": [0, 1, 6, 11, 22, 23, 24, 27, 35, 37, 44, 86, 87, 97, 103, 106, 113, 127, 130, 133, 147, 152, 154], "seashor": [42, 70], "seat": [49, 101, 144, 146, 151], "sec": [2, 4, 12, 17, 39, 40, 97, 124, 125, 127, 130, 131], "second": [1, 2, 3, 5, 6, 8, 10, 11, 13, 16, 17, 18, 19, 21, 27, 58, 59, 83, 84, 86, 91, 103, 106, 109, 110, 113, 130, 133, 147, 164], "secondari": [0, 12, 86, 108, 139, 147], "secondary_kernel": 12, "secondary_offload_min_prior": [86, 147], "secondaryoffloadminprior": 0, "secondli": [91, 130], "section": [2, 12, 14, 15, 16, 20, 21, 26, 27, 29, 30, 31, 32, 33, 35, 36, 39, 75, 80, 83, 84, 86, 91, 96, 98, 103, 106, 117, 118, 119, 124, 125, 127, 128, 129, 130, 131, 133, 142, 151, 152, 155, 161, 164], "section_s": 133, "secur": [53, 152], "securityprotocol": 53, "see": [0, 1, 2, 3, 5, 6, 7, 11, 12, 14, 15, 16, 19, 20, 21, 22, 24, 26, 27, 29, 30, 31, 32, 33, 34, 35, 36, 39, 40, 42, 46, 49, 58, 70, 75, 77, 80, 81, 86, 89, 91, 92, 96, 98, 99, 100, 101, 105, 106, 108, 113, 117, 118, 120, 121, 124, 125, 127, 130, 131, 132, 133, 134, 135, 139, 141, 143, 152, 157, 161, 162, 169], "seed": [0, 9, 22, 24, 47, 48, 106, 137, 142, 147, 152], "seek": [29, 30, 31, 32], "seem": [33, 39, 64, 109, 124, 126], "seen": [2, 16, 19, 39, 113, 124], "segment": 152, "select": [0, 1, 7, 12, 13, 15, 17, 19, 20, 23, 35, 37, 39, 83, 84, 92, 97, 104, 106, 124, 131, 133, 136, 138, 139, 147, 154, 164, 170], "selectcontextid": 0, "selectgenidx": 0, "selective_scan": 133, "self": [0, 11, 34, 37, 58, 60, 75, 83, 90, 95, 96, 105, 107, 115, 117, 118, 124, 133, 135, 136, 138, 143, 147, 148, 153, 164, 169, 170], "self_attent": 118, "self_attention_mask": 134, "self_attention_packed_mask": 134, "self_attn": [118, 148, 153], "selfidx": 0, "sell": [39, 124], "semianalysi": 12, "semicolon": 98, "senat": [49, 101, 144, 146, 151], "send": [0, 9, 11, 12, 13, 16, 17, 20, 21, 27, 29, 30, 31, 32, 33, 84, 89, 102, 117, 125, 126, 133, 151, 152], "sens": 127, "sensit": [8, 13, 16, 20, 127], "sent": [0, 15, 16, 17, 26, 29, 30, 31, 32, 59, 84, 113, 147], "sentenc": [0, 9, 29, 31, 60, 106, 147, 151, 160], "sep": 147, "separ": [8, 10, 12, 16, 17, 20, 23, 33, 35, 40, 75, 76, 78, 83, 84, 86, 92, 98, 111, 113, 124, 133, 136, 138, 147, 150, 151, 155, 156, 158, 164], "separate_match_rewrit": 107, "seq": [1, 39, 75, 77, 105, 124, 133, 152, 157], "seq_idx": 138, "seq_len": [40, 75, 133, 134, 155], "seq_length": 133, "seq_lens_cuda": [75, 155], "seqlen": [0, 12, 133], "seqslot": 1, "sequenc": [0, 1, 2, 3, 4, 5, 6, 8, 10, 11, 12, 13, 14, 15, 16, 17, 19, 21, 22, 27, 29, 30, 31, 32, 33, 37, 39, 40, 58, 60, 66, 67, 75, 76, 77, 84, 86, 91, 92, 95, 96, 97, 103, 105, 106, 107, 108, 109, 113, 117, 124, 125, 128, 129, 132, 133, 134, 138, 139, 147, 150, 152, 155, 156, 157, 160, 169], "sequence_length": [133, 134, 138, 143], "sequence_length_buff": 138, "sequence_limit_length": 138, "sequenceindex": [0, 103], "sequencelengthscba": 1, "sequencelimitlength": 1, "sequenti": [0, 8, 14, 59, 84, 102, 113, 139], "seri": [11, 12, 30, 81, 82, 150, 152, 162, 163], "serial": [20, 22, 23, 133, 135, 138, 147], "serializ": 147, "serialize_as_ani": 147, "serialize_engin": 138, "serializeds": 0, "serializedst": 0, "serv": [0, 6, 7, 8, 9, 10, 12, 16, 18, 19, 20, 24, 29, 30, 31, 32, 33, 41, 42, 43, 45, 47, 48, 52, 69, 70, 71, 72, 73, 75, 85, 90, 91, 92, 94, 97, 101, 102, 103, 105, 108, 117, 131, 147, 149, 150, 152, 154, 155, 162, 166], "server": [0, 4, 16, 20, 26, 40, 41, 42, 43, 45, 47, 48, 69, 70, 71, 72, 73, 97, 102, 109, 113, 117, 120, 150, 151, 152, 160], "server_rol": 27, "server_start_timeout": 27, "servic": [11, 17, 20, 21, 36, 60, 84, 152], "session": [98, 105, 124, 138, 147], "set": [0, 1, 2, 9, 11, 12, 13, 15, 16, 17, 18, 19, 20, 21, 22, 23, 27, 29, 30, 31, 32, 33, 35, 36, 38, 40, 44, 50, 51, 53, 59, 61, 62, 63, 75, 78, 83, 84, 86, 87, 90, 91, 93, 95, 96, 98, 101, 102, 103, 104, 105, 106, 107, 108, 111, 112, 113, 114, 116, 118, 119, 123, 125, 127, 130, 131, 132, 133, 134, 135, 136, 138, 139, 143, 146, 147, 151, 152, 158, 160, 164, 166, 170], "set_api_statu": 34, "set_attn_processor": 135, "set_context_fmha": 136, "set_default_max_input_len": 147, "set_dora_plugin": 136, "set_fp8_rowwise_quant_plugin": 136, "set_from_opt": 1, "set_if_not_exist": 135, "set_input_shap": 138, "set_lora_plugin": 136, "set_nccl_plugin": 136, "set_qserve_plugin": 136, "set_rank": 135, "set_rel_attn_t": 134, "set_runtime_knobs_from_build_config": 147, "set_shap": 138, "set_smooth_quant_plugin": 136, "setadditionalmodeloutput": [0, 103], "setallottedtimem": 0, "setattentiondpeventsgatherperiodm": 0, "setattr": 11, "setbackend": 0, "setbackendtyp": 0, "setbadword": 0, "setbatchingtyp": 0, "setbeamsearchdiversityr": 0, "setbeamwidth": [0, 1], "setbeamwidtharrai": 0, "setbitto": 0, "setcachesaltid": 0, "setcachest": 0, "setcachetransceiverconfig": [0, 84], "setclientid": 0, "setcommst": 0, "setcommunicationmod": 0, "setcommunicationtyp": 0, "setcontextfmha": 1, "setcontextphaseparam": 0, "setcopyonpartialreus": 0, "setcrossattentionmask": 0, "setcrosskvcachefract": 0, "setcudagraphcaches": 0, "setcudagraphmod": 0, "setdatatyp": 1, "setdebugconfig": 0, "setdebuginputtensor": 0, "setdebugoutputtensor": 0, "setdebugtensornam": 0, "setdebugtensorsmaxiter": 0, "setdecodingconfig": 0, "setdecodingmod": 0, "setdeviceid": 0, "seteagleconfig": 0, "setearlystop": 0, "setembeddingbia": 0, "setenableblockreus": 0, "setenablechunkedcontext": 0, "setenablecontextfmhafp32acc": 0, "setenablepartialreus": 0, "setenabletrtoverlap": 0, "setencodedvocab": 0, "setencoderhiddens": 1, "setencoderinputfeatur": 0, "setencoderinputtokenid": 0, "setencoderoutputlength": 0, "setendid": 0, "seteventbuffermaxs": 0, "setexecutionconfig": 1, "setextendedruntimeperfknobconfig": 0, "setexternaldrafttokensconfig": 0, "setfailfastonattentionwindowtoolarg": 0, "setfreegpumemoryfract": 0, "setfrequencypenalti": 0, "setfrom": 0, "setfrominput": 1, "setgathergenerationlogit": 0, "setgemmallreducedtyp": 1, "setgenerationstep": 1, "setgpuweightsperc": [0, 114], "setguideddecodingconfig": 0, "setguideddecodingparam": 0, "sethostcaches": 0, "setinittozero": 1, "setisorchestr": 0, "setiterstatsmaxiter": 0, "setkvcacheconfig": 0, "setkvcacheretentionconfig": 0, "setkvcachetyp": 1, "setlanguageadapteruid": 0, "setlayertyp": 1, "setlengthpenalti": 0, "setlevel": 1, "setlogitsdtyp": 1, "setlogitspostprocessor": 0, "setlogitspostprocessorconfig": 0, "setlogitspostprocessornam": 0, "setlookaheadconfig": 0, "setlookaheaddecodingconfig": 0, "setloraconfig": 0, "setloramodul": 1, "setmanagedweightsmap": 1, "setmanageweightstyp": 1, "setmaxattentionwindowvec": 0, "setmaxbatchs": [0, 1], "setmaxbeamwidth": [0, 1], "setmaxdraftpathlen": 1, "setmaxdrafttoken": 1, "setmaxencoderlen": 1, "setmaxgputotalbyt": 0, "setmaxinputlen": 1, "setmaxlorarank": 1, "setmaxnumpath": 1, "setmaxnumtoken": [0, 1], "setmaxpagesperblock": 1, "setmaxpositionembed": 1, "setmaxpromptembeddingtables": 1, "setmaxqueues": 0, "setmaxseqidlemicrosecond": 0, "setmaxsequencelen": 1, "setmaxtoken": 0, "setmaxtokensinbuff": 0, "setmedusachoic": 0, "setmem": 1, "setmemorytyp": 1, "setminp": 0, "setmintoken": 0, "setmlphiddens": 1, "setmodelnam": 1, "setmodelvari": 1, "setmropeconfig": 0, "setmultiblockmod": 0, "setmultimodalembed": 0, "setmultimodalinput": 0, "setnbcrosskvhead": 1, "setnbkvhead": 1, "setnorepeatngrams": 0, "setnormalizelogprob": 0, "setnumcopystream": 1, "setnumdecodingenginetoken": 1, "setnumkvheadspercrosslay": 1, "setnumkvheadsperlay": 1, "setnumlanguag": 1, "setnumnod": 0, "setnumreturnsequ": 0, "setonboardblock": 0, "setorchestratorconfig": 0, "setorchleadercomm": 0, "setoutputconfig": 0, "setpadid": 0, "setpagedcontextfmha": 1, "setpagewidth": 1, "setparallelconfig": 0, "setparticipantid": 0, "setpath": 1, "setpeftcacheconfig": 0, "setpositionid": 0, "setppreducescatt": 1, "setpresencepenalti": 0, "setprior": 0, "setprocessorbatch": 0, "setprocessormap": 0, "setprompttableoffload": 0, "setprompttuningconfig": 0, "setquantmod": 1, "setrecvpollperiodm": 0, "setrepetitionpenalti": 0, "setrepl": [0, 103], "setrequeststatsmaxiter": 0, "setrequesttyp": 0, "setreturnallgeneratedtoken": 0, "setrnnconfig": 1, "setrotaryembeddingdim": 1, "setsamplingconfig": 0, "setschedulerconfig": 0, "setse": 0, "setsecondaryoffloadminprior": 0, "setsinktokenlength": 0, "setsizeperhead": 1, "setskipcrossattnblock": [0, 1], "setslotsperpag": 1, "setspawnprocess": 0, "setspecdecconfig": 0, "setspeculativedecodingmod": 1, "setspeculativedecodingmodul": 1, "setstoptokenid": 0, "setstopword": 0, "setstream": 0, "settemperatur": 0, "setter": [0, 106], "settokenizerstr": 0, "settokensperblock": 1, "settopk": 0, "settopp": 0, "settoppdecai": 0, "settoppmin": 0, "settoppresetid": 0, "settotalnumpag": 1, "setup": [1, 9, 17, 18, 23, 26, 29, 30, 31, 32, 33, 53, 61, 62, 63, 81, 101, 105, 126, 127, 138, 139, 146, 150, 151, 152, 162], "setup_embedding_parallel_mod": 147, "setup_fake_prompt": 138, "setup_fake_prompts_qwen2vl": 138, "setup_fake_prompts_vila": 138, "setup_input": 138, "setupbuff": 1, "setupcacheindirect": 1, "setupcacheindirectionbuff": 1, "setupspeculativedecod": 1, "setupspeculativedecodingbuff": 1, "setuptool": [81, 101, 162], "setusecrossattent": 1, "setusegpudirectstorag": 0, "setusemrop": 1, "setusepositionembed": 1, "setuseshapeinfer": 1, "setusetokentypeembed": 1, "setuseuvm": 0, "setvirtualmemoryalloc": 1, "setworkerexecutablepath": 0, "setzero": [0, 1], "seven": 19, "sever": [0, 1, 8, 10, 11, 16, 17, 18, 19, 20, 21, 37, 40, 44, 59, 75, 84, 91, 105, 107, 113, 116, 127, 128, 129, 130, 131, 133, 139, 143, 150, 155], "sft": [64, 82, 163], "sglang": [10, 11, 16, 90, 166], "sh": [18, 26, 29, 30, 31, 32, 33, 36, 93, 117, 120, 152, 167], "shah": 152, "shall": [1, 119, 139], "shape": [0, 1, 10, 12, 13, 15, 75, 91, 105, 107, 110, 116, 117, 131, 133, 135, 138, 139, 141, 143, 147, 152, 155, 169], "shape_cast_dtyp": 133, "shapeequ": 1, "shard": [13, 22, 35, 77, 81, 82, 92, 118, 124, 128, 133, 134, 157, 162, 163], "shard_map": 118, "sharding_along_vocab": 147, "sharding_dim": [133, 134], "share": [1, 2, 7, 8, 10, 13, 14, 15, 16, 17, 18, 19, 21, 23, 24, 75, 83, 84, 86, 87, 96, 98, 102, 103, 105, 107, 108, 109, 110, 113, 119, 126, 127, 133, 134, 136, 152, 164], "share_embed": 152, "share_weight": 134, "shared_embedding_t": 152, "shared_expert_output": 133, "shared_fc1": 15, "shared_fc2": 15, "shared_ptr": [0, 1], "sharedconstptr": 1, "sharedptr": 1, "shelf": [10, 81, 152, 162], "shell": [35, 36, 99], "sherlock113": 152, "shift": [14, 16, 111], "ship": [21, 119], "shm": [16, 143], "short": [8, 11, 16, 21, 66, 75, 91, 105, 124, 127, 130], "short_factor": 133, "short_mscal": [133, 134], "shorter": [40, 60, 75, 91, 105], "shortli": 27, "shot": [29, 31, 32, 152], "should": [0, 1, 2, 10, 11, 12, 15, 16, 20, 22, 24, 29, 30, 31, 32, 33, 35, 39, 40, 44, 49, 51, 53, 61, 62, 63, 64, 65, 75, 80, 83, 86, 91, 96, 98, 100, 101, 103, 107, 109, 110, 111, 119, 124, 125, 126, 131, 132, 133, 134, 138, 139, 144, 146, 147, 148, 151, 152, 153, 155, 164, 169, 170], "should_early_stop": 11, "should_skip_modul": [83, 164], "should_stop": 138, "shouldus": [75, 105], "show": [4, 8, 10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 21, 27, 29, 30, 31, 32, 33, 35, 40, 49, 59, 84, 86, 87, 91, 103, 117, 125, 130, 131, 136, 139, 142, 145, 151, 160], "showcas": [52, 59, 127, 131], "shown": [5, 11, 12, 14, 16, 17, 26, 29, 30, 31, 32, 39, 40, 83, 84, 91, 98, 111, 124, 125, 127, 130, 131, 133, 164], "shrunk": 133, "shuffl": 133, "shutdown": [0, 59, 146, 147], "shutdown_abort": 147, "si": [75, 105], "sibl": 117, "side": [12, 16, 20, 37, 78, 103, 133, 147, 158], "side_stream_id": 133, "sidestreamidtyp": 133, "sigh": 64, "sigmoid": [117, 133], "signal": [0, 11, 12, 20], "signatur": [10, 34, 107, 133], "signifi": [91, 130], "signific": [5, 8, 10, 11, 12, 14, 15, 16, 20, 37, 64, 75, 84, 102, 103, 105, 108, 126, 127, 130, 131], "significantli": [7, 8, 12, 13, 14, 15, 16, 17, 20, 26, 29, 30, 31, 32, 33, 37, 59, 75, 78, 87, 90, 91, 92, 125, 126, 127, 130, 131, 139, 155, 158, 166], "silicon": [15, 21], "silu": [117, 133, 134], "similar": [0, 2, 3, 5, 8, 10, 11, 12, 14, 16, 19, 20, 21, 26, 38, 39, 44, 75, 96, 105, 106, 107, 113, 123, 124, 132, 133, 151, 154, 170], "similarli": [10, 83, 84, 94, 113, 164], "simpl": [9, 11, 12, 16, 18, 19, 20, 21, 33, 40, 49, 58, 76, 78, 82, 94, 96, 98, 100, 107, 108, 113, 117, 144, 145, 146, 151, 156, 158, 160, 163], "simple_shard_onli": 161, "simpler": [16, 113], "simpleschedul": 170, "simplest": [96, 99, 133], "simpli": [11, 12, 39, 40, 78, 87, 91, 96, 105, 113, 124, 130, 136, 143, 146, 148, 151, 153, 158], "simplic": [19, 119], "simplifi": [8, 10, 24, 37, 39, 58, 81, 82, 91, 105, 119, 124, 130, 133, 146, 152, 162, 163], "simul": 59, "simultan": [8, 88, 95, 113, 130, 165], "sin": [0, 133, 134], "sinc": [0, 1, 2, 8, 10, 12, 14, 15, 16, 19, 20, 22, 26, 27, 29, 30, 31, 32, 33, 39, 44, 59, 60, 75, 78, 87, 91, 96, 98, 104, 105, 107, 109, 113, 114, 119, 120, 124, 125, 126, 127, 130, 131, 133, 135, 136, 139, 147, 154, 158, 169, 170], "sincer": [10, 11, 15], "sinco": 134, "singl": [0, 1, 2, 5, 6, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 26, 27, 29, 30, 31, 32, 33, 37, 38, 39, 42, 51, 59, 70, 75, 78, 81, 84, 86, 91, 92, 94, 96, 102, 103, 104, 105, 106, 108, 113, 115, 117, 119, 123, 124, 127, 131, 133, 135, 136, 138, 139, 141, 147, 148, 150, 151, 152, 153, 154, 155, 158, 160, 162, 169], "singleton": [107, 133], "sink": [0, 1, 75, 105, 138, 147], "sink_token_len": 138, "sink_token_length": [75, 86, 105, 138, 147], "sinktokenlength": [0, 1], "sinusoid": 134, "sit": [30, 64, 119], "situaiton": 40, "situat": [20, 33, 64, 91, 113, 125, 130], "six": 14, "size": [0, 1, 2, 4, 5, 7, 8, 10, 12, 13, 14, 15, 16, 19, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 37, 38, 39, 40, 44, 59, 61, 62, 63, 67, 75, 76, 77, 78, 80, 84, 87, 88, 91, 92, 96, 97, 102, 105, 106, 108, 109, 110, 111, 113, 114, 123, 124, 125, 126, 127, 128, 131, 133, 134, 135, 136, 138, 143, 147, 152, 155, 156, 157, 158, 161, 165, 170], "size_t": [0, 1], "size_typ": [0, 1], "sizeof": 1, "sizeperhead": [0, 1], "sizetype32": [0, 1], "sizetype64": [0, 1], "skip": [0, 1, 2, 10, 22, 35, 58, 59, 76, 77, 83, 84, 98, 107, 118, 121, 133, 147, 152, 156, 157, 164, 170], "skip_attn": [133, 134], "skip_cross_attn_block": [135, 138], "skip_cross_kv": [134, 138], "skip_encod": 138, "skip_loading_weight": [76, 80, 156, 160, 161], "skip_special_token": [147, 152], "skip_tokenizer_init": [44, 147], "skipcrossattnblock": [0, 1], "sku": [125, 127, 130, 131], "skylin": 33, "skyscrap": 33, "skywork": [141, 142, 152], "sleep": 121, "slice": [1, 92, 104, 118, 133, 147, 152], "slice_shap": 118, "sliceinputtyp": 133, "slicen": 1, "slide": [0, 19, 85, 97, 108, 132, 133, 138, 149, 152], "slider": [2, 13, 39, 124], "sliding_window": 135, "sliding_window_caus": 133, "slight": [2, 14, 15, 127, 130, 131], "slighter": 10, "slightli": [0, 10, 21, 27, 93, 110, 111, 127, 131], "slope": [75, 105, 133], "slot": [0, 1, 10, 16, 92, 152], "slot_map": [133, 135], "slotid": 16, "slotidx": 1, "slotsperpag": 1, "slow": [20, 82, 103, 109, 126, 147], "slowdown": 20, "slower": [15, 37, 59, 108, 119, 126], "slowest": [8, 75, 105], "slurm": [16, 20, 26, 97, 101, 117, 143, 152], "slurm_script": [20, 92], "sm": [21, 142, 152], "sm100": [93, 142, 149], "sm120": [93, 142, 152], "sm121": 152, "sm80": [142, 152], "sm86": [142, 152], "sm89": [142, 152], "sm90": [93, 142, 149, 152], "small": [7, 10, 12, 13, 14, 15, 16, 19, 20, 21, 59, 60, 75, 88, 92, 105, 109, 111, 113, 117, 125, 127, 130, 131, 133, 139, 140, 142, 143, 152, 165], "smaller": [1, 2, 11, 14, 19, 20, 21, 23, 38, 39, 80, 87, 91, 95, 113, 123, 124, 126, 130, 131, 132, 133, 139, 152, 161], "smallest": [0, 1, 108, 133], "smart": [17, 84, 133, 152], "smaug": [142, 152], "smi": [2, 13, 18, 21, 29, 30, 31, 32, 33, 39, 84, 102, 124, 139], "smile": 64, "smith": [49, 54, 55, 56, 57, 101, 144, 146, 151], "smooth": [119, 136, 147, 152], "smooth_quant_gemm_plugin": 136, "smooth_quant_plugin": 136, "smoother": 2, "smoothli": 11, "smoothquant": [7, 107, 152], "smoothquant_v": 147, "snap": 147, "snapshot": [10, 124], "snapshot_download": 64, "snip": [39, 124], "snippet": [10, 21, 124, 152, 170], "snshrivas10": 64, "so": [0, 1, 2, 10, 11, 12, 13, 14, 15, 16, 20, 21, 27, 29, 30, 31, 32, 33, 35, 39, 40, 44, 50, 58, 75, 86, 91, 94, 95, 96, 98, 103, 105, 107, 110, 113, 119, 120, 124, 126, 127, 130, 131, 132, 133, 134, 135, 138, 139, 142, 147, 148, 152, 153, 160, 169], "soc": 58, "socketst": 0, "softmax": [14, 15, 75, 105, 117, 133, 155], "softplu": 133, "softwar": [15, 16, 19, 21, 29, 30, 31, 32, 100, 101, 103, 105, 117, 152], "sol": 17, "sole": 24, "solid": [12, 128, 129], "solut": [10, 11, 16, 20, 21, 86, 92, 143, 146, 154], "some": [0, 2, 8, 10, 11, 12, 13, 14, 15, 16, 17, 20, 23, 24, 26, 27, 30, 35, 36, 40, 60, 64, 75, 78, 83, 84, 86, 89, 91, 98, 101, 102, 103, 104, 105, 106, 107, 109, 113, 114, 116, 117, 119, 121, 127, 128, 129, 131, 132, 133, 136, 139, 143, 146, 147, 148, 152, 153, 154, 158, 170], "some_uri": 36, "someon": 33, "someone\u56fd\u5916": 33, "someth": [11, 44, 117], "sometim": [16, 17, 35, 39, 84, 124], "song": [39, 124], "soon": [0, 3, 4, 5, 6, 7, 20, 44, 86, 94], "sophist": [8, 20, 78, 158], "sora": [42, 70], "sort": [0, 1, 8, 103, 106, 133], "sota": 152, "sourc": [2, 3, 6, 10, 11, 13, 15, 16, 22, 23, 27, 29, 30, 31, 32, 36, 41, 42, 43, 45, 47, 48, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 78, 81, 83, 99, 100, 101, 112, 115, 116, 118, 119, 133, 134, 135, 136, 137, 138, 147, 150, 152, 158, 162, 164], "source_dir": 36, "source_param1": [83, 164], "source_param2": [83, 164], "source_root": [61, 62, 63], "sourcetaskvalu": 1, "south": [27, 30], "southeast": 30, "southern": 30, "southwest": 30, "soyer": [115, 117, 143], "sp": [78, 158], "sp_kwarg": [78, 158], "space": [16, 17, 20, 59, 81, 84, 86, 89, 91, 96, 98, 110, 130, 139, 147, 162, 169], "spaces_between_special_token": [147, 152], "span": [13, 14, 16, 17, 87, 119], "spars": [15, 20, 52, 92, 113, 133, 147, 152], "sparse_attention_config": [67, 147], "sparse_fc1": 15, "sparse_fc2": 15, "sparseattentionconfig": 147, "sparsiti": [16, 20, 21, 23, 147], "spatial_norm_dim": 134, "spawn": [57, 94, 125, 143, 146], "spawnprocess": 0, "speakleash": 142, "spec": [16, 23, 152], "spec_config": [19, 68], "spec_dec_mod": 147, "spec_decode_algo": [14, 19], "spec_decode_nextn": 14, "spec_decoding_generation_length": [133, 134, 135], "spec_decoding_is_generation_length_vari": [133, 134, 135], "spec_decoding_max_generation_length": [133, 134], "spec_decoding_packed_mask": [133, 134, 135], "spec_decoding_param": [134, 135], "spec_decoding_position_offset": [133, 134, 135], "spec_decoding_us": [133, 134], "specconfig": [96, 152], "specdec": 0, "specdecconfig": 0, "specdecfastlogitsinfo": 0, "specdecodinggenerationlength": 1, "specdecodinggenerationlengthshost": 1, "specdecodingpackedmask": 1, "specdecodingparam": 134, "specdecodingpositionoffset": 1, "specdecodingstat": 0, "special": [3, 10, 11, 12, 14, 23, 75, 83, 84, 102, 105, 110, 117, 118, 136, 147, 152, 164], "specif": [0, 1, 4, 7, 8, 10, 11, 13, 15, 16, 17, 18, 20, 21, 22, 26, 27, 29, 30, 31, 32, 33, 34, 35, 39, 58, 60, 66, 78, 83, 84, 86, 88, 92, 95, 98, 101, 104, 106, 107, 108, 110, 111, 112, 113, 116, 119, 124, 126, 127, 131, 133, 146, 147, 148, 150, 151, 152, 153, 154, 158, 164, 165], "specifi": [0, 1, 2, 10, 11, 16, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 35, 37, 38, 39, 40, 44, 53, 60, 64, 73, 75, 76, 77, 78, 79, 81, 82, 84, 86, 87, 96, 98, 103, 105, 106, 107, 108, 110, 113, 118, 119, 123, 124, 126, 127, 130, 132, 133, 135, 136, 138, 139, 143, 146, 147, 151, 152, 155, 156, 157, 158, 159, 162, 163, 164], "specmetadata": 96, "spectrum": [8, 150], "specul": [0, 1, 13, 16, 17, 18, 37, 52, 81, 95, 97, 103, 122, 124, 125, 133, 135, 144, 146, 147, 150, 152, 162, 168], "speculative_config": [2, 9, 13, 14, 29, 68, 96, 147], "speculative_decod": 152, "speculative_decoding_draft_tokens_extern": 135, "speculative_decoding_mod": [23, 124, 147], "speculative_model": 96, "speculative_model_dir": [9, 18, 68, 147], "speculative_model_format": 147, "speculativeconfig": 147, "speculativedecod": 0, "speculativedecodingconfig": 0, "speculativedecodingfastlogitsinfo": 0, "speculativedecodingmetr": 0, "speculativedecodingmod": [96, 135, 147, 152], "speculativedecodingmodul": 152, "speculativedecodingoutput": 1, "speed": [4, 12, 13, 14, 15, 16, 18, 23, 39, 40, 96, 117, 124, 131, 136, 152], "speedup": [2, 4, 6, 7, 8, 10, 11, 13, 15, 17, 18, 21], "spent": 0, "spirit": 16, "split": [1, 12, 19, 33, 39, 75, 87, 91, 92, 104, 105, 110, 117, 124, 126, 127, 133, 139, 147, 150, 152], "split_input_id": 138, "split_prompt_by_imag": 138, "split_siz": 133, "split_size_or_sect": 133, "splitlin": 147, "splittransposecpu": 1, "splittransposecpuinn": 1, "splitwis": 102, "spot": [16, 20, 130], "sq": [7, 141, 152], "sqrt": [75, 105, 133], "sqsh": 98, "sqsh_path": 98, "squar": [20, 26, 33, 91, 130, 133], "squared_relu": 133, "squash": 98, "squeez": [1, 133, 138], "src": [1, 117, 133], "src_seq_len": 133, "srcdesc": 0, "srctype": 1, "srun": [27, 61, 62, 63, 101, 117, 143], "ssd": [9, 18, 21], "ssh": 36, "sshd": 120, "ssid": 53, "ssm": [81, 133, 147, 162], "ssm_state": 135, "stabil": [8, 13, 16, 81, 94, 97, 112, 147, 162], "stabl": [16, 23, 34, 75, 91, 105, 118, 125, 130, 131, 133, 136, 147, 152], "stack": [9, 11, 13, 18, 21, 26, 29, 30, 31, 32, 82, 98, 118, 133, 151, 163], "stacklevel": 34, "stackoverflow": 36, "stage": [0, 11, 14, 17, 18, 22, 34, 40, 75, 78, 81, 84, 89, 94, 96, 97, 105, 107, 139, 152, 155, 158, 162], "stage_list": 35, "stai": [4, 7, 16, 126, 131], "stall": 16, "stand": 117, "standalon": 119, "standard": [0, 3, 10, 11, 12, 16, 18, 21, 22, 40, 76, 78, 81, 82, 89, 113, 117, 133, 156, 158, 162, 163], "starcod": [142, 152], "starcoder1": 141, "starcoder2": [82, 141, 152, 163], "starrickliu": 152, "start": [0, 2, 11, 12, 14, 16, 20, 21, 23, 28, 34, 35, 36, 40, 41, 42, 43, 45, 47, 48, 50, 51, 63, 64, 69, 70, 71, 72, 73, 84, 91, 92, 98, 103, 105, 107, 109, 120, 121, 124, 125, 126, 130, 132, 133, 135, 137, 138, 139, 147, 150, 152], "start_dim": 133, "start_load_kv": 58, "started_loading_req_id": 58, "startswith": 147, "startup": [26, 29, 30, 31, 32, 33, 84, 102, 143], "stat": [0, 147, 152], "state": [0, 1, 2, 11, 12, 13, 14, 16, 17, 20, 23, 29, 30, 31, 32, 33, 39, 40, 49, 50, 51, 59, 75, 81, 83, 86, 91, 92, 96, 97, 101, 103, 104, 105, 107, 108, 109, 113, 124, 125, 130, 132, 133, 136, 144, 146, 147, 151, 152, 162, 164, 170], "state_dict": [83, 164], "state_dtyp": 138, "state_or_ptr": 133, "state_s": 138, "statement": [11, 146], "staten": [30, 33], "stateptr": 0, "states": 1, "static": [0, 1, 15, 20, 23, 77, 86, 103, 112, 113, 133, 134, 135, 136, 138, 147, 152, 157], "static_batch": [132, 147], "static_cast": [1, 141], "staticbatchingstat": 0, "statist": [0, 18, 22, 27, 39, 59, 92, 103, 113, 124, 147, 152], "statu": [1, 9, 12, 16, 21, 29, 30, 31, 32, 33, 34, 97, 143, 147, 152], "std": [0, 1, 16, 103], "stddev": [47, 48], "stderr": [29, 31, 32], "stdev": [2, 22, 38, 39, 40, 62, 123, 124, 125], "stdin": 22, "stdit": 152, "stdout": [2, 22, 38, 39, 40, 62, 123, 124, 125], "steadi": 40, "steady_clock": 0, "stem": [12, 20], "step": [0, 1, 3, 10, 11, 12, 13, 14, 21, 26, 34, 37, 39, 40, 59, 66, 75, 76, 90, 91, 93, 94, 96, 97, 101, 105, 106, 107, 109, 113, 116, 117, 119, 121, 124, 125, 133, 138, 143, 147, 152, 154, 155, 156, 166, 169, 170], "stick": 33, "still": [2, 8, 10, 11, 12, 13, 14, 15, 16, 17, 20, 21, 34, 39, 87, 105, 118, 119, 124, 125, 127, 133, 138, 139, 152], "stine": 29, "stoica": 11, "stop": [0, 1, 11, 16, 21, 30, 37, 58, 60, 90, 91, 95, 103, 106, 107, 113, 124, 130, 138, 147, 150, 151, 152, 166, 168], "stop_reason": [18, 21, 29, 30, 31, 32, 33, 147, 151, 152], "stop_token_id": [103, 147], "stop_words_data": 138, "stop_words_list": 138, "stopping_criteria": 138, "stoppingcriteria": [138, 152], "stoppingcriterialist": 138, "stoptokenid": [0, 103], "stopword": [0, 106], "stopwordslen": 1, "stopwordslist": 1, "stopwordsptr": 1, "storag": [0, 1, 8, 9, 18, 21, 83, 108, 110, 146, 147, 164], "store": [0, 1, 4, 10, 12, 13, 14, 16, 18, 22, 27, 34, 35, 37, 39, 51, 59, 75, 83, 86, 87, 91, 92, 96, 105, 108, 109, 110, 117, 124, 132, 133, 135, 139, 141, 147, 148, 152, 153, 155, 164, 169], "store_tru": 59, "stori": [64, 66], "str": [11, 34, 55, 56, 58, 60, 64, 65, 66, 67, 68, 72, 83, 84, 116, 119, 133, 134, 135, 136, 138, 147, 164], "straight": 98, "straightforward": [10, 14, 20, 21, 33], "strateg": 8, "strategi": [0, 2, 7, 14, 16, 17, 20, 37, 39, 44, 84, 94, 95, 97, 111, 113, 124, 128, 133, 135, 139, 146, 147, 150, 152], "stream": [0, 1, 9, 10, 11, 12, 15, 16, 21, 22, 23, 34, 38, 39, 44, 47, 48, 52, 58, 60, 84, 102, 103, 117, 123, 133, 138, 139, 143, 147, 152], "stream_interv": [20, 21, 29, 30, 33, 147], "stream_ptr": [60, 95], "streaming_llm": 152, "streamingllm": [23, 136, 152], "streamlin": [37, 39, 81, 82, 124, 146, 151, 162, 163], "streamptr": [0, 1, 103], "street": [33, 64], "strenum": [137, 147], "stretch": 30, "strict": [13, 14, 16, 29, 31, 32, 78, 147, 158], "strict_bound": 133, "strict_dtyp": [133, 134], "strictbasemodel": 147, "stricter": 13, "strictli": [10, 39, 124, 147], "stride": [1, 133, 134], "strike": [16, 86, 113], "string": [0, 1, 20, 24, 35, 39, 53, 73, 86, 103, 116, 124, 133, 136, 138, 147], "string_valu": 109, "string_view": 1, "stringptrmap": 1, "stringvec": 0, "strip": [23, 147, 152], "strip_plan": 23, "strive": [81, 94, 150, 162], "strong": [16, 19], "strongli": 127, "strongly_typ": [147, 152], "struct": [0, 1, 108], "structur": [0, 10, 15, 19, 21, 24, 33, 39, 81, 86, 95, 96, 97, 104, 107, 108, 113, 124, 133, 139, 147, 152, 162], "structural_tag": [95, 147], "struggl": 64, "student": [33, 49, 54, 55, 57, 101, 144, 146, 151], "studi": [15, 91, 97, 125, 127, 128, 129, 131], "studio": 36, "style": [13, 75, 96, 105, 113, 152], "sub": [11, 16, 116, 119, 133, 147], "subclass": [1, 11, 60, 96, 119, 147, 148, 153], "subcommad": 124, "subcommand": [22, 40, 152], "subcompon": [83, 164], "subdirectori": [39, 124], "subgraph": [107, 133], "subject": [3, 5, 6, 7, 34, 75, 81, 84, 102, 133, 145, 147, 162], "submiss": 124, "submit": [11, 26, 29, 30, 31, 32, 34, 51, 110, 124, 147], "submit_sync": 147, "submittransferrequest": 0, "submodul": [2, 18, 96, 98, 148, 152, 153], "suboptim": [12, 21, 117], "subscript": 133, "subsequ": [8, 10, 11, 12, 14, 20, 35, 37, 58, 98, 109, 110, 113, 125, 152], "subset": [0, 14, 39, 92, 95, 96, 103, 106, 117, 119, 124, 133, 147, 168], "substanti": [8, 13, 15, 17, 84, 102, 109, 113], "substitut": [36, 147], "substr": [35, 147], "subsystem": 152, "subtract": 107, "succe": [1, 8, 139, 150, 152], "succeed": 138, "success": [1, 4, 10, 13, 20, 26, 29, 30, 31, 32, 35, 40, 103, 147], "successfulli": [1, 59, 113, 121, 127], "suddenli": 11, "sudo": [2, 13, 39, 81, 101, 124, 162], "suffer": [13, 16, 20], "suffici": [11, 12, 20, 91, 126, 127], "suffix": [11, 29, 30, 31, 32, 96, 147], "sugar": 11, "suggest": [7, 11, 16, 60, 64, 105, 127, 146], "suit": [8, 16, 17, 21, 24, 37, 39, 40, 86, 105, 124], "suitabl": [16, 17, 36, 58, 84, 89, 147], "sum": [1, 26, 29, 30, 31, 32, 107, 115, 133, 169], "sum_": 8, "sum_of_token": 133, "summar": [5, 7, 16, 24, 40, 87, 88, 105, 113, 114, 115, 116, 124, 132, 139, 165], "summari": [9, 11, 16, 33, 108, 113], "summat": 133, "sunjiabin17": 152, "sunset": 66, "super": [11, 33, 58, 82, 83, 107, 115, 118, 119, 142, 143, 148, 149, 153, 163, 164, 170], "superchip": 142, "superior": 8, "superjomn": 66, "supplementari": 134, "suppli": [35, 60, 96, 110, 160], "support": [0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 13, 15, 16, 21, 22, 23, 24, 26, 27, 31, 32, 33, 35, 36, 37, 40, 44, 53, 61, 62, 63, 64, 66, 67, 75, 78, 83, 86, 87, 91, 92, 94, 95, 96, 97, 100, 101, 102, 103, 104, 105, 106, 108, 109, 110, 111, 112, 113, 114, 116, 119, 120, 122, 125, 127, 130, 131, 132, 133, 134, 136, 143, 145, 146, 147, 148, 151, 152, 153, 154, 155, 158, 160, 164, 167, 168, 169, 170], "support_partial_config": [78, 158], "supports_backend": 147, "supportsinflightbatch": 1, "suppos": [96, 148, 153], "suprem": [49, 101, 144, 146, 151], "sure": [2, 10, 14, 16, 18, 21, 29, 30, 31, 32, 33, 39, 58, 78, 84, 91, 96, 98, 101, 102, 119, 121, 124, 132, 133, 146, 152, 158], "surfac": 34, "surpass": [75, 87, 105], "surprisingli": 10, "surround": [75, 105, 152], "survei": 10, "swa": 108, "swap": [16, 108], "swapcas": 147, "sweep": [4, 17, 30, 33, 117, 130], "sweet": 130, "swept": [5, 10], "swiftli": [16, 19], "swiglu": [23, 133, 136, 152], "switch": [4, 7, 13, 15, 17, 24, 26, 84, 88, 98, 104, 109, 111, 112, 132, 139, 152, 165], "swizzl": 12, "sxm": [4, 23, 40, 125, 127, 128, 129], "sy": [20, 58, 152], "symbol": 0, "sync": 138, "sync_quant_config_with_kv_cache_config_dtyp": 147, "synchron": [1, 8, 10, 16, 20, 37, 58, 103, 117, 143, 147, 152], "syncmessag": 0, "syntact": 11, "syntax": [97, 133, 151], "synthet": [2, 22, 26, 39, 40, 47, 48, 124], "synthetic_128_128": [39, 76, 124, 156], "synthetic_2048_2048": 125, "synthetic_2048_2048_1000": 125, "synthetic_lora_data": [39, 124], "system": [2, 4, 8, 10, 11, 12, 14, 15, 16, 18, 20, 21, 24, 26, 29, 30, 31, 32, 35, 40, 41, 42, 50, 51, 53, 58, 59, 61, 62, 63, 69, 70, 73, 78, 83, 90, 92, 97, 98, 108, 109, 117, 126, 142, 145, 150, 151, 152, 158, 160, 166], "system_prompt": 24, "systemat": [13, 16, 17], "t": [0, 1, 8, 9, 11, 13, 15, 16, 20, 27, 29, 30, 31, 32, 33, 37, 38, 39, 44, 58, 60, 61, 62, 63, 68, 75, 84, 87, 91, 92, 96, 101, 102, 105, 113, 117, 119, 120, 123, 124, 126, 130, 131, 133, 135, 138, 143, 147, 152, 160], "t4": 21, "t5": [93, 105, 106, 141, 142, 152], "t_": [14, 26, 29, 30, 31, 32], "t_2": 14, "t_5": 14, "tab": 147, "tabl": [0, 4, 7, 23, 40, 92, 96, 106, 109, 124, 133, 134, 138, 142, 143, 147, 149, 152], "tabsiz": 147, "tackl": 15, "tactic": [15, 23], "tag": [0, 1, 11, 29, 30, 31, 32, 35, 36, 95, 98, 101, 120, 147, 152], "tagentrymap": 1, "tail": [8, 12], "tailor": [7, 127, 131], "take": [0, 1, 10, 11, 12, 14, 16, 17, 18, 19, 21, 29, 30, 31, 32, 33, 37, 64, 75, 84, 86, 91, 96, 102, 105, 106, 107, 109, 111, 116, 119, 124, 125, 126, 130, 133, 134, 136, 147, 152, 169], "takeawai": 33, "taken": [3, 4, 16, 118, 133], "talk": [16, 33, 64], "tanh": [133, 134], "tar": 24, "target": [0, 1, 2, 8, 15, 16, 17, 19, 22, 23, 44, 60, 78, 83, 98, 118, 124, 131, 132, 136, 147, 152, 158, 164], "target_input_len": 22, "target_model": 96, "target_module_nam": [83, 164], "target_output_len": 22, "targetcach": 1, "targetpageid": 1, "targetprob": 1, "targettaskvalu": 1, "tarot": 64, "tarot_lora_dir": 64, "task": [0, 1, 8, 10, 11, 16, 19, 20, 21, 22, 24, 29, 31, 32, 37, 39, 55, 56, 60, 61, 62, 63, 87, 88, 90, 109, 110, 113, 115, 116, 124, 134, 138, 141, 146, 147, 152, 165, 166, 169], "task_collect": 11, "task_handl": 11, "task_id": [39, 110, 124], "task_vocab_s": 134, "taskid": [0, 1], "taskidtyp": 1, "tasklayermoduleconfig": 1, "tasklayermoduleconfigbind": 1, "tasklayermoduleconfiglistptr": 1, "taskshost": 1, "taskstatu": 11, "taskvalu": 1, "taskvalueptr": 1, "taslid": 1, "tayef": 152, "tb": [12, 147], "tconstptr": 1, "tcp": 121, "tdp": 40, "team": [8, 10, 11, 12, 13, 14, 15, 16, 17, 20, 29, 30, 31, 32, 35, 84, 116, 119, 121, 142, 152], "teamwork": 12, "teardown": 1, "tech": [14, 16, 17, 84, 152], "technic": [8, 14, 15, 16, 20, 33, 92, 108], "techniqu": [3, 8, 10, 12, 13, 14, 15, 16, 17, 52, 75, 86, 87, 88, 91, 93, 105, 107, 113, 117, 126, 127, 128, 129, 132, 141, 152, 165], "technologi": [13, 18, 49, 54, 55, 57, 58, 60, 101, 144, 146, 150, 151], "tediou": 35, "tee": 59, "tekit_2025": 124, "tell": [42, 58, 64, 66, 70, 96, 131, 151, 160], "temb": 134, "temp": [66, 138], "temperatur": [0, 1, 9, 11, 18, 21, 29, 31, 32, 39, 41, 42, 43, 44, 49, 54, 55, 56, 57, 60, 65, 66, 67, 73, 78, 84, 95, 101, 106, 124, 125, 132, 138, 144, 146, 147, 151, 152, 158, 168], "tempfil": 58, "templat": [0, 1, 24, 26, 35, 83, 117, 118, 147, 164], "tempor": [8, 138], "temporari": [83, 84, 102, 164], "temporarili": 12, "temporarydirectori": 58, "ten": [7, 10, 14, 16, 113], "tenant": 86, "tend": [11, 19, 132], "tensor": [1, 2, 3, 4, 5, 6, 10, 13, 14, 15, 16, 18, 19, 20, 21, 22, 24, 27, 29, 30, 31, 32, 33, 40, 57, 58, 60, 83, 84, 91, 93, 95, 102, 106, 111, 116, 117, 118, 124, 127, 128, 129, 131, 133, 134, 135, 136, 138, 141, 143, 147, 148, 150, 152, 153, 155, 164], "tensor_dict": 138, "tensor_input": 107, "tensor_parallel_s": [57, 61, 62, 63, 67, 92, 125, 126, 127, 131, 132, 147], "tensor_shap": 118, "tensorconstptr": 1, "tensorflow": 21, "tensorinfo": 138, "tensorloc": 133, "tensormap": 1, "tensorparallel": [0, 1, 106], "tensorptr": [0, 1], "tensorrt": [1, 3, 6, 8, 13, 15, 22, 23, 24, 26, 27, 28, 34, 35, 36, 37, 38, 40, 41, 42, 43, 44, 45, 47, 48, 49, 50, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 75, 76, 78, 82, 83, 84, 86, 87, 91, 94, 96, 99, 100, 101, 103, 105, 106, 107, 108, 110, 111, 114, 115, 122, 123, 127, 128, 129, 131, 132, 133, 136, 138, 141, 143, 145, 146, 147, 148, 151, 153, 154, 155, 156, 158, 163, 167, 169, 170], "tensorrt_llm": [0, 1, 2, 10, 18, 26, 27, 29, 30, 31, 32, 33, 34, 36, 37, 39, 40, 44, 49, 50, 51, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 75, 78, 80, 81, 83, 88, 91, 93, 95, 96, 98, 99, 101, 103, 105, 106, 107, 110, 112, 114, 115, 117, 118, 119, 120, 121, 124, 125, 127, 131, 132, 133, 134, 135, 136, 137, 138, 143, 144, 145, 146, 147, 148, 151, 152, 153, 154, 155, 158, 161, 162, 164, 165, 167, 168, 169], "tensorrt_llm_gpt": 117, "tensorrt_llm_rouge1_threshold": 116, "tensorrtllm_backend": [110, 152], "tensortrt": 98, "tep": [84, 102], "tep4": 17, "term": [11, 12, 16, 17, 20, 33, 35, 100, 101, 117, 132, 133, 146], "termin": [0, 9, 11, 29, 30, 31, 32, 33, 40, 98, 109, 121, 151, 152], "test": [1, 7, 11, 13, 14, 17, 19, 20, 22, 24, 39, 40, 42, 67, 70, 75, 80, 81, 82, 92, 94, 96, 97, 98, 101, 105, 124, 125, 127, 128, 129, 130, 131, 132, 142, 147, 152, 160, 161, 162, 163, 169], "test_beam_search_larg": 152, "test_cli_flow": 152, "test_e2": 152, "test_generate_with_se": 152, "test_gpt_ib_ptun": 35, "test_graph_rewrit": 107, "test_list": 35, "test_llm_api": 34, "test_llm_openai_triton_1gpu": 35, "test_llm_qwen2audio_single_gpu": 35, "test_openai": 35, "test_qwen2audio": 35, "test_star_attention_input": 67, "test_text": 58, "test_triton": 35, "test_trt_llm": [114, 115, 116], "testb": 8, "testgpt2": 152, "texec": [0, 84], "text": [0, 11, 17, 19, 22, 23, 26, 27, 29, 30, 31, 32, 34, 35, 37, 39, 40, 42, 44, 49, 50, 51, 52, 57, 58, 59, 65, 66, 67, 68, 70, 75, 89, 94, 95, 101, 103, 105, 106, 109, 124, 125, 132, 136, 138, 143, 144, 146, 147, 151, 152], "text0": 58, "text1": 58, "text_complet": [29, 31, 32], "text_diff": 147, "text_hidden_s": 135, "text_to_token": 60, "textattack": [142, 149], "textprompt": 147, "tg_group": 133, "tgt": [117, 133], "tgt_len": [133, 134], "tgt_seq_len": 133, "th": [1, 14, 116, 133], "than": [0, 1, 3, 4, 5, 7, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 23, 37, 39, 40, 59, 75, 84, 86, 87, 91, 92, 95, 98, 102, 103, 105, 106, 107, 109, 113, 117, 124, 125, 126, 127, 130, 132, 133, 138, 139, 143, 147, 152, 155], "thank": [8, 10, 14, 16, 152], "thankfulli": 10, "thecodewrangl": 152, "theft": 86, "thei": [0, 1, 10, 11, 12, 13, 14, 15, 16, 20, 21, 26, 30, 33, 35, 36, 39, 75, 83, 86, 89, 91, 96, 98, 103, 105, 106, 110, 117, 118, 119, 124, 125, 127, 130, 131, 132, 133, 135, 141, 147, 152], "them": [0, 2, 8, 11, 12, 13, 14, 15, 16, 17, 19, 20, 21, 29, 33, 37, 38, 58, 61, 62, 63, 66, 78, 83, 84, 91, 96, 103, 104, 107, 113, 114, 123, 124, 126, 128, 129, 130, 132, 133, 138, 139, 147, 148, 153, 158, 164], "themselv": 35, "theoret": [10, 12, 16, 97, 139], "theori": [21, 132], "therebi": [84, 102, 132], "therefor": [2, 11, 19, 20, 40, 106, 114, 119, 133, 143, 169], "thermal": [39, 124], "theta": 133, "thi": [0, 1, 2, 3, 4, 5, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 44, 46, 49, 50, 51, 53, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 73, 75, 78, 80, 81, 82, 83, 84, 86, 87, 88, 90, 91, 92, 93, 94, 95, 96, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 138, 139, 141, 143, 144, 145, 146, 147, 148, 151, 152, 153, 154, 155, 158, 160, 161, 162, 163, 164, 165, 166, 169, 170], "thin": 119, "thing": [17, 33, 49, 54, 55, 57, 91, 101, 106, 121, 130, 131, 144, 146, 151], "think": [11, 13, 14, 15, 33, 96, 128, 129, 149], "thinking_budget": 24, "third": [10, 11, 12, 17, 19, 35, 86, 100, 101, 103, 147, 152], "this_modul": 58, "thorough": [16, 21], "those": [2, 12, 13, 14, 15, 16, 20, 23, 26, 27, 29, 30, 31, 32, 35, 38, 75, 78, 81, 87, 91, 92, 103, 104, 105, 106, 116, 117, 123, 125, 131, 133, 134, 141, 147, 158, 162], "though": [10, 11, 14, 16, 17, 84, 91, 119, 130, 139], "thought": [11, 33, 97], "thread": [0, 1, 10, 12, 16, 20, 44, 75, 105, 111, 124, 138, 146, 147, 152], "three": [7, 8, 11, 12, 13, 15, 17, 22, 75, 89, 92, 103, 116, 132, 133, 141, 147, 148, 153, 154, 155], "threshold": [0, 8, 13, 14, 34, 86, 133, 138, 147], "threw": 1, "throttl": [39, 124], "through": [0, 1, 2, 8, 10, 11, 12, 13, 16, 17, 18, 20, 21, 23, 27, 30, 33, 34, 75, 81, 83, 86, 91, 92, 96, 98, 101, 105, 106, 107, 111, 112, 113, 117, 118, 124, 125, 126, 127, 130, 131, 134, 150, 151, 152, 162, 164], "throughout": [8, 20, 125, 128, 129], "throughput": [0, 3, 4, 5, 9, 10, 12, 14, 16, 17, 18, 20, 37, 38, 62, 65, 75, 76, 84, 88, 89, 90, 91, 92, 97, 102, 103, 105, 123, 127, 130, 131, 132, 152, 155, 156, 165, 166], "throw": [0, 1], "thrown": 1, "thu": [2, 10, 13, 15, 16, 20, 36, 96, 98, 109, 119, 133, 139], "thumb": [105, 126, 143], "ti": [14, 21, 75, 105], "tightli": 21, "tiiuae": [39, 124], "tile": 15, "tilen": 152, "time": [0, 1, 2, 5, 7, 8, 10, 12, 13, 14, 15, 16, 17, 19, 20, 21, 22, 23, 24, 33, 37, 38, 39, 40, 49, 54, 55, 56, 57, 64, 84, 86, 87, 90, 92, 96, 97, 98, 101, 102, 103, 105, 109, 110, 111, 113, 114, 117, 123, 124, 125, 127, 128, 130, 132, 133, 136, 138, 143, 144, 146, 147, 150, 151, 152, 166, 169], "time_": 8, "time_embed_dim": 134, "time_encod": 138, "time_i": 8, "time_point": 0, "timedelta": 147, "timedout": 0, "timelin": [10, 17, 84, 116, 152], "timeout": [0, 8, 16, 27, 44, 84, 102, 147, 152], "timeout_it": [8, 30, 147], "timepoint": 0, "timestamp": 0, "timestep": [134, 135], "timestepembed": 134, "timingmetr": 0, "tini": 64, "tinyllama": [17, 27, 37, 41, 43, 47, 49, 50, 51, 53, 54, 55, 56, 57, 60, 64, 65, 66, 69, 71, 73, 77, 78, 81, 82, 84, 101, 144, 146, 151, 157, 158, 162, 163], "tip": [12, 97], "titl": [53, 95, 136, 147], "titlecas": 147, "tle": 114, "tllm": [85, 147, 149], "tllm_benchmark_req_queues_s": 20, "tllm_checkpoint_16gpu_tp8_pp2": 126, "tllm_ckpt_dir": 115, "tllm_engine_dir": 115, "tllm_kei": [118, 134], "tllm_llmapi_build_cach": 152, "tllm_llmapi_enable_nvtx": [38, 123], "tllm_log_level": [59, 143], "tllm_nvtx_debug": [20, 38, 123], "tllm_override_layer_num": 152, "tllm_profile_record_gc": [20, 38, 123], "tllm_profile_start_stop": [20, 38, 123], "tllm_to_externel_key_dict": 118, "tllm_torch_profile_trac": [38, 123], "tllm_trace_model_forward": 152, "tllm_weight": 118, "tllmruntim": [1, 106, 143], "tlntin": 152, "tma": [12, 152], "tmp": [26, 29, 30, 31, 32, 33, 38, 39, 62, 76, 110, 114, 123, 124, 126, 156], "tmp9so41y3r": [39, 124], "tmpowsrb_f4": [39, 124], "tmpxhdvasex": [39, 124], "to_arrai": 133, "to_dict": [135, 147], "to_json": 147, "to_json_fil": 135, "to_layer_quant_config": 135, "to_legacy_set": 136, "to_python": 147, "to_str": [0, 1, 103], "to_trt": 135, "tobyt": 1, "todo": [1, 30, 66, 133], "togeth": [3, 10, 11, 12, 13, 17, 18, 21, 23, 66, 75, 91, 103, 105, 106, 110, 117, 136, 138, 141, 150, 152], "toggl": [38, 59, 86, 94, 123], "toi": [91, 130], "toitensor": 0, "tojsonstr": 0, "tok": [3, 5, 6, 17, 26, 29, 30, 31, 32, 97, 131], "token": [0, 1, 2, 3, 6, 7, 9, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21, 22, 23, 24, 27, 33, 37, 38, 39, 40, 47, 48, 51, 53, 58, 59, 60, 62, 66, 67, 75, 76, 77, 78, 84, 86, 87, 89, 91, 92, 96, 97, 103, 104, 105, 106, 108, 109, 113, 117, 120, 123, 124, 125, 127, 128, 131, 133, 134, 136, 138, 139, 141, 147, 148, 152, 153, 154, 155, 156, 157, 158], "token_count": 60, "token_drop": 134, "token_end": 147, "token_id": [29, 31, 32, 44, 95, 147], "token_ids_diff": 147, "token_norm_dist": 97, "token_range_retention_config": [51, 147], "token_start": 147, "token_type_id": [135, 138], "token_unif_dist": 97, "tokenend": 0, "tokenextraid": 1, "tokenextraidtyp": 1, "tokenid": 1, "tokenidtyp": [0, 1], "tokenization_utils_bas": 147, "tokenized_request": [29, 31, 32], "tokenizer_dir": [115, 117, 143, 147], "tokenizer_image_token": 138, "tokenizer_max_seq_length": [127, 135, 137, 147], "tokenizer_mod": 147, "tokenizer_revis": 147, "tokenizer_str": [0, 103], "tokenizerbas": 147, "tokenizerstr": [0, 103], "tokenlogprob": 147, "tokenrangeretentionconfig": [0, 51, 86, 147], "tokenrangeretentionprior": 0, "tokens_": 8, "tokens_i": 8, "tokens_per_block": [23, 58, 59, 80, 87, 108, 109, 136, 138, 147, 152, 169], "tokenselectedexpert": 12, "tokensperblock": [0, 1, 27, 106], "tokensperstep": 1, "tokensprompt": 147, "tokenstart": 0, "tokyo": [42, 70], "toler": [7, 12, 16, 152], "toml": [83, 164], "tomodulenam": 1, "tomoduletyp": 1, "tonylek": 152, "too": [0, 2, 10, 15, 16, 20, 27, 33, 50, 84, 96, 102, 103, 105, 125, 130, 138, 143, 147], "took": 125, "tool": [2, 10, 11, 15, 16, 20, 21, 22, 29, 30, 31, 32, 36, 86, 89, 92, 116, 124, 150, 151, 152], "tool_cal": [18, 21, 30, 33, 151], "toolcal": 11, "toolkit": [7, 13, 18, 21, 29, 30, 31, 32, 33, 101, 119, 154], "toolset": 152, "top": [0, 8, 9, 10, 12, 14, 15, 16, 17, 31, 32, 35, 37, 66, 75, 84, 92, 95, 96, 104, 105, 106, 113, 117, 133, 147, 152, 168], "top1": 13, "top_k": [9, 66, 78, 95, 106, 138, 147, 152, 158, 168], "top_k_valu": 66, "top_p": [9, 11, 21, 30, 33, 49, 54, 55, 56, 57, 60, 65, 66, 67, 95, 101, 106, 125, 132, 138, 144, 146, 147, 151, 168], "top_p_decai": [138, 147], "top_p_min": [138, 147], "top_p_reset_id": [138, 147], "top_p_valu": 66, "topenkoff": 152, "topic": [16, 26, 131], "topk": [0, 1, 13, 15, 20, 104, 106, 113, 133, 147, 152], "topk_logit": 103, "topklastdim": 133, "topklogit": 103, "topkmedusahead": 1, "topktopp": [0, 106], "topmodelmixin": [119, 135], "topn": 13, "topologi": [16, 20], "topp": [0, 1, 106, 152], "toppdecai": [0, 1, 106], "toppmin": [0, 1, 106, 147], "toppresetid": [0, 1, 106], "topr": 147, "torch": [10, 58, 60, 75, 76, 77, 78, 80, 81, 82, 83, 85, 94, 95, 98, 101, 105, 118, 124, 133, 138, 143, 147, 148, 149, 152, 153, 156, 157, 158, 160, 161, 162, 163, 164], "torch_compile_config": [147, 152], "torchaudio": 101, "torchcompileconfig": [97, 147], "torchllmarg": [26, 29, 30, 31, 32, 33, 58, 97, 147, 152], "torchsampl": [9, 96, 147], "torchvis": 101, "toronto": 30, "tostr": [0, 1], "total": [0, 1, 2, 8, 10, 14, 16, 17, 20, 22, 23, 24, 27, 33, 35, 39, 40, 67, 75, 91, 104, 105, 106, 113, 116, 118, 124, 125, 126, 139, 147, 169], "total_lat": [3, 6], "total_token": [18, 21, 29, 30, 31, 32, 33, 151], "totalaccepteddrafttoken": 0, "totaldrafttoken": 0, "totalgentoken": 1, "totalnumpag": 1, "totensor": 0, "touch": [120, 148, 153], "tourist": 33, "toward": [16, 90, 94, 96, 166], "tp": [0, 2, 3, 4, 5, 6, 7, 12, 13, 14, 15, 16, 17, 18, 21, 22, 27, 39, 40, 62, 76, 84, 94, 102, 104, 106, 110, 117, 124, 125, 133, 147, 150, 152, 156], "tp1": [3, 4, 5, 10, 40], "tp2": [17, 40, 84, 124, 152], "tp4": [10, 12, 13, 30, 40], "tp4ep2": 13, "tp8": [5, 13, 15, 30, 40], "tp8ep2": 13, "tp_1_pp_1": 124, "tp_dim": [118, 134], "tp_group": [133, 134], "tp_rank": [118, 133, 134], "tp_size": [9, 18, 21, 22, 24, 26, 27, 40, 45, 61, 63, 104, 110, 116, 117, 118, 119, 124, 126, 133, 134, 137, 152], "tp_split_dim": 134, "tpot": [2, 6, 8, 17, 33, 40, 84], "tprank": 1, "tps_": 8, "tpsize": 1, "tqdm": [118, 147, 152], "trace": [16, 22, 23, 24, 27, 38, 97, 119, 123, 143], "track": [1, 16, 34, 36, 75, 81, 91, 105, 108, 133, 147, 162], "trade": [2, 15, 21, 37, 96, 109], "tradeoff": [7, 13, 14, 97, 127], "tradit": [0, 8, 11, 92], "traffic": [16, 17, 76, 84, 86, 92, 156, 160], "trail": 147, "train": [4, 7, 11, 14, 19, 30, 96, 113, 115, 116, 117, 119, 124, 133, 143, 148, 153], "trainabl": [88, 165], "trait": 152, "trampolin": 10, "transa": 133, "transb": 133, "transceiv": [0, 147], "transfer": [0, 15, 16, 17, 20, 84, 102, 117, 147, 152], "transfer_mod": [86, 147], "transferdesc": 0, "transfermod": 0, "transferop": 0, "transferrequest": 0, "transferstatu": 0, "transform": [0, 20, 22, 23, 24, 27, 37, 44, 60, 75, 76, 78, 81, 82, 83, 89, 91, 104, 105, 113, 115, 116, 117, 118, 135, 139, 142, 143, 147, 148, 152, 153, 154, 156, 158, 160, 161, 162, 163, 164, 169], "transformerstoken": 147, "transit": [8, 11], "translat": [20, 21, 36, 81, 88, 132, 147, 152, 162, 165], "transmiss": [17, 20, 84, 102, 111], "transmit": [12, 84, 102, 111], "transpar": [10, 16, 19, 20], "transparent_hugepag": 20, "transport": 12, "transpos": [1, 116, 133], "transposit": 133, "travel": 30, "travers": [10, 117], "treat": [13, 20, 75, 87, 105, 133, 147], "tree": [0, 9, 10, 11, 22, 30, 86, 96, 99, 124, 138, 143, 169], "tremend": 10, "trend": 19, "tri": [15, 170], "tricki": 135, "trigger": [10, 12, 16, 20, 23, 34, 44, 75, 83, 97, 105, 107, 117, 136, 146, 147], "trigger_completion_at_end": 133, "trim": 1, "trimpool": 1, "trip": 33, "triton": [10, 35, 78, 80, 82, 97, 109, 110, 113, 117, 147, 150, 152, 158, 161, 163], "triton_serv": 35, "tritonserv": 152, "trivial": 117, "troubleshoot": [97, 152], "trt": [0, 4, 11, 22, 27, 46, 75, 80, 81, 82, 84, 91, 96, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 117, 118, 120, 124, 130, 133, 135, 137, 138, 139, 143, 152, 155, 158, 161, 162, 163], "trt_ckpt": [110, 114, 116, 143], "trt_engin": [110, 114, 116, 143], "trt_llm_data": [36, 67], "trt_llm_disable_load_weights_in_parallel": 18, "trt_root": 2, "trt_tensor": [117, 133], "trtdatatyp": 1, "trtgptmodel": 139, "trtgptmodeloptionalparam": 152, "trtgptmodelv1": 152, "trtllm": [2, 9, 11, 12, 14, 16, 18, 20, 21, 29, 30, 31, 32, 33, 36, 39, 40, 41, 42, 43, 44, 45, 47, 48, 52, 61, 67, 69, 70, 71, 72, 73, 77, 82, 83, 87, 92, 97, 102, 109, 110, 114, 115, 116, 117, 119, 124, 127, 128, 129, 130, 131, 139, 143, 147, 152, 157, 162, 163, 164], "trtllm_deep_ep_token_limit": 152, "trtllm_dg_jit_use_nvcc": 2, "trtllm_disable_kv_cache_transfer_overlap": [84, 102], "trtllm_disable_unified_convert": 118, "trtllm_enable_kvcache_receive_parallel": [84, 102], "trtllm_enable_mmha_multi_block_debug": 124, "trtllm_enable_pdl": [2, 9, 12, 13, 14, 18, 21, 124], "trtllm_force_xqa": [75, 105], "trtllm_kvcache_send_max_concurrency_num": [84, 102], "trtllm_kvcache_transfer_buffer_s": [84, 102], "trtllm_kvcache_transfer_use_async_buff": [84, 102], "trtllm_llama_eager_fusion_dis": 152, "trtllm_mmha_blocks_per_sequ": 124, "trtllm_mmha_kernel_block_s": 124, "trtllm_model": 118, "trtllm_modules_to_hf_modul": [39, 88, 124, 138, 165], "trtllm_pdl_overlap_ratio": 124, "trtllm_precompiled_loc": 98, "trtllm_prefetch_ratio": 124, "trtllm_request_kv_cache_concurr": [84, 102], "trtllm_try_zcopy_for_kvcache_transf": [84, 102], "trtllm_use_precompil": 98, "trtllmarg": [97, 147], "trtllmattent": [97, 155], "trtllmattentionwrapp": 152, "trtllmsampler": 147, "trtllmworker": 11, "trtlmmdatatyp": 0, "true": [0, 1, 2, 8, 9, 11, 13, 14, 15, 16, 18, 19, 21, 26, 27, 29, 30, 31, 32, 33, 38, 40, 44, 56, 58, 59, 60, 62, 65, 66, 68, 75, 77, 78, 84, 87, 90, 92, 95, 96, 103, 106, 107, 109, 113, 116, 123, 124, 127, 131, 133, 134, 135, 136, 138, 139, 143, 147, 152, 157, 158, 166], "true_output_valu": 133, "true_valu": 133, "truli": 20, "truncat": [147, 152], "truncate_prompt_token": [147, 152], "truncation_sid": 11, "trust": [15, 22, 147], "trust_remote_cod": [9, 11, 18, 21, 22, 24, 26, 27, 147, 152], "truth": [81, 162], "try": [0, 1, 12, 16, 18, 19, 20, 21, 29, 30, 31, 32, 33, 40, 50, 64, 73, 86, 89, 99, 103, 115, 119, 127, 130, 131, 132, 139, 143, 145, 146, 147, 151, 167], "tsuji": [39, 124], "ttensor": 1, "ttft": [8, 12, 17, 33, 39, 40, 84, 91, 127, 130, 131, 132, 152], "ttim": 152, "ttl": 13, "tunabl": [128, 129], "tune": [0, 4, 7, 8, 13, 15, 16, 17, 21, 22, 23, 26, 39, 40, 88, 91, 92, 96, 97, 103, 113, 124, 127, 129, 131, 134, 135, 138, 139, 147, 150, 152, 160, 165], "tuner": 0, "tupl": [0, 1, 58, 133, 134, 138, 147, 170], "turn": [10, 15, 17, 21, 87, 98, 105, 106, 109, 113, 127, 138, 139, 152], "turn1": 19, "turn2": 19, "turnaround": 35, "tushar": 152, "tutori": 26, "tweak": 132, "twice": [10, 117], "two": [0, 4, 8, 9, 11, 12, 13, 14, 15, 16, 17, 19, 20, 22, 23, 27, 30, 33, 34, 35, 39, 42, 59, 70, 75, 83, 84, 85, 86, 87, 91, 92, 95, 97, 98, 102, 103, 104, 105, 106, 107, 109, 110, 111, 113, 114, 116, 117, 119, 124, 125, 127, 130, 132, 133, 134, 136, 146, 147, 149, 151, 152, 154, 164, 168, 169, 170], "twofold": 113, "twoshot": [111, 133, 147], "txt": [2, 21, 22, 24, 35, 38, 39, 62, 76, 94, 98, 119, 123, 124, 125, 152, 156], "type": [1, 4, 7, 9, 11, 12, 15, 17, 18, 21, 22, 23, 26, 27, 29, 30, 31, 32, 33, 35, 39, 41, 42, 43, 47, 48, 53, 58, 60, 64, 65, 66, 67, 68, 70, 73, 75, 77, 82, 83, 84, 86, 91, 93, 95, 96, 97, 102, 103, 105, 106, 107, 110, 116, 117, 124, 127, 131, 133, 135, 136, 137, 138, 141, 142, 143, 147, 148, 151, 152, 153, 154, 155, 157, 160, 163, 164, 169], "typedef": [0, 1], "typeerror": 147, "typenam": [0, 1, 117], "typetrait": 0, "typic": [0, 7, 12, 14, 15, 16, 17, 20, 26, 27, 29, 30, 31, 32, 33, 84, 89, 102, 107, 115, 117, 119, 126, 127, 131, 132, 136, 138, 139, 146, 148, 152, 153], "typo": 152, "u": [1, 8, 12, 15, 16, 17, 29, 30, 31, 32, 33, 39, 40, 49, 54, 55, 56, 57, 101, 107, 120, 124, 144, 146, 147, 151, 152], "ub": [111, 133, 147], "ub_oneshot": 124, "ub_tp_siz": 124, "ubuntu": [101, 145, 152], "uc_handl": 1, "uc_ptr": 1, "uc_va": 1, "ucsd": 11, "ucx": [0, 17, 84, 102, 147, 152], "ucx_cuda_ipc_enable_mnnvl": [84, 102], "ucx_max_rndv_rail": [84, 102], "ucx_net_devic": [84, 102], "ucx_rndv_schem": [84, 102], "ue8m0": 93, "uid": [0, 36, 138], "uint16_t": 0, "uint32": 1, "uint32_t": [0, 1, 133], "uint64": [1, 109], "uint64_t": [0, 1], "uint8": 1, "uint8_t": [0, 1], "uintptr_t": [0, 1], "uk": 15, "uk_bgemm": 13, "ulimit": [9, 18, 21, 26, 98, 143, 151], "ultim": 126, "ultra": [82, 163], "ulyss": 152, "unabl": 130, "unaccept": 127, "unaffect": 20, "unaligneddata": 12, "unari": 133, "unaryoper": 133, "unbind": 133, "unblock": [10, 20], "uncas": [142, 147, 149], "uncertain_word": 11, "uncertainti": 113, "unchang": [16, 35, 78, 113, 131, 133, 158], "uncom": [36, 98], "uncommon": 117, "undefin": 133, "under": [0, 7, 8, 9, 17, 19, 20, 21, 23, 31, 32, 34, 35, 39, 40, 81, 84, 87, 89, 94, 98, 124, 143, 146, 147, 152, 162], "undergo": [81, 82, 162, 163], "underli": [0, 1, 16, 17, 83, 84, 107, 113, 147, 164], "underlying_type_t": 1, "underlyingtyp": [0, 1], "underscor": 127, "understand": [8, 16, 35, 38, 39, 78, 91, 92, 98, 123, 158], "understood": [130, 147], "underutil": [20, 113], "underwai": 17, "uneven": [8, 152], "unevenli": 13, "unexpect": [10, 20, 143, 147, 152], "unfinish": 0, "unfortun": 20, "unfus": [133, 152], "unfuse_qkv_project": 135, "ungath": 1, "unguid": 53, "unicast": 1, "unicastconfigur": 1, "unicod": 147, "unicodeencodeerror": 147, "unif": 152, "unifi": [7, 24, 81, 83, 116, 119, 152, 162, 164], "uniform": [8, 22, 39, 40, 124, 133], "uniformli": 8, "uniniti": [75, 155], "union": [133, 147], "uniqu": [0, 1, 23, 39, 83, 87, 105, 106, 108, 110, 113, 116, 124, 147, 164], "unique_ptr": [0, 1], "uniqueconstptr": 1, "uniqueptr": 1, "uniquetoken": [0, 1], "unit": [1, 15, 18, 21, 29, 30, 31, 32, 33, 34, 39, 49, 50, 51, 58, 97, 98, 101, 108, 118, 124, 125, 132, 144, 146, 151, 152], "unittest": [34, 35, 67], "univers": [8, 49, 54, 55, 57, 101, 144, 146, 151], "unknown": [1, 22, 147], "unleash": 19, "unless": [0, 33, 34, 44, 91, 126, 131, 132, 147], "unlik": [14, 33, 78, 95, 109, 113, 158], "unlock": 16, "unmatch": 12, "unnecessari": [10, 11, 33, 107, 148, 152, 153, 170], "unneed": [13, 75, 105], "unordered_map": [0, 1, 103], "unpack": 10, "unpatchifi": 135, "unpredict": 8, "unsaf": [84, 102], "unsaferemov": 1, "unsatisfactori": 20, "unschedul": [91, 130], "unset": [16, 84, 102, 132], "unsign": 1, "unspecifi": [23, 24, 27, 133], "unsqueez": [1, 133], "unstabl": [119, 147], "unsupport": [35, 152], "untest": [85, 149], "until": [0, 1, 8, 10, 11, 12, 16, 26, 29, 30, 31, 32, 86, 96, 103, 106, 109, 113, 147], "untouch": [133, 147], "unus": [0, 39, 124], "up": [0, 2, 4, 5, 8, 9, 10, 12, 13, 14, 15, 16, 17, 18, 20, 21, 22, 23, 27, 29, 30, 31, 32, 33, 37, 39, 53, 58, 60, 75, 83, 84, 91, 96, 97, 102, 105, 106, 110, 113, 124, 130, 131, 136, 147, 151, 152, 164, 169], "up_proj": 118, "upcast": 133, "upcast_attent": 134, "upcast_softmax": 134, "upcom": [7, 81, 162, 169], "updat": [0, 2, 5, 10, 14, 15, 16, 18, 19, 23, 33, 37, 81, 83, 89, 90, 92, 98, 108, 113, 117, 118, 119, 120, 133, 136, 138, 143, 147, 162, 164, 166, 169], "update_forward_ref": 147, "update_from_dict": 147, "update_key_map": 118, "update_kv_cache_typ": 147, "update_output_ids_by_offset": 138, "update_resourc": [154, 169], "update_state_after_alloc": 58, "update_strategi": 133, "updatenumreturnbeam": 0, "updatespositionid": 1, "upfront": 33, "upgrad": [81, 101, 152, 162], "uplift": [127, 130, 131], "upon": [1, 10, 12, 18, 20, 37, 40, 83, 113, 131, 143, 152], "upper": [8, 68, 124, 133, 139, 147], "uppercas": 147, "upsampl": 12, "upstat": 33, "uq_qr_gemm": 13, "url": [17, 27, 42, 47, 48, 70, 84, 98, 101, 152], "us": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 19, 20, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 34, 37, 38, 39, 40, 44, 46, 49, 51, 52, 53, 56, 58, 59, 61, 62, 63, 64, 66, 67, 75, 76, 77, 78, 79, 80, 81, 82, 84, 86, 88, 89, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 108, 109, 111, 112, 116, 117, 118, 119, 120, 121, 123, 124, 125, 126, 127, 128, 129, 130, 133, 134, 135, 136, 138, 141, 143, 144, 145, 147, 148, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 165, 168, 169, 170], "usabl": [11, 20, 145, 147], "usag": [0, 3, 6, 15, 17, 18, 20, 21, 22, 23, 27, 29, 30, 31, 32, 33, 34, 49, 67, 75, 86, 87, 92, 97, 99, 102, 105, 107, 108, 117, 119, 124, 131, 132, 133, 144, 146, 147, 151, 152, 155], "use_beam_hyp": 138, "use_beam_search": [95, 147, 152], "use_cach": [133, 134, 135], "use_context_fmha_for_gener": 152, "use_custom_all_reduc": 152, "use_diff_of_squar": 133, "use_dynamic_tre": 147, "use_embedding_shar": 152, "use_fast": 11, "use_fp32_acc": 133, "use_fp8": 134, "use_fp8_context_fmha": [23, 105, 124, 136, 152], "use_fused_mlp": [23, 124, 136, 152], "use_gemm_allreduce_plugin": 138, "use_gpt_attention_plugin": 138, "use_gpu_direct_storag": 138, "use_implicit_relative_attent": 134, "use_kv_cach": [134, 138, 152], "use_logn_sc": 134, "use_lora": 135, "use_lora_plugin": 138, "use_low_precision_moe_combin": 147, "use_mamba_conv1d_plugin": 138, "use_meta_recip": 147, "use_modelopt_quant": 119, "use_mrop": 147, "use_mtp_vanilla": 147, "use_one_more_block": 138, "use_paged_context_fmha": [23, 75, 105, 109, 124, 127, 131, 136], "use_parallel_embed": [116, 117, 135], "use_preload": 135, "use_prompt_tun": [135, 152], "use_py_sess": 143, "use_refit": 147, "use_relaxed_acceptance_for_think": [13, 14, 68, 96, 147], "use_runtime_default": 138, "use_safetensors_load": 135, "use_strip_plan": 147, "use_torch_sampl": 9, "use_tqdm": 147, "use_uvm": [86, 147], "use_variable_beam_width_search": 138, "usebantoken": 0, "usebanword": 0, "usecrossattent": 1, "usedefaultvalu": 1, "usednumblock": [0, 27], "usedraftlogit": 1, "usedraftlogitshost": 1, "usedynamictre": 0, "usedynamictreehost": 1, "useexpliciteosstop": 0, "usefrequencypenalti": 0, "usegemmallreduceplugin": 1, "usegptattentionplugin": [1, 106], "usegpudirectstorag": 0, "uselanguageadapt": 1, "useloraplugin": 1, "usemambaconv1dplugin": 1, "usemaxlengthstop": 0, "useminlen": 0, "useminlength": 0, "useminp": 0, "usemrop": 1, "usenorepeatngrams": 0, "useoccurrencepenalti": 0, "usepackedinput": 1, "usepagedst": 1, "usepenalti": 0, "usepositionembed": 1, "usepresencepenalti": 0, "useprogthread": 0, "useprompttun": 1, "user": [0, 2, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 26, 27, 29, 30, 31, 32, 33, 34, 36, 37, 38, 39, 40, 41, 42, 53, 69, 70, 73, 75, 78, 83, 84, 86, 87, 91, 92, 98, 100, 102, 103, 105, 106, 107, 109, 110, 111, 112, 117, 118, 119, 120, 123, 124, 130, 131, 132, 133, 135, 136, 139, 141, 143, 146, 147, 150, 151, 152, 158, 160, 164], "user_buff": [23, 127, 136], "user_provid": [135, 147], "userandomacceptancethreshold": 1, "userbuff": [147, 152], "userepetitionpenalti": 0, "usernam": [9, 18], "userprovideddecodingconfig": [96, 97, 147], "useshapeinfer": 1, "usespecdecod": 1, "usestopword": 0, "usetemp": 0, "usetemperatur": 0, "usetokentypeembed": 1, "useuvm": 0, "usevariablebeamwidthsearch": 0, "using_oss_cutlass_": 112, "using_oss_cutlass_low_latency_gemm": 112, "using_oss_cutlass_moe_gemm": 112, "usr": [2, 30, 33, 41, 42, 43, 45, 47, 48, 116, 124], "usual": [14, 33, 40, 87, 96, 101, 117, 119, 125, 131, 133, 136, 147, 169], "utf": 147, "utf8": 147, "util": [0, 1, 2, 3, 8, 11, 12, 13, 15, 16, 17, 18, 21, 23, 24, 27, 29, 30, 31, 32, 33, 37, 38, 39, 49, 65, 75, 76, 78, 84, 87, 89, 90, 91, 96, 105, 106, 113, 117, 123, 124, 127, 131, 132, 136, 139, 147, 150, 152, 155, 156, 158, 166], "uv": 15, "uv_gemm": 13, "uvicorn": 26, "uvm": [0, 1, 20, 147], "v": [1, 2, 3, 4, 7, 9, 11, 13, 15, 18, 19, 21, 29, 30, 31, 32, 33, 34, 36, 75, 91, 97, 105, 106, 110, 133, 138, 141, 142, 143, 148, 149, 153, 155], "v0": [3, 4, 5, 6, 39, 40, 82, 90, 91, 110, 124, 142, 149, 152, 163, 166], "v1": [9, 12, 17, 18, 21, 27, 29, 30, 31, 32, 33, 37, 40, 41, 42, 43, 47, 49, 50, 51, 53, 54, 55, 56, 57, 60, 64, 65, 66, 69, 70, 71, 72, 73, 77, 78, 81, 82, 84, 88, 101, 142, 144, 146, 149, 151, 152, 157, 158, 160, 162, 163, 165], "v10": 152, "v100": 152, "v12": 152, "v2": [7, 12, 15, 34, 82, 83, 93, 141, 142, 152, 163, 164], "v3": [8, 14, 16, 27, 38, 92, 96, 123, 141, 142, 149, 152], "v9": 5, "v_dim": 133, "v_head_dim": [133, 134], "v_proj": [39, 83, 88, 118, 124, 148, 153, 164, 165], "vacat": [49, 101, 144, 146, 151], "valid": [0, 1, 8, 10, 12, 14, 16, 20, 29, 30, 31, 32, 40, 78, 82, 83, 84, 87, 92, 96, 103, 113, 133, 136, 138, 147, 152, 158, 160, 163, 164], "validate_and_init_token": 147, "validate_assign": 136, "validate_attention_dp_config": 147, "validate_auto_parallel": 147, "validate_batch_wait_max_tokens_ratio": 147, "validate_batch_wait_timeout_it": 147, "validate_batch_wait_timeout_m": 147, "validate_build_config_remain": 147, "validate_build_config_with_runtime_param": 147, "validate_capture_num_token": 147, "validate_checkpoint_format": 147, "validate_cuda_graph_config": 147, "validate_cuda_graph_max_batch_s": 147, "validate_dtyp": 147, "validate_dtype_not_auto": 136, "validate_enable_build_cach": 147, "validate_gpus_per_nod": 147, "validate_kv_cache_dtyp": 147, "validate_load_balanc": 147, "validate_lora_config_consist": 147, "validate_max_attention_window": 147, "validate_max_gpu_total_byt": 147, "validate_model": 147, "validate_model_format_misc": 147, "validate_parallel_config": 147, "validate_peft_cache_config": 147, "validate_positive_valu": 147, "validate_quant_config": 147, "validate_runtime_arg": 147, "validate_speculative_config": 147, "validate_stream_interv": 147, "validate_torch_compile_max_num_stream": 147, "validatevec": 1, "validationerror": 147, "validmpiconfig": 1, "valu": [0, 1, 2, 3, 4, 8, 10, 11, 12, 14, 15, 17, 19, 20, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 33, 34, 35, 37, 39, 44, 59, 60, 66, 75, 77, 78, 84, 86, 87, 89, 91, 96, 102, 105, 106, 108, 109, 110, 111, 114, 116, 117, 118, 124, 125, 127, 130, 132, 133, 135, 136, 137, 138, 139, 141, 143, 147, 152, 155, 157, 158, 169, 170], "valuabl": [8, 13, 16, 17], "value_typ": 0, "valueerror": [67, 68, 147], "valuestatu": 1, "vanilla": [67, 75, 105, 147, 152, 155], "vanillaattent": [75, 155], "var": 133, "vari": [5, 8, 11, 16, 17, 18, 20, 26, 29, 30, 31, 32, 84, 130, 131, 149, 169], "variabl": [0, 1, 2, 5, 12, 13, 16, 17, 20, 29, 36, 38, 39, 61, 62, 63, 79, 86, 97, 101, 106, 108, 118, 123, 124, 143, 146, 147, 152, 159], "variabledraftlength": 1, "varianc": [8, 15, 127, 130, 131, 133], "variant": [0, 3, 10, 14, 15, 21, 40, 75, 94, 96, 103, 105, 119, 133, 152, 155], "variat": 8, "varieti": [39, 82, 89, 93, 124, 125, 152, 163], "variou": [11, 16, 17, 20, 22, 26, 36, 39, 40, 66, 75, 77, 81, 82, 83, 89, 98, 105, 113, 124, 127, 130, 146, 147, 150, 151, 152, 157, 162, 163], "varnam": 1, "vartyp": 1, "vastli": 8, "vboost": [2, 13, 39, 124], "vbw": 152, "ve": [12, 13, 64], "vec": [0, 1], "vec2": 133, "veclogprob": 0, "vectoken": 0, "vectokenextraid": [0, 1], "vector": [0, 1, 15, 103, 105, 106, 108, 110, 133], "vecuniquetoken": [0, 1], "vehicl": 18, "verbatim": 135, "verbos": [22, 23, 24, 27, 39, 79, 124, 147, 159], "veri": [7, 12, 14, 16, 18, 19, 20, 21, 29, 30, 31, 32, 33, 59, 75, 91, 96, 105, 116, 117, 125, 126, 127, 152], "verif": [0, 10, 11, 14, 19, 113, 147], "verifi": [9, 10, 12, 14, 18, 19, 20, 21, 24, 33, 35, 59, 113, 131, 133, 152], "verification_batch": 19, "verificationsets": 0, "verl": 94, "vermont": 30, "versa": [15, 109], "version": [0, 1, 2, 10, 12, 13, 15, 16, 20, 21, 24, 27, 29, 31, 32, 34, 36, 39, 44, 75, 84, 86, 91, 96, 98, 99, 102, 105, 106, 116, 118, 119, 124, 125, 133, 143, 145, 147, 152], "versu": [11, 33], "vertic": 133, "vertical_strid": 134, "vgqa": 108, "via": [0, 10, 13, 16, 17, 20, 21, 26, 29, 30, 31, 32, 33, 35, 39, 61, 62, 63, 64, 78, 80, 81, 82, 84, 96, 98, 100, 111, 112, 113, 124, 127, 128, 129, 131, 132, 133, 145, 147, 152, 158, 161, 162, 163], "vice": [15, 109], "vicuna": 113, "video": [22, 27, 39, 42, 70, 124, 138, 142, 149, 152], "video_grid_thw": 138, "video_path": 138, "video_preprocess": 138, "video_url": [27, 42, 70], "view": [1, 10, 14, 16, 18, 33, 78, 133, 138, 158], "vila": [42, 70, 93, 140, 141, 142, 149, 152], "vinyl": [39, 124], "violat": [10, 152], "virtual": [0, 1, 134], "virtualmemorymanagertest": 1, "vision": [21, 22, 81, 89, 93, 138, 141, 142, 147, 149, 152, 162], "vision_grid_thw": 138, "vision_length": 133, "vision_model_typ": 135, "vision_start": 133, "vision_token_mask": 134, "visit": [13, 81, 113, 152, 162], "visual": [8, 18, 36, 91, 130, 147, 152], "visual_engine_dir": 138, "visual_featur": 138, "visualize_network": [23, 147, 152], "vit": [147, 152], "vital": [7, 107], "vl": [26, 27, 39, 42, 48, 70, 89, 124, 140, 142, 152], "vllm": [10, 11, 152], "vlm": [81, 142, 152, 162], "vocab": [96, 133, 138], "vocab_embed": [115, 118], "vocab_s": [0, 10, 12, 116, 118, 134, 135, 138, 147, 148, 153], "vocab_size_pad": 138, "vocabs": [1, 106], "vocabsizepad": [0, 1], "vocabulari": [0, 1, 10, 40, 106, 109, 113, 134, 138], "void": [0, 1, 12, 103, 117], "volatil": 8, "volta": 152, "volum": [1, 8, 39, 97, 98, 111, 124], "volumenonneg": 1, "vonjackustc": 152, "vote": [49, 101, 144, 146, 151], "vram": 59, "vswa": 108, "vt": 33, "vulner": 152, "vultureprim": 152, "w": [1, 2, 6, 9, 10, 13, 15, 18, 19, 21, 22, 27, 29, 30, 31, 32, 33, 84, 133, 135, 141, 142, 152], "w1": 133, "w4a": [141, 152], "w4a16": [7, 22, 93, 116, 135, 147], "w4a16_awq": [22, 44, 116, 119, 147], "w4a16_gptq": [22, 116, 147], "w4a16_mxfp4": 147, "w4a8": [7, 93, 152], "w4a8_awq": [22, 116, 119, 147], "w4a8_mxfp4_fp8": [147, 152], "w4a8_mxfp4_mxfp8": 147, "w4a8_nvfp4_fp8": 147, "w4a8_qserve_per_channel": 147, "w4a8_qserve_per_group": 147, "w4aint8": 152, "w8a": 141, "w8a16": [7, 22, 116, 135, 147], "w8a16_gptq": 147, "w8a8": [4, 7], "w8a8_sq_per_channel": [116, 147], "w8a8_sq_per_channel_per_tensor_plugin": [135, 147], "w8a8_sq_per_channel_per_token_plugin": [135, 147], "w8a8_sq_per_tensor_per_token_plugin": [135, 147], "w8a8_sq_per_tensor_plugin": [135, 147], "wa": [0, 1, 8, 10, 12, 14, 15, 16, 20, 30, 39, 40, 59, 75, 86, 91, 99, 101, 103, 105, 106, 116, 124, 125, 127, 130, 131, 132, 134, 141, 143, 147, 148, 152, 153, 170], "wai": [10, 11, 12, 13, 14, 15, 16, 17, 19, 20, 39, 57, 58, 60, 75, 84, 91, 96, 99, 100, 105, 106, 107, 111, 124, 125, 127, 133, 139, 146, 152], "wait": [0, 1, 10, 11, 15, 16, 20, 26, 37, 44, 90, 103, 119, 124, 133, 147, 150, 166], "wait_for_layer_load": 58, "wait_for_sav": 58, "waiv": 97, "walk": [18, 21, 42, 64, 70, 125, 126, 127], "wall": 33, "wang1120": 152, "wangkuiyi": 152, "want": [2, 11, 13, 14, 16, 18, 20, 21, 29, 30, 31, 32, 33, 38, 39, 58, 75, 83, 87, 91, 98, 105, 113, 119, 121, 123, 124, 127, 130, 132, 133, 143, 147, 148, 152, 153], "war": 1, "warm": [22, 33, 84, 102, 169], "warmup": [2, 16, 20, 21, 22, 38, 75, 123, 124, 125, 152, 155, 169], "warn": [22, 23, 24, 26, 27, 34, 40, 60, 75, 79, 105, 124, 139, 147, 159], "warn_on_unstable_feature_usag": 147, "warp": [111, 152], "washington": 30, "wast": [11, 15, 20, 35, 37], "watch": 131, "watt": 21, "wdkv": 13, "wdq": 13, "we": [1, 2, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 26, 27, 29, 30, 31, 32, 33, 38, 39, 40, 49, 54, 55, 57, 58, 64, 81, 82, 84, 86, 87, 91, 94, 96, 98, 101, 102, 104, 106, 107, 110, 111, 112, 113, 114, 116, 119, 120, 121, 123, 124, 125, 126, 127, 130, 131, 133, 138, 143, 144, 146, 147, 148, 151, 152, 153, 162, 163], "web": 121, "weekli": [29, 30, 31, 32], "weig": 133, "weight": [0, 1, 3, 4, 7, 9, 12, 13, 14, 16, 18, 19, 21, 22, 23, 24, 26, 27, 29, 30, 31, 32, 40, 57, 76, 77, 82, 88, 92, 93, 104, 110, 119, 125, 126, 127, 133, 134, 135, 136, 138, 147, 150, 152, 156, 157, 163, 165], "weight_index": 133, "weight_load": [83, 134, 164], "weight_mapp": [83, 164], "weight_only_groupwise_quant_matmul": 141, "weight_only_groupwise_quant_matmul_plugin": 136, "weight_only_precis": 152, "weight_only_quant_matmul_plugin": 136, "weight_spars": [23, 147], "weight_stream": [23, 114, 147], "weightonlygroupwisequantmatmulplugin": 141, "weights_dict": [83, 119, 164], "weights_scaling_factor": [116, 118], "weightsinpoint": 1, "weightsoutpoint": 1, "welcom": [11, 16, 81, 94, 162], "well": [4, 10, 12, 16, 19, 20, 21, 38, 44, 75, 86, 91, 102, 105, 106, 117, 123, 130, 131, 141, 142, 147, 151], "were": [0, 1, 3, 7, 10, 12, 15, 17, 20, 40, 91, 96, 112, 113, 116, 119, 126, 130, 147, 152], "weren": 101, "west": 30, "wget": [24, 143], "what": [11, 15, 16, 21, 27, 30, 33, 35, 38, 39, 42, 64, 66, 68, 70, 84, 86, 88, 91, 97, 98, 102, 103, 123, 124, 125, 127, 130, 131, 136, 147, 165], "whatev": 1, "wheel": [98, 100, 101, 152], "when": [0, 1, 2, 6, 7, 8, 9, 10, 11, 12, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 26, 27, 29, 30, 31, 32, 33, 34, 35, 36, 38, 39, 44, 50, 51, 59, 60, 75, 78, 84, 86, 87, 91, 92, 95, 96, 98, 101, 102, 103, 104, 105, 106, 108, 109, 110, 111, 113, 117, 118, 119, 120, 123, 124, 125, 127, 130, 131, 132, 133, 134, 135, 136, 138, 139, 141, 143, 147, 148, 150, 152, 153, 155, 158, 168, 169], "whenev": [1, 75], "where": [0, 1, 3, 7, 8, 10, 12, 13, 14, 15, 16, 17, 20, 22, 29, 30, 31, 32, 33, 37, 39, 40, 41, 43, 44, 51, 59, 64, 69, 71, 75, 76, 84, 86, 87, 92, 96, 102, 105, 106, 108, 109, 111, 112, 113, 116, 117, 124, 127, 130, 132, 133, 138, 141, 147, 151, 152, 156, 160, 170], "wherea": [0, 17, 84, 116, 130], "whether": [0, 1, 8, 11, 12, 16, 17, 19, 23, 24, 75, 77, 84, 86, 103, 105, 110, 126, 127, 131, 133, 134, 136, 138, 147, 150, 154, 155, 157], "which": [0, 1, 3, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 26, 27, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 50, 59, 60, 65, 66, 75, 77, 78, 81, 82, 83, 84, 86, 87, 89, 91, 92, 93, 95, 96, 98, 102, 103, 104, 105, 106, 107, 109, 110, 113, 116, 117, 118, 119, 123, 124, 125, 127, 130, 131, 132, 133, 135, 136, 138, 139, 141, 146, 147, 151, 152, 154, 155, 157, 158, 162, 163, 164, 167, 168, 170], "while": [0, 1, 3, 4, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 26, 29, 30, 31, 32, 33, 35, 37, 39, 59, 75, 81, 83, 84, 87, 90, 94, 98, 101, 102, 104, 107, 108, 109, 111, 112, 113, 117, 119, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 139, 141, 146, 152, 155, 162, 164], "whisper": [141, 142, 152], "whisperencod": 135, "whitespac": 147, "whl": [2, 98, 101], "who": [10, 11, 12, 14, 33, 146], "whole": [1, 11, 33, 87, 133, 147], "whose": [8, 10, 13, 16, 17, 19, 35, 66, 84, 102, 109, 116, 134, 147], "why": [0, 15, 18, 21, 35, 91, 117, 127, 130, 131, 133, 139, 147], "wide": [0, 12, 14, 18, 82, 97, 125, 150, 152, 163], "wide_ep": [20, 92], "wideep": [29, 147], "widespread": 104, "width": [0, 1, 26, 48, 75, 82, 84, 95, 105, 106, 134, 138, 139, 147, 152, 163], "wildcard": 35, "win": 147, "window": [0, 1, 8, 19, 23, 27, 39, 67, 85, 97, 108, 113, 124, 133, 136, 138, 147, 149, 152], "window_s": [67, 105, 147], "windows": 0, "wip": [13, 85, 160], "wireless": 53, "wirelessaccesspoint": 53, "wise": [16, 20, 92, 93, 107, 133, 147, 152], "wish": [83, 98, 109], "with_ssh": 120, "with_traceback": 147, "within": [8, 10, 11, 12, 15, 16, 20, 21, 29, 30, 31, 32, 33, 39, 66, 75, 84, 86, 89, 97, 102, 105, 108, 111, 113, 117, 124, 126, 127, 130, 131, 133, 147, 151, 169], "without": [0, 1, 2, 7, 10, 11, 12, 13, 16, 17, 19, 20, 23, 29, 30, 33, 34, 35, 37, 44, 59, 60, 75, 81, 82, 87, 88, 90, 91, 94, 103, 105, 111, 113, 117, 118, 124, 127, 131, 133, 135, 147, 148, 152, 153, 155, 160, 162, 163, 165, 166], "wkr": 13, "wo": [13, 118, 152], "wo_gemm": [13, 20], "won": [20, 30, 33, 126, 138], "word": [0, 11, 95, 103, 105, 106, 133, 138, 147, 150, 152, 168], "word_dict": 138, "word_embed": 118, "word_embeddings_layernorm": 118, "work": [2, 8, 10, 12, 16, 19, 21, 22, 30, 33, 37, 39, 40, 44, 58, 61, 62, 63, 68, 75, 83, 84, 86, 87, 88, 97, 98, 99, 101, 102, 105, 106, 107, 108, 111, 113, 117, 119, 128, 129, 133, 138, 141, 143, 147, 148, 152, 153, 165], "workaround": [2, 21, 29, 118, 146, 152], "workdir": [27, 61, 62, 63, 98], "worker": [17, 23, 27, 37, 84, 94, 96, 117, 124, 139, 147, 152], "worker_cl": 11, "worker_not_support": 11, "worker_tag": 11, "workerexecutablepath": 0, "workertag": 11, "workflow": [2, 11, 12, 14, 16, 17, 24, 40, 44, 75, 76, 78, 81, 82, 84, 94, 97, 105, 106, 115, 116, 125, 127, 128, 129, 133, 143, 146, 147, 152, 156, 158, 162, 163], "workload": [8, 10, 11, 12, 15, 16, 17, 18, 19, 20, 23, 26, 38, 39, 40, 76, 84, 91, 92, 104, 111, 117, 123, 124, 125, 127, 128, 129, 130, 131, 150, 156], "workspac": [1, 12, 16, 22, 23, 27, 36, 39, 88, 124, 133, 139, 147, 152, 165], "workstat": 4, "world": [0, 2, 8, 14, 16, 21, 23, 30, 39, 61, 62, 63, 77, 78, 87, 88, 107, 124, 125, 126, 127, 133, 150, 157, 158, 165], "world_config": 138, "world_siz": [78, 80, 116, 119, 133, 152, 158, 160, 161], "worldconfig": [0, 106, 138], "worldsiz": 1, "worri": 11, "wors": [20, 23, 113, 127, 136], "worst": [16, 91, 130, 131], "worth": [105, 108, 127, 131], "would": [0, 10, 12, 14, 16, 39, 84, 91, 96, 107, 113, 124, 125, 127, 130, 132, 133, 147, 148, 153], "wpa2": 53, "wqr": 13, "wrap": [0, 1, 23, 117, 125, 133, 136, 138, 146, 152], "wrapped_properti": 147, "wrapper": [1, 10, 16, 29, 30, 31, 32, 33, 75, 82, 107, 119, 155, 163], "write": [0, 1, 11, 12, 13, 16, 23, 58, 66, 109, 118, 133, 143, 152, 160], "write_interv": 147, "written": [11, 12, 22, 117, 124, 133], "wrong": [113, 152], "wsl": 152, "wuk": 13, "wuq": 13, "wuv": 13, "www": 152, "x": [0, 1, 9, 10, 16, 26, 27, 29, 30, 31, 32, 33, 40, 86, 98, 99, 103, 106, 110, 114, 124, 133, 134, 135, 141, 147, 151, 152], "x64": 21, "x86": 109, "x86_64": [101, 142], "xcomposer2": 152, "xf": 24, "xgrammar": [0, 10, 53, 73, 95, 103, 147, 152], "xl": 152, "xml": 103, "xmlcharrefreplac": 147, "xor": 133, "xqa": [97, 152], "xxx": [118, 119, 143], "xxx_plugin": 136, "xy": 133, "y": [2, 6, 10, 11, 16, 39, 81, 93, 98, 99, 101, 103, 120, 124, 133, 135, 141, 147, 162], "y_bia": 133, "yaml": [9, 16, 17, 18, 20, 21, 22, 24, 26, 27, 34, 35, 39, 40, 46, 73, 83, 84, 92, 96, 124, 152, 164], "yarn": [133, 152], "ye": [33, 84, 85, 102, 133, 139, 140, 149], "yeah": [33, 64], "yelp": [142, 149], "yen": [39, 124], "yet": [0, 4, 12, 13, 16, 19, 27, 84, 99, 106, 119, 133, 160, 170], "yield": [8, 11, 15, 19, 44, 109, 127, 130], "yiyixu": [42, 70], "yml": [2, 14, 26, 27, 29, 30, 31, 32, 33, 35, 36, 40, 45, 84, 124], "york": [29, 30, 31, 32, 33, 41, 43, 69, 71, 151, 160], "you": [2, 7, 9, 11, 13, 14, 15, 16, 17, 18, 20, 21, 23, 24, 26, 27, 29, 30, 31, 32, 33, 35, 39, 40, 41, 42, 44, 46, 50, 51, 53, 61, 62, 63, 64, 68, 69, 70, 73, 75, 76, 77, 78, 81, 83, 84, 86, 87, 88, 89, 91, 93, 94, 95, 96, 97, 98, 99, 101, 103, 104, 105, 106, 107, 109, 110, 112, 113, 116, 117, 119, 120, 121, 124, 126, 127, 128, 129, 130, 131, 132, 133, 138, 139, 143, 145, 146, 147, 148, 151, 152, 153, 155, 156, 157, 158, 160, 162, 164, 165, 167], "your": [2, 7, 9, 11, 12, 14, 16, 21, 23, 26, 29, 30, 31, 32, 33, 34, 35, 38, 39, 40, 44, 46, 50, 64, 66, 75, 76, 77, 78, 81, 82, 83, 84, 91, 92, 95, 98, 101, 102, 109, 110, 111, 113, 119, 120, 121, 123, 124, 125, 126, 127, 128, 129, 130, 131, 143, 146, 147, 148, 151, 153, 155, 156, 157, 158, 162, 163, 164, 169], "your_data_path": [2, 14], "your_dockerhub_usernam": [120, 121], "your_model_dir": 14, "your_model_path": [2, 16], "your_public_kei": 121, "your_work_path": 2, "yourself": [76, 156, 167], "yuhuili": 68, "yyi": 143, "z": [11, 98, 99, 133], "zars19": 152, "zero": [0, 1, 10, 59, 90, 103, 118, 133, 134, 141, 143, 146, 147, 166], "zero_is_placehold": 133, "zfill": 147, "zhang": 11, "zhuang": 11, "zip": 58, "zjli2013": 152, "zoo": [21, 60, 152], "zoom": [8, 16], "\u00b5": 20, "\u7f8e\u56fd\u7684\u9996\u90fd\u5728\u54ea\u91cc": 72}, "titles": ["Executor", "Runtime", "How to get best performance on DeepSeek-R1 in TensorRT LLM", "Falcon-180B on a single H200 GPU with INT4 AWQ, and 6.7x faster Llama-70B over A100", "H100 has 4.6x A100 Performance in TensorRT LLM, achieving 10,000 tok/s at 100ms to first token", "H200 achieves nearly 12,000 tokens/sec on Llama2-13B with TensorRT LLM", "New XQA-kernel provides 2.4x more Llama-70B throughput within the same latency budget", "Speed up inference with SOTA quantization techniques in TRT-LLM", "ADP Balance Strategy", "Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM)", "Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly", "Inference Time Compute Implementation in TensorRT LLM", "Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary)", "Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs", "DeepSeek R1 MTP Implementation and Optimization", "Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers", "Scaling Expert Parallelism in TensorRT LLM (Part 1: Design and Implementation of Large-scale EP)", "Disaggregated Serving in TensorRT LLM", "How to launch Llama4 Maverick + Eagle3 TensorRT LLM server", "N-Gram\u202fSpeculative\u202fDecoding\u202fin TensorRT LLM", "Scaling Expert Parallelism in TensorRT LLM (Part 2: Performance Status and Optimization)", "Running a High Performance GPT-OSS-120B Inference Server with TensorRT LLM", "trtllm-bench", "trtllm-build", "trtllm-eval", "trtllm-serve", "Run benchmarking with trtllm-serve auto_deploy pip auto_deploy Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers
diff --git a/latest/torch/auto_deploy/advanced/benchmarking_with_trtllm_bench.html b/latest/torch/auto_deploy/advanced/benchmarking_with_trtllm_bench.html
index b83c78a491..77a7cec2c0 100644
--- a/latest/torch/auto_deploy/advanced/benchmarking_with_trtllm_bench.html
+++ b/latest/torch/auto_deploy/advanced/benchmarking_with_trtllm_bench.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Configuration Examples
@@ -618,7 +614,7 @@ python build_and_run_ad.py The configuration system follows a precedence order in which higher priority sources override lower priority ones:
CLI Arguments (highest priority) - Direct command line arguments
YAML Configs - Files specified via --yaml-configs --args.yaml-configs
YAML Configs - Files specified via --yaml-extra --args.yaml-extra
Default Settings (lowest priority) - Built-in defaults from the config classes
Deep Merging : Unlike simple overwriting, deep merging recursively combines nested dictionaries. For example:
@@ -639,12 +635,12 @@ python build_and_run_ad.py
Nested Config Behavior : When using nested configurations, outer YAML configuration files become initialization settings for inner objects, giving them higher precedence:
-# The outer yaml-configs affects the entire ExperimentConfig
-# The inner args.yaml-configs affects only the AutoDeployConfig
+# The outer yaml-extra affects the entire ExperimentConfig
+# The inner args.yaml-extra affects only the AutoDeployConfig
python build_and_run_ad.py \
--model "meta-llama/Meta-Llama-3.1-8B-Instruct" \
- --yaml-configs experiment_config.yaml \
- --args.yaml-configs autodeploy_config.yaml \
+ --yaml-extra experiment_config.yaml \
+ --args.yaml-extra autodeploy_config.yaml \
--args.world-size= 8 # CLI override beats both YAML configs
build_and_run_ad.py
diff --git a/latest/torch/auto_deploy/advanced/logging.html b/latest/torch/auto_deploy/advanced/logging.html
index cac9a1be41..94876b3a3c 100644
--- a/latest/torch/auto_deploy/advanced/logging.html
+++ b/latest/torch/auto_deploy/advanced/logging.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
-
+
@@ -331,6 +331,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters
+
Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading
@@ -361,6 +362,7 @@
Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware
+
Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -403,6 +405,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -419,6 +422,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
Example autodeploy_config.yaml
-# Compilation backend for AutoDeploy
-compile_backend : torch-opt # options: torch-simple, torch-compile, torch-cudagraph, torch-opt
+# runtime engine
+runtime : trtllm
-# Runtime engine
-runtime : trtllm # options: trtllm, demollm
+# model loading
+skip_loading_weights : false
-# Model loading
-skip_loading_weights : false # set true for architecture-only perf runs
+# Sequence configuration
+max_batch_size : 256
-# KV cache memory
-free_mem_ratio : 0.8 # fraction of free GPU mem for KV cache
+# multi-gpu execution
+world_size : 1
-# CUDA graph optimization
-cuda_graph_batch_sizes : [ 1 , 2 , 4 , 8 , 16 , 32 , 64 ]
-
-# Attention backend
-attn_backend : flashinfer # recommended for best performance
+# transform options
+transforms :
+ insert_cached_attention :
+ # attention backend
+ backend : flashinfer
+ resize_kv_cache :
+ # fraction of free memory to use for kv-caches
+ free_mem_ratio : 0.8
+ compile_model :
+ # compilation backend
+ backend : torch-opt
+ # CUDA Graph optimization
+ cuda_graph_batch_sizes : [ 1 , 2 , 4 , 8 , 16 , 32 , 64 , 128 , 256 ]
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters
+
Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading
@@ -361,6 +362,7 @@
Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware
+
Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -403,6 +405,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -419,6 +422,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers llm = LLM (
model =< HF_MODEL_CARD_OR_DIR > ,
world_size =< DESIRED_WORLD_SIZE > ,
- compile_backend = "torch-compile" ,
- model_kwargs = { "num_hidden_layers" : 2 }, # test with smaller model configuration
- attn_backend = "flashinfer" , # choose between "triton" and "flashinfer"
- attn_page_size = 64 , # page size for attention (tokens_per_block, should be == max_seq_len for triton)
- skip_loading_weights = False ,
model_factory = "AutoModelForCausalLM" , # choose appropriate model factory
- mla_backend = "MultiHeadLatentAttention" , # for models that support MLA
- free_mem_ratio = 0.8 , # fraction of available memory for cache
- simple_shard_only = False , # tensor parallelism sharding strategy
+ model_kwargs = { "num_hidden_layers" : 2 }, # test with smaller model configuration
+ transforms = {
+ "insert_cached_attention" : { "backend" : "flashinfer" }, # or "triton"
+ "insert_cached_mla_attention" : { "backend" : "MultiHeadLatentAttention" },
+ "resize_kv_cache" : { "free_mem_ratio" : 0.8 },
+ "compile_model" : { "backend" : "torch-compile" },
+ "detect_sharding" : { "simple_shard_only" : False },
+
+ },
+ attn_page_size = 64 , # page size for attention
+ skip_loading_weights = False ,
max_seq_len =< MAX_SEQ_LEN > ,
max_batch_size =< MAX_BATCH_SIZE > ,
)
@@ -632,9 +639,9 @@
diff --git a/latest/torch/auto_deploy/auto-deploy.html b/latest/torch/auto_deploy/auto-deploy.html
index e24703299b..8a2a0801ce 100644
--- a/latest/torch/auto_deploy/auto-deploy.html
+++ b/latest/torch/auto_deploy/auto-deploy.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
diff --git a/latest/torch/auto_deploy/support_matrix.html b/latest/torch/auto_deploy/support_matrix.html
index 8d56adb99d..77a81c0ad5 100644
--- a/latest/torch/auto_deploy/support_matrix.html
+++ b/latest/torch/auto_deploy/support_matrix.html
@@ -59,7 +59,7 @@
@@ -69,7 +69,7 @@
-
+
@@ -335,6 +335,7 @@
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters
+
Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading
@@ -365,6 +366,7 @@
Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware
+
Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Feature Combination Matrix — TensorRT LLM
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
- Feature Combination Matrix
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-Feature Combination Matrix
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
@@ -423,6 +426,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -403,6 +405,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype)
+
Ray Orchestrator (Prototype) Developer Guide
Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -403,6 +405,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -419,6 +422,7 @@
Running GPT-OSS-120B with Eagle3 Speculative Decoding on GB200/B200 (TensorRT LLM) Combining Guided Decoding and Speculative Decoding: Making CPU and GPU Cooperate Seamlessly Inference Time Compute Implementation in TensorRT LLM Scaling Expert Parallelism in TensorRT LLM (Part 3: Pushing the Performance Boundary) Pushing Latency Boundaries: Optimizing DeepSeek-R1 Performance on NVIDIA B200 GPUs DeepSeek R1 MTP Implementation and Optimization Optimizing DeepSeek R1 Throughput on NVIDIA Blackwell GPUs: A Deep Dive for Developers Generate text with guided decoding Control generated text using logits processor Generate text with multiple LoRA adapters Sparse Attention Speculative Decoding KV Cache Connector KV Cache Offloading Quick Start Recipe for Llama3.3 70B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for Llama4 Scout 17B on TensorRT LLM - Blackwell & Hopper Hardware Quick Start Recipe for GPT-OSS on TensorRT-LLM - Blackwell Hardware Quick Start Recipe for Qwen3 Next on TensorRT LLM - Blackwell & Hopper Hardware
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
@@ -407,6 +409,7 @@
Speculative Decoding Checkpoint Loading AutoDeploy (Prototype) Ray Orchestrator (Prototype) Developer Guide
+ 
+
+
+ 
+
+
+
+
+
+
+
+
+ 
+
+
+ 