mirror of
https://github.com/NVIDIA/TensorRT-LLM.git
synced 2026-01-14 06:27:45 +08:00
4e370a509a
* refactor: Copy sequence lengths once in decoder setup Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com> * refactor: Update DecoderInputBuffers to remove duplicated buffers - Renamed and reorganized buffer variables in decoderBuffers.h and decoderBuffers.cpp for better readability. - Adjusted references in generateRequestOptions.cpp to align with the new buffer structure. Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com> * refactor: Move getEmbeddingBias to anonymous namespace Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com> * refactor: Filter context requests Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com> * refactor: GenerateRequestOptions using more fine-grained functions - Added a new method `createDecoderRequests` to encapsulate the logic for creating decoder requests from finished context requests. - Updated the `operator()` method to utilize the new method, improving code clarity and maintainability. Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com> Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com> * refactor: Update TRTLLMDecoder - Updated the `generate_request_options` call. - Updated the `make_decoding_batch_input_output` call. Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com> * refactor: Remove const where we modify input buffers - Changed `DecoderInputBuffers` parameters from const references to non-const references in multiple functions to allow modifications. - Updated related function calls to ensure compatibility with the new parameter types. Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com> * fixup! refactor: Copy sequence lengths once in decoder setup Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com> --------- Signed-off-by: Robin Kobus <19427718+Funatiq@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| _torch | ||
| auto_parallel | ||
| bench | ||
| commands | ||
| evaluate | ||
| executor | ||
| inputs | ||
| layers | ||
| llmapi | ||
| models | ||
| plugin | ||
| quantization | ||
| runtime | ||
| scaffolding | ||
| serve | ||
| tools | ||
| __init__.py | ||
| _common.py | ||
| _dlpack_utils.py | ||
| _ipc_utils.py | ||
| _mnnvl_utils.py | ||
| _utils.py | ||
| builder.py | ||
| disaggregated_params.py | ||
| functional.py | ||
| graph_rewriting.py | ||
| logger.py | ||
| lora_manager.py | ||
| mapping.py | ||
| module.py | ||
| network.py | ||
| parameter.py | ||
| profiler.py | ||
| prompt_adapter_manager.py | ||
| python_plugin.py | ||
| sampling_params.py | ||
| top_model_mixin.py | ||
| version.py | ||