mirror of
https://github.com/NVIDIA/TensorRT-LLM.git
synced 2026-01-14 06:27:45 +08:00
bc5811da65
* initial ucx support Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> * fixes to support dynloading and ucx connection establishment - not stable yet Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> * update Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> * more connection bringup fixes - faillig on connection vector build Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> * executor test pass Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> * update Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> * passed full benchmark Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> * changing to TLLM_THROW and removing cout Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> * stoping progress thread at ucxComm destructor Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> * fixing build with ENABLE_UCX=0 to not build ucx traget at all and removing includes for ucxConnection for cache transceiver, also delete commented cold code Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> * fix copyrights Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> * adding ucx flavor to cache transceiver test and insertto the CI pipeline Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> * allowing sending non ib interfaces IPs Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> * setting UCX port reuse for the tests in pipeline Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> * code review fixes Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> * querying ep after GID message is sent to avoid UCX Errors Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> * fixing more CR issues Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> * querying ep to not fail is ep_not_connected yet Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> * remove mpi dependency and debug Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com> * debug to info Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com> * mpirun n 2 Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com> * remove mpi comm split when disaggOrchestrator mode Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com> * waive disagg_mtp test Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com> * use future instead of thread Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com> * use future_promise instead of cv wait Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com> * connectionId type Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com> * improve test Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com> * imporve test 2 Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com> * gtest_skip Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com> --------- Signed-off-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com> Co-authored-by: roeya <165803633+RoeyAzran1992@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| utils | ||
| allocateKvCache.cpp | ||
| assignReqSeqSlots.cpp | ||
| cacheFormatter.cpp | ||
| cacheFormatter.h | ||
| cacheTransceiver.cpp | ||
| capacityScheduler.cpp | ||
| CMakeLists.txt | ||
| contextProgress.cpp | ||
| createNewDecoderRequests.cpp | ||
| dataTransceiver.cpp | ||
| dataTransceiver.h | ||
| dataTransceiverImpl.cpp | ||
| dataTransceiverImpl.h | ||
| decoderBuffers.cpp | ||
| encoderBuffers.cpp | ||
| encoderBuffers.h | ||
| evictionPolicy.cpp | ||
| generateRequestOptions.cpp | ||
| guidedDecoder.cpp | ||
| handleContextLogits.cpp | ||
| handleGenerationLogits.cpp | ||
| kvCacheEventManager.cpp | ||
| kvCacheManager.cpp | ||
| kvCacheTransferManager.cpp | ||
| llmRequest.cpp | ||
| logitsPostProcessor.cpp | ||
| loraBuffers.cpp | ||
| loraBuffers.h | ||
| makeDecodingBatchInputOutput.cpp | ||
| medusaBuffers.cpp | ||
| microBatchScheduler.cpp | ||
| mlaCacheFormatter.cpp | ||
| mlaCacheFormatter.h | ||
| pauseRequests.cpp | ||
| peftCacheManager.cpp | ||
| promptTuningBuffers.cpp | ||
| promptTuningBuffers.h | ||
| rnnStateBuffers.cpp | ||
| rnnStateBuffers.h | ||
| rnnStateManager.cpp | ||
| runtimeBuffers.cpp | ||
| sequenceSlotManager.cpp | ||
| transformerBuffers.cpp | ||
| trtEncoderModel.cpp | ||
| trtEncoderModel.h | ||
| trtGptModel.h | ||
| trtGptModelFactory.h | ||
| trtGptModelInflightBatching.cpp | ||
| trtGptModelInflightBatching.h | ||
| trtGptModelV1.cpp | ||
| trtGptModelV1.h | ||