kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
release/0.19
TensorRT-LLMs/docs/source/installation/grace-hopper.md
bhsueh_NV 7413cb555a
relax the limitation of setuptools (#2992) 
Signed-off-by: bhsueh <11360707+byshiue@users.noreply.github.com>
2025-03-24 13:36:10 +08:00

750 B
Raw Permalink Blame History

(grace-hopper)=

Installing on Grace Hopper

  1. Install TensorRT-LLM (tested on Ubuntu 24.04).

    pip3 install torch==2.6.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126

sudo apt-get -y install libopenmpi-dev && pip3 install --upgrade pip setuptools && pip3 install tensorrt_llm

    If using the PyTorch NGC Container image, the prerequisite step for installing CUDA-enabled PyTorch package is not required.

  2. Sanity check the installation by running the following in Python (tested on Python 3.12):

        :language: python
    :linenos: