kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
ab5b19e027
TensorRT-LLMs/docs/source/installation/grace-hopper.md
Kaiyu Xie ab5b19e027
Update TensorRT-LLM (#2820)
2025-02-25 21:21:49 +08:00

792 B
Raw Blame History

(grace-hopper)=

Installing on Grace Hopper

  1. Install TensorRT-LLM (tested on Ubuntu 24.04).

    pip3 install torch==2.6.0 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126

sudo apt-get -y install libopenmpi-dev && pip3 install --upgrade pip setuptools && pip3 install tensorrt_llm --extra-index-url https://pypi.nvidia.com

    If using the PyTorch NGC Container image, the prerequisite step for installing CUDA-enabled PyTorch package is not required.

  2. Sanity check the installation by running the following in Python (tested on Python 3.12):

        :language: python
    :linenos: