kanshan/TensorRT-LLMs
1
0
Fork 0
mirror of https://github.com/NVIDIA/TensorRT-LLM.git synced 2026-01-14 06:27:45 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
16d2467ea8
TensorRT-LLMs/docs/source/installation/grace-hopper.md
Dan Blanaru 16d2467ea8 Update TensorRT-LLM (#2755) 
* Update TensorRT-LLM

---------

Co-authored-by: Denis Kayshev <topenkoff@gmail.com>
Co-authored-by: akhoroshev <arthoroshev@gmail.com>
Co-authored-by: Patrick Reiter Horn <patrick.horn@gmail.com>

Update
2025-02-11 03:01:00 +00:00

730 B
Raw Blame History

(grace-hopper)=

Installing on Grace Hopper

  1. Install TensorRT-LLM (tested on Ubuntu 24.04).

    pip3 install torch==2.5.1 torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

sudo apt-get -y install libopenmpi-dev && pip3 install --upgrade setuptools && pip3 install tensorrt_llm

    If using the PyTorch NGC Container image, the prerequisite step for installing CUDA-enabled PyTorch package is not required.

  2. Sanity check the installation by running the following in Python (tested on Python 3.12):

    :language: python
    :linenos: