TensorRT-LLMs/examples/openai_triton/README.md
2023-10-10 23:22:17 -07:00

431 B

Integration for OpenAI Triton

The typical approach to integrate a kernel into TensorRT-LLM is to create TensorRT plugins. Specially for integrating OpenAI Triton kernels, there are two methods:

  1. Creating TensorRT plugin manually, you can refer to manual plugin example for details,
  2. Generate the TensorRT plugins automatically, please refer to automatic plugin example for details.