TensorRT
Optimize supported networks and execute the resulting inference engines on NVIDIA GPUs.
Published by NVIDIA
How to use it
Import a supported ONNX graph and build an inference engine with the target input shapes and precision. TensorRT can then execute the compiled engine through its native runtime.
Keep the original model alongside the built engine. Unless you have verified the relevant compatibility options, rebuild the engine for a new runtime version or GPU target and compare output against the original model.
- Hardware
- NVIDIA CUDA
- Model formats
- ONNX, TensorRT plan
- Operating systems
- Linux, Windows
- Access
- Runs locally on a supported NVIDIA GPU with compatible drivers.