nvdiffrast

nvdiffrast is a library from NVIDIA Research that provides high-performance differentiable rasterization primitives — the fundamental building blocks for constructing custom differentiable rendering pipelines. Rather than offering a complete renderer with built-in shading, materials, and lighting, nvdiffrast exposes four core operations (vertex transformation, rasterization, interpolation, and anti-aliasing) as differentiable modules that researchers compose into application-specific pipelines.

The library's key technical contribution is its approach to differentiable anti-aliasing, which addresses the discontinuity problem that plagues naive differentiable rasterizers. When object silhouettes move across pixel boundaries during gradient-based optimization, standard rasterization produces zero or incorrect gradients. nvdiffrast solves this by analytically computing coverage at silhouette edges, producing clean gradients that enable reliable optimization of geometry, texture, and camera parameters.

nvdiffrast achieves its performance through custom CUDA kernels with OpenGL or Vulkan interop for hardware-accelerated rasterization, making it fast enough for training loops that require millions of rendering iterations. It supports both PyTorch and TensorFlow as frontend frameworks, though the PyTorch path receives more active maintenance. The library has become a standard backend for several high-profile NVIDIA research projects including nvdiffrec (3D reconstruction) and GET3D (3D generative model), and is integrated into NVIDIA Kaolin as the default differentiable rasterization engine.