Want smarter insights in your inbox? Sign up for our weekly newsletters to get only what matters to enterprise AI, data, and security leaders. Subscribe Now
Facebook AI Research (FAIR) today unveiled PyTorch3D, a library that enables researchers and developers to combine deep learning and 3D objects.
As part of the release, Facebook is also open-sourcing Mesh R-CNN, a model introduced last year capable of rendering 3D objects from 2D shapes in images of interior spaces. PyTorch3D was inspired by Mesh R-CNN and recent 3D work by Facebook AI Research, FAIR engineer Nikhila Ravi said.
Working in 3D is important for rendering 3D objects or scenes that appear in mixed reality or virtual reality. It can also be used to tackle AI challenges like robotic grasping or helping autonomous vehicles understand the position of nearby objects.
PyTorch3D comes with frequently used 3D operators and loss functions for 3D data and a differentiable mesh renderer for creating 3D objects. PyTorch3D also has a differentiable rendering API, some CUDA support, and heterogeneous batching capabilities unavailable in any existing 3D library, Ravi told VentureBeat in a phone interview.
AI Scaling Hits Its Limits
Power caps, rising token costs, and inference delays are reshaping enterprise AI. Join our exclusive salon to discover how top teams are:
- Turning energy into a strategic advantage
- Architecting efficient inference for real throughput gains
- Unlocking competitive ROI with sustainable AI systems
Secure your spot to stay ahead: https://bit.ly/4mwGngO
“With PyTorch3D, researchers can input all these functions and use them with the existing deep learning system in PyTorch and it greatly reduces on the time to work on 3D planning research, which requires a lot of expertise in order to get started, and we want to try and reduce that ramp-up time,” she said.
PyTorch3D uses meshes, a data format for interoperability of vertices and faces that make up 3D objects, and can use a patch tensor to collapse all vertices for meshes in a batch into a single tensor involved with batching, a common process for deep learning research.
The premiere of PyTorch3D follows the launch of the PyRobot robotics framework last year, and FAIR 3D research that extracts characters from real-world videos.
