Rasterize a set of 3D Gaussians (N) to a batch of image planes (C). This function provides a handful features for 3D Gaussian rasterization, which we detail in the following notes. A complete profiling of the these features can be found in the :ref:`profiling` page. .. note::
(
means: Tensor, # [..., N, 3]
quats: Tensor, # [..., N, 4]
scales: Tensor, # [..., N, 3]
opacities: Tensor, # [..., N]
colors: Tensor, # [..., (C,) N, D] or [..., (C,) N, K, 3]
viewmats: Tensor, # [..., C, 4, 4]
Ks: Tensor, # [..., C, 3, 3]
width: int,
height: int,
near_plane: float = 0.01,
far_plane: float = 1e10,
radius_clip: float = 0.0,
eps2d: float = 0.3,
sh_degree: Optional[int] = None,
packed: bool = True,
tile_size: int = 16,
backgrounds: Optional[Tensor] = None,
render_mode: Literal["RGB", "D", "ED", "RGB+D", "RGB+ED"] = "RGB",
sparse_grad: bool = False,
absgrad: bool = False,
rasterize_mode: Literal["classic", "antialiased"] = "classic",
channel_chunk: int = 32,
distributed: bool = False,
camera_model: Literal["pinhole", "ortho", "fisheye", "ftheta"] = "pinhole",
segmented: bool = False,
covars: Optional[Tensor] = None,
with_ut: bool = False,
with_eval3d: bool = False,
# distortion
radial_coeffs: Optional[Tensor] = None, # [..., C, 6] or [..., C, 4]
tangential_coeffs: Optional[Tensor] = None, # [..., C, 2]
thin_prism_coeffs: Optional[Tensor] = None, # [..., C, 4]
ftheta_coeffs: Optional[FThetaCameraDistortionParameters] = None,
# rolling shutter
rolling_shutter: RollingShutterType = RollingShutterType.GLOBAL,
viewmats_rs: Optional[Tensor] = None, # [..., C, 4, 4]
)
| 31 | |
| 32 | |
| 33 | def rasterization( |
| 34 | means: Tensor, # [..., N, 3] |
| 35 | quats: Tensor, # [..., N, 4] |
| 36 | scales: Tensor, # [..., N, 3] |
| 37 | opacities: Tensor, # [..., N] |
| 38 | colors: Tensor, # [..., (C,) N, D] or [..., (C,) N, K, 3] |
| 39 | viewmats: Tensor, # [..., C, 4, 4] |
| 40 | Ks: Tensor, # [..., C, 3, 3] |
| 41 | width: int, |
| 42 | height: int, |
| 43 | near_plane: float = 0.01, |
| 44 | far_plane: float = 1e10, |
| 45 | radius_clip: float = 0.0, |
| 46 | eps2d: float = 0.3, |
| 47 | sh_degree: Optional[int] = None, |
| 48 | packed: bool = True, |
| 49 | tile_size: int = 16, |
| 50 | backgrounds: Optional[Tensor] = None, |
| 51 | render_mode: Literal["RGB", "D", "ED", "RGB+D", "RGB+ED"] = "RGB", |
| 52 | sparse_grad: bool = False, |
| 53 | absgrad: bool = False, |
| 54 | rasterize_mode: Literal["classic", "antialiased"] = "classic", |
| 55 | channel_chunk: int = 32, |
| 56 | distributed: bool = False, |
| 57 | camera_model: Literal["pinhole", "ortho", "fisheye", "ftheta"] = "pinhole", |
| 58 | segmented: bool = False, |
| 59 | covars: Optional[Tensor] = None, |
| 60 | with_ut: bool = False, |
| 61 | with_eval3d: bool = False, |
| 62 | # distortion |
| 63 | radial_coeffs: Optional[Tensor] = None, # [..., C, 6] or [..., C, 4] |
| 64 | tangential_coeffs: Optional[Tensor] = None, # [..., C, 2] |
| 65 | thin_prism_coeffs: Optional[Tensor] = None, # [..., C, 4] |
| 66 | ftheta_coeffs: Optional[FThetaCameraDistortionParameters] = None, |
| 67 | # rolling shutter |
| 68 | rolling_shutter: RollingShutterType = RollingShutterType.GLOBAL, |
| 69 | viewmats_rs: Optional[Tensor] = None, # [..., C, 4, 4] |
| 70 | ) -> Tuple[Tensor, Tensor, Dict]: |
| 71 | """Rasterize a set of 3D Gaussians (N) to a batch of image planes (C). |
| 72 | |
| 73 | This function provides a handful features for 3D Gaussian rasterization, which |
| 74 | we detail in the following notes. A complete profiling of the these features |
| 75 | can be found in the :ref:`profiling` page. |
| 76 | |
| 77 | .. note:: |
| 78 | **Multi-GPU Distributed Rasterization**: This function can be used in a multi-GPU |
| 79 | distributed scenario by setting `distributed` to True. When `distributed` is True, |
| 80 | a subset of total Gaussians could be passed into this function in each rank, and |
| 81 | the function will collaboratively render a set of images using Gaussians from all ranks. Note |
| 82 | to achieve balanced computation, it is recommended (not enforced) to have similar number of |
| 83 | Gaussians in each rank. But we do enforce that the number of cameras to be rendered |
| 84 | in each rank is the same. The function will return the rendered images |
| 85 | corresponds to the input cameras in each rank, and allows for gradients to flow back to the |
| 86 | Gaussians living in other ranks. For the details, please refer to the paper |
| 87 | `On Scaling Up 3D Gaussian Splatting Training <https://arxiv.org/abs/2406.18533>`_. |
| 88 | |
| 89 | .. note:: |
| 90 | **Batch Rasterization**: This function allows for rasterizing a set of 3D Gaussians |
searching dependent graphs…