Generate camera rays (origin and direction) in the world coordinate. The function reproduces `o3d.t.geometry.RaycastingScene.create_rays_pinhole` when offset = ''center. Args: cam_poses: (m, 4, 4) homegeneous matrix that transforms the camera coord to world coor
(
cam_poses: torch.Tensor, # (m, 4, 4) target camera pose, cam to world
intrinsics: torch.Tensor, # (m, 3, 3) intrinsic matrix of the camrea
width_px: int,
height_px: int,
subsample: int = 1, # only trace 1 ray every subsample sensor pixels
offsets: T.Union[float, str, torch.Tensor] = 'center',
device=torch.device('cpu'),
)
| 1535 | |
| 1536 | |
| 1537 | def generate_camera_rays( |
| 1538 | cam_poses: torch.Tensor, # (m, 4, 4) target camera pose, cam to world |
| 1539 | intrinsics: torch.Tensor, # (m, 3, 3) intrinsic matrix of the camrea |
| 1540 | width_px: int, |
| 1541 | height_px: int, |
| 1542 | subsample: int = 1, # only trace 1 ray every subsample sensor pixels |
| 1543 | offsets: T.Union[float, str, torch.Tensor] = 'center', |
| 1544 | device=torch.device('cpu'), |
| 1545 | ): |
| 1546 | """ |
| 1547 | Generate camera rays (origin and direction) in the world coordinate. |
| 1548 | The function reproduces `o3d.t.geometry.RaycastingScene.create_rays_pinhole` |
| 1549 | when offset = ''center. |
| 1550 | |
| 1551 | Args: |
| 1552 | cam_poses: |
| 1553 | (m, 4, 4) homegeneous matrix that transforms the camera coord to world coord |
| 1554 | Note that to use the rays to render an image, you can have the y axis inverted in the cam_poses. |
| 1555 | intrinsics: |
| 1556 | (m, 3, 3) intrinsic matrix for each camera pose |
| 1557 | width_px: |
| 1558 | number of pixels on the sensor (before subsample) |
| 1559 | height_px: |
| 1560 | number of pixels on the sensor (before subsample) |
| 1561 | subsample: |
| 1562 | subsample the sensor (camera ray) |
| 1563 | offsets: |
| 1564 | float or (m, h, w) that will be added to the pixel location on the sensor |
| 1565 | If 0 or 'center', ray will be coming from the center of a pixel |
| 1566 | 'rand': offset = [-0.5, 0.5) |
| 1567 | device: |
| 1568 | |
| 1569 | Returns: |
| 1570 | ray_origins_w: (m, h', w', 3) |
| 1571 | ray_directions_w: (m, h', w', 3) normalized to unit norm |
| 1572 | |
| 1573 | Note: |
| 1574 | The function does not flip the y axis (which should be handled by the image coordinate) |
| 1575 | To use the rays to render an image, you can have the y axis inverted in the cam_poses. |
| 1576 | """ |
| 1577 | |
| 1578 | m = cam_poses.size(0) |
| 1579 | |
| 1580 | # generate u v w on the sensor coord |
| 1581 | u, v = torch.meshgrid( |
| 1582 | torch.arange(0, width_px, subsample, dtype=cam_poses.dtype, device=device), |
| 1583 | torch.arange(0, height_px, subsample, dtype=cam_poses.dtype, device=device), |
| 1584 | indexing='xy', |
| 1585 | ) # u: (h', w') for x, v: (h', w') for y in the sensor coord |
| 1586 | u = u + 0.5 # (h', w') |
| 1587 | v = v + 0.5 # (h', w') |
| 1588 | |
| 1589 | uv = torch.stack((u, v), dim=-1).to(dtype=cam_poses.dtype) # (h', w', 2) |
| 1590 | uv_shape = uv.shape # (h', w', 2) |
| 1591 | uv = uv.expand(m, *uv_shape) # (m, h', w', 2) |
| 1592 | if isinstance(offsets, str): |
| 1593 | if offsets == 'center': |
| 1594 | pass |
no test coverage detected