Back-project the center point of the voxel into the depth map. Transform the obtained depth in the camera coordinate system to the world Args: centers (tensor, num_voxels*3): voxel centers c2w (tensor, 4*4): camera coordinate to world coordinate. K (array, 3
(centers, c2w, K, depth, truncation)
| 4 | |
| 5 | |
| 6 | def back_project(centers, c2w, K, depth, truncation): |
| 7 | """ |
| 8 | Back-project the center point of the voxel into the depth map. |
| 9 | Transform the obtained depth in the camera coordinate system to the world |
| 10 | |
| 11 | Args: |
| 12 | centers (tensor, num_voxels*3): voxel centers |
| 13 | c2w (tensor, 4*4): camera coordinate to world coordinate. |
| 14 | K (array, 3*3): camera reference |
| 15 | depth (tensor, w*h): depth ground true. |
| 16 | truncation (float): truncation value. |
| 17 | Returns: |
| 18 | initsdf (tensor,num_voxels): Each vertex of the voxel corresponds to the depth value of the depth map, |
| 19 | if it exceeds the boundary, it will be 0 |
| 20 | seen_iter_mask (tensor,num_voxels): True if two points match |
| 21 | (1).The voxel is mapped to the corresponding pixel in the image, and does not exceed the image boundary |
| 22 | (2).The initialized sdf value should be within the cutoff distance. |
| 23 | """ |
| 24 | H, W = depth.shape |
| 25 | w2c = torch.linalg.inv(c2w.float()) |
| 26 | K = torch.from_numpy(K).cuda() |
| 27 | ones = torch.ones_like(centers[:, 0]).reshape(-1, 1).float() |
| 28 | homo_points = torch.cat([centers, ones], dim=-1).unsqueeze(-1).float() |
| 29 | homo_cam_points = w2c @ homo_points # (N,4,1) = (4,4) * (N,4,1) |
| 30 | cam_points = homo_cam_points[:, :3] # (N,3,1) |
| 31 | uv = K.float() @ cam_points.float() |
| 32 | z = uv[:, -1:] + 1e-8 |
| 33 | uv = uv[:, :2] / z # (N,2) |
| 34 | uv = uv.round() |
| 35 | cur_mask_seen = (uv[:, 0] < W) & (uv[:, 0] > 0) & (uv[:, 1] < H) & (uv[:, 1] > 0) |
| 36 | cur_mask_seen = (cur_mask_seen & (z[:, :, 0] > 0)).reshape(-1) # (N_mask,1) -> (N_mask) |
| 37 | uv = (uv[cur_mask_seen].int()).squeeze(-1) # (N_mask,2) |
| 38 | depth = depth.transpose(-1, -2) # (W,H) |
| 39 | |
| 40 | initsdf = torch.zeros((centers.shape[0], 1), device=centers.device) |
| 41 | |
| 42 | voxel_depth = torch.index_select(depth, dim=0, index=uv[:, 0]).gather(dim=1, index=uv[:, 1].reshape(-1, |
| 43 | 1).long()) # (N_mask,1) |
| 44 | |
| 45 | initsdf[cur_mask_seen] = (voxel_depth - cam_points[cur_mask_seen][:, 2]) / truncation # (N,1) |
| 46 | seen_iter_mask = cur_mask_seen |
| 47 | |
| 48 | return initsdf.squeeze(-1), seen_iter_mask |
| 49 | |
| 50 | |
| 51 | @torch.no_grad() |