Interpolate a uv_map given ray_tracing results and a mesh Args: mesh: o3d mesh raycast_results: (*, h', w') the output of ray casting of o3d, texture_maps: a list of texture maps (h, w, c) that we want to get the values using uv_
(
mesh: o3d.geometry.TriangleMesh,
raycast_results: T.Dict[str, T.Any],
texture_maps: T.List[np.ndarray],
merge_textures: bool = False
)
| 93 | |
| 94 | |
| 95 | def interp_texture_map_from_ray_tracing_results( |
| 96 | mesh: o3d.geometry.TriangleMesh, |
| 97 | raycast_results: T.Dict[str, T.Any], |
| 98 | texture_maps: T.List[np.ndarray], |
| 99 | merge_textures: bool = False |
| 100 | ) -> T.List[np.ndarray]: |
| 101 | """ |
| 102 | Interpolate a uv_map given ray_tracing results and a mesh |
| 103 | |
| 104 | Args: |
| 105 | mesh: |
| 106 | o3d mesh |
| 107 | raycast_results: |
| 108 | (*, h', w') the output of ray casting of o3d, |
| 109 | texture_maps: |
| 110 | a list of texture maps (h, w, c) that we want to get the values using uv_mapping |
| 111 | For example, if it is the rgb albedo, it is be retrieved with |
| 112 | :py:`texture = np.asarray(mesh.textures[0]) / 255. # (h,w,3)`. |
| 113 | It can also be surface normals, features, etc. |
| 114 | If `None`, not uv interpolation is performed. |
| 115 | merge_textures: |
| 116 | If True, assume each texture map corresponds to a material_ids in the mesh |
| 117 | For each texture map, only apply if the material_id is matched with the pixel |
| 118 | Sum all contribution from all texture maps |
| 119 | |
| 120 | Returns: |
| 121 | a list of uv-interpolated values from each texture map. (*, dim_texture) |
| 122 | The size `*` is determined by the rays's shape to ray tracing. |
| 123 | If merge_textures is true, return sum of uv-interpolated values from all texture map instead. (dim_texture) |
| 124 | """ |
| 125 | |
| 126 | hit_map = 1 - np.isinf(raycast_results['t_hit'].numpy()) # (*, h',w') |
| 127 | |
| 128 | # barycentric coordinates of the intersection in the intersected triangle |
| 129 | barycentric_coords = raycast_results['primitive_uvs'].numpy() # (*, h',w',2), the third weight is 1-sum() |
| 130 | # (*, h',w',2) -> (*, h',w',3) |
| 131 | barycentric_coords = np.concatenate( |
| 132 | (1 - np.sum(barycentric_coords, axis=-1, keepdims=True), barycentric_coords), |
| 133 | # note that in open 3d, the primitive_uvs indicates last two coordinates rather than first two |
| 134 | # (barycentric_coords, 1 - np.sum(barycentric_coords, axis=-1, keepdims=True)), |
| 135 | axis=-1, |
| 136 | ) # (*, h', w', 3) |
| 137 | |
| 138 | # the intersected triangle index |
| 139 | primitive_ids = raycast_results['primitive_ids'].numpy() # (*, h', w'), |
| 140 | # fillin a dummy primitive_id for the rays that go to inf |
| 141 | primitive_ids[primitive_ids == o3d.t.geometry.RaycastingScene.INVALID_ID] = 0 # (*, h', w') |
| 142 | |
| 143 | # get the uv coordinates of the vertices of each triangle |
| 144 | triangle_uvs = np.asarray(mesh.triangle_uvs) # (num_triangles*3, 2) |
| 145 | triangle_uvs = np.reshape(triangle_uvs, (-1, 3, 2)) # (num_triangles, 3, 2) |
| 146 | |
| 147 | # compute the uv coordinate on the texture map |
| 148 | vertex_uvs = triangle_uvs[primitive_ids] # (*, h', w', 3, 2) |
| 149 | uvs = np.sum(np.expand_dims(barycentric_coords, axis=-1) * vertex_uvs, axis=-2) # (*, h', w', 2) |
| 150 | |
| 151 | # compute the material id for each pixel in the image |
| 152 | if merge_textures: |
no test coverage detected