Class HypersimDataset

pointersect/data/hypersim_dataset.py:91–591

Dataset content: https://github.com/apple/ml-hypersim We will load the data to the following convention: For world coordinate: x-axis to right, y-axis to up, and z-axis to us. For image coordinate: u-axis to right, v-axis to down. Origin (0,0,) is at top left of the image. We w

class HypersimDataset:
    """
    Dataset content: https://github.com/apple/ml-hypersim

    We will load the data to the following convention:
    For world coordinate: x-axis to right, y-axis to up, and z-axis to us.
    For image coordinate: u-axis to right, v-axis to down. Origin (0,0,) is at top left of the image.
    We will incorporate the flipping of y->v axis in H_c2w (the camera pose homogeneous matrix
    from the camera coordinate to the world coordinate).
    """

    def __init__(
            self,
            scene_dir: str,
            camera_info: T.Dict[str, T.Any],
            cam_idx: int = -1,
            num_images_per_item: int = None,
            image_subsample: int = 1,
            tonemap: bool = True,
            tonemap_gamma: float = 1. / 2.2,
            tonemap_percentile: float = 90,
            tonemap_percentile_target: float = 0.7,
            tonemap_scale: float = None,
            num_hold_out_items: int = 0,
    ):
        """

        Args:
            scene_dir:
                the root dir of the scene,
                e.g., /mnt/task_runtime/trove/ml-hypersim-1.0.0/data/raw/ai_001_001.
            camera_info:
                a dict containing the corresponding row of metadata_camera_parameters.csv
            cam_idx:
                camera index, e.g., 0, 1.  If -1, combine all cameras.
            num_images_per_item:
                number of images/frame_ids to group and form a single sample.
                If None, use all
                If negative, it will divide the total samples into `-num_images_per_item` samples.
            image_subsample: int = 1

            tonemap_gamma:
                for tone mapping hdr color
            tonemap_percentile:
                for tone mapping hdr color
            tonemap_percentile_target:
                for tone mapping hdr color.  We will set the n-th percentile brightness to the target.

            num_hold_out_items:
                number of hold out items to keep in the last sample
        Ref:
        https://github.com/apple/ml-hypersim/blob/main/contrib/mikeroberts3000/jupyter/01_casting_rays_that_match_hypersim_images.ipynb
        """

        self.scene_dir = scene_dir
        self.camera_info = camera_info
        self.cam_idx = cam_idx
        self.image_subsample = image_subsample