Convert the video from its original fps to the target_fps. If the video support selective decoding (contain decoding information in the video head), the perform temporal selective decoding and sample a clip from the video with the PyAV decoder. If the video does not support selectiv
(
container,
sampling_rate,
num_frames,
clip_idx,
num_clips=10,
target_fps=30,
use_offset=False,
)
| 249 | |
| 250 | |
| 251 | def pyav_decode( |
| 252 | container, |
| 253 | sampling_rate, |
| 254 | num_frames, |
| 255 | clip_idx, |
| 256 | num_clips=10, |
| 257 | target_fps=30, |
| 258 | use_offset=False, |
| 259 | ): |
| 260 | """ |
| 261 | Convert the video from its original fps to the target_fps. If the video |
| 262 | support selective decoding (contain decoding information in the video head), |
| 263 | the perform temporal selective decoding and sample a clip from the video |
| 264 | with the PyAV decoder. If the video does not support selective decoding, |
| 265 | decode the entire video. |
| 266 | |
| 267 | Args: |
| 268 | container (container): pyav container. |
| 269 | sampling_rate (int): frame sampling rate (interval between two sampled |
| 270 | frames. |
| 271 | num_frames (int): number of frames to sample. |
| 272 | clip_idx (int): if clip_idx is -1, perform random temporal sampling. If |
| 273 | clip_idx is larger than -1, uniformly split the video to num_clips |
| 274 | clips, and select the clip_idx-th video clip. |
| 275 | num_clips (int): overall number of clips to uniformly sample from the |
| 276 | given video. |
| 277 | target_fps (int): the input video may has different fps, convert it to |
| 278 | the target video fps before frame sampling. |
| 279 | Returns: |
| 280 | frames (tensor): decoded frames from the video. Return None if the no |
| 281 | video stream was found. |
| 282 | fps (float): the number of frames per second of the video. |
| 283 | decode_all_video (bool): If True, the entire video was decoded. |
| 284 | """ |
| 285 | # Try to fetch the decoding information from the video head. Some of the |
| 286 | # videos does not support fetching the decoding information, for that case |
| 287 | # it will get None duration. |
| 288 | fps = float(container.streams.video[0].average_rate) |
| 289 | frames_length = container.streams.video[0].frames |
| 290 | duration = container.streams.video[0].duration |
| 291 | |
| 292 | if duration is None: |
| 293 | # If failed to fetch the decoding information, decode the entire video. |
| 294 | decode_all_video = True |
| 295 | video_start_pts, video_end_pts = 0, math.inf |
| 296 | else: |
| 297 | # Perform selective decoding. |
| 298 | decode_all_video = False |
| 299 | start_idx, end_idx = get_start_end_idx( |
| 300 | frames_length, |
| 301 | sampling_rate * num_frames / target_fps * fps, |
| 302 | clip_idx, |
| 303 | num_clips, |
| 304 | use_offset=use_offset, |
| 305 | ) |
| 306 | timebase = duration / frames_length |
| 307 | video_start_pts = int(start_idx * timebase) |
| 308 | video_end_pts = int(end_idx * timebase) |
no test coverage detected