If input_dir is None, download pretrained weight from google cloud and load.

take labels of images and sentences as a softlabel e.g., image_label = [1, 0, 1, -1], sentence_label = [0, 0, 1, -1] this pair has sim

do preprocessing to split raw reports into sentence segments for sentence-image contrastive pretraining.

Args: x: (B, H, W, C) window_size (int): window size Returns: windows: (num_windows*B, window_size, window_size, C)

Returns the correct learning rate scheduler. Available scheduler: constantlr, warmupconstant, warmuplinear, warmupcosine, warmupcosinewithhar

pad img to square.

pad img to square.

pad img to square.

pad img to square.

Generate text prompts for each CheXpert classification task Parameters ---------- n: int number of prompts per class Returns

handle key mismatch of medclip and the vision encoder.

handle key mismatch of medclip and the vision encoder.

pad img to square.

output_path: model save path checkpoint_path: model load and continue to learn path

Args: windows: (num_windows*B, window_size, window_size, C) window_size (int): Window size H (int): Height of image

Main method to prepare for the model one or several image(s). <Tip warning={true}> NumPy arrays and PyTorch tensors are con

specify class_names if doing zero-shot classification. mode: `binary`, 'multiclass`, or `multilabel`, if set None, the method will aut

support data list in mimic-cxr-train, chexpert-train

Args: use_EDA: easy data augmentation from textaugment

support data list in mimic-5x200, chexpert-5x200, rsna-balanced-test, covid-test args: imgtransform: a torchvision transform

support data list in mimic-5x200, mimic-5x200-finetune, chexpert-5x200, chexpert-5x200-finetune, rsna-balanced-test, rsna-balanced-train, covi

support data list in mimic-5x200, mimic-5x200-finetune, chexpert-5x200, chexpert-5x200-finetune, rsna-balanced-test, rsna-balanced-train, covi

args: checkpoint: load from the vision encoder checkpoint medclip_checkpoint: load from the vision-text dual encoders checkpoint

args: vision_model: the medclip vision model that encodes input images into embeddings. num_class: number of classes to predict

clean up raw reports into sentences

Args: x: input features with shape of (num_windows*B, N, C) mask: (0/-inf) mask with shape of (num_windows, Wh*Ww, Wh

x: B, H*W, C

args: pixel_values: tensor with shape [bs, 3, img_size, img_size]

args: pixel_values: tensor with shape [bs, 3, img_size, img_size]

take image pixel values (after transform) and prompt_inputs (a dict of {'class1':{'input_ids':...,'attention_mask':,...}), 'class2':...}

take image pixel values (after transform) and prompt_inputs (a dict of {'class1':{'input_ids':...,'attention_mask':,...}), 'class2':...}

args: labels: the image corresponds to which classes of diagnoses text_labels: the text corresponds to which classes of diagnoses