Makes broadcastable attention and causal masks so that future and masked tokens are ignored. Arguments: attention_mask (

Makes broadcastable attention and causal masks so that future and masked tokens are ignored. Arguments: attention_mask (

Given a URL, returns the URL it redirects to or the original URL in case of no indirection

Get runner class from config. Default to epoch-based runner.

罗列一些 hdfs 路径下的文件。 Args: folders (List): hdfs文件路径的list Returns: 一个list of hdfs 路径

enumerate available model architectures based on config files

Returns the names of available CLIP models

return all models having the specified pretrain tag

Load a CLIP model Parameters ---------- name : str A model name listed by `clip.available_models()`, or the path to a model checkp

Load preprocessor configs and construct preprocessors. If no preprocessor is specified, return BaseProcessor, which does not do any preproce

Prepare masked tokens inputs/labels for masked language modeling: 80% MASK, 10% random, 10% original.

Merge datasets in download_path to storage_path

feature module name accessor

Move files from download_path to storage_path

Move files from download_path to storage_path

Move files from download_path to storage_path

Generate the first token of answers using decoder and select ${num_ans_candidates} most probable ones. Then select answers from answe