MCPcopy Create free account

hub / github.com/google-research/language / functions

Functions6,154 in github.com/google-research/language

Methodpostprocess_fn
Function that prepares model's input and output for serialization.
language/mentionmemory/encoders/base_encoder.py:85
Methodpostprocess_fn
Function that prepares model's input and output for serialization.
language/mentionmemory/tasks/downstream_encoder_task.py:62
Methodpostprocess_fn
Function that prepares model's input and output for serialization.
language/mentionmemory/tasks/mention_encoder_task.py:303
Methodpostprocess_program
Postprocesses a predicted SPARQL program.
language/compir/dataset_parsers/cfq_parser.py:189
Methodpostprocess_program
No postprocessing for SCAN programs.
language/compir/dataset_parsers/scan_parser.py:178
Methodpostprocess_program
Replaces back T5 OOV tokens.
language/compir/dataset_parsers/sql_parser.py:70
Functionpostprocess_question
Add a ? at the end of SQuAD questions and start questions with Wh* word.
language/bert_extraction/steal_bert_qa/data_generation/preprocess_util.py:155
Functionpostprocess_question_boolq
Start BoolQ questions with a yes/no question starter word.
language/bert_extraction/steal_bert_qa/data_generation/preprocess_util.py:168
Functionpostprocess_wikidiff
Applies normalization to outputs.
language/fruit/postprocessors.py:24
Functionprecision
Computes the precision of a document set. Args: document_list: The list of document over which to compute the precision. gold_answer: The l
language/search_agents/muzero/utils.py:111
Methodpredict
Scores claim against candidates and returns ordered candidates. Args: claim: The claim to match candidates: The candidates to rank
language/serene/text_matcher.py:87
Functionpredict_and_evaluate
Runs a testing and evaluation loop. Args: config: Configuration to use. workdir: Working directory for checkpoints and TF summaries. If thi
language/gscan/xattn_model/predict.py:195
Methodpredict_fn
Model-specific prediction function. Args: model_config: contains model config hyperparameters. model_params: contains model par
language/mentionmemory/tasks/memory_generation_task.py:80
Functionpredict_step
Compute the prediction for the given model in inference mode.
language/gscan/xattn_model/predict.py:72
Methodpredict_t5
(self, query_string: str)
language/search_agents/t5/run_inference_beam.py:161
Functionpreproc_doc
Convert document to list of TF Examples for binary order classification. Args: document: a CCNews article (ie. a list of sentences) Return
language/conpono/cpc/preproc/ccnews_preproc_pipeline.py:114
Functionpreproc_doc
Convert document to list of TF Examples for binary order classification. Args: document: a CCNews article (ie. a list of sentences) Return
language/conpono/cpc/preproc/raw_books_preproc_pipeline.py:128
Functionpreproc_doc
Convert document to list of TF Examples for binary order classification. Args: document_tuple: a chapter from one book as a list of lines
language/conpono/cpc/preproc/books_preproc_pipeline.py:115
Functionpreproc_doc
Convert document to list of TF Examples for binary order classification. Args: document_tuple: a wikipedia article as a list of lines and a t
language/conpono/cpc/preproc/wiki_preproc_pipeline.py:101
Functionpreproc_doc
Convert document to list of TF Examples for binary order classification. Args: document: a chapter from one book as a list of lines Return
language/conpono/create_pretrain_data/books_preproc_pipeline.py:88
Functionpreproc_doc
Convert document to list of TF Examples for binary order classification. Args: document: a wikipedia article as a list of lines Returns:
language/conpono/create_pretrain_data/wiki_preproc_pipeline.py:97
Methodpreprocess
(self, example)
language/gscan/xattn_model/dataset/gscan_dataset.py:64
Functionpreprocess_example
Returns a preprocessed example according to the funcall format.
language/casper/augment/casper_formatters.py:126
Methodpreprocess_fn
(sample: Dict[Text, tf.Tensor])
language/mentionmemory/utils/data_utils_test.py:40
Methodpreprocess_fn
Performs preprocessing for individual sample.
language/mentionmemory/tasks/mention_memory_task.py:355
Methodpreprocess_fn
Performs preprocessing for individual sample.
language/mentionmemory/tasks/mauto_task.py:170
Methodpreprocess_fn
(sample: Dict[str, tf.Tensor])
language/mentionmemory/tasks/example_task.py:94
Methodpreprocess_fn
Performs preprocessing for individual sample.
language/mentionmemory/tasks/entity_qa_task.py:63
Methodpreprocess_fn
(example: Dict[Text, tf.Tensor])
language/mentionmemory/tasks/text_classifier.py:157
Methodpreprocess_fn
Performs preprocessing for individual sample.
language/mentionmemory/tasks/memory_generation_task.py:130
Methodpreprocess_fn
Performs preprocessing for individual sample.
language/mentionmemory/tasks/mention_encoder_task.py:60
Functionpreprocess_mapper
Model-specific preprocessing of features from the dataset.
language/capwap/datasets/text_dataset.py:33
Functionpreprocess_mapper
Model-specific preprocessing of features from the dataset.
language/capwap/datasets/wsp_dataset.py:91
Functionpreprocess_mapper
Model-specific preprocessing of features from the dataset.
language/capwap/datasets/rc_dataset.py:95
Functionpreprocess_mapper
Model-specific preprocessing of features from the dataset.
language/capwap/datasets/captions_dataset.py:61
Functionpreprocess_mapper
Model-specific preprocessing of features from the dataset.
language/capwap/datasets/vqa_dataset.py:93
Methodpreprocess_program
Switches OOV T5 tokens to in-vocabulary tokens.
language/compir/dataset_parsers/cfq_parser.py:35
Methodpreprocess_program
No preprocessing for SCAN programs.
language/compir/dataset_parsers/scan_parser.py:31
Methodpreprocess_program
Switches OOV T5 tokens and performs lowercasing.
language/compir/dataset_parsers/sql_parser.py:25
Functionpreprocess_text
Tokenize and convert raw text to ids. Args: text: String to be tokenized. tokenizer: An instance of Tokenizer class. Returns: tokens
language/labs/drkit/bert_utils.py:35
Functionpreprocess_text
Tokenize and convert raw text to ids. Args: text: String to be tokenized. tokenizer: An instance of Tokenizer class. Returns: tokens
language/labs/drkit/bert_utils_v2.py:35
Functionpreprocess_with_list_input_fn
(*args)
language/mentionmemory/utils/data_utils.py:144
Methodpretraining_loss
(self, sample, training=True)
language/search_agents/muzero/network.py:65
Methodpretty_print
Pretty-prints BERT inputs for human inspection. NOTE: this method assumes that every attribute of BertInputs is a Numpy array, not a Tensor.
language/conpono/reconstruct/preprocess.py:434
Functionprint_avg
(examples, metrics)
language/quest/eval/eval_utils.py:46
Functionprint_avg_by_template
Prints results partitioned by template.
language/quest/eval/eval_utils.py:40
Functionprint_nested_list
(nested_lists, indent=0)
language/compgen/csl/tasks/smcalflow/tools/generate_target_cfg.py:170
Functionprint_predictions
Prints out a sample of correct and incorrect predictions.
language/fruit/metrics.py:146
Methodprint_relation
Print the distantly supervised relation mention.
language/labs/drkit/wikidata/preprocessing/distantly_supervise.py:198
Functionprint_report
()
language/realm/profile.py:60
Functionprint_table
Print the list of strings with evenly spaced columns.
language/boolq/utils/py_utils.py:53
Functionprint_text
Print text.
language/labs/exemplar_decoding/models/common.py:100
Functionprint_trie
Recursively prints Trie for debugging purposes.
language/compgen/csl/cky/cfg_parser.py:64
Functionprint_trie
Recursively prints Trie for debugging purposes.
language/compgen/nqg/common/cky/trie_utils.py:60
Methodprocess
Convert a json claim to a list of claim-evidence example pairs. Sketch of this method: 1. Get the gold examples for the claim 2. Get the
language/serene/fever_tfds.py:599
Methodprocess
Given a Wikipedia page, return an example for each sentence. Args: wikipedia_url: A Wikipedia url (title) *args: Unused, kept for API
language/serene/wiki_tfds.py:37
Methodprocess
Tries to convert an example, skipping it if conversion fails.
language/compgen/csl/model/data/write_examples_beam.py:57
Methodprocess
(self, elements)
language/compgen/csl/model/inference/eval_model_beam.py:107
Methodprocess
(self, element, *args, **kwargs)
language/search_agents/muzero/run_inference_beam.py:217
Methodprocess
(self, element: str)
language/search_agents/t5/run_inference_beam.py:254
Methodprocess
Processes element.
language/fruit/beam_pipelines.py:367
Methodprocess
(self, element)
language/fruit/beam_pipelines.py:658
Methodprocess
(self, element)
language/fruit/beam_pipelines.py:766
Methodprocess
(self, element)
language/fruit/beam_pipelines.py:793
Methodprocess
(self, example)
language/xsp/model/run_inference.py:310
Methodprocess
(self, example)
language/xsp/data_preprocessing/convert_to_tfrecords.py:200
Functionprocess_annotation
Process annotation dictionary to generate references. Not all annotations will be a reference set to which prediction will be evaluated against.
language/decontext/eval_util.py:200
Functionprocess_batchwise_mention_targets
Processes mention targets and subsamples/pads as necessary. This function does two things. First, it selects which mentions to mark as mentions f
language/mentionmemory/utils/mention_preprocess_utils.py:400
Functionprocess_dataset
Function that processes a dataset without multiprocessing.
language/xsp/data_preprocessing/convert_to_tfrecords.py:456
Methodprocess_el_im_loss
(loss, weight, prefix='')
language/mentionmemory/tasks/mention_memory_task.py:165
Methodprocess_feature
Write a InputFeature to the TFRecordWriter as a tf.train.Example.
language/labs/drkit/input_fns.py:101
Methodprocess_feature
Writes InputFeatures to the TFRecordWriter as a tf.train.Example.
language/canine/tydiqa/tf_io.py:140
Functionprocess_from_queue
Processes items in a queue.
language/realm/parallel.py:28
Functionprocess_metrics
Flatten 2-level dictionary of metrics and divide values by denominator.
language/mentionmemory/utils/metric_utils.py:283
Methodprocess_retrievals
(prefix='')
language/mentionmemory/encoders/mention_memory_encoder.py:92
Functionprocess_snapshot_pipeline
Pipeline for processing Wikipedia snapshots.
language/fruit/beam_pipelines.py:492
Functionprocess_source
(source)
language/compgen/nqg/tasks/spider/nqg_tokenization.py:60
Functionprocess_target
Preprocess target for space-separated tokenization.
language/compgen/nqg/tasks/spider/nqg_tokenization.py:67
Functionprocess_tfr
Processes examples in the TFRecord file src_file and writes to tgt_file. Args: src_file: Source TFRecord file tgt_file: Target TFRecord fil
language/casper/augment/patch_guiding_tag.py:71
Functionprocess_wikilinks
Returns clean text and extracted wikilinks from wikitext.
language/fruit/wiki_utils.py:271
Functionprocess_wikitext
Processes Wikitext to human readable text.
language/fruit/wiki_utils.py:127
Functionprofiled_function
(fn)
language/realm/profile.py:34
Functionpush_examples
Create a server and continuously push examples from a generator. Args: example_generator: Generator that yields examples. port: Port for th
language/realm/preprocessing.py:77
Functionquantize_bbox
Transform bounding box coordinates into a position in an N x N grid. Args: bboxes: <float32> [batch, 4] Object bounding boxes, each with (x_min
language/capwap/utils/image_utils.py:73
Methodquery2box_generator
Convert query2box features to EmQL features. We simply hard-code their input format for parsing, since there's not a simple way to automa
language/emql/data_loader.py:1074
Methodquery_and_docs_feature_structure
Nested structure of the features produced by featurize_query_and_docs. The leaves are all 'placeholders' with fully specified shapes and dtypes.
language/realm/featurization.py:357
Methodquery_layer
(self)
language/labs/exemplar_decoding/models/attention.py:188
Functionragged_sparse_lookup
Extract rows from sparse matrix and return as another sparse matrix. Args: ragged_ind: RaggedTensor mapping entities to mention indices. (E x M
language/labs/drkit/model_fns.py:25
Functionrandom_string
Return a moderately randomized string, possibly with a prefix. Helpful for generating random directories to write different models to Args:
language/serene/util.py:128
Functionrc_example
Builds an Example proto for an image-caption pair.
language/capwap/preprocessing/coco_synthetic_qa.py:73
Functionread
Read jsonl file to a List of Dicts.
language/quest/common/jsonl_utils.py:21
Functionread_candidates_from_one_split
Read candidates from a single jsonl file.
language/canine/tydiqa/postproc.py:39
Functionread_celestial_body
()
language/conpono/evals/coherence_eval.py:111
Functionread_documents
(filepath, limit=None)
language/quest/common/document_utils.py:31
Functionread_entries
Reads TyDi QA examples from JSONL files. Args: input_jsonl_pattern: Glob of the gzipped JSONL files to read. tokenizer: Used to create spec
language/canine/tydiqa/tf_io.py:33
Functionread_examples
(corpus)
language/compgen/nqg/tasks/geoquery/xml_file_utils.py:55
Functionread_examples
(filepath)
language/quest/common/example_utils.py:61
Functionread_file
Read the contents of filename (str) and split into documents by chapter.
language/conpono/cpc/preproc/ccnews_preproc_pipeline.py:49
Functionread_file
Read the contents of filename (str) and split into documents by chapter.
language/conpono/cpc/preproc/raw_books_preproc_pipeline.py:58
Functionread_file
Read the contents of filename (str) and split into documents by chapter.
language/conpono/cpc/preproc/books_preproc_pipeline.py:59
← previousnext →5,001–5,100 of 6,154, ranked by callers