Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/google-research/language
/ functions
Functions
6,154 in github.com/google-research/language
⨍
Functions
6,154
◇
Types & classes
800
↳
Endpoints
12
Method
postprocess_fn
Function that prepares model's input and output for serialization.
language/mentionmemory/encoders/base_encoder.py:85
Method
postprocess_fn
Function that prepares model's input and output for serialization.
language/mentionmemory/tasks/downstream_encoder_task.py:62
Method
postprocess_fn
Function that prepares model's input and output for serialization.
language/mentionmemory/tasks/mention_encoder_task.py:303
Method
postprocess_program
Postprocesses a predicted SPARQL program.
language/compir/dataset_parsers/cfq_parser.py:189
Method
postprocess_program
No postprocessing for SCAN programs.
language/compir/dataset_parsers/scan_parser.py:178
Method
postprocess_program
Replaces back T5 OOV tokens.
language/compir/dataset_parsers/sql_parser.py:70
Function
postprocess_question
Add a ? at the end of SQuAD questions and start questions with Wh* word.
language/bert_extraction/steal_bert_qa/data_generation/preprocess_util.py:155
Function
postprocess_question_boolq
Start BoolQ questions with a yes/no question starter word.
language/bert_extraction/steal_bert_qa/data_generation/preprocess_util.py:168
Function
postprocess_wikidiff
Applies normalization to outputs.
language/fruit/postprocessors.py:24
Function
precision
Computes the precision of a document set. Args: document_list: The list of document over which to compute the precision. gold_answer: The l
language/search_agents/muzero/utils.py:111
Method
predict
Scores claim against candidates and returns ordered candidates. Args: claim: The claim to match candidates: The candidates to rank
language/serene/text_matcher.py:87
Function
predict_and_evaluate
Runs a testing and evaluation loop. Args: config: Configuration to use. workdir: Working directory for checkpoints and TF summaries. If thi
language/gscan/xattn_model/predict.py:195
Method
predict_fn
Model-specific prediction function. Args: model_config: contains model config hyperparameters. model_params: contains model par
language/mentionmemory/tasks/memory_generation_task.py:80
Function
predict_step
Compute the prediction for the given model in inference mode.
language/gscan/xattn_model/predict.py:72
Method
predict_t5
(self, query_string: str)
language/search_agents/t5/run_inference_beam.py:161
Function
preproc_doc
Convert document to list of TF Examples for binary order classification. Args: document: a CCNews article (ie. a list of sentences) Return
language/conpono/cpc/preproc/ccnews_preproc_pipeline.py:114
Function
preproc_doc
Convert document to list of TF Examples for binary order classification. Args: document: a CCNews article (ie. a list of sentences) Return
language/conpono/cpc/preproc/raw_books_preproc_pipeline.py:128
Function
preproc_doc
Convert document to list of TF Examples for binary order classification. Args: document_tuple: a chapter from one book as a list of lines
language/conpono/cpc/preproc/books_preproc_pipeline.py:115
Function
preproc_doc
Convert document to list of TF Examples for binary order classification. Args: document_tuple: a wikipedia article as a list of lines and a t
language/conpono/cpc/preproc/wiki_preproc_pipeline.py:101
Function
preproc_doc
Convert document to list of TF Examples for binary order classification. Args: document: a chapter from one book as a list of lines Return
language/conpono/create_pretrain_data/books_preproc_pipeline.py:88
Function
preproc_doc
Convert document to list of TF Examples for binary order classification. Args: document: a wikipedia article as a list of lines Returns:
language/conpono/create_pretrain_data/wiki_preproc_pipeline.py:97
Method
preprocess
(self, example)
language/gscan/xattn_model/dataset/gscan_dataset.py:64
Function
preprocess_example
Returns a preprocessed example according to the funcall format.
language/casper/augment/casper_formatters.py:126
Method
preprocess_fn
(sample: Dict[Text, tf.Tensor])
language/mentionmemory/utils/data_utils_test.py:40
Method
preprocess_fn
Performs preprocessing for individual sample.
language/mentionmemory/tasks/mention_memory_task.py:355
Method
preprocess_fn
Performs preprocessing for individual sample.
language/mentionmemory/tasks/mauto_task.py:170
Method
preprocess_fn
(sample: Dict[str, tf.Tensor])
language/mentionmemory/tasks/example_task.py:94
Method
preprocess_fn
Performs preprocessing for individual sample.
language/mentionmemory/tasks/entity_qa_task.py:63
Method
preprocess_fn
(example: Dict[Text, tf.Tensor])
language/mentionmemory/tasks/text_classifier.py:157
Method
preprocess_fn
Performs preprocessing for individual sample.
language/mentionmemory/tasks/memory_generation_task.py:130
Method
preprocess_fn
Performs preprocessing for individual sample.
language/mentionmemory/tasks/mention_encoder_task.py:60
Function
preprocess_mapper
Model-specific preprocessing of features from the dataset.
language/capwap/datasets/text_dataset.py:33
Function
preprocess_mapper
Model-specific preprocessing of features from the dataset.
language/capwap/datasets/wsp_dataset.py:91
Function
preprocess_mapper
Model-specific preprocessing of features from the dataset.
language/capwap/datasets/rc_dataset.py:95
Function
preprocess_mapper
Model-specific preprocessing of features from the dataset.
language/capwap/datasets/captions_dataset.py:61
Function
preprocess_mapper
Model-specific preprocessing of features from the dataset.
language/capwap/datasets/vqa_dataset.py:93
Method
preprocess_program
Switches OOV T5 tokens to in-vocabulary tokens.
language/compir/dataset_parsers/cfq_parser.py:35
Method
preprocess_program
No preprocessing for SCAN programs.
language/compir/dataset_parsers/scan_parser.py:31
Method
preprocess_program
Switches OOV T5 tokens and performs lowercasing.
language/compir/dataset_parsers/sql_parser.py:25
Function
preprocess_text
Tokenize and convert raw text to ids. Args: text: String to be tokenized. tokenizer: An instance of Tokenizer class. Returns: tokens
language/labs/drkit/bert_utils.py:35
Function
preprocess_text
Tokenize and convert raw text to ids. Args: text: String to be tokenized. tokenizer: An instance of Tokenizer class. Returns: tokens
language/labs/drkit/bert_utils_v2.py:35
Function
preprocess_with_list_input_fn
(*args)
language/mentionmemory/utils/data_utils.py:144
Method
pretraining_loss
(self, sample, training=True)
language/search_agents/muzero/network.py:65
Method
pretty_print
Pretty-prints BERT inputs for human inspection. NOTE: this method assumes that every attribute of BertInputs is a Numpy array, not a Tensor.
language/conpono/reconstruct/preprocess.py:434
Function
print_avg
(examples, metrics)
language/quest/eval/eval_utils.py:46
Function
print_avg_by_template
Prints results partitioned by template.
language/quest/eval/eval_utils.py:40
Function
print_nested_list
(nested_lists, indent=0)
language/compgen/csl/tasks/smcalflow/tools/generate_target_cfg.py:170
Function
print_predictions
Prints out a sample of correct and incorrect predictions.
language/fruit/metrics.py:146
Method
print_relation
Print the distantly supervised relation mention.
language/labs/drkit/wikidata/preprocessing/distantly_supervise.py:198
Function
print_report
()
language/realm/profile.py:60
Function
print_table
Print the list of strings with evenly spaced columns.
language/boolq/utils/py_utils.py:53
Function
print_text
Print text.
language/labs/exemplar_decoding/models/common.py:100
Function
print_trie
Recursively prints Trie for debugging purposes.
language/compgen/csl/cky/cfg_parser.py:64
Function
print_trie
Recursively prints Trie for debugging purposes.
language/compgen/nqg/common/cky/trie_utils.py:60
Method
process
Convert a json claim to a list of claim-evidence example pairs. Sketch of this method: 1. Get the gold examples for the claim 2. Get the
language/serene/fever_tfds.py:599
Method
process
Given a Wikipedia page, return an example for each sentence. Args: wikipedia_url: A Wikipedia url (title) *args: Unused, kept for API
language/serene/wiki_tfds.py:37
Method
process
Tries to convert an example, skipping it if conversion fails.
language/compgen/csl/model/data/write_examples_beam.py:57
Method
process
(self, elements)
language/compgen/csl/model/inference/eval_model_beam.py:107
Method
process
(self, element, *args, **kwargs)
language/search_agents/muzero/run_inference_beam.py:217
Method
process
(self, element: str)
language/search_agents/t5/run_inference_beam.py:254
Method
process
Processes element.
language/fruit/beam_pipelines.py:367
Method
process
(self, element)
language/fruit/beam_pipelines.py:658
Method
process
(self, element)
language/fruit/beam_pipelines.py:766
Method
process
(self, element)
language/fruit/beam_pipelines.py:793
Method
process
(self, example)
language/xsp/model/run_inference.py:310
Method
process
(self, example)
language/xsp/data_preprocessing/convert_to_tfrecords.py:200
Function
process_annotation
Process annotation dictionary to generate references. Not all annotations will be a reference set to which prediction will be evaluated against.
language/decontext/eval_util.py:200
Function
process_batchwise_mention_targets
Processes mention targets and subsamples/pads as necessary. This function does two things. First, it selects which mentions to mark as mentions f
language/mentionmemory/utils/mention_preprocess_utils.py:400
Function
process_dataset
Function that processes a dataset without multiprocessing.
language/xsp/data_preprocessing/convert_to_tfrecords.py:456
Method
process_el_im_loss
(loss, weight, prefix='')
language/mentionmemory/tasks/mention_memory_task.py:165
Method
process_feature
Write a InputFeature to the TFRecordWriter as a tf.train.Example.
language/labs/drkit/input_fns.py:101
Method
process_feature
Writes InputFeatures to the TFRecordWriter as a tf.train.Example.
language/canine/tydiqa/tf_io.py:140
Function
process_from_queue
Processes items in a queue.
language/realm/parallel.py:28
Function
process_metrics
Flatten 2-level dictionary of metrics and divide values by denominator.
language/mentionmemory/utils/metric_utils.py:283
Method
process_retrievals
(prefix='')
language/mentionmemory/encoders/mention_memory_encoder.py:92
Function
process_snapshot_pipeline
Pipeline for processing Wikipedia snapshots.
language/fruit/beam_pipelines.py:492
Function
process_source
(source)
language/compgen/nqg/tasks/spider/nqg_tokenization.py:60
Function
process_target
Preprocess target for space-separated tokenization.
language/compgen/nqg/tasks/spider/nqg_tokenization.py:67
Function
process_tfr
Processes examples in the TFRecord file src_file and writes to tgt_file. Args: src_file: Source TFRecord file tgt_file: Target TFRecord fil
language/casper/augment/patch_guiding_tag.py:71
Function
process_wikilinks
Returns clean text and extracted wikilinks from wikitext.
language/fruit/wiki_utils.py:271
Function
process_wikitext
Processes Wikitext to human readable text.
language/fruit/wiki_utils.py:127
Function
profiled_function
(fn)
language/realm/profile.py:34
Function
push_examples
Create a server and continuously push examples from a generator. Args: example_generator: Generator that yields examples. port: Port for th
language/realm/preprocessing.py:77
Function
quantize_bbox
Transform bounding box coordinates into a position in an N x N grid. Args: bboxes: <float32> [batch, 4] Object bounding boxes, each with (x_min
language/capwap/utils/image_utils.py:73
Method
query2box_generator
Convert query2box features to EmQL features. We simply hard-code their input format for parsing, since there's not a simple way to automa
language/emql/data_loader.py:1074
Method
query_and_docs_feature_structure
Nested structure of the features produced by featurize_query_and_docs. The leaves are all 'placeholders' with fully specified shapes and dtypes.
language/realm/featurization.py:357
Method
query_layer
(self)
language/labs/exemplar_decoding/models/attention.py:188
Function
ragged_sparse_lookup
Extract rows from sparse matrix and return as another sparse matrix. Args: ragged_ind: RaggedTensor mapping entities to mention indices. (E x M
language/labs/drkit/model_fns.py:25
Function
random_string
Return a moderately randomized string, possibly with a prefix. Helpful for generating random directories to write different models to Args:
language/serene/util.py:128
Function
rc_example
Builds an Example proto for an image-caption pair.
language/capwap/preprocessing/coco_synthetic_qa.py:73
Function
read
Read jsonl file to a List of Dicts.
language/quest/common/jsonl_utils.py:21
Function
read_candidates_from_one_split
Read candidates from a single jsonl file.
language/canine/tydiqa/postproc.py:39
Function
read_celestial_body
()
language/conpono/evals/coherence_eval.py:111
Function
read_documents
(filepath, limit=None)
language/quest/common/document_utils.py:31
Function
read_entries
Reads TyDi QA examples from JSONL files. Args: input_jsonl_pattern: Glob of the gzipped JSONL files to read. tokenizer: Used to create spec
language/canine/tydiqa/tf_io.py:33
Function
read_examples
(corpus)
language/compgen/nqg/tasks/geoquery/xml_file_utils.py:55
Function
read_examples
(filepath)
language/quest/common/example_utils.py:61
Function
read_file
Read the contents of filename (str) and split into documents by chapter.
language/conpono/cpc/preproc/ccnews_preproc_pipeline.py:49
Function
read_file
Read the contents of filename (str) and split into documents by chapter.
language/conpono/cpc/preproc/raw_books_preproc_pipeline.py:58
Function
read_file
Read the contents of filename (str) and split into documents by chapter.
language/conpono/cpc/preproc/books_preproc_pipeline.py:59
← previous
next →
5,001–5,100 of 6,154, ranked by callers