Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/google-research/language
/ functions
Functions
6,154 in github.com/google-research/language
⨍
Functions
6,154
◇
Types & classes
800
↳
Endpoints
12
↓ 1 callers
Function
remove_punc
(text)
language/bert_extraction/steal_bert_qa/data_generation/preprocess_util.py:209
↓ 1 callers
Function
remove_punc
(text)
language/search_agents/muzero/utils.py:43
↓ 1 callers
Function
remove_punc
(text)
language/asqa/scoring.py:40
↓ 1 callers
Function
remove_punc
(text)
language/capwap/utils/metric_utils.py:184
↓ 1 callers
Function
remove_repetive_unigram
(text)
language/labs/exemplar_decoding/utils/data.py:77
↓ 1 callers
Function
rename_top_lf_labels
Rename the labels of the TopLFs in place. Args: lfs: List of TopLF objects. label_map: Mapping from old labels to new labels.
language/casper/augment/casper_formatters.py:141
↓ 1 callers
Function
render_entities_html
Add <b> tags to mark entities in string passage.
language/labs/drkit/hotpotqa/demo.py:959
↓ 1 callers
Function
repetitive_ngrams
(n, text)
language/labs/exemplar_decoding/models/model_function.py:31
↓ 1 callers
Function
replace_punctuation
(s)
language/decontext/eval_util.py:50
↓ 1 callers
Function
rescore_sparse
Rescore mentions in sparse tensor with given batch of queries.
language/labs/drkit/model_fns.py:379
↓ 1 callers
Function
reserved_wordpieces
Returns list of wordpieces reserved with special function.
language/relation_learning/data/fewrel.py:33
↓ 1 callers
Function
resolve_objects
Combine consecutive objects across years into one fact. Args: facts: A list of fact tuples. Returns: a list of fact tuples with consecut
language/templama/templama.py:268
↓ 1 callers
Function
resplit_ct
Resplit the CuratedTrec dataset.
language/orqa/preprocessing/create_data_splits.py:87
↓ 1 callers
Function
resplit_nq
Resplit the Natural Questions dataset.
language/orqa/preprocessing/create_data_splits.py:53
↓ 1 callers
Function
resplit_wb
Resplit the WebQuestions dataset.
language/orqa/preprocessing/create_data_splits.py:70
↓ 1 callers
Function
results_file_metrics
(output_dir, global_step, eval_name)
language/multivec/models/metrics.py:37
↓ 1 callers
Function
results_file_outputs
(output_dir, global_step, eval_name)
language/multivec/models/metrics.py:33
↓ 1 callers
Method
retrieval_score
Return the retrieval score @k for documents_list.
language/search_agents/muzero/types.py:236
↓ 1 callers
Function
retrieve
Do retrieval.
language/orqa/models/orqa_model.py:88
↓ 1 callers
Function
retrieve_latest_revids
Gets the latest revision id for an article.
language/fruit/scripts/get_topics.py:58
↓ 1 callers
Function
retrieve_topics
Gets the topics for a collection of revision ids.
language/fruit/scripts/get_topics.py:82
↓ 1 callers
Method
rir_from_lossy
Prepares data when the input is both lossy and reversible.
language/compir/dataset_parsers/dataset_parser.py:204
↓ 1 callers
Method
rir_to_lossy
Transforms programs to both a lossy and reversible IR.
language/compir/dataset_parsers/dataset_parser.py:132
↓ 1 callers
Function
rnn_attention
A basic RNN with attention. Args: seqs: <tf.float32>[batch_size, total_len, pattern_size] Sequences of patterns, where `total_len` is the
language/labs/memory/baseline_models.py:96
↓ 1 callers
Function
rnn_differentiable_plasticity
Implements differentiable plasticity from Miconi et al. 2018. Differences with FW: (1) learnable plasticity_coeffs (2) learnable fast learning ra
language/labs/memory/differentiable_plasticity.py:30
↓ 1 callers
Function
rnn_explicit_mem
Implements a fixed size memory baseline with a simple r/w rule. Args: seqs: <tf.float32>[batch_size, seq_len, pattern_size] Sequences of
language/labs/memory/explicit_mem.py:25
↓ 1 callers
Function
rnn_fast_weights
Implements a RNN with fast weights. The RNN updates a fast-weights "memory" matrix A via Hebbian-style updates as described by [Ba et al. 2018] U
language/labs/memory/baseline_models.py:167
↓ 1 callers
Function
rouge
Main function for rouge scoring. If two references are provided, the best score is chosen for each instance. Args: hypotheses: {key: answe
language/asqa/scoring.py:95
↓ 1 callers
Function
rouge_l_metric
(predictions, prediction_len, labels, label_len, vocab, use_bpe=False)
language/labs/exemplar_decoding/utils/rouge_utils.py:181
↓ 1 callers
Function
rouge_score
Compute ROUGE score for predictions.
language/capwap/utils/metric_utils.py:310
↓ 1 callers
Function
run_dev_eval
Run evaluation on eval set.
language/multivec/models/ranking_model_experiment_inbatch.py:763
↓ 1 callers
Function
run_episode
Runs and mutates `episode`. Returns MCTS visualizations for each step.
language/search_agents/muzero/run_inference_beam.py:130
↓ 1 callers
Function
run_inference
Determine set of parses given model. The set of output parses will be pruned by score, such that each cell of the chart contains only at most `no
language/compgen/csl/model/inference/inference_parser.py:108
↓ 1 callers
Function
run_main
()
language/realm/example_generator.py:478
↓ 1 callers
Function
run_main
()
language/realm/generate_retrieval_corpus.py:185
↓ 1 callers
Function
run_main
()
language/realm/refresh_doc_embeds.py:193
↓ 1 callers
Function
run_model
Run experiment with tf.estimator.
language/emql/main.py:103
↓ 1 callers
Method
run_model
Initialize and run the model once, perform sanity checks.
language/mentionmemory/tasks/mention_based_entity_qa_task_test.py:174
↓ 1 callers
Function
safe_path
Return a path that is safe to write to by making intermediate dirs. Args: path: Path to ensure is safe Returns: The original path, conve
language/serene/util.py:34
↓ 1 callers
Function
sample_mask_indices
Samples indices to mask. Args: tokens (Tensor): 1-D string Tensor. mask_rate (float): percentage of tokens to mask. mask_blacklist (Ten
language/conpono/reconstruct/preprocess.py:337
↓ 1 callers
Function
sample_rand_from_docs
(other_docs, num)
language/conpono/cpc/preproc/preprocessing_utils.py:121
↓ 1 callers
Function
sample_rand_from_paragraphs
(document, paragraph, num)
language/conpono/cpc/preproc/preprocessing_utils.py:112
↓ 1 callers
Function
sample_rule
Sample a rule. Rules with nonterminals are scaled with nonterminal_coef.
language/compgen/csl/cky/cfg_sampler.py:23
↓ 1 callers
Function
sanitize_char
Optionally normalize chars we don't want in log messages.
language/canine/tydiqa/debug.py:49
↓ 1 callers
Function
sanitize_sentence
Replace all OOV tokens with random tokens from vocab.
language/bert_extraction/steal_bert_classifier/data_generation/preprocess_util.py:112
↓ 1 callers
Function
save_as_tsv
Save the embeddings and their metadata to TSV for embedding projector. Args: embeddings: Numpy 2d Array of embeddings metadata: List of met
language/serene/export_embeddings.py:35
↓ 1 callers
Method
save_dataset
Saves the current generated data to a file.
language/gscan/data/dataset.py:202
↓ 1 callers
Method
save_sharded_array
(array: np.ndarray, filename: str)
language/mentionmemory/tasks/memory_generation_task.py:319
↓ 1 callers
Function
save_tf_examples
Saves examples as TFRecords.
language/multivec/preprocessing/create_training_data.py:1090
↓ 1 callers
Method
save_to_file
(self, filename_prefix)
language/serene/preprocessing.py:633
↓ 1 callers
Function
score
Compares answer to ground truth to return TP / FP stats.
language/labs/drkit/evaluate.py:495
↓ 1 callers
Method
score_application
Computes scores for a single anchored rule applications. Args: wordpiece_encodings: <float>[max_num_wordpieces, bert_dims] applicatio
language/compgen/nqg/model/parser/nqg_model.py:68
↓ 1 callers
Method
score_claim_to_wiki
Score all claims to wikipedia and return the predictions. Args: n: Number of predictions to make per claim Returns: Top ranked w
language/serene/wiki_index.py:312
↓ 1 callers
Function
score_file
Score each line in a file and return the scores.
language/labs/consistent_zero_shot_nmt/bin/t2t_decoder.py:115
↓ 1 callers
Function
score_fn
Returns scalar score for anchored rule application.
language/compgen/nqg/model/parser/inference/inference_wrapper.py:42
↓ 1 callers
Function
score_matrix
Creates a matrix of match scores.
language/fruit/rendering_utils.py:334
↓ 1 callers
Function
score_qa_accuracy
Compute QA metrics. Args: predictions: {qa_key: short_answer} output of ROBERTA predictions of short answers to disambiguated questions
language/asqa/scoring.py:273
↓ 1 callers
Method
score_word
Scores a word as the sum of its n-letter sequences.
language/wino_dict/create_new_words.py:131
↓ 1 callers
Function
select_random
Selecting the given number of items at random.
language/multivec/preprocessing/create_training_data.py:245
↓ 1 callers
Function
send_initial_inference_request
Initial inference for the agent, used at the beginning of MCTS.
language/search_agents/muzero/run_inference_beam.py:63
↓ 1 callers
Method
serialize_para
Create JSON object for input paragraph.
language/labs/drkit/wikidata/preprocessing/distantly_supervise.py:203
↓ 1 callers
Method
serialize_trained
Save the current value of all trainable relations in a file. Args: output_file: Filename string or FileLike object. session: (DEPRECA
language/nql/nql/__init__.py:1459
↓ 1 callers
Function
serving_fn
()
language/orqa/models/orqa_model.py:471
↓ 1 callers
Method
setUp
(self)
language/search_agents/muzero/state_tree_test.py:68
↓ 1 callers
Method
setUp
(self)
language/mentionmemory/utils/mention_utils_test.py:183
↓ 1 callers
Method
setUp
(self)
language/nql/nql/dataset_test.py:32
↓ 1 callers
Method
setUp
(self)
language/fruit/rendering_utils_test.py:149
↓ 1 callers
Function
set_random_seed
Set the random seed from flag everywhere.
language/labs/consistent_zero_shot_nmt/bin/t2t_datagen.py:136
↓ 1 callers
Function
setup_graph
Sets up the Tenorflow graph for inference.
language/xsp/model/run_inference.py:183
↓ 1 callers
Function
shard_id
Get an integer scalar Tensor indicating the index of the current shard.
language/common/utils/tpu_utils.py:37
↓ 1 callers
Method
sharded_iterator
Returns an iterator of iterators of at most size `shard_size`.
language/canine/tydiqa/run_tydi_lib.py:351
↓ 1 callers
Function
shared_qry_encoder_v2
Embed query into a BOW and shared dense representation.
language/labs/drkit/model_fns.py:635
↓ 1 callers
Function
should_skip_context
(e, idx)
language/question_answering/bert_joint/run_nq.py:227
↓ 1 callers
Method
shutdown
Shuts down all relevant resources.
language/realm/parallel.py:113
↓ 1 callers
Function
single_eval
Run one evaluation using given checkpoint.
language/labs/drkit/run_dualencoder_qa.py:1098
↓ 1 callers
Method
sorted_unique_documents
Unique documents seen until 'step' ordered by mr score.
language/search_agents/muzero/types.py:218
↓ 1 callers
Method
source_data_files
Files to be passed to compile_data.
language/labs/consistent_zero_shot_nmt/data_generators/translate_iwslt17.py:224
↓ 1 callers
Function
span_candidates
Generate span candidates. Args: masks: <int32> [num_retrievals, max_sequence_len] max_span_width: int Returns: starts: <int32> [num_
language/orqa/models/orqa_model.py:43
↓ 1 callers
Function
sparse_batched_matmul
Batch multiply sparse tensor x with dense tensor y. Args: x: <tf.float32>[B, M, N] SparseTensor. y: <tf.float32>[B, N, K] DenseTensor. R
language/labs/exemplar_decoding/models/common.py:225
↓ 1 callers
Function
sparse_ragged_mul
Multiply two sparse matrices, one of which is in ragged form. Args: sp_tensor: SparseTensor of shape B x E. ragged_ind: RaggedTensor mappin
language/labs/drkit/model_fns.py:70
↓ 1 callers
Function
sparse_tile_batch
Tile SparseTensor along batch dimension. This is a sparse equivalent for tf.contrib.seq2seq.tile_batch. NOTE: Only rank 3 tensors are supported
language/labs/exemplar_decoding/models/common.py:265
↓ 1 callers
Function
spec_as_shape
Convert a type_spec to a tf shape. Args: spec: a single specification for tuple_generator_builder context: a NQL context Returns: te
language/nql/nql/dataset.py:163
↓ 1 callers
Function
spec_as_tf_type
Convert a type_spec to a tf type. Args: spec: a single specification for tuple_generator_builder Returns: type specification required by
language/nql/nql/dataset.py:146
↓ 1 callers
Function
spider_db_to_foreign_key_tuples
Return list of abstract_sql.ForiegnKeyRelation.
language/xsp/data_preprocessing/abstract_sql_converters.py:313
↓ 1 callers
Function
spider_db_to_table_tuples
Return list of abstract_sql.TableSchema from spider json.
language/xsp/data_preprocessing/abstract_sql_converters.py:280
↓ 1 callers
Function
split_line_by_sentences
(line)
language/conpono/cpc/preproc/raw_books_preproc_pipeline.py:54
↓ 1 callers
Function
split_line_by_sentences
(line)
language/conpono/create_pretrain_data/books_preproc_pipeline.py:84
↓ 1 callers
Function
split_line_by_sentences
(line)
language/conpono/create_pretrain_data/wiki_preproc_pipeline.py:89
↓ 1 callers
Function
split_tables
Splits tables from the text if they exist.
language/fruit/wiki_utils.py:172
↓ 1 callers
Method
split_terms_on_location
(self)
language/gscan/data/grammar.py:83
↓ 1 callers
Function
sq_diff
(var_name)
language/bert_extraction/steal_bert_classifier/utils/model_diff.py:126
↓ 1 callers
Function
sql_spans_to_string
Converts list of SqlSpan tuples to string.
language/xsp/data_preprocessing/abstract_sql.py:456
↓ 1 callers
Function
state_tree_state_part
Computes the state tree part of the BERT state.
language/search_agents/muzero/bert_state_lib.py:142
↓ 1 callers
Function
step_fn
(step: int)
language/mentionmemory/utils/optim_utils.py:45
↓ 1 callers
Function
str_em
Compute STR-EM metric. Args: predictions: {key: answer} dictionary of predicted answers asqa: dict representation of asqa target_keys:
language/asqa/scoring.py:161
↓ 1 callers
Function
string_acc
Computes string accuracy between two SQL queries.
language/xsp/evaluation/official_evaluation.py:75
↓ 1 callers
Function
strip_accents_and_punct
Strips accents from a piece of text.
language/labs/drkit/evaluate.py:513
↓ 1 callers
Function
sub_select_examples
Select a sub-set of the examples if we have a limit on examples.
language/multivec/preprocessing/create_training_data.py:204
↓ 1 callers
Method
submit_from_generator
(self, generator_fn, **generator_kwargs)
language/realm/parallel.py:137
↓ 1 callers
Function
sum_by_batch_position
Sum values per position in the batch.
language/mentionmemory/utils/mention_utils.py:38
← previous
next →
2,601–2,700 of 6,154, ranked by callers