Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/google-research/language
/ functions
Functions
6,154 in github.com/google-research/language
⨍
Functions
6,154
◇
Types & classes
800
↳
Endpoints
12
Function
read_file
Read the contents of filename (str) and split into documents by chapter.
language/conpono/cpc/preproc/wiki_preproc_pipeline.py:52
Function
read_file
Read the contents of filename (str) and split into documents by chapter.
language/conpono/create_pretrain_data/books_preproc_pipeline.py:62
Function
read_file
Read the contents of filename (str) and split into documents by chapter.
language/conpono/create_pretrain_data/wiki_preproc_pipeline.py:54
Function
read_json
Read json file from path and return. Args: path: Path of file to read Returns: JSON object from file
language/serene/util.py:104
Function
read_jsonl
Yields entries from the JSONL files.
language/casper/retrieve/query_retrievers.py:43
Function
read_jsonlines
Read jsonlines file from the path. Args: path: Path to json file Returns: List of objects decoded from each line
language/serene/util.py:88
Function
read_orig_examples
Reads and deserializes JSONLs from files.
language/casper/augment/cached_retrieval_to_dataset_lib.py:52
Function
read_rules
Read rule txt file to list of rules.
language/compgen/csl/qcfg/qcfg_file.py:21
Function
read_rules
Read rule txt file to list of rules.
language/compgen/nqg/model/qcfg/qcfg_file.py:22
Function
read_sparse_matrix_dict
Read a dictionary of relations from a file. Args: input_file: Filename string or FileLike object. Returns: A dictionary mapping relation
language/nql/nql/io.py:97
Function
read_symbol_table_dict
Read a dictionary of SymbolTable values from a file. Args: input_file: Filename string or FileLike object. restrict_to: If defined, a list
language/nql/nql/io.py:130
Function
read_tsv
Read file to list of rows.
language/quest/common/tsv_utils.py:20
Function
read_txt
Read file to list of lines.
language/compgen/csl/common/txt_utils.py:20
Function
read_txt
Read newline separated text file.
language/quest/common/tsv_utils.py:50
Function
read_tydi_examples
Read a TyDi json file into a list of `TyDiExample`. Delegates to `preproc.create_entry_from_json` to add special tokens to input and handle chara
language/canine/tydiqa/preproc.py:693
Function
read_wiki_a
()
language/conpono/evals/coherence_eval.py:106
Method
rec_hist_cast
(idx, stack, leaves, visit_count)
language/search_agents/muzero/types.py:369
Method
reccast
(node, stack, leaves)
language/search_agents/muzero/types.py:359
Function
redirect_table_pipeline
Beam pipeline for producing a redirect table as a TSV.
language/fruit/beam_pipelines.py:78
Function
redirect_to_id
Creates mapping of redirect titles to article ids.
language/fruit/beam_pipelines.py:65
Function
reduce_fn
(_, x)
language/boolq/utils/ops.py:187
Function
regex_match_score
(prediction, ground_truth)
language/orqa/utils/eval_utils.py:49
Function
register_encoder
Register encoder. Encoder should implement BaseEncoder abstraction. Used as decorator, for example: @register_encoder('my_encoder') class My
language/mentionmemory/encoders/encoder_registry.py:26
Function
register_task
Register task. Task should implement BaseTask abstraction. Used as decorator, for example: @register_task('my_task') class MyTask(BaseTask):
language/mentionmemory/tasks/task_registry.py:26
Function
reject
Reject span sample.
language/capwap/utils/text_utils.py:203
Function
rekey_mentions
Re-keys mentions in preparation for article merge.
language/fruit/beam_pipelines.py:378
Function
relative_positions_adjacency_matrix
Generates batch of relative position matrices. Args: length: Integer specifying maximum sequence length in batch. batch_size: Integer speci
language/xsp/model/common_layers.py:402
Function
remove_bad_sqlparse_keywords
Removes keywords from the sqlparse keyword lists. This is a relatively hacky solution (removing from the globally-scoped internal keyword lists)
language/xsp/data_preprocessing/sqlparse_keyword_utils.py:25
Function
remove_repetitive_trigram
Select from the beam a prediction without repetitive trigrams.
language/labs/exemplar_decoding/models/model_function.py:50
Function
replace_entities
Replace entity references with something more copy friendly.
language/compgen/nqg/tasks/geoquery/entity_utils.py:60
Function
replace_from_clause
Replace fully-specified FROM clause(s) with under-specified FROM clause(s). Arguments: sql_spans: List of SqlSpan tuples with fully-specified F
language/xsp/data_preprocessing/abstract_sql.py:502
Function
report_ci
Report confidence interval and some other statistics from bootstrap samples. Note: rel_threshold is a good approximation only if L is not close to
language/multiberts/multibootstrap.py:159
Function
reset
()
language/realm/profile.py:27
Method
reset
Resets the environment to play a query. Args: index: If set as an integer, requests this specific query from the environment. If
language/search_agents/muzero/env.py:620
Function
reshape_from_matrix
Reshapes a rank 2 tensor back to its original rank >= 2 tensor.
language/canine/bert_modeling.py:1011
Function
reshape_to_matrix
Reshapes a >= rank 2 tensor to a rank 2 tensor (i.e., a matrix).
language/canine/bert_modeling.py:997
Function
restore_entities
Restore entities in funql.
language/compgen/nqg/tasks/geoquery/entity_utils.py:74
Function
restore_from_clause
Restores fully-specified FROM clause. Args: sql_spans: List of SqlSpan tuples with underspecified FROM clause(s). fk_relations: List of For
language/xsp/data_preprocessing/abstract_sql.py:642
Function
restore_from_clauses
Loads an original dataset and matches with a predictions file. The input and output is a text file containing model predictions. This is a newlin
language/xsp/evaluation/restore_from_asql.py:125
Function
restore_funql
Recursively parse FunQL string back to original string.
language/compgen/nqg/tasks/geoquery/funql_normalization.py:158
Function
restore_predicted_sql
Restore FROM clause in predicted SQL. TODO(petershaw): Add call to this function from run_inference.py. Args: sql_string: SQL query as strin
language/xsp/data_preprocessing/abstract_sql_converters.py:696
Method
retrieve
(self, query_batch)
language/realm/retrieval.py:58
Method
retrieve
(self, query_batch)
language/realm/retrieval.py:113
Method
retrieve_batch
(self, examples: List[Example])
language/casper/retrieve/query_retrievers.py:318
Function
rewire_summary_calls
Rewires histogram() to nop and scalar() to tpu_summary_scalar. Summaries are not yet supported on TPUs (b/64980426). So we provide this context m
language/xsp/model/tpu_utils.py:33
Function
rhs_can_maybe_derive
Return False if rhs_a cannot be used in a derivation of rhs_b. This function uses a fast approximation based on terminal sequence overlap to iden
language/compgen/csl/induction/rule_utils.py:47
Function
rhs_can_maybe_derive
Return False if rhs_a cannot be used in a derivation of rhs_b. This function uses a fast approximation based on terminal sequence overlap to iden
language/compgen/nqg/model/induction/rule_utils.py:68
Function
rhs_contains
Returns True if sublist is contained in list_to_search.
language/compgen/nqg/model/induction/rule_utils.py:58
Function
rhs_count
Returns count of occurances of sublist in list_to_search.
language/compgen/nqg/model/induction/rule_utils.py:47
Function
rhs_remove
Remove occurrences of sublist in rhs.
language/compgen/csl/induction/rule_utils.py:156
Function
rhs_replace
Replace occurrences of sublist in rhs with replacement.
language/compgen/csl/induction/rule_utils.py:136
Function
rouge_l
Rouge-L. Args: eval_sentences: prediction to be evaluated. eval_len: lengths of the predictions. ref_sentences: reference sentences.
language/labs/exemplar_decoding/utils/rouge_utils.py:118
Function
rouge_n
Rouge N.
language/labs/exemplar_decoding/utils/rouge_utils.py:143
Function
rule_codelength
Computes the codelength for a given rule.
language/compgen/nqg/model/induction/codelength_utils.py:27
Function
rule_from_string
Parse rule in format 'source SEPARATOR target'.
language/compgen/csl/qcfg/qcfg_rule.py:143
Function
rule_from_string
Parse rule in format 'source SEPARATOR target'.
language/compgen/nqg/model/qcfg/qcfg_rule.py:80
Function
rules_to_txt_file
Write rules to txt file.
language/compgen/csl/targets/target_grammar.py:63
Function
rules_to_txt_file
Write rules to txt file.
language/compgen/nqg/model/parser/inference/targets/target_grammar.py:65
Method
run_episode
( self, q0: str, answers: Sequence[str], get_documents_fn: Callable[[str, str], List[environment_p
language/search_agents/t5/run_inference_beam.py:165
Function
run_experiment
Run an experiment using estimators. This is a light wrapper around typical estimator usage to avoid boilerplate code. Please use the following co
language/common/utils/experiment_utils.py:80
Method
run_given_entities
Run model given the passage title entities.
language/labs/drkit/hotpotqa/demo.py:829
Function
run_inference
Determine one-best parse using score_fn. Args: source: Input string. rules: Set of QCFGRules. score_fn: Function with inputs (rule, spa
language/compgen/nqg/model/parser/inference/inference_parser.py:81
Function
run_one_hot_embeddings
Extract only the word embeddings of the original BERT model.
language/bert_extraction/steal_bert_classifier/embedding_perturbations/embedding_util.py:184
Function
safe_copy
Copy safely from src to dst by creating intermediate directories. Args: src: File to copy dst: Where to copy to overwrite: Whether to o
language/serene/util.py:52
Function
safe_cumprod
Computes cumprod of x in logspace using cumsum to avoid underflow. The cumprod function and its gradient can result in numerical instabilities wh
language/labs/exemplar_decoding/models/attention.py:487
Function
same_entity_set_retrieval_loss
Computes same-entity-set-retrieval loss. We want to maximize attention scores received by memories which passages have large enough entity overla
language/mentionmemory/modules/mention_losses.py:546
Function
sample
Sample data from CFG. Args: parser_rules: A list of CFGRule instances. start_idx: Index of non-terminal that is start symbol. rule_valu
language/compgen/csl/cky/cfg_sampler.py:39
Function
sample
Sample length from truncated Geo(p).
language/capwap/utils/text_utils.py:216
Method
sample
(self, time, outputs, state, name=None)
language/labs/consistent_zero_shot_nmt/modules/helpers.py:45
Method
sample
(self, time, outputs, state, name=None)
language/labs/consistent_zero_shot_nmt/modules/helpers.py:114
Method
sample
(self, time, outputs, state, name=None)
language/labs/consistent_zero_shot_nmt/modules/helpers.py:157
Method
sample
Sample a source and target pair. Args: score_fn: A scoring function that takes parent_rule (QCFGRule), nt_idx, child_rule (QCFGRule
language/compgen/csl/augment/joint_sampler.py:476
Method
sample_batch
()
language/mentionmemory/tasks/mention_based_entity_qa_task_test.py:239
Method
sample_batch
()
language/mentionmemory/tasks/mention_memory_task_test.py:202
Function
sample_example
(i, sampler)
language/compgen/csl/augment/generate_synthetic_examples_beam.py:57
Function
sample_fn
(x)
language/conpono/reconstruct/model_builder.py:67
Function
sample_from_rollouts
Sample a single example from the given rollouts. Args: rollouts: Instance of RolloutOutputs. baseline: <float32> [batch_size] Baseline valu
language/capwap/utils/reward_utils.py:564
Function
sample_next_sequence
Sample a full sequence of random tokens using uniform or multinomials.
language/bert_extraction/steal_bert_classifier/data_generation/preprocess_util.py:149
Function
sample_thief_data
Sample sentences from thief dataset. Remove OOV if sanitize is True.
language/bert_extraction/steal_bert_classifier/data_generation/preprocess_util.py:123
Method
save
(self, data_dir)
language/serene/text_matcher.py:122
Method
save
(self, file_path)
language/serene/training.py:136
Method
save
Save JointSampler to JSON file.
language/compgen/csl/augment/joint_sampler.py:403
Method
save
(self)
language/compgen/csl/augment/sampler_utils.py:191
Function
save_checkpoint_if_best
Copies a checkpoint if it is the best so far. Args: eval_result: The new eval result. checkpoint_path: Specific checkpoint path to compare
language/multivec/models/checkpoint_utils.py:115
Function
save_params
Serialize params to model directory.
language/capwap/utils/experiment_utils.py:132
Function
save_samples_to_json
Save samples to a json file.
language/mentionmemory/utils/data_utils.py:413
Method
save_to_file
(self, filename_prefix)
language/serene/tokenizers.py:48
Method
save_to_file
(self, filename_prefix)
language/serene/tokenizers.py:64
Method
save_to_file
(self, filename_prefix)
language/serene/tokenizers.py:142
Method
save_to_file
(self, filename_prefix)
language/serene/tokenizers.py:194
Method
save_to_file
(self, filename_prefix)
language/serene/preprocessing.py:307
Function
save_weights
Save serialized weight dictionary.
language/mentionmemory/utils/checkpoint_utils.py:27
Function
scaffold_fn
Init op run on host.
language/capwap/models/reinforce_model.py:224
Function
scaffold_fn
Init op run on host.
language/capwap/models/supervised_model.py:169
Function
scaffold_fn
Init op run on host.
language/capwap/evaluation/infer_wsp_captions.py:152
Function
score_annotations
Computes scored annotations from the annotation pairs. Args: paired_annotations: Pairs of gold and predicted annotations. qsim_model_name:
language/diffqg/metrics.py:401
Function
score_classification
Computes the classification score.
language/decontext/eval_util.py:350
Function
score_endpoints
Compute two scores over context words based on the input embeddings. Args: question_emb: <float32> [batch_size, max_question_len, hidden_size]
language/question_answering/decatt_docreader/layers/document_reader.py:99
Method
score_fn
(parent_rule, nt_idx, child_rule)
language/compgen/csl/augment/qcfg_sampler_test.py:52
Method
score_fn
(parent_rule, nt_idx, child_rule)
language/compgen/csl/augment/joint_sampler_test.py:114
← previous
next →
5,101–5,200 of 6,154, ranked by callers