MCPcopy Create free account

hub / github.com/google-research/language / functions

Functions6,154 in github.com/google-research/language

Functionread_file
Read the contents of filename (str) and split into documents by chapter.
language/conpono/cpc/preproc/wiki_preproc_pipeline.py:52
Functionread_file
Read the contents of filename (str) and split into documents by chapter.
language/conpono/create_pretrain_data/books_preproc_pipeline.py:62
Functionread_file
Read the contents of filename (str) and split into documents by chapter.
language/conpono/create_pretrain_data/wiki_preproc_pipeline.py:54
Functionread_json
Read json file from path and return. Args: path: Path of file to read Returns: JSON object from file
language/serene/util.py:104
Functionread_jsonl
Yields entries from the JSONL files.
language/casper/retrieve/query_retrievers.py:43
Functionread_jsonlines
Read jsonlines file from the path. Args: path: Path to json file Returns: List of objects decoded from each line
language/serene/util.py:88
Functionread_orig_examples
Reads and deserializes JSONLs from files.
language/casper/augment/cached_retrieval_to_dataset_lib.py:52
Functionread_rules
Read rule txt file to list of rules.
language/compgen/csl/qcfg/qcfg_file.py:21
Functionread_rules
Read rule txt file to list of rules.
language/compgen/nqg/model/qcfg/qcfg_file.py:22
Functionread_sparse_matrix_dict
Read a dictionary of relations from a file. Args: input_file: Filename string or FileLike object. Returns: A dictionary mapping relation
language/nql/nql/io.py:97
Functionread_symbol_table_dict
Read a dictionary of SymbolTable values from a file. Args: input_file: Filename string or FileLike object. restrict_to: If defined, a list
language/nql/nql/io.py:130
Functionread_tsv
Read file to list of rows.
language/quest/common/tsv_utils.py:20
Functionread_txt
Read file to list of lines.
language/compgen/csl/common/txt_utils.py:20
Functionread_txt
Read newline separated text file.
language/quest/common/tsv_utils.py:50
Functionread_tydi_examples
Read a TyDi json file into a list of `TyDiExample`. Delegates to `preproc.create_entry_from_json` to add special tokens to input and handle chara
language/canine/tydiqa/preproc.py:693
Functionread_wiki_a
()
language/conpono/evals/coherence_eval.py:106
Methodrec_hist_cast
(idx, stack, leaves, visit_count)
language/search_agents/muzero/types.py:369
Methodreccast
(node, stack, leaves)
language/search_agents/muzero/types.py:359
Functionredirect_table_pipeline
Beam pipeline for producing a redirect table as a TSV.
language/fruit/beam_pipelines.py:78
Functionredirect_to_id
Creates mapping of redirect titles to article ids.
language/fruit/beam_pipelines.py:65
Functionreduce_fn
(_, x)
language/boolq/utils/ops.py:187
Functionregex_match_score
(prediction, ground_truth)
language/orqa/utils/eval_utils.py:49
Functionregister_encoder
Register encoder. Encoder should implement BaseEncoder abstraction. Used as decorator, for example: @register_encoder('my_encoder') class My
language/mentionmemory/encoders/encoder_registry.py:26
Functionregister_task
Register task. Task should implement BaseTask abstraction. Used as decorator, for example: @register_task('my_task') class MyTask(BaseTask):
language/mentionmemory/tasks/task_registry.py:26
Functionreject
Reject span sample.
language/capwap/utils/text_utils.py:203
Functionrekey_mentions
Re-keys mentions in preparation for article merge.
language/fruit/beam_pipelines.py:378
Functionrelative_positions_adjacency_matrix
Generates batch of relative position matrices. Args: length: Integer specifying maximum sequence length in batch. batch_size: Integer speci
language/xsp/model/common_layers.py:402
Functionremove_bad_sqlparse_keywords
Removes keywords from the sqlparse keyword lists. This is a relatively hacky solution (removing from the globally-scoped internal keyword lists)
language/xsp/data_preprocessing/sqlparse_keyword_utils.py:25
Functionremove_repetitive_trigram
Select from the beam a prediction without repetitive trigrams.
language/labs/exemplar_decoding/models/model_function.py:50
Functionreplace_entities
Replace entity references with something more copy friendly.
language/compgen/nqg/tasks/geoquery/entity_utils.py:60
Functionreplace_from_clause
Replace fully-specified FROM clause(s) with under-specified FROM clause(s). Arguments: sql_spans: List of SqlSpan tuples with fully-specified F
language/xsp/data_preprocessing/abstract_sql.py:502
Functionreport_ci
Report confidence interval and some other statistics from bootstrap samples. Note: rel_threshold is a good approximation only if L is not close to
language/multiberts/multibootstrap.py:159
Functionreset
()
language/realm/profile.py:27
Methodreset
Resets the environment to play a query. Args: index: If set as an integer, requests this specific query from the environment. If
language/search_agents/muzero/env.py:620
Functionreshape_from_matrix
Reshapes a rank 2 tensor back to its original rank >= 2 tensor.
language/canine/bert_modeling.py:1011
Functionreshape_to_matrix
Reshapes a >= rank 2 tensor to a rank 2 tensor (i.e., a matrix).
language/canine/bert_modeling.py:997
Functionrestore_entities
Restore entities in funql.
language/compgen/nqg/tasks/geoquery/entity_utils.py:74
Functionrestore_from_clause
Restores fully-specified FROM clause. Args: sql_spans: List of SqlSpan tuples with underspecified FROM clause(s). fk_relations: List of For
language/xsp/data_preprocessing/abstract_sql.py:642
Functionrestore_from_clauses
Loads an original dataset and matches with a predictions file. The input and output is a text file containing model predictions. This is a newlin
language/xsp/evaluation/restore_from_asql.py:125
Functionrestore_funql
Recursively parse FunQL string back to original string.
language/compgen/nqg/tasks/geoquery/funql_normalization.py:158
Functionrestore_predicted_sql
Restore FROM clause in predicted SQL. TODO(petershaw): Add call to this function from run_inference.py. Args: sql_string: SQL query as strin
language/xsp/data_preprocessing/abstract_sql_converters.py:696
Methodretrieve
(self, query_batch)
language/realm/retrieval.py:58
Methodretrieve
(self, query_batch)
language/realm/retrieval.py:113
Methodretrieve_batch
(self, examples: List[Example])
language/casper/retrieve/query_retrievers.py:318
Functionrewire_summary_calls
Rewires histogram() to nop and scalar() to tpu_summary_scalar. Summaries are not yet supported on TPUs (b/64980426). So we provide this context m
language/xsp/model/tpu_utils.py:33
Functionrhs_can_maybe_derive
Return False if rhs_a cannot be used in a derivation of rhs_b. This function uses a fast approximation based on terminal sequence overlap to iden
language/compgen/csl/induction/rule_utils.py:47
Functionrhs_can_maybe_derive
Return False if rhs_a cannot be used in a derivation of rhs_b. This function uses a fast approximation based on terminal sequence overlap to iden
language/compgen/nqg/model/induction/rule_utils.py:68
Functionrhs_contains
Returns True if sublist is contained in list_to_search.
language/compgen/nqg/model/induction/rule_utils.py:58
Functionrhs_count
Returns count of occurances of sublist in list_to_search.
language/compgen/nqg/model/induction/rule_utils.py:47
Functionrhs_remove
Remove occurrences of sublist in rhs.
language/compgen/csl/induction/rule_utils.py:156
Functionrhs_replace
Replace occurrences of sublist in rhs with replacement.
language/compgen/csl/induction/rule_utils.py:136
Functionrouge_l
Rouge-L. Args: eval_sentences: prediction to be evaluated. eval_len: lengths of the predictions. ref_sentences: reference sentences.
language/labs/exemplar_decoding/utils/rouge_utils.py:118
Functionrouge_n
Rouge N.
language/labs/exemplar_decoding/utils/rouge_utils.py:143
Functionrule_codelength
Computes the codelength for a given rule.
language/compgen/nqg/model/induction/codelength_utils.py:27
Functionrule_from_string
Parse rule in format 'source SEPARATOR target'.
language/compgen/csl/qcfg/qcfg_rule.py:143
Functionrule_from_string
Parse rule in format 'source SEPARATOR target'.
language/compgen/nqg/model/qcfg/qcfg_rule.py:80
Functionrules_to_txt_file
Write rules to txt file.
language/compgen/csl/targets/target_grammar.py:63
Functionrules_to_txt_file
Write rules to txt file.
language/compgen/nqg/model/parser/inference/targets/target_grammar.py:65
Methodrun_episode
( self, q0: str, answers: Sequence[str], get_documents_fn: Callable[[str, str], List[environment_p
language/search_agents/t5/run_inference_beam.py:165
Functionrun_experiment
Run an experiment using estimators. This is a light wrapper around typical estimator usage to avoid boilerplate code. Please use the following co
language/common/utils/experiment_utils.py:80
Methodrun_given_entities
Run model given the passage title entities.
language/labs/drkit/hotpotqa/demo.py:829
Functionrun_inference
Determine one-best parse using score_fn. Args: source: Input string. rules: Set of QCFGRules. score_fn: Function with inputs (rule, spa
language/compgen/nqg/model/parser/inference/inference_parser.py:81
Functionrun_one_hot_embeddings
Extract only the word embeddings of the original BERT model.
language/bert_extraction/steal_bert_classifier/embedding_perturbations/embedding_util.py:184
Functionsafe_copy
Copy safely from src to dst by creating intermediate directories. Args: src: File to copy dst: Where to copy to overwrite: Whether to o
language/serene/util.py:52
Functionsafe_cumprod
Computes cumprod of x in logspace using cumsum to avoid underflow. The cumprod function and its gradient can result in numerical instabilities wh
language/labs/exemplar_decoding/models/attention.py:487
Functionsame_entity_set_retrieval_loss
Computes same-entity-set-retrieval loss. We want to maximize attention scores received by memories which passages have large enough entity overla
language/mentionmemory/modules/mention_losses.py:546
Functionsample
Sample data from CFG. Args: parser_rules: A list of CFGRule instances. start_idx: Index of non-terminal that is start symbol. rule_valu
language/compgen/csl/cky/cfg_sampler.py:39
Functionsample
Sample length from truncated Geo(p).
language/capwap/utils/text_utils.py:216
Methodsample
(self, time, outputs, state, name=None)
language/labs/consistent_zero_shot_nmt/modules/helpers.py:45
Methodsample
(self, time, outputs, state, name=None)
language/labs/consistent_zero_shot_nmt/modules/helpers.py:114
Methodsample
(self, time, outputs, state, name=None)
language/labs/consistent_zero_shot_nmt/modules/helpers.py:157
Methodsample
Sample a source and target pair. Args: score_fn: A scoring function that takes parent_rule (QCFGRule), nt_idx, child_rule (QCFGRule
language/compgen/csl/augment/joint_sampler.py:476
Methodsample_batch
()
language/mentionmemory/tasks/mention_based_entity_qa_task_test.py:239
Methodsample_batch
()
language/mentionmemory/tasks/mention_memory_task_test.py:202
Functionsample_example
(i, sampler)
language/compgen/csl/augment/generate_synthetic_examples_beam.py:57
Functionsample_fn
(x)
language/conpono/reconstruct/model_builder.py:67
Functionsample_from_rollouts
Sample a single example from the given rollouts. Args: rollouts: Instance of RolloutOutputs. baseline: <float32> [batch_size] Baseline valu
language/capwap/utils/reward_utils.py:564
Functionsample_next_sequence
Sample a full sequence of random tokens using uniform or multinomials.
language/bert_extraction/steal_bert_classifier/data_generation/preprocess_util.py:149
Functionsample_thief_data
Sample sentences from thief dataset. Remove OOV if sanitize is True.
language/bert_extraction/steal_bert_classifier/data_generation/preprocess_util.py:123
Methodsave
(self, data_dir)
language/serene/text_matcher.py:122
Methodsave
(self, file_path)
language/serene/training.py:136
Methodsave
Save JointSampler to JSON file.
language/compgen/csl/augment/joint_sampler.py:403
Methodsave
(self)
language/compgen/csl/augment/sampler_utils.py:191
Functionsave_checkpoint_if_best
Copies a checkpoint if it is the best so far. Args: eval_result: The new eval result. checkpoint_path: Specific checkpoint path to compare
language/multivec/models/checkpoint_utils.py:115
Functionsave_params
Serialize params to model directory.
language/capwap/utils/experiment_utils.py:132
Functionsave_samples_to_json
Save samples to a json file.
language/mentionmemory/utils/data_utils.py:413
Methodsave_to_file
(self, filename_prefix)
language/serene/tokenizers.py:48
Methodsave_to_file
(self, filename_prefix)
language/serene/tokenizers.py:64
Methodsave_to_file
(self, filename_prefix)
language/serene/tokenizers.py:142
Methodsave_to_file
(self, filename_prefix)
language/serene/tokenizers.py:194
Methodsave_to_file
(self, filename_prefix)
language/serene/preprocessing.py:307
Functionsave_weights
Save serialized weight dictionary.
language/mentionmemory/utils/checkpoint_utils.py:27
Functionscaffold_fn
Init op run on host.
language/capwap/models/reinforce_model.py:224
Functionscaffold_fn
Init op run on host.
language/capwap/models/supervised_model.py:169
Functionscaffold_fn
Init op run on host.
language/capwap/evaluation/infer_wsp_captions.py:152
Functionscore_annotations
Computes scored annotations from the annotation pairs. Args: paired_annotations: Pairs of gold and predicted annotations. qsim_model_name:
language/diffqg/metrics.py:401
Functionscore_classification
Computes the classification score.
language/decontext/eval_util.py:350
Functionscore_endpoints
Compute two scores over context words based on the input embeddings. Args: question_emb: <float32> [batch_size, max_question_len, hidden_size]
language/question_answering/decatt_docreader/layers/document_reader.py:99
Methodscore_fn
(parent_rule, nt_idx, child_rule)
language/compgen/csl/augment/qcfg_sampler_test.py:52
Methodscore_fn
(parent_rule, nt_idx, child_rule)
language/compgen/csl/augment/joint_sampler_test.py:114
← previousnext →5,101–5,200 of 6,154, ranked by callers