Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/google-research/language
/ functions
Functions
6,154 in github.com/google-research/language
⨍
Functions
6,154
◇
Types & classes
800
↳
Endpoints
12
↓ 2 callers
Method
postprocess
Apply postpostprocess_fn to a chart cell.
language/compgen/csl/cky/cfg_parser.py:110
↓ 2 callers
Method
pprintable
Convert to a nested dictionary structure. Returns: a structure that the pprint module can render nicely.
language/nql/nql/__init__.py:1918
↓ 2 callers
Function
predict
Compute the class logit predictions. Args: is_train: bool, are we in train mode embeddings: `embedding_utils.PretrainedWordEmbeddings` to e
language/boolq/run_recurrent_model_boolq.py:319
↓ 2 callers
Method
preprocess
Appiles preprocessing for programs in all examples.
language/compir/dataset_parsers/dataset_parser.py:55
↓ 2 callers
Method
preprocess_fn
(sample: Dict[str, tf.Tensor])
language/mentionmemory/tasks/base_task.py:121
↓ 2 callers
Function
print_avg_by_key
Prints avg results partitioned according to some `key_fn`. Args: examples: List of Example instances. metrics: List of float values corresp
language/quest/eval/eval_utils.py:20
↓ 2 callers
Method
process
Coverts an NQ example in a list of serialized tf examples.
language/question_answering/bert_joint/run_nq.py:746
↓ 2 callers
Function
process_one_example
Process one example.
language/gscan/xattn_model/dataset/preprocess.py:318
↓ 2 callers
Function
process_tsv
Processes examples in the TSV file src_file and writes to tgt_file. Args: src_file: Source TSV file tgt_file: Target TSV file affected_
language/casper/augment/patch_guiding_tag.py:44
↓ 2 callers
Function
random_choice_1d
Samples num_to_sample elements from [0; num_values) without replacement.
language/mentionmemory/utils/mention_preprocess_utils.py:36
↓ 2 callers
Function
raw_data_to_tfrecord
Process data into TFRecord compatible format to prepare for writing.
language/gscan/xattn_model/dataset/preprocess.py:289
↓ 2 callers
Function
rc_span
Computes exact match score from QA model run on context. Args: question_ids: <int32> [batch_size, question_len] question_mask: <int32> [bat
language/capwap/utils/reward_utils.py:155
↓ 2 callers
Function
read_hotpot_examples
Read hotpot examples from file.
language/labs/drkit/hotpotqa/answer_extractor.py:281
↓ 2 callers
Function
read_nq_entry
Converts a NQ entry into a list of NqExamples.
language/question_answering/bert_joint/run_nq.py:477
↓ 2 callers
Function
read_object
Read R-CNN object data from HDF5 file.
language/capwap/utils/io_utils.py:171
↓ 2 callers
Function
read_squad_examples
Read a SQuAD json file into a list of SquadExample.
language/bert_extraction/steal_bert_qa/models/run_squad.py:238
↓ 2 callers
Function
read_squad_examples
Read a SQuAD json file into a list of SquadExample.
language/conpono/evals/run_squad.py:226
↓ 2 callers
Function
read_txt
Read file to list of lines.
language/compgen/csl/tasks/smcalflow/tools/merge_dataset.py:33
↓ 2 callers
Function
recover_tokenization
(string)
language/labs/exemplar_decoding/utils/data.py:88
↓ 2 callers
Method
render
(self, mode='human')
language/search_agents/muzero/env.py:972
↓ 2 callers
Function
replace_and_adjust
Remove subj/obj annotations and adjust token mapping accordingly.
language/mentionmemory/data/prepare_webred.py:91
↓ 2 callers
Function
request_helper
Helps rerun requests upon failure.
language/fruit/scripts/get_topics.py:47
↓ 2 callers
Method
reset
(self)
language/search_agents/muzero/state_tree.py:303
↓ 2 callers
Method
reset_to_previous_finished_query
(self)
language/search_agents/muzero/state_tree.py:334
↓ 2 callers
Method
restore_checkpoint
Restore model parameters from checkpoint.
language/compgen/csl/model/inference/inference_wrapper.py:70
↓ 2 callers
Method
restore_checkpoint
Restore model parameters from checkpoint.
language/compgen/nqg/model/parser/inference/inference_wrapper.py:80
↓ 2 callers
Function
result_table_to_string
Converts a resulting SQL table to a human-readable string.
language/xsp/evaluation/official_evaluation.py:80
↓ 2 callers
Method
retrieve
Retrieves candidates for a batch of queries. Args: query_batch (list[Query]): a list of queries. Returns: a batch of lists, wher
language/realm/retrieval.py:35
↓ 2 callers
Method
retrieve_batch
Returns the neighbors and distances from a batch of examples. Args: examples: A batch of examples. Returns: a list of RetrievalR
language/casper/retrieve/query_retrievers.py:206
↓ 2 callers
Function
rhs_replace
Replace occurrences of sublist in rhs with replacement.
language/compgen/nqg/model/induction/rule_utils.py:140
↓ 2 callers
Function
rouge_n_metric
(predictions, prediction_len, labels, label_len, n, vocab, use_bpe=False, predict_mode=Fals
language/labs/exemplar_decoding/utils/rouge_utils.py:172
↓ 2 callers
Function
rr
Return the reciprical rank of the first positive candidate.
language/multivec/models/metrics.py:42
↓ 2 callers
Function
run_bert_embeddings
Extract only the word embeddings of the original BERT model.
language/bert_extraction/steal_bert_classifier/embedding_perturbations/embedding_util.py:204
↓ 2 callers
Function
run_inference
Run inference.
language/compgen/csl/model/inference/eval_model_beam.py:121
↓ 2 callers
Function
run_inference
Run inference.
language/compgen/csl/model/inference/eval_model.py:70
↓ 2 callers
Function
run_inference
Run inference.
language/compgen/nqg/model/parser/inference/eval_model.py:203
↓ 2 callers
Method
run_model
Initialize and run the model once, perform sanity checks.
language/mentionmemory/tasks/mention_memory_task_test.py:121
↓ 2 callers
Function
sample_neighbor
Randomly sample a neighbor to encode.
language/labs/exemplar_decoding/utils/data.py:176
↓ 2 callers
Function
sample_next_token
(vocab, probs, scheme)
language/bert_extraction/steal_bert_classifier/data_generation/preprocess_util.py:142
↓ 2 callers
Method
sample_non_nearby_position
Sample an available position that is not next to the given position.
language/gscan/data/world.py:71
↓ 2 callers
Method
save_rules
(self)
language/compgen/csl/induction/induction_utils.py:76
↓ 2 callers
Method
save_to_dict
(self)
language/compgen/csl/augment/joint_sampler.py:85
↓ 2 callers
Function
save_to_h5py
(outfn, output_array, field_name)
language/multivec/predict/encode_blocks.py:147
↓ 2 callers
Method
score_batch
Scores multiple annotations at once, useful for batching model calls. Note this method does not handle batching, and the length of the list will
language/diffqg/metrics.py:172
↓ 2 callers
Method
select_action
Select action that maximizes improvememt in objective.
language/compgen/csl/induction/greedy_policy.py:100
↓ 2 callers
Method
serialize_query
Create JSON object with distantly supervised fact.
language/labs/drkit/wikidata/preprocessing/distantly_supervise.py:227
↓ 2 callers
Method
setUp
(self)
language/nql/nql/util_test.py:146
↓ 2 callers
Method
shard_emb
Get the path for the sharded embedding. Args: shard: Shard to get embedding path for tmp: Whether to return /tmp version of path
language/serene/wiki_index.py:128
↓ 2 callers
Method
shard_sentence_ids
Get the path for the sharded sentence_ids. Args: shard: Shard to get sentence_ids path for tmp: Whether to return /tmp version of pat
language/serene/wiki_index.py:154
↓ 2 callers
Method
shard_urls
Get the path for the sharded urls. Args: shard: Shard to get urls path for tmp: Whether to return /tmp version of path Returns:
language/serene/wiki_index.py:141
↓ 2 callers
Function
shared_qry_encoder
Embed query into a BOW and shared dense representation.
language/labs/drkit/model_fns.py:652
↓ 2 callers
Function
simplify
Pre-process answer string.
language/labs/drkit/evaluate.py:527
↓ 2 callers
Function
single_eval
Run one evaluation using given checkpoint.
language/labs/drkit/run_dualencoder_lsf.py:2087
↓ 2 callers
Function
single_eval
Run one evaluation using given checkpoint.
language/labs/drkit/run_multihop_follow.py:526
↓ 2 callers
Function
sp_sp_matmul
Element-wise multiply two SparseTensors of same shape.
language/labs/drkit/model_fns.py:324
↓ 2 callers
Method
span_tokenize
( self, text, *args, **kwargs, )
language/fruit/rendering_utils.py:316
↓ 2 callers
Method
special_episode_statistics_learner
Everything you want logged for episodes. These are only used for logging. Actual rewards for learning need to be returned via `step`. A
language/search_agents/muzero/env.py:537
↓ 2 callers
Function
split_line_by_sentences
(line)
language/conpono/cpc/preproc/books_preproc_pipeline.py:111
↓ 2 callers
Function
split_line_by_sentences
(line)
language/conpono/cpc/preproc/wiki_preproc_pipeline.py:93
↓ 2 callers
Function
split_on_whitespace
(str_tensor)
language/question_answering/decatt_docreader/datasets/nq_short_pipeline_dataset.py:42
↓ 2 callers
Function
stack
(examples)
language/mentionmemory/tasks/mention_based_entity_qa_task_test.py:33
↓ 2 callers
Function
stack
(examples)
language/mentionmemory/tasks/mention_memory_task_test.py:37
↓ 2 callers
Function
symbols_to_logits_fn
Go from targets to logits. Args: current_index: Integer corresponding to 0-indexed decoder step. logit_indices: Tensor of shape [batc
language/xsp/model/transformer.py:387
↓ 2 callers
Method
tf_op
Apply any shape-preserving function to the underlying expression.
language/nql/nql/__init__.py:507
↓ 2 callers
Method
to_dict
Serializes this instance to a Python dictionary.
language/canine/config_utils.py:148
↓ 2 callers
Method
to_json
(self)
language/xsp/data_preprocessing/schema_utils.py:40
↓ 2 callers
Method
to_json
Returns the JSON form of this class.
language/xsp/data_preprocessing/nl_to_sql_example.py:37
↓ 2 callers
Method
token_ids_to_str
Converts a list of token IDs back to a human-readable string. WARNING: this is ONLY for debug purposes. It does NOT perfectly invert BERT tok
language/realm/featurization.py:448
↓ 2 callers
Function
token_to_char_offset
Converts a token index to the char offset within the candidate.
language/question_answering/bert_joint/run_nq.py:282
↓ 2 callers
Function
tokenize
Tokenizes text, optionally looking up special tokens separately. Args: tokenizer: a tokenizer from bert.tokenization.FullTokenizer text: te
language/question_answering/bert_joint/run_nq.py:707
↓ 2 callers
Method
tokenize
Tokenize one sentence.
language/gscan/xattn_model/dataset/preprocess.py:168
↓ 2 callers
Method
tokenize
Tokenizes a SQL query into a list of SQL tokens.
language/xsp/data_preprocessing/sql_utils.py:37
↓ 2 callers
Function
tokens_to_features_dict
Converts a list of Tokens into a dict of TF Features.
language/realm/generate_retrieval_corpus.py:84
↓ 2 callers
Function
total_size
Return the total size of all relations in their initial encoding.
language/nql/demos/gridworld_scaling/scaling_eval.py:255
↓ 2 callers
Function
train
Run one training loop over given TFRecords file.
language/labs/drkit/run_dualencoder_qa.py:1087
↓ 2 callers
Function
train_for_n_steps_fn
(iterator, steps)
language/compgen/csl/model/training/training_utils.py:105
↓ 2 callers
Function
train_for_n_steps_fn
(iterator, steps)
language/compgen/nqg/model/parser/training/training_utils.py:86
↓ 2 callers
Function
train_model
Run model training.
language/compgen/csl/model/training/train_model.py:51
↓ 2 callers
Function
train_model
Run model training.
language/compgen/nqg/model/parser/training/train_model.py:64
↓ 2 callers
Function
truncate_or_pad
(seq, seq_len, pad_val)
language/realm/featurization.py:622
↓ 2 callers
Function
txt_line_lang_iterator
Iterate through lines of file.
language/labs/consistent_zero_shot_nmt/data_generators/translate_multilingual.py:174
↓ 2 callers
Function
unflatten_beam_dim
Unflattens the first, flat batch*beam dimension of a non-scalar array.
language/gscan/xattn_model/model/decode.py:67
↓ 2 callers
Function
unwrap_np
(array)
language/serene/callbacks.py:54
↓ 2 callers
Function
update_bert_input_mask
Utility function to update the BERT input mask for an incoming shard.
language/bert_extraction/steal_bert_classifier/embedding_perturbations/embedding_util.py:264
↓ 2 callers
Function
update_token_type_ids
Utility function to update token_type_ids list for an incoming shard.
language/bert_extraction/steal_bert_classifier/embedding_perturbations/embedding_util.py:282
↓ 2 callers
Function
update_values
Update stored values with this time step.
language/capwap/utils/transformer_utils.py:608
↓ 2 callers
Method
valid_slot_obj
Do these object and qualifiers satisfy numeric or date conditions.
language/templama/sling2facts.py:423
↓ 2 callers
Method
visualize_data_example
Visualize a data exmaple.
language/gscan/data/dataset.py:284
↓ 2 callers
Method
visualize_data_examples
Visualize data examples. Sample examples from each split if not any.
language/gscan/data/dataset.py:301
↓ 2 callers
Function
wrap_with_passage_texts
Add passage text to the records.
language/multivec/preprocessing/create_training_data.py:318
↓ 2 callers
Method
zeros_numpy_array
A zero valued row vector encoding this entity. Args: type_name: the string type name of the named entity as_matrix: if true, return a
language/nql/nql/__init__.py:884
↓ 1 callers
Method
GetDocuments
(self, request: environment_pb2.GetDocumentsRequest, context)
language/search_agents/environment_server.py:42
↓ 1 callers
Method
GetQuery
(self, request: environment_pb2.GetQueryRequest, context)
language/search_agents/environment_server.py:46
↓ 1 callers
Method
PushExample
(self, example, timeout)
language/realm/preprocessing.py:44
↓ 1 callers
Method
QueueSize
(self)
language/realm/preprocessing.py:51
↓ 1 callers
Method
__add__
Add the underlying expressions. Args: other: A NeuralQueryExpression Returns: A NeuralQueryExpression.
language/nql/nql/__init__.py:402
↓ 1 callers
Method
__init__
(self, args, output_size, build_bias, weights=None
language/labs/exemplar_decoding/models/linear.py:50
↓ 1 callers
Method
__init__
Construct the AttentionMechanism mechanism. Args: num_units: The depth of the attention mechanism. memory: The memory to query; usual
language/labs/exemplar_decoding/models/attention.py:314
↓ 1 callers
Method
__init__
(self, name="BasicRNNDecoder")
language/labs/consistent_zero_shot_nmt/modules/decoders.py:45
↓ 1 callers
Method
__init__
(self, name="BaseLanguageModel")
language/labs/consistent_zero_shot_nmt/modules/language_models.py:38
← previous
next →
1,101–1,200 of 6,154, ranked by callers