Functions — github.com/ConnorJL/GPT2

hub / github.com/ConnorJL/GPT2 / functions

Functions101 in github.com/ConnorJL/GPT2

⨍ Functions 101◇ Types & classes 5

↓ 5 callersFunctiondropout

(x, pdrop, train)

models/gpt2/gpt2.py:157

↓ 5 callersFunctionmkdir

(fp)

datasets/openwebtext/extract_text.py:94

↓ 5 callersFunctionshape_list

Deal with dynamic shape in tensorflow cleanly.

models/gpt2/gpt2.py:7

↓ 4 callersFunctionconv1d

(x, scope, nf, *, w_init_stdev=0.02, params=None, scale=False)

models/gpt2/gpt2.py:47

↓ 4 callersMethoddelete

Delete the TPU node.

experimental/tpu_survival.py:116

↓ 4 callersMethodtpu_name

Format tpu_name to be used in creation and deletion calls.

experimental/tpu_survival.py:53

↓ 3 callersFunctionbpe_text

(batch_size, files, iterations, stitch, amount=1024, batch=True)

inputs.py:53

↓ 3 callersFunctionnorm

Normalize to mean = 0, std = 1, then do a diagonal affine transform.

models/gpt2/gpt2.py:21

↓ 2 callersFunction_get_x

(i)

inputs.py:73

↓ 2 callersMethod_resource_apply_dense

(self, grad, handle)

optimizers.py:255

↓ 2 callersMethod_should_use_factored_second_moment_estimate

Should we use a factored second moment estimator. Based on the shape of the variable. Args: shape: a list of integers

optimizers.py:207

↓ 2 callersFunctionadafactor_decay_rate_pow

(exponent)

optimizers.py:329

↓ 2 callersMethodcreate

Create a TPU node.

experimental/tpu_survival.py:129

↓ 2 callersMethoddecode

(self, tokens)

models/gpt2/encoder.py:104

↓ 2 callersMethodencode

(self, text)

datasets/openwebtext/encoder.py:97

↓ 2 callersMethodencode

(self, text)

models/gpt2/encoder.py:97

↓ 2 callersFunctionextract_month

(url_file_name)

datasets/openwebtext/extract_text.py:75

↓ 2 callersFunctionget_pairs

Return set of symbol pairs in a word. Word is represented as tuple of symbols (symbols being variable-length strings).

datasets/openwebtext/encoder.py:32

↓ 2 callersFunctionget_pairs

Return set of symbol pairs in a word. Word is represented as tuple of symbols (symbols being variable-length strings).

models/gpt2/encoder.py:32

↓ 2 callersMethodget_value

(self)

experimental/experiments.py:69

↓ 2 callersMethodkill_current_task

Kill the current running task.

experimental/tpu_survival.py:81

↓ 2 callersFunctionparse_archive

(archive_fp, out_dir, n_procs, chunk_size=100)

datasets/openwebtext/extract_text.py:47

↓ 2 callersFunctionreduce_rms

(x)

optimizers.py:335

↓ 2 callersFunctionsave

(runners)

experimental/overrunner.py:38

↓ 2 callersFunctionstep

(params, tokens, past=None)

models/gpt2/sample.py:34

↓ 2 callersFunctionstep_num

()

optimizers.py:332

↓ 1 callersMethod_apply_dense

(self, grad, var)

optimizers.py:231

↓ 1 callersFunction_bytes_feature

Returns a bytes_list from a string / byte.

datasets/openwebtext/create_tfrecords.py:27

↓ 1 callersMethod_decay_rate_default

(self)

optimizers.py:312

↓ 1 callersFunction_int64_feature

Returns an int64_list from a bool / enum / int / uint.

datasets/openwebtext/create_tfrecords.py:23

↓ 1 callersMethod_learning_rate_default

(self, multiply_by_parameter_scale)

optimizers.py:315

↓ 1 callersMethod_parameter_scale

Estimate the scale of the parameters from the current values. We include a minimum value of 0.001 to give it a chance to escape 0 if i

optimizers.py:242

↓ 1 callersFunction_top_k

()

models/gpt2/sample.py:10

↓ 1 callersFunctionadafactor_decay_rate_adam

(beta2)

optimizers.py:322

↓ 1 callersFunctionattention_mask

1's in the lower triangle, counting from the lower right corner. Same as tf.matrix_band_part(tf.ones([nd, ns]), -1, ns-nd), but doesn't produce g

models/gpt2/gpt2.py:64

↓ 1 callersFunctionattn

(x, scope, n_state, *, past, params, train=False)

models/gpt2/gpt2.py:75

↓ 1 callersFunctionblock

(x, scope, *, past, params, train=False)

models/gpt2/gpt2.py:134

↓ 1 callersMethodbpe

(self, token)

datasets/openwebtext/encoder.py:56

↓ 1 callersMethodbpe

(self, token)

models/gpt2/encoder.py:56

↓ 1 callersFunctionbytes_to_unicode

Returns list of utf-8 byte and a corresponding list of unicode strings. The reversible bpe codes work on unicode strings. This means you

datasets/openwebtext/encoder.py:11

↓ 1 callersFunctionbytes_to_unicode

Returns list of utf-8 byte and a corresponding list of unicode strings. The reversible bpe codes work on unicode strings. This means you

models/gpt2/encoder.py:11

↓ 1 callersFunctioncast_like

Cast x to y's dtype, if necessary.

optimizers.py:338

↓ 1 callersFunctionchunks

(l, n)

datasets/openwebtext/create_tfrecords.py:32

↓ 1 callersFunctionchunks

Yield successive n-sized chunks from l.

datasets/openwebtext/extract_text.py:82

↓ 1 callersFunctioncosine_decay_with_warmup

(global_step, learning_rate_base, total_steps,

optimizers.py:66

↓ 1 callersFunctioncreate_train_op

(loss, params)

optimizers.py:5

↓ 1 callersMethoddump_dict

(self)

experimental/tpu_survival.py:148

↓ 1 callersFunctionexpand_tile

Add a new axis of given size.

models/gpt2/gpt2.py:146

↓ 1 callersFunctiongelu

(x)

models/gpt2/gpt2.py:18

↓ 1 callersFunctiongenerate_experiments

(base, parameters, number)

experimental/experiments.py:100

↓ 1 callersFunctionget_processed_files

(out_dir)

datasets/openwebtext/extract_text.py:42

↓ 1 callersFunctionlist_tpus

List existing TPU nodes in the project/location. Args: project: (str) GCP project id. location: (str) GCP compute location, such as "us-ce

experimental/tpu_survival.py:162

↓ 1 callersFunctionmask_attn_weights

(w)

models/gpt2/gpt2.py:89

↓ 1 callersFunctionmerge_heads

(x)

models/gpt2/gpt2.py:85

↓ 1 callersFunctionmerge_states

Smash the last two dimensions of x into a single dimension.

models/gpt2/gpt2.py:42

↓ 1 callersFunctionmlp

(x, scope, n_state, *, params, train=False)

models/gpt2/gpt2.py:125

↓ 1 callersFunctionmultihead_attn

(q, k, v)

models/gpt2/gpt2.py:97

↓ 1 callersFunctionopenwebtext

(params, eval=False, batch=True)

inputs.py:8

↓ 1 callersFunctionopenwebtext_long

(params, eval=False, batch=True)

inputs.py:20

↓ 1 callersFunctionperplexity_metric

(loss)

metric_fns.py:3

↓ 1 callersFunctionpositions_for

(tokens, past_length)

models/gpt2/gpt2.py:152

↓ 1 callersMethodrun_task

Call a subprocess to run the training task on the current TPU node.

experimental/tpu_survival.py:95

↓ 1 callersFunctionsave_parsed_text

(parsed_entries, out_dir)

datasets/openwebtext/extract_text.py:35

↓ 1 callersFunctionsoftmax

(x, axis=-1)

models/gpt2/gpt2.py:13

↓ 1 callersFunctionsplit_states

Reshape the last dimension of x into [n, x.shape[-1]/n].

models/gpt2/gpt2.py:37

↓ 1 callersFunctiontop_k_logits

(logits, k)

models/gpt2/sample.py:5

↓ 1 callersMethodtpu_cidr_block

Format CIDR block to be used in creation calls.

experimental/tpu_survival.py:57

↓ 1 callersMethodupdate_state

Poll the TPU nodes and update self.state.

experimental/tpu_survival.py:62

Method__init__

Construct a new Adafactor optimizer. See class comment. Args: multiply_by_parameter_scale: a boolean learning_rate: an

optimizers.py:163

Method__init__

(self, encoder, bpe_merges, errors='replace')

datasets/openwebtext/encoder.py:44

Method__init__

(self, encoder, bpe_merges, errors='replace')

models/gpt2/encoder.py:44

Method__init__

(self, name, distribution, model=True, dtype=None, values=None)

experimental/experiments.py:49

Method__init__

(self, project=None, location=None, id=None, params=None, d=None)

experimental/tpu_survival.py:23

Method_apply_sparse

(self, grad, var)

optimizers.py:234

Function_assert_float_dtype

(dtype)

models/gpt2/gpt2.py:162

Method_create_slots

(self, var_list)

optimizers.py:217

Function_parse_function

(example_proto)

inputs.py:57

Method_resource_apply_sparse

(self, grad, handle, indices)

models/gpt2/sample.py:57

Functioncond

(*args)

models/gpt2/sample.py:68

Functioncreate_file

(args)

datasets/openwebtext/create_tfrecords.py:47

Functioncreate_tpu

Create a TPU node. Args: project: (str) GCP project id. location: (str) GCP compute location, such as "us-central1-b". tpu_name: (str)

experimental/tpu_survival.py:180

Methoddecode

(self, tokens)

datasets/openwebtext/encoder.py:104

Functiondelete_tpu

List existing TPU nodes in the project/location. Args: project: (str) GCP project id. location: (str) GCP compute location, such as "us-ce

experimental/tpu_survival.py:236

Functiondo_work

(f)

datasets/openwebtext/run_newspaper_extract.py:21

Functionextract_archive

(archive_fp, outdir=".")

datasets/openwebtext/extract_text.py:88

datasets/openwebtext/encoder.py:109

Functionget_encoder

(encoder_path)

models/gpt2/encoder.py:109

Functionget_tpu

List existing TPU nodes in the project/location. Args: project: (str) GCP project id. location: (str) GCP compute location, such as "us-ce

experimental/tpu_survival.py:217

Functiongpt2_model

(features, labels, mode, params)

model_fns.py:9

Functiongpt2_pred_input

(params, text=None)

inputs.py:110

Functiongpt2_predict

(network, text, params)

predict_fns.py:11

Functionmodel

(X, params, labels=None, past=None, scope='model', reuse=False, train=False)

models/gpt2/gpt2.py:168

Functionopenwebtext_longbiased

datasets/openwebtext/extract_text.py:27

Functionpast_shape

(*, params, batch_size=None, sequence=None)

models/gpt2/gpt2.py:143

Functionsample_sequence

(*, params, length, start_token=None, batch_size=None, context=None, temperature=1, top_k=0)

models/gpt2/sample.py:25

next →1–100 of 101, ranked by callers