Apply rotary embeddings to input tensors using the given frequency tensor. This function applies rotary embeddings to the given query 'xq' a

1x1 convolution

This long function is unfortunately doing something very simple and is being very defensive: We are separating out all parameters of the mode

Override of `Tracer.create_proxy`. This override intercepts the recording of every operation and stores away the current traced modul

Args: rank: Unique identifier of each process world_size: Total number of processes

Approximated function.

Training code. Calculates actor and critic loss and performs backprop.

r""" This function is mostly borrowed from the Reinforcement Learning example. See https://github.com/pytorch/examples/tree/main/reinf

Initialize a Transformer model from a ModelArgs object. Args: model_args (ModelArgs): Model configuration arguments.

Builds a batch i.e. (x, f(x)) pair.

create date and time for file save uniqueness example: 2022-05-07-08:31:12_PM'

establish current policies for mixed precision and fsdp wrapping

To ease the accessibility of data based on the class, we will use `group_examples` to group examples based on class.

[Note: On ``init_weights`` vs. ``reset_parameters``] Modules may define ``reset_parameters`` to initialize parameter values.

Loads the Cora dataset. The dataset is downloaded from https://linqs-data.soe.ucsc.edu/public/lbc/cora.tgz.

The graph convolutional operation rquires the normalized adjacency matrix: D^(-1/2) * A * D^(-1/2). This step scales the adjacency matrix su

Builds features i.e. a matrix with columns [x, x^2, x^3, x^4].

Reinitialize learnable parameters.

Reshape frequency tensor for broadcasting it with another tensor. This function reshapes the frequency tensor to have the same shape as the

r""" Run one episode. The agent will tell each oberser to run n_steps.