Initialize any policy-specific parameters that depend on information from the bandit environment.

Randomly initialize the starting GMM parameters.

r""" Compute the (unnormalized) log posterior for the label at index `class_idx` in :py:attr:`labels <numpy_ml.linear_models.GaussianN

Actual computation of NCE loss

Regularized Frobenius loss

Return the least-squares reconstruction loss between X and Xhat

Optimize alpha using Blei's O(n) Newton-Raphson modification for a Hessian with special structure

Optimize model parameter beta β_{t, n} ∝ \sum_{d=1}^D \sum_{i=1}^{N_d} ϕ_{d, t, n} [ i = n]

Optimize variational parameter gamma γ_t = α_t + \sum_{n=1}^{N_d} ϕ_{t, n}

Optimize variational parameter phi ϕ_{t, n} ∝ β_{t, w_n} e^( Ψ(γ_t) )

Replace `bigram` with a single token and update vocab accordingly

Update `Q` using weighted importance sampling. Notes ----- In importance sampling updates, we account for the fact t

Update the `Q` function using the TD(0) Q-learning update: Q[s, a] <- Q[s, a] + lr * ( r + temporal_discount * m

r""" Update the `Q` function using an on-policy first-visit Monte Carlo update. Notes ----- The on-policy fir

Update the Q function using the expected SARSA on-policy TD(0) update: Q[s, a] <- Q[s, a] + lr * [ r + temporal_

Execute a bandit action and return the received reward.

Reset any model-specific parameters. This gets called within the public `self.reset()` method.

Returns a sample from the distribution q(t | x) = N(t_mean, diag(exp(t_log_var))) using the reparameterization trick.

Sample `n_samples_per_episode` thetas from a multivariate Gaussian with mean `theta_mean` and covariance `diag(theta_var)`

Find the optimal split rule (feature index and split threshold) for the data according to `self.criterion`.

Select an arm based on the current context

Perform random-sample one-step tabular Q-planning with prioritized sweeping. Notes ----- This approach uses

Transform a single text string to a list of byte-pair IDs

Update Q using a full-backup version of the TD(0) Q-learning update: Q(s, a) = Q(s, a) + lr * sum_{r, s'} [

Sherman-Morrison update for a single example

Woodbury update for multiple examples

Perform the fast HALS update for H

Perform the fast HALS update for W

Update the policy parameters after an interaction

Return the aggregated prediction across all trees in the RF for each problem. Parameters ---------- predictions : :p

r""" Generate actions according to a softmax policy. Notes ----- The softmax policy assumes that the pmf over actions

r""" Execute the behavior policy--an :math:`\epsilon`-soft policy used to generate actions during training. Parameters

Select an arm and sample from its payoff distribution. Parameters ---------- bandit : :class:`Bandit <numpy_ml.bandi

Get information on `env`'s action space. Parameters ---------- md_action : bool Whether the `env`'s action space is multidim

Compute the gradient of the loss with respect to the layer parameters. Parameters ---------- dLdy : :py:class:`ndarr

Compute the gradient of the loss with respect to the module parameters. Parameters ---------- dLdy : :py:class:`ndar

Run a backward pass across all timesteps in the input. Parameters ---------- dLdA : :py:class:`ndarray <numpy.ndarra

r""" Backprop from layer outputs to inputs. Parameters ---------- dLdY : :py:class:`ndarray <numpy.ndarray>` of shape

r""" Backprop from layer outputs to inputs. Parameters ---------- dLdY : :py:class:`ndarray <numpy.ndarray>` of shape

r""" Backprop from layer outputs to inputs. Parameters ---------- dLdY : :py:class:`ndarray <numpy.ndarray>` of shape

Backprop from layer outputs to inputs. Parameters ---------- dLdY : :py:class:`ndarray <numpy.ndarray>` of shape `(n

Backprop from layer outputs to inputs. Parameters ---------- dLdY : :py:class:`ndarray <numpy.ndarray>` of shape `(n

Backprop from layer outputs to inputs. Parameters ---------- dLdY : :py:class:`ndarray <numpy.ndarray>` of shape `(n

Backprop from layer outputs to inputs. Parameters ---------- dLdY : :py:class:`ndarray <numpy.ndarray>` of shape `(n

Backprop from layer outputs to embedding weights. Notes ----- Because the items in `X` are interpreted as indices, w

Backprop from layer outputs to inputs. Parameters ---------- dLdy : :py:class:`ndarray <numpy.ndarray>` of shape `(n

Compute the gradient of the loss with respect to the layer parameters. Notes ----- Relies on :meth:`~numpy_ml.neural

Compute the gradient of the loss with respect to the layer parameters. Notes ----- Relies on :meth:`~numpy_ml.neural

Backprop from layer outputs to inputs Parameters ---------- dLdY : :py:class:`ndarray <numpy.ndarray>` of shape `(n_

Compute the gradient of the loss with respect to the layer parameters. Notes ----- Relies on :meth:`~numpy_ml.neural

Backprop for a single timestep. Parameters ---------- dLdAt : :py:class:`ndarray <numpy.ndarray>` of shape `(n_ex, n

Backprop for a single timestep. Parameters ---------- dLdAt : :py:class:`ndarray <numpy.ndarray>` of shape `(n_ex, n

VAE backward pass

Compute the gradient of the loss wrt the current network parameters.

Decode bytes (represented as an integer between 0 and 255) to characters in the specified encoding.

Compute the padding necessary to ensure that convolving `X` with a 1D kernel of shape `kernel_shape` and stride `stride` produces outputs wit

A simple sinusoidal filter applied in the Mel-frequency domain. Notes ----- Cepstral lifting helps to smooth the spectral envelope a

Compute the Chebyshev (:math:`L_\infty`) distance between two real vectors Notes ----- The Chebyshev distance between two vectors **

A list of the learned byte pair codewords, decoded into human-readable format

Compare the performance of multiple policies on the same bandit environment, generating a plot for each. Parameters

Return the distribution over proposed next words under the `N`-gram language model. Parameters ---------- wo

A faster (but more memory intensive) implementation of a 1D "convolution" (technically, cross-correlation) of input `X` with a collection of

A slow but more straightforward implementation of a 2D "convolution" (technically, cross-correlation) of input `X` with a collection of kerne

r""" Given the HMM parameterized by :math:`(A, B, \pi)` and an observation sequence :math:`O = o_1, \ldots, o_T`, compute the most pro

Perform a "deconvolution" (more accurately, a transposed convolution) of an input volume `X` with a weight kernel `W`, incorporating stride,

Compute the Euclidean (`L2`) distance between two real vectors Notes ----- The Euclidean distance between two vectors **x** and **y*