| 4 | |
| 5 | |
| 6 | class LinearRegression: |
| 7 | def __init__(self, fit_intercept=True): |
| 8 | r""" |
| 9 | A weighted linear least-squares regression model. |
| 10 | |
| 11 | Notes |
| 12 | ----- |
| 13 | In weighted linear least-squares regression [1]_, a real-valued target |
| 14 | vector, **y**, is modeled as a linear combination of covariates, **X**, |
| 15 | and model coefficients, :math:`\beta`: |
| 16 | |
| 17 | .. math:: |
| 18 | |
| 19 | y_i = \beta^\top \mathbf{x}_i + \epsilon_i |
| 20 | |
| 21 | In this equation :math:`\epsilon_i \sim \mathcal{N}(0, \sigma^2_i)` is |
| 22 | the error term associated with example :math:`i`, and |
| 23 | :math:`\sigma^2_i` is the variance of the corresponding example. |
| 24 | |
| 25 | Under this model, the maximum-likelihood estimate for the regression |
| 26 | coefficients, :math:`\beta`, is: |
| 27 | |
| 28 | .. math:: |
| 29 | |
| 30 | \hat{\beta} = \Sigma^{-1} \mathbf{X}^\top \mathbf{Wy} |
| 31 | |
| 32 | where :math:`\Sigma^{-1} = (\mathbf{X}^\top \mathbf{WX})^{-1}` and |
| 33 | **W** is a diagonal matrix of weights, with each entry inversely |
| 34 | proportional to the variance of the corresponding measurement. When |
| 35 | **W** is the identity matrix the examples are weighted equally and the |
| 36 | model reduces to standard linear least squares [2]_. |
| 37 | |
| 38 | References |
| 39 | ---------- |
| 40 | .. [1] https://en.wikipedia.org/wiki/Weighted_least_squares |
| 41 | .. [2] https://en.wikipedia.org/wiki/General_linear_model |
| 42 | |
| 43 | Parameters |
| 44 | ---------- |
| 45 | fit_intercept : bool |
| 46 | Whether to fit an intercept term in addition to the model |
| 47 | coefficients. Default is True. |
| 48 | |
| 49 | Attributes |
| 50 | ---------- |
| 51 | beta : :py:class:`ndarray <numpy.ndarray>` of shape `(M, K)` or None |
| 52 | Fitted model coefficients. |
| 53 | sigma_inv : :py:class:`ndarray <numpy.ndarray>` of shape `(N, N)` or None |
| 54 | Inverse of the data covariance matrix. |
| 55 | """ |
| 56 | self.beta = None |
| 57 | self.sigma_inv = None |
| 58 | self.fit_intercept = fit_intercept |
| 59 | |
| 60 | self._is_fit = False |
| 61 | |
| 62 | def update(self, X, y, weights=None): |
| 63 | r""" |
no outgoing calls