Gaussian Mixture Model: clusters with Gaussian prior. Finds clusters by repeatedly performing Expectation–Maximization (EM) algorithm on the dataset. GMM assumes the datasets is distributed in multivariate Gaussian, and tries to find the underlying structure of the Gaussian, i.e. mean a
| 11 | |
| 12 | |
| 13 | class GaussianMixture(BaseEstimator): |
| 14 | """Gaussian Mixture Model: clusters with Gaussian prior. |
| 15 | |
| 16 | Finds clusters by repeatedly performing Expectation–Maximization (EM) algorithm |
| 17 | on the dataset. GMM assumes the datasets is distributed in multivariate Gaussian, |
| 18 | and tries to find the underlying structure of the Gaussian, i.e. mean and covariance. |
| 19 | E-step computes the "responsibility" of the data to each cluster, given the mean |
| 20 | and covariance; M-step computes the mean, covariance and weights (prior of each |
| 21 | cluster), given the responsibilities. It iterates until the total likelihood |
| 22 | changes less than the tolerance. |
| 23 | |
| 24 | |
| 25 | Parameters |
| 26 | ---------- |
| 27 | |
| 28 | K : int |
| 29 | The number of clusters into which the dataset is partitioned. |
| 30 | max_iters: int |
| 31 | The maximum iterations of assigning points to the perform EM. |
| 32 | Short-circuited by the assignments converging on their own. |
| 33 | init: str, default 'random' |
| 34 | The name of the method used to initialize the first clustering. |
| 35 | |
| 36 | 'random' - Randomly select values from the dataset as the K centroids. |
| 37 | 'kmeans' - Initialize the centroids, covariances, weights with KMeams's clusters. |
| 38 | tolerance: float, default 1e-3 |
| 39 | The tolerance of difference of the two latest likelihood for convergence. |
| 40 | """ |
| 41 | |
| 42 | y_required = False |
| 43 | |
| 44 | def __init__(self, K=4, init="random", max_iters=500, tolerance=1e-3): |
| 45 | self.K = K |
| 46 | self.max_iters = max_iters |
| 47 | self.init = init |
| 48 | self.assignments = None |
| 49 | self.likelihood = [] |
| 50 | self.tolerance = tolerance |
| 51 | |
| 52 | def fit(self, X, y=None): |
| 53 | """Perform Expectation–Maximization (EM) until converged.""" |
| 54 | self._setup_input(X, y) |
| 55 | self._initialize() |
| 56 | for _ in range(self.max_iters): |
| 57 | self._E_step() |
| 58 | self._M_step() |
| 59 | if self._is_converged(): |
| 60 | break |
| 61 | |
| 62 | def _initialize(self): |
| 63 | """Set the initial weights, means and covs (with full covariance matrix). |
| 64 | |
| 65 | weights: the prior of the clusters (what percentage of data does a cluster have) |
| 66 | means: the mean points of the clusters |
| 67 | covs: the covariance matrix of the clusters |
| 68 | """ |
| 69 | self.weights = np.ones(self.K) |
| 70 | if self.init == "random": |