A model is a bag-of-word representation of a corpus of documents, where each document vector is a bag of (word, relevance)-items. Vectors can then be compared for similarity using a distance metric. The weighting scheme can be: relative TF, TFIDF (default), BINA
(self, documents=[], weight=TFIDF)
| 768 | class Model(object): |
| 769 | |
| 770 | def __init__(self, documents=[], weight=TFIDF): |
| 771 | """ A model is a bag-of-word representation of a corpus of documents, |
| 772 | where each document vector is a bag of (word, relevance)-items. |
| 773 | Vectors can then be compared for similarity using a distance metric. |
| 774 | The weighting scheme can be: relative TF, TFIDF (default), BINARY, None, |
| 775 | where None means that the original weights are used. |
| 776 | """ |
| 777 | self.description = "" # Description of the dataset: author e-mail, etc. |
| 778 | self._documents = readonlylist() # List of documents (read-only). |
| 779 | self._index = {} # Document.name => Document |
| 780 | self._df = {} # Cache of document frequency per word. |
| 781 | self._cos = {} # Cache of ((d1.id, d2.id), relevance)-items (cosine similarity). |
| 782 | self._ig = {} # Cache of (word, information gain)-items. |
| 783 | self._vector = None # Cache of model vector with all the features in the model. |
| 784 | self._lsa = None # LSA matrix with reduced dimensionality. |
| 785 | self._weight = weight # Weight used in Document.vector (TF, TFIDF, BINARY or None). |
| 786 | self._update() |
| 787 | self.extend(documents) |
| 788 | |
| 789 | @property |
| 790 | def documents(self): |
nothing calls this directly
no test coverage detected