Return a sorted set of stopwords if ``text`` has less than ``min_tokens`` tokens and contains STOPWORDS or None. Stopwords in short texts may make exact matching inaccurate.
(text, min_tokens=4)
| 2213 | |
| 2214 | |
| 2215 | def get_stopwords_in_short_text(text, min_tokens=4): |
| 2216 | """ |
| 2217 | Return a sorted set of stopwords if ``text`` has less than ``min_tokens`` tokens and contains |
| 2218 | STOPWORDS or None. |
| 2219 | Stopwords in short texts may make exact matching inaccurate. |
| 2220 | """ |
| 2221 | tokens = list(index_tokenizer(text, stopwords=frozenset(), preserve_case=False)) |
| 2222 | if len(tokens) < min_tokens: |
| 2223 | tokens = set(tokens) |
| 2224 | return tokens.intersection(STOPWORDS) |
| 2225 | |
| 2226 | |
| 2227 | def has_only_lower_license_keys(license_expression, licensing=Licensing()): |
no test coverage detected