MCPcopy
hub / github.com/unclecode/crawl4ai / chunk

Method chunk

crawl4ai/chunking_strategy.py:25–32  ·  view source on GitHub ↗
(self, text: str)

Source from the content-addressed store, hash-verified

23 self.patterns = patterns
24
25 def chunk(self, text: str) -> list:
26 paragraphs = [text]
27 for pattern in self.patterns:
28 new_paragraphs = []
29 for paragraph in paragraphs:
30 new_paragraphs.extend(re.split(pattern, paragraph))
31 paragraphs = new_paragraphs
32 return paragraphs
33
34# NLP-based sentence chunking
35class NlpSentenceChunking(ChunkingStrategy):

Callers 4

run_oldMethod · 0.45
process_htmlMethod · 0.45
aprocess_htmlMethod · 0.45
process_htmlMethod · 0.45

Calls

no outgoing calls

Tested by

no test coverage detected