MCPcopy Create free account
hub / github.com/Datakitpage/Datakit / processBatch

Function processBatch

frontend/src/hooks/stream/useStreamingCSVParser.ts:240–268  ·  view source on GitHub ↗
()

Source from the content-addressed store, hash-verified

238
239 // Helper function to process a batch
240 async function processBatch() {
241 // Update stats
242 stats.rowsProcessed += currentBatch.length;
243
244 // Calculate processing speed and estimated time
245 updateProcessingStats(stats);
246
247 // Call custom chunk handler if provided
248 if (onChunkParsed) {
249 await onChunkParsed(currentBatch, headers, stats.chunksProcessed === 1);
250 }
251
252 // Clear the batch
253 const processedBatch = currentBatch;
254 currentBatch = [];
255
256 // Yield control to browser if enough time has passed
257 const currentTime = Date.now();
258 if (currentTime - lastYieldTime > 16) { // ~60fps
259 await new Promise(resolve => {
260 if (typeof requestAnimationFrame !== 'undefined') {
261 requestAnimationFrame(resolve);
262 } else {
263 setTimeout(resolve, 0);
264 }
265 });
266 lastYieldTime = currentTime;
267 }
268 }
269
270 } catch (err) {
271 const errorMessage = err instanceof Error ? err.message : 'Unknown error parsing CSV';

Callers 1

parseCSVStreamFunction · 0.85

Calls 1

updateProcessingStatsFunction · 0.85

Tested by

no test coverage detected