MCPcopy Create free account
hub / github.com/ByConity/ByConity / appendChunk

Method appendChunk

src/Processors/Transforms/WindowTransform.cpp:1338–1596  ·  view source on GitHub ↗

Source from the content-addressed store, hash-verified

1336}
1337
1338void WindowTransform::appendChunk(Chunk & chunk)
1339{
1340// fmt::print(stderr, "new chunk, {} rows, finished={}\n", chunk.getNumRows(),
1341// input_is_finished);
1342// fmt::print(stderr, "chunk structure '{}'\n", chunk.dumpStructure());
1343
1344 // First, prepare the new input block and add it to the queue. We might not
1345 // have it if it's end of data, though.
1346 if (!input_is_finished)
1347 {
1348 if (!chunk.hasRows())
1349 {
1350 // Joins may generate empty input chunks when it's not yet end of
1351 // input. Just ignore them. They probably shouldn't be sending empty
1352 // chunks up the pipeline, but oh well.
1353 return;
1354 }
1355
1356 blocks.push_back({});
1357 auto & block = blocks.back();
1358
1359 // Use the number of rows from the Chunk, because it is correct even in
1360 // the case where the Chunk has no columns. Not sure if this actually
1361 // happens, because even in the case of `count() over ()` we have a dummy
1362 // input column.
1363 block.rows = chunk.getNumRows();
1364
1365 // If we have a (logically) constant column, some Chunks will have a
1366 // Const column for it, and some -- materialized. Such difference is
1367 // generated by e.g. MergingSortedAlgorithm, which mostly materializes
1368 // the constant ORDER BY columns, but in some obscure cases passes them
1369 // through, unmaterialized. This mix is a pain to work with in Window
1370 // Transform, because we have to compare columns across blocks, when e.g.
1371 // searching for peer group boundaries, and each of the four combinations
1372 // of const and materialized requires different code.
1373 // Another problem with Const columns is that the aggregate functions
1374 // can't work with them, so we have to materialize them like the
1375 // Aggregator does.
1376 // Likewise, aggregate functions can't work with LowCardinality,
1377 // so we have to materialize them too.
1378 // Just materialize everything.
1379 auto columns = chunk.detachColumns();
1380 block.original_input_columns = columns;
1381 for (auto & column : columns)
1382 column = recursiveRemoveLowCardinality(std::move(column)->convertToFullColumnIfConst());
1383 block.input_columns = std::move(columns);
1384
1385 // Initialize output columns.
1386 for (auto & ws : workspaces)
1387 {
1388 block.output_columns.push_back(ws.aggregate_function->getReturnType()
1389 ->createColumn());
1390 block.output_columns.back()->reserve(block.rows);
1391 }
1392
1393 // As a debugging aid, assert that all chunks have the same C++ type of
1394 // columns, that also matches the input header, because we often have to
1395 // work across chunks.

Callers

nothing calls this directly

Calls 15

assertSameColumnsFunction · 0.85
isCancelledFunction · 0.85
hasRowsMethod · 0.80
detachColumnsMethod · 0.80
push_backMethod · 0.45
getNumRowsMethod · 0.45
createColumnMethod · 0.45
getReturnTypeMethod · 0.45
reserveMethod · 0.45
getColumnsMethod · 0.45

Tested by

no test coverage detected