MCPcopy Create free account
hub / github.com/apache/arrow / _perform_join_asof

Function _perform_join_asof

python/pyarrow/acero.py:263–351  ·  view source on GitHub ↗

Perform asof join of two tables or datasets. The result will be an output table with the result of the join operation Parameters ---------- left_operand : Table or Dataset The left operand for the join operation. left_on : str The left key (or keys) on whic

(left_operand, left_on, left_by,
                       right_operand, right_on, right_by,
                       tolerance, use_threads=True,
                       output_type=Table)

Source from the content-addressed store, hash-verified

261
262
263def _perform_join_asof(left_operand, left_on, left_by,
264 right_operand, right_on, right_by,
265 tolerance, use_threads=True,
266 output_type=Table):
267 """
268 Perform asof join of two tables or datasets.
269
270 The result will be an output table with the result of the join operation
271
272 Parameters
273 ----------
274 left_operand : Table or Dataset
275 The left operand for the join operation.
276 left_on : str
277 The left key (or keys) on which the join operation should be performed.
278 left_by: str or list[str]
279 The left key (or keys) on which the join operation should be performed.
280 right_operand : Table or Dataset
281 The right operand for the join operation.
282 right_on : str or list[str]
283 The right key (or keys) on which the join operation should be performed.
284 right_by: str or list[str]
285 The right key (or keys) on which the join operation should be performed.
286 tolerance : int
287 The tolerance to use for the asof join. The tolerance is interpreted in
288 the same units as the "on" key.
289 output_type: Table or InMemoryDataset
290 The output type for the exec plan result.
291
292 Returns
293 -------
294 result_table : Table or InMemoryDataset
295 """
296 if not isinstance(left_operand, (Table, ds.Dataset)):
297 raise TypeError(f"Expected Table or Dataset, got {type(left_operand)}")
298 if not isinstance(right_operand, (Table, ds.Dataset)):
299 raise TypeError(f"Expected Table or Dataset, got {type(right_operand)}")
300
301 if not isinstance(left_by, (tuple, list)):
302 left_by = [left_by]
303 if not isinstance(right_by, (tuple, list)):
304 right_by = [right_by]
305
306 # AsofJoin does not return on or by columns for right_operand.
307 right_columns = [
308 col for col in right_operand.schema.names
309 if col not in [right_on] + right_by
310 ]
311 columns_collisions = set(left_operand.schema.names) & set(right_columns)
312 if columns_collisions:
313 raise ValueError(
314 f"Columns {columns_collisions} present in both tables. "
315 "AsofJoin does not support column collisions."
316 )
317
318 # Add the join node to the execplan
319 if isinstance(left_operand, ds.Dataset):
320 left_source = _dataset_to_decl(

Callers

nothing calls this directly

Calls 7

_dataset_to_declFunction · 0.85
TableSourceNodeOptionsFunction · 0.85
InMemoryDatasetMethod · 0.80
TypeErrorFunction · 0.50
typeEnum · 0.50
DeclarationClass · 0.50
to_tableMethod · 0.45

Tested by

no test coverage detected