MCPcopy Create free account
hub / github.com/apache/arrow / _perform_join

Function _perform_join

python/pyarrow/acero.py:83–261  ·  view source on GitHub ↗

Perform join of two tables or datasets. The result will be an output table with the result of the join operation Parameters ---------- join_type : str One of supported join types. left_operand : Table or Dataset The left operand for the join operation.

(join_type, left_operand, left_keys,
                  right_operand, right_keys,
                  left_suffix=None, right_suffix=None,
                  use_threads=True, coalesce_keys=False,
                  output_type=Table, filter_expression=None)

Source from the content-addressed store, hash-verified

81
82
83def _perform_join(join_type, left_operand, left_keys,
84 right_operand, right_keys,
85 left_suffix=None, right_suffix=None,
86 use_threads=True, coalesce_keys=False,
87 output_type=Table, filter_expression=None):
88 """
89 Perform join of two tables or datasets.
90
91 The result will be an output table with the result of the join operation
92
93 Parameters
94 ----------
95 join_type : str
96 One of supported join types.
97 left_operand : Table or Dataset
98 The left operand for the join operation.
99 left_keys : str or list[str]
100 The left key (or keys) on which the join operation should be performed.
101 right_operand : Table or Dataset
102 The right operand for the join operation.
103 right_keys : str or list[str]
104 The right key (or keys) on which the join operation should be performed.
105 left_suffix : str, default None
106 Which suffix to add to left column names. This prevents confusion
107 when the columns in left and right operands have colliding names.
108 right_suffix : str, default None
109 Which suffix to add to the right column names. This prevents confusion
110 when the columns in left and right operands have colliding names.
111 use_threads : bool, default True
112 Whether to use multithreading or not.
113 coalesce_keys : bool, default False
114 If the duplicated keys should be omitted from one of the sides
115 in the join result.
116 output_type: Table or InMemoryDataset
117 The output type for the exec plan result.
118 filter_expression : pyarrow.compute.Expression
119 Residual filter which is applied to matching row.
120
121 Returns
122 -------
123 result_table : Table or InMemoryDataset
124 """
125 if not isinstance(left_operand, (Table, ds.Dataset)):
126 raise TypeError(f"Expected Table or Dataset, got {type(left_operand)}")
127 if not isinstance(right_operand, (Table, ds.Dataset)):
128 raise TypeError(f"Expected Table or Dataset, got {type(right_operand)}")
129
130 # Prepare left and right tables Keys to send them to the C++ function
131 left_keys_order = {}
132 if not isinstance(left_keys, (tuple, list)):
133 left_keys = [left_keys]
134 for idx, key in enumerate(left_keys):
135 left_keys_order[key] = idx
136
137 right_keys_order = {}
138 if not isinstance(right_keys, (list, tuple)):
139 right_keys = [right_keys]
140 for idx, key in enumerate(right_keys):

Callers 5

test_joins_corner_casesFunction · 0.90
test_joinsFunction · 0.90

Calls 10

_dataset_to_declFunction · 0.85
TableSourceNodeOptionsFunction · 0.85
HashJoinNodeOptionsFunction · 0.85
lenFunction · 0.85
InMemoryDatasetMethod · 0.80
TypeErrorFunction · 0.50
typeEnum · 0.50
DeclarationClass · 0.50
appendMethod · 0.45
to_tableMethod · 0.45

Tested by 5

test_joins_corner_casesFunction · 0.72
test_joinsFunction · 0.72