内部函数

内部函数

内部函数

内部函数

内部函数

normal select elements

pobject as side input for select elements

注册一个在 pipeline.run() 执行之后的 hook. hook 执行顺序: 注册的 name 进行 sorted 排序结果 todo: deal with callback with parameters. Users can always u

注册一个在 pipeline.run() 执行之前的 hook. hook 执行顺序: 注册的 name 进行 sorted 排序结果 todo: deal with callback with parameters. Users can always u

Add egg file to executor/flume_worker's python path :return: None

Pass sys default encoding to remote side.

test accumulate

test aggregate

test as_object

test cartesian

fail to run in local

test combine

test diff

test filter

test this

test flat_map

pass has no good way to test it

test full join

test group_by

test group_by_key

test intersection

test is empty

test join

test left_join

test map

test max

test max_elements

test min

test min elements

test reduce

test right_join

test sort

test sort by

test substract

test sum

test transform

test union

向Pipeline添加一个压缩文件，使得该文件在运行期自动被解包 Args: file_path (str): 文件路径，目前仅支持HDFS resource_path (str): 运行期访问该文件解压后的路径

save the ptype cache node id for use

Add a dynamic library file(.so) to be packed with the job and set it to LD_LIBRARY_PATH at runtime. Args: file_pat

向Pipeline添加单个文件，使得该文件能够在运行期被访问 Args: file_path(str): 需要添加的文件路径，可以是本地路径或者 HDFS 路径. resource_path(str): local 引擎运行

backend_parser

Indicate the beginning of a group Args: keys (list): keys of the group to start inputs (list): re-iterable inp

Case: test binary operations

config

返回给定PCollection中元素的数量 Args: pcollection (PCollection): 输入PCollection **options: 可配置选项 Returns: PObject: 元素数量

used to create a pipeline

Deserialize the processor This method is used for stream processing, Override this function to deserialize the processor on failover. @param buffer:

distribute as batch

execuate when task cleanup.

execuate when task setup.