MCPcopy Create free account

hub / github.com/baidu/bigflow / functions

Functions5,490 in github.com/baidu/bigflow

⨍ Functions 5,490◇ Types & classes 1,763
↓ 2 callersMethodfinish_shuffle
flume/runtime/common/shuffle_executor.h:144
↓ 2 callersMethodfs_get
Wraps console command 'hadoop fs -get <source> <target>' Args: source (str): path of source target (str): path
bigflow_python/python/bigflow/util/hadoop_client.py:116
↓ 2 callersMethodfs_mv
Wraps console command 'hadoop fs -mv <source> <target>' Args: source (str): path of source target (str): path
bigflow_python/python/bigflow/util/hadoop_client.py:146
↓ 2 callersFunctiongenerate_line
(byte_list)
resource_library.py:41
↓ 2 callersMethodget
inner function
bigflow_python/python/bigflow/pipeline/local_pipeline.py:62
↓ 2 callersMethodget_counters
bigflow_python/rpc/impls/backends_impl.h:344
↓ 2 callersFunctionget_entity_config_str
bigflow_python/common/python.cpp:259
↓ 2 callersFunctionget_entity_reflect_str
bigflow_python/common/python.cpp:248
↓ 2 callersFunctionget_formatted_exception
bigflow_python/common/python.cpp:304
↓ 2 callersFunctionget_next_uri
flume/runtime/io/io_format.cpp:565
↓ 2 callersFunctionget_out_fields_serde
内部函数
bigflow_python/python/bigflow/future/fields.py:101
↓ 2 callersFunctionget_serde_of_field
get serde of field
bigflow_python/python/bigflow/future/fields.py:78
↓ 2 callersMethodget_size
获得所有读取文件在文件系统中的大小 Returns: int: 文件大小，以字节为单位
bigflow_python/python/bigflow/input.py:180
↓ 2 callersMethodget_timestamp
bigflow_python/delegators/python_time_reader_delegator.cpp:72
↓ 2 callersFunctionhelper
(keys, current)
bigflow_python/python/bigflow/util/utils.py:71
↓ 2 callersFunctionhtobe64
flume/runtime/common/local_dataset.cpp:195
↓ 2 callersMethodinitialize
flume/runtime/spark/cache_input_executor.cpp:37
↓ 2 callersMethodinput_executor
flume/runtime/spark/spark_task.cpp:116
↓ 2 callersMethodinput_size
(self)
bigflow_python/python/bigflow/core/logical_plan.py:612
↓ 2 callersFunctionis_python_bool
bigflow_python/common/python.h:108
↓ 2 callersMethodis_stateful
ProcessNode is stateful
bigflow_python/python/bigflow/core/logical_plan.py:589
↓ 2 callersFunctionjni_throw_bigflow_runtime_exception
bigflow_python/bigflow_python_runtime.cpp:68
↓ 2 callersMethodkey_compare
flume/runtime/common/shuffle_executor.h:284
↓ 2 callersMethodkey_name
flume/runtime/common/shuffle_runner.h:136
↓ 2 callersMethodlaunch
bigflow_python/rpc/impls/backends_impl.h:95
↓ 2 callersFunctionlist_append_item
bigflow_python/common/python.cpp:62
↓ 2 callersFunctionmin
Implementation of transforms.max()
bigflow_python/python/bigflow/transform_impls/min.py:26
↓ 2 callersMethodmin
取最小元素, 等同于 :func:`bigflow.transforms.min(self, key) <bigflow.transforms.min>` Args: key (function, optiona
bigflow_python/python/bigflow/pcollection.py:318
↓ 2 callersFunctionnew_shuffle_executor
flume/runtime/common/shuffle_executor.cpp:336
↓ 2 callersMethodnode
返回PTable所对应的Node Returns: LogicalPlan.Node: node Raises: BigflowPlanningException: 若无法得到Node
bigflow_python/python/bigflow/ptable.py:78
↓ 2 callersMethodon_input_done
called at input scope level
flume/runtime/common/executor_impl.h:315
↓ 2 callersMethodon_input_ready
called at input scope level
flume/runtime/common/executor_impl.h:304
↓ 2 callersMethodopen
用户可以重写该方法。 传入参数partition表示这是第几个partition
bigflow_python/python/bigflow/output.py:453
↓ 2 callersMethodopen
(self)
bigflow_python/python/bigflow/util/process_util.py:59
↓ 2 callersMethodoperator[]
flume/util/bitset.h:227
↓ 2 callersFunctionprint_exec
(stream)
bigflow_python/python/bigflow/core/serde/cloudpickle.py:862
↓ 2 callersMethodpriority_output
flume/runtime/common/executor_impl.h:341
↓ 2 callersMethodrecent_cmds
get all cmds
bigflow_python/python/bigflow/test/mock_hadoop_client.py:114
↓ 2 callersFunctionreduce
对于属于PCollection，使用给定的fn将所有元素规约为单个元素 假设输入类型为I，fn的期望签名为 fn(I1, I2) => I，即输出的类型必须与输入相同 (fn的第一个参数允许被修改) Args: pcollection (PCollecti
bigflow_python/python/bigflow/transforms.py:815
↓ 2 callersFunctionregister_classes
bigflow_python/register.cpp:102
↓ 2 callersMethodregister_pipeline
bigflow_python/rpc/impls/backends_impl.h:33
↓ 2 callersMethodrelative_path
flume/runtime/resource.cpp:230
↓ 2 callersMethodrelease_count
flume/util/cached_allocator.h:95
↓ 2 callersMethodrun_check_parallelize_case
no comment
bigflow_python/python/bigflow/pipeline/test/pipeline_test.py:414
↓ 2 callersMethodsave_function_tuple
Pickles an actual func object. A func comprises: code, globals, defaults, closure, and dict. We extract and save these, injecting r
bigflow_python/python/bigflow/core/serde/cloudpickle.py:275
↓ 2 callersMethodsave_global
(self, obj, name=None, pack=struct.pack)
bigflow_python/python/bigflow/core/serde/cloudpickle.py:407
↓ 2 callersFunctionselect
对每条数据选择一些字段进行变换 Args: pcollection (SchemaPCollection): 输入数据集SchemaPCollection(用来表示结构化的, 带字段的PCollection), 可以当作每个元素是一个dict的PCollectio
bigflow_python/python/bigflow/schema.py:251
↓ 2 callersMethodset
(self, obj, val)
bigflow_python/python/bigflow/test/magic.py:40
↓ 2 callersMethodset_debug_info
flume/core/logical_plan.h:256
↓ 2 callersMethodset_done_fn
bigflow_python/functors/py_functor_caller.cpp:58
↓ 2 callersMethodset_effective_key_num
flume/core/logical_plan.h:476
↓ 2 callersMethodset_emitter
bigflow_python/delegators/python_processor_delegator.h:111
↓ 2 callersMethodset_environment
(self, env)
bigflow_python/python/bigflow/core/logical_plan.py:989
↓ 2 callersMethodset_exception_handler
bigflow_python/python_interpreter.cpp:127
↓ 2 callersMethodset_exception_handler_with_error_msg
bigflow_python/python_interpreter.cpp:137
↓ 2 callersMethodset_fini_hook
向Pipeline设置一个结束钩子，使得Pipeline能够在进程结束前依次执行 Args: name (str): 钩子名称 fn (callable): 方法名称
bigflow_python/python/bigflow/pipeline/pipeline_base.py:181
↓ 2 callersMethodset_least_prepared_inputs
flume/core/logical_plan.h:470
↓ 2 callersMethodset_objector
flume/core/logical_plan.h:259
↓ 2 callersFunctionset_of
Return an optional set serde
bigflow_python/python/bigflow/serde.py:754
↓ 2 callersMethodset_partition
flume/runtime/common/shuffle_executor.cpp:107
↓ 2 callersMethodshuffle_input_executor
flume/runtime/spark/spark_task.cpp:124
↓ 2 callersMethodsort_and_group_by
(self, key_reader_obj)
bigflow_python/python/bigflow/core/logical_plan.py:278
↓ 2 callersMethodstart_shuffle
flume/runtime/common/shuffle_executor.h:138
↓ 2 callersMethodtearDown
(self)
bigflow_python/python/bigflow/test/test_base.py:80
↓ 2 callersFunctionthrow_exception_to_client_without_abortion
bigflow_python/common/python.cpp:398
↓ 2 callersFunctionto_list_pobject
对于给定的PCollection，聚合为PObject，PObject的内容为list Args: pvalue (PCollection): 输入 **options: 可配置选项 Returns: PObject: 聚合后的li
bigflow_python/python/bigflow/transforms.py:1209
↓ 2 callersFunctionto_pcollection
inner fn
bigflow_python/python/bigflow/transform_impls/cartesian.py:60
↓ 2 callersMethodto_proto_message
(self)
bigflow_python/python/bigflow/core/entity.py:167
↓ 2 callersFunctiontuple_to_dict
从每个元素是tuple的PCollection转化成一个SchemaPCollection。 Args: pcollection (PCollection): 输入PCollection，每个元素是一个tuple fields (list): fi
bigflow_python/python/bigflow/schema.py:203
↓ 2 callersMethodunion
(self, a, b)
bigflow_python/python/bigflow/pipeline/test/testdata/disjointset.py:35
↓ 2 callersMethodunit
flume/planner/testing/plan_test_helper.h:358
↓ 2 callersMethodunregister_pipeline
bigflow_python/rpc/impls/backends_impl.h:71
↓ 2 callersMethodvertex_concurrency
get vertex concurrency
bigflow_python/python/bigflow/test/mock_hadoop_client.py:77
↓ 2 callersMethodwait_status
等待指定的运行状态 Args: status: 要等待的运行状态 options: timeout - 超时时间，默认1800s inspection_
bigflow_python/python/bigflow/pipeline/pipeline_status.py:55
↓ 2 callersMethodwildcard_case
Case: test wildcard
bigflow_python/python/bigflow/test/input_test.py:210
↓ 2 callersMethodwindow_by
(self, window, trigger=None)
bigflow_python/python/bigflow/core/logical_plan.py:355
↓ 2 callersFunctionworking_scope
Returns the working scope of a given PType instance Args: pvalue (PType): PType instance Returns: LogicalPlan.Scope: work
bigflow_python/python/bigflow/util/broadcast.py:55
↓ 2 callersFunctionwrite_doodle
flume/runtime/common/cache_iterator_test.cpp:106
↓ 1 callersMethodAbort
flume/runtime/spark/spark_backend.cpp:234
↓ 1 callersMethodAccept
flume/planner/rule_dispatcher.h:59
↓ 1 callersMethodAddDynamicLibraryFromBytes
flume/runtime/resource.cpp:254
↓ 1 callersMethodAddFileFromBytes
flume/runtime/resource.cpp:296
↓ 1 callersMethodAddJavaLibrary
flume/runtime/resource.cpp:263
↓ 1 callersFunctionAddJavaLibraryFromBytes
flume/runtime/spark/spark_backend.cpp:71
↓ 1 callersMethodAddKey
flume/core/testing/processor_test_helper.cpp:44
↓ 1 callersMethodAddPreparedInput
flume/core/testing/processor_test_helper.cpp:48
↓ 1 callersMethodAddPythonLibraryFromBytes
flume/runtime/resource.cpp:279
↓ 1 callersMethodAddReference
flume/runtime/common/local_dataset.cpp:223
↓ 1 callersMethodAllocate
flume/util/cached_allocator.h:42
↓ 1 callersMethodAllocateDataset
flume/runtime/common/local_dataset.cpp:171
↓ 1 callersFunctionBasicMessageForProcessorExecutor
flume/runtime/executor_factory_test.cpp:84
↓ 1 callersMethodBeginGroup
flume/core/testing/processor_test_helper_test.cpp:45
↓ 1 callersMethodBeginGroup
flume/runtime/common/executor_impl.h:288
↓ 1 callersMethodBeginGroup
flume/runtime/common/partial_executor.cpp:174
↓ 1 callersMethodBeginSubGroup
flume/runtime/common/executor_base_test.cpp:90
↓ 1 callersMethodBoolSerde
bigflow_python/objectors/bool_serde.h:41
↓ 1 callersMethodBuildPlan
flume/planner/spark/spark_planner.cpp:171
↓ 1 callersMethodBuildPlan
flume/planner/monitor/monitor_planner.cpp:86
↓ 1 callersMethodBuildPlan
flume/planner/local/local_planner.cpp:149
↓ 1 callersFunctionCacheFrom
flume/planner/spark/testing/testing_helper.cpp:106
← previousnext →901–1,000 of 5,490, ranked by callers