MCPcopy Create free account

hub / github.com/baidu/bigflow / functions

Functions5,490 in github.com/baidu/bigflow

↓ 10 callersMethodRemoveUnit
flume/planner/plan.cpp:117
↓ 10 callersMethodRequireBinary
flume/runtime/testing/mock_listener.h:73
↓ 10 callersMethodRun
flume/planner/spark/init_rdd_info_pass.cpp:130
↓ 10 callersMethodRun
flume/planner/common/cache_analysis.cpp:60
↓ 10 callersMethodSerialize
flume/runtime/io/io_format.cpp:60
↓ 10 callersMethodSetup
flume/runtime/common/shuffle_impl.h:145
↓ 10 callersMethodStartShuffle
flume/runtime/common/shuffle_impl.h:169
↓ 10 callersMethodadd_dispatcher
flume/runtime/common/executor_impl_test.cpp:565
↓ 10 callersMethodadd_file
向Pipeline添加单个文件,使得该文件能够在运行期被访问 Args: source (str): 需要添加的文件路径或者比特数据流 resource_path (str): 计算引擎运行时访问该文件的本地路径, 应是相
bigflow_python/python/bigflow/pipeline/pipeline_base.py:108
↓ 10 callersMethodchange_identity
flume/planner/unit.cpp:166
↓ 10 callersMethodconcurrency
flume/core/logical_plan.h:184
↓ 10 callersMethoddeserialize
deserialize
bigflow_python/python/bigflow/serde.py:136
↓ 10 callersMethodemit
bigflow_python/processors/processor.cpp:120
↓ 10 callersMethodend_sub_group
flume/runtime/common/executor_impl.h:359
↓ 10 callersMethodfather
flume/core/logical_plan.h:181
↓ 10 callersMethodget
inner function
bigflow_python/python/bigflow/pipeline/spark_pipeline.py:75
↓ 10 callersFunctionis_list
bigflow_python/common/python.cpp:116
↓ 10 callersFunctioniterator
bigflow_python/common/iterator.h:57
↓ 10 callersMethodleave_scope
(self)
bigflow_python/python/bigflow/core/logical_plan.py:242
↓ 10 callersFunctionlist_len
bigflow_python/common/python.cpp:71
↓ 10 callersMethodpartition
对输出结果进行分组 Args: n (int): 输出结果的分组个数,具体表现为产生n个输出文件,文件内容为各组数据 partition_fn (callable): 用于指定分组方式的函数 Retur
bigflow_python/python/bigflow/output.py:205
↓ 10 callersMethodsetConfig
no comment
bigflow_python/python/bigflow/test/test_base.py:231
↓ 10 callersMethodset_is_prepared
flume/core/logical_plan.h:431
↓ 9 callersMethodAcquireArena
flume/runtime/common/local_dataset.cpp:375
↓ 9 callersMethodAddCachedNodeId
flume/runtime/session.cpp:48
↓ 9 callersMethodBeginKeys
flume/runtime/common/file_cache_manager.cpp:157
↓ 9 callersMethodCombineWith
process this node in COMMON scope
flume/core/logical_plan.h:286
↓ 9 callersMethodDirectOutputs
flume/planner/local/local_planner_test.cpp:123
↓ 9 callersMethodDone
bigflow_python/functors/py_functor_caller.cpp:44
↓ 9 callersMethodGetAllUnits
flume/planner/plan.cpp:125
↓ 9 callersMethodGetStatus
flume/runtime/backend.cpp:218
↓ 9 callersMethodIsAcceptMore
flume/runtime/common/single_dispatcher.cpp:177
↓ 9 callersMethodNext
flume/runtime/common/cache_iterator.cpp:110
↓ 9 callersMethodRun
flume/planner/spark/build_transfer_executor_pass.cpp:152
↓ 9 callersFunctionSerialize
flume/runtime/common/write_cache_executor_test.cpp:92
↓ 9 callersMethodSetCachePathForId
flume/runtime/session.cpp:65
↓ 9 callersFunctionTaskMessageOf
flume/planner/monitor/build_physical_plan_pass.cpp:49
↓ 9 callersMethodTell
flume/runtime/io/gzip_file.cpp:159
↓ 9 callersMethodToString
flume/util/reflection_test.cpp:163
↓ 9 callersMethodUnion
flume/planner/pass_manager.cpp:96
↓ 9 callersMethodadd_node
flume/core/logical_plan.h:228
↓ 9 callersMethodadd_record
flume/runtime/common/transfer_encoding_test.cpp:68
↓ 9 callersMethodadd_split
flume/runtime/common/split_combiner.cpp:31
↓ 9 callersMethodall
flume/util/bitset.h:220
↓ 9 callersMethodbegin_group
bigflow_python/processors/pipe_processor.cpp:286
↓ 9 callersFunctioncp_long
bigflow_python/common/barshal_objector.cpp:92
↓ 9 callersMethodempty
(self)
bigflow_python/python/bigflow/rpc/backend_parser.py:111
↓ 9 callersMethodend_group
bigflow_python/processors/pipe_processor.cpp:347
↓ 9 callersMethodfinish_group
flume/runtime/common/executor_impl.h:68
↓ 9 callersMethodfrom_size
flume/core/logical_plan.h:539
↓ 9 callersMethodignore_group
(self)
bigflow_python/python/bigflow/core/logical_plan.py:600
↓ 9 callersFunctionkey
flume/runtime/spark/cache_input_executor_test.cpp:71
↓ 9 callersFunctionof
创建FieldsDictSerde,用来序列化、反序列化有指定字段的字典。 因为字段已知,则key无需序列化,序列化出来的数据会小于marshal序列化后的结果。 Args: fields_to_types (dict): 可以传入一个字段名组成的列表,
bigflow_python/python/bigflow/schema.py:80
↓ 9 callersFunctionrecord
flume/runtime/common/cache_iterator_test.cpp:92
↓ 9 callersMethodref
flume/runtime/common/partial_executor.cpp:238
↓ 9 callersMethodset_shuffle_type
(self, value)
bigflow_python/python/bigflow/core/logical_plan.py:678
↓ 9 callersMethodsetup
bigflow_python/objectors/set_serde.cpp:29
↓ 9 callersMethodsinker
(self)
bigflow_python/python/bigflow/core/logical_plan.py:507
↓ 9 callersFunctionsort
对于输入PCollection,将其进行排序 Args: pcollection (PCollection): 输入PCollection reverse (bool): 若True则降序排列,否则为升序排列 Returns: P
bigflow_python/python/bigflow/transforms.py:840
↓ 9 callersMethodtake
给定PCollection中的任意n个元素,等同于 :func:`bigflow.transforms.take` Args: n (int or PObject): 元素数量 **options: 可配置参数
bigflow_python/python/bigflow/pcollection.py:385
↓ 9 callersMethodvalue
flume/runtime/spark/kv_buffer.cpp:86
↓ 9 callersMethodvertex
flume/planner/testing/plan_desc.cpp:80
↓ 8 callersMethodAddSunkNodeId
flume/runtime/session.cpp:44
↓ 8 callersMethodClose
flume/runtime/io/io_format.cpp:1085
↓ 8 callersMethodDone
bigflow_python/delegators/python_processor_delegator.cpp:75
↓ 8 callersMethodDump
flume/runtime/common/local_dataset.cpp:385
↓ 8 callersMethodEndGroup
flume/runtime/common/write_cache_executor.cpp:87
↓ 8 callersMethodEndKeys
flume/runtime/common/file_cache_manager.cpp:172
↓ 8 callersMethodRead
flume/runtime/common/file_cache_manager.cpp:119
↓ 8 callersMethodRegister
flume/planner/pass_manager.h:263
↓ 8 callersMethodSourcesBetween
flume/planner/local/local_planner_test.cpp:107
↓ 8 callersMethodToProtoMessage
flume/core/logical_plan.cpp:91
↓ 8 callersMethodWithConcurrency
flume/core/logical_plan.h:221
↓ 8 callersMethodWriteRecord
flume/runtime/io/io_format.cpp:1254
↓ 8 callersMethod__set_scope_type
(self, scope_type)
bigflow_python/python/bigflow/core/logical_plan.py:785
↓ 8 callersMethodas_type
通过kv_deserializer反序列化读取的(Key, Value) kv_deserializer的期望签名为: kv_deserializer(key: str, value: str) => object
bigflow_python/python/bigflow/input.py:548
↓ 8 callersMethodclose
(self)
bigflow_python/python/bigflow/core/tests/test_helper.py:83
↓ 8 callersMethodencode
flume/core/timer_queue.h:72
↓ 8 callersFunctionfind_or_insert_vertex
(node_id, vertices)
bigflow_python/python/bigflow/core/tests/test_helper.py:194
↓ 8 callersMethodget_dealt_side_inputs_tuple
(side_input_tuple)
bigflow_python/python/bigflow/util/side_input_util.py:61
↓ 8 callersMethodhas_next
flume/runtime/spark/kv_buffer.cpp:64
↓ 8 callersMethodkey_reader
(self)
bigflow_python/python/bigflow/core/logical_plan.py:681
↓ 8 callersMethodloader
(self)
bigflow_python/python/bigflow/core/logical_plan.py:443
↓ 8 callersMethodreplace
flume/runtime/common/executor_impl.cpp:207
↓ 8 callersMethodrunner
flume/runtime/common/executor_impl.h:329
↓ 8 callersMethodset_init_hook
向Pipeline设置一个初始化钩子,使得Pipeline能够在进程启动时依次执行(按钩子名字排序) Args: name (str): 钩子名称, 用户应使用 str 作为钩子名字, 同时钩子应使用 ascii 码中字符.
bigflow_python/python/bigflow/pipeline/pipeline_base.py:172
↓ 8 callersFunctionsplit_of
flume/runtime/common/split_combiner_test.cpp:51
↓ 8 callersMethodtest
flume/util/bitset.h:207
↓ 8 callersMethodto_plan
flume/planner/testing/plan_desc.cpp:201
↓ 8 callersFunctiontransform
对给定PCollection进行任意的变换,结果为另一个PCollection transform有两种形式,形式一: 基本原型为`transform(pcollection, initializer, transformer, finalizer, *side_in
bigflow_python/python/bigflow/transforms.py:986
↓ 8 callersFunctiontuple_len
bigflow_python/common/python.cpp:75
↓ 7 callersFunctionCacheRecordObjectorEntity
flume/planner/common/cache_util.h:153
↓ 7 callersMethodClear
flume/runtime/util/object_holder.h:45
↓ 7 callersMethodDebugString
flume/planner/spark/spark_planner.cpp:193
↓ 7 callersFunctionEffectiveKeyNum
flume/planner/testing/tag_desc.cpp:385
↓ 7 callersMethodGetScopeLevel
flume/runtime/common/single_dispatcher.cpp:168
↓ 7 callersFunctionGetToftStylePath
flume/util/path_util.cpp:43
↓ 7 callersMethodGetWriter
flume/runtime/common/file_cache_manager.cpp:223
↓ 7 callersMethodIntSerde
bigflow_python/objectors/int_serde.h:41
↓ 7 callersFunctionIsHdfsPath
flume/util/path_util.cpp:99
← previousnext →301–400 of 5,490, ranked by callers