MCPcopy Create free account
hub / github.com/alibaba/GraphScope / to_data_sink

Function to_data_sink

python/graphscope/framework/dag_utils.py:883–913  ·  view source on GitHub ↗

Dump result to `fd` by drivers in vineyard. Parameters: result (:class:`graphscope.framework.context.ResultDAGNode`): Dataframe or numpy or result hold the object id of vineyard dataframe. fd (str): Such as `hdfs:///tmp/result_path` kwargs (dict, optional): S

(result, fd, storage_options=None, write_options=None, **kwargs)

Source from the content-addressed store, hash-verified

881
882
883def to_data_sink(result, fd, storage_options=None, write_options=None, **kwargs):
884 """Dump result to `fd` by drivers in vineyard.
885
886 Parameters:
887 result (:class:`graphscope.framework.context.ResultDAGNode`):
888 Dataframe or numpy or result hold the object id of vineyard dataframe.
889 fd (str): Such as `hdfs:///tmp/result_path`
890 kwargs (dict, optional): Storage options with respect to output storage type
891
892 Returns:
893 An op to dump result to `fd`.
894 """
895 if storage_options is None:
896 storage_options = {}
897 storage_options.update(kwargs)
898 if write_options is None:
899 write_options = {}
900 write_options.update(kwargs)
901 config = {
902 types_pb2.STORAGE_OPTIONS: utils.s_to_attr(json.dumps(storage_options)),
903 types_pb2.WRITE_OPTIONS: utils.s_to_attr(json.dumps(write_options)),
904 types_pb2.FD: utils.s_to_attr(str(fd)),
905 }
906 op = Operation(
907 result.session_id,
908 types_pb2.DATA_SINK,
909 config=config,
910 inputs=[result.op],
911 output_types=types_pb2.NULL_OUTPUT,
912 )
913 return op
914
915
916def output(

Callers

nothing calls this directly

Calls 2

OperationClass · 0.90
updateMethod · 0.45

Tested by

no test coverage detected