MCPcopy Create free account

hub / github.com/Yelp/mrjob / functions

Functions4,468 in github.com/Yelp/mrjob

Methodarchives
(self)
tests/test_job.py:1668
Methodargs_for_step
(*args, **kwargs)
tests/test_hadoop.py:1204
MethodassertJobHandlesUndecodableInput
(self, job_args=())
tests/test_job.py:413
MethodassertJobHandlesUnencodableOutput
(self, job_args=())
tests/test_job.py:437
MethodassertScriptDownloads
(path, name=None)
tests/test_emr.py:1554
MethodassertScriptDownloads
(path, name=None)
tests/test_dataproc.py:1040
Methodauth_default
(self, scopes=None)
tests/mock_google/case.py:96
Methodc_combine_counts
Sum up all those 1s before passing data off to the reducer
mrjob/examples/mr_next_word_stats.py:56
Methodcall_and_maybe_retry
(*args, **kwargs)
mrjob/retry.py:102
Methodcan_handle_path
(self, path)
mrjob/fs/hadoop.py:72
Methodcan_handle_path
(self, path)
mrjob/fs/gcs.py:95
Methodcan_handle_path
(self, path)
mrjob/fs/local.py:33
Methodcan_handle_path
(self, path)
mrjob/fs/ssh.py:167
Methodcan_handle_path
(self, path)
mrjob/fs/s3.py:105
Methodchange_back_directory
()
tests/test_local.py:301
Methodcheck_script
(path)
tests/test_emr.py:5427
Methodcluster_client
(self)
mrjob/dataproc.py:375
Methodcluster_client
(self, channel=None, credentials=None)
tests/mock_google/case.py:107
Methodcombiner
Re-define this to define the combiner for a one-step job. Yields one or more tuples of ``(out_key, out_value)`` :param key: A key wh
mrjob/job.py:295
Methodcombiner
(self, word, counts)
mrjob/examples/mr_words_containing_u_freq_count.py:36
Methodcombiner
(self, word, counts)
mrjob/examples/mr_word_freq_count.py:29
Methodcombiner
(self, key, values)
tests/mr_hadoop_format_job.py:30
Methodcombiner
(self, key, values)
tests/mr_two_step_job.py:27
Methodcombiner
(self, key, values)
tests/test_job.py:150
Methodcombiner
(self, name, counts)
tests/mr_word_count.py:32
Methodcombiner
(self, key, values)
tests/mr_two_step_hadoop_format_job.py:60
Methodcombiner
(self, word, counts)
tests/spark/test_harness.py:59
Methodcombiner
(self, word, counts)
tests/spark/test_harness.py:66
Methodcombiner
(self, word, counts)
tests/spark/test_harness.py:74
Methodcombiner
(self, word, values)
tests/spark/test_harness.py:86
Methodcombiner_cmd
Re-define this to define the combiner for a one-step job **as a shell command.** If you define your mapper this way, the command will
mrjob/job.py:451
Methodcombiner_cmd
(self)
tests/mr_word_freq_count_with_combiner_cmd.py:26
Methodcombiner_cmd
(self)
tests/test_job.py:1058
Methodcombiner_count_words
(self, word, counts)
mrjob/examples/mr_most_used_word.py:61
Methodcombiner_final
Re-define this to define an action to run after the combiner reaches the end of input. Yields one or more tuples of ``(out_key, out_v
mrjob/job.py:440
Methodcombiner_init
Re-define this to define an action to run before the combiner processes any input. One use for this function is to initialize combine
mrjob/job.py:426
Methodcombiner_init
(self)
tests/test_job.py:147
Methodcombiner_pre_filter
Re-define this to specify a shell command to filter the combiner's input before it gets to your job's combiner in a one-step job. For
mrjob/job.py:464
Methodcombiner_pre_filter
(self)
tests/test_job.py:1076
Methodcombiner_pre_filter
(self)
tests/spark/test_harness.py:97
Methodconfigure_args
(self)
mrjob/examples/mr_spark_most_used_word.py:34
Methodconfigure_args
(self)
mrjob/examples/mr_grep.py:23
Methodconfigure_args
(self)
mrjob/examples/mr_jar_step_example.py:46
Methodconfigure_args
(self)
mrjob/examples/mr_most_used_word.py:37
Methodconfigure_args
(self)
mrjob/examples/mr_log_sampler.py:48
Methodconfigure_args
Add command-line options specific to this script.
mrjob/examples/mr_text_classifier.py:163
Methodconfigure_args
(self)
tests/mr_jar_and_streaming.py:25
Methodconfigure_args
(self)
tests/mr_filter_job.py:29
Methodconfigure_args
(self)
tests/mr_spark_script.py:24
Methodconfigure_args
(self)
tests/mr_pass_thru_arg_test.py:19
Methodconfigure_args
(self)
tests/test_job.py:2059
Methodconfigure_args
(self)
tests/mr_spark_harness.py:34
Methodconfigure_args
(self)
tests/test_sim.py:169
Methodconfigure_args
(self)
tests/mr_spark_jar.py:24
Methodconfigure_args
(self)
tests/mr_cmd_job.py:29
Methodconfigure_args
(self)
tests/mr_spark_os_walk.py:25
Methodconfigure_args
(self)
tests/mr_runner.py:21
Methodconfigure_args
(self)
tests/mr_doubler.py:23
Methodconfigure_args
(self)
tests/mr_jar_with_generic_args.py:26
Methodconfigure_args
(self)
tests/mr_tower_of_powers.py:33
Methodconfigure_args
(self)
tests/mr_just_a_jar.py:24
Methodconfigure_args
(self)
tests/mr_null_spark.py:23
Methodconfigure_args
(self)
tests/test_runner.py:1024
Methodcount_by_host_reducer
(self, host, phone_urls)
mrjob/examples/mr_phone_to_url.py:107
Methodcount_ngram_freq
Reducer: Combine information about how many times each ngram appears for docs in/not in each category. Dump ngrams that appear in very
mrjob/examples/mr_text_classifier.py:262
Methodcounters
(self)
mrjob/hadoop.py:572
Methodcounters
Get counters associated with this run in this form:: [{'group name': {'counter1': 1, 'counter2': 2}}, {'group name': ...}]
mrjob/runner.py:659
Methodcounters
(self)
mrjob/dataproc.py:1021
Methodcounters
(self)
mrjob/spark/runner.py:213
Methodcreated
(cluster)
tests/mock_boto3/emr.py:1314
Functiondecorator
(f)
mrjob/cmd.py:53
Methoddelete
(self)
tests/mock_google/storage.py:144
Methoddelete_bucket
(self, bucket)
mrjob/fs/gcs.py:282
Functiondepart_noop
(self, node)
docs/options_extension.py:165
Methoddirs
(self)
tests/test_job.py:1582
Functiondoctree_resolved
(app, doctree, fromdocname)
docs/options_extension.py:81
Methoddownload_to_file
(self, file_obj)
tests/mock_google/storage.py:165
Methoddu
(self, path_glob)
mrjob/fs/composite.py:135
Methoddu
Get the size of a file or directory (recursively), or 0 if it doesn't exist.
mrjob/fs/hadoop.py:190
Methoddu
Get the size of all files matching path_glob.
mrjob/fs/gcs.py:98
Methoddu
(self, path_glob)
mrjob/fs/local.py:36
Methoddu
(self, path_glob)
mrjob/fs/ssh.py:170
Methoddu
Get the size of all files matching path_glob.
mrjob/fs/s3.py:108
Methoddummy_mapper_init
(self)
tests/mr_sort_values_and_more.py:30
Methodemit
(self, record)
mrjob/util.py:45
Functionerror
(msg)
mrjob/options.py:1720
Methodexists
(self, path_glob)
mrjob/fs/composite.py:141
Methodexists
Does the given path exist? If dest is a directory (ends with a "/"), we check if there are any files starting with that path.
mrjob/fs/hadoop.py:300
Methodexists
Does the given path exist? If dest is a directory (ends with a "/"), we check if there are any files starting with that path.
mrjob/fs/gcs.py:194
Methodexists
(self, path_glob)
mrjob/fs/local.py:58
Methodexists
(self, path_glob)
mrjob/fs/ssh.py:205
Methodexists
Does the given path exist? If dest is a directory (ends with a "/"), we check if there are any files starting with that path.
mrjob/fs/s3.py:178
Methodextract_phone_and_url_mapper
Read in .wet file, and extract phone ant URL
mrjob/examples/mr_phone_to_url.py:82
Methodfake_create_mrjob_zip
(mocked_runner, *args, **kwargs)
tests/mock_boto3/case.py:75
Methodfake_getuser
()
tests/test_runner.py:123
Methodfiles
(self)
tests/test_job.py:1494
Methodfilter_path
(path)
mrjob/bin.py:526
Methodforbidding_boto3_client
(service_name, **kwargs)
tests/test_emr.py:580
Methodfs
:py:class:`mrjob.fs.base.Filesystem` object for HDFS and the local filesystem.
mrjob/hadoop.py:191
Methodfs
:py:class:`~mrjob.fs.base.Filesystem` object for the local filesystem.
mrjob/runner.py:470
← previousnext →1,501–1,600 of 4,468, ranked by callers