MCPcopy Create free account

hub / github.com/ArchieGertsman/spark-sched-sim / functions

Functions248 in github.com/ArchieGertsman/spark-sched-sim

⨍ Functions 248◇ Types & classes 41
↓ 1 callersMethod_validate_cache
(self, obs: dict[str, Any])
schedulers/decima/env_wrapper.py:145
↓ 1 callersMethod_write_stats
( self, epoch: int, learning_stats: dict, stats_list: Iterable[dict],
trainers/trainer.py:305
↓ 1 callersMethodadd_job_pool
(self, pool_key: JobPoolKey)
spark_sched_sim/components/executor_tracker.py:72
↓ 1 callersMethodadd_reset
(self, step: int)
trainers/rollout_worker.py:41
↓ 1 callersMethodadd_stage_pool
(self, pool_key: StagePoolKey)
spark_sched_sim/components/executor_tracker.py:82
↓ 1 callersMethodattach_executor
(self, executor: Executor)
spark_sched_sim/components/job.py:81
↓ 1 callersMethodaverage
(self, ts_list, ys_list)
trainers/utils/baselines.py:12
↓ 1 callersFunctionavg_num_jobs
(env)
spark_sched_sim/metrics.py:17
↓ 1 callersFunctioncollate_dag_batches
collates the dag batches from each observation into one large dag batch
schedulers/decima/utils.py:172
↓ 1 callersFunctioncollate_edge_masks
collates list of edge mask batches from each message passing path. Since the message passing depth varies between observations, edge mask batches
schedulers/decima/utils.py:150
↓ 1 callersFunctioncollate_edges
( dag_batches: Iterable[pyg.data.Batch], num_nodes_per_obs: Tensor )
schedulers/decima/utils.py:209
↓ 1 callersMethodcollect_rollout
(self)
trainers/rollout_worker.py:115
↓ 1 callersMethodcollect_stats
(self)
trainers/rollout_worker.py:122
↓ 1 callersMethodcommon_pool_has_executors
(self)
spark_sched_sim/components/executor_tracker.py:113
↓ 1 callersMethoddevice
(self)
schedulers/scheduler.py:34
↓ 1 callersMethodexecutor_location
(self, executor_id: int)
spark_sched_sim/components/executor_tracker.py:143
↓ 1 callersMethodextend
(self, new_data)
trainers/utils/returns_calculator.py:11
↓ 1 callersMethodget_parent_stages
(self, stage: Stage)
spark_sched_sim/components/job.py:78
↓ 1 callersMethodget_source
(self)
spark_sched_sim/components/executor_tracker.py:96
↓ 1 callersMethodget_source_commitments
(self)
spark_sched_sim/components/executor_tracker.py:131
↓ 1 callersMethodget_source_pool
(self)
spark_sched_sim/components/executor_tracker.py:140
↓ 1 callersMethodjob_sequence
generates a sequence of job arrivals over time, which follow a Poisson process parameterized by `self.job_arrival_rate`
spark_sched_sim/data_samplers/tpch.py:54
↓ 1 callersMethodlaunch_next_task
(self)
spark_sched_sim/components/stage.py:53
↓ 1 callersFunctionmain
()
examples.py:26
↓ 1 callersFunctionmake_data_sampler
(data_sampler_cfg)
spark_sched_sim/data_samplers/__init__.py:9
↓ 1 callersFunctionmake_edge_mask
(edge_links: ndarray, node_mask: ndarray)
schedulers/decima/utils.py:234
↓ 1 callersFunctionmake_parser
()
cfg_loader.py:16
↓ 1 callersFunctionnp_to_nx
(edge_links: ndarray, num_nodes: int)
schedulers/decima/utils.py:270
↓ 1 callersMethodnum_commitments_to_stage
(self, stage_pool_key: StagePoolKey)
spark_sched_sim/components/executor_tracker.py:119
↓ 1 callersMethodnum_executors_moving_to_stage
(self, stage_pool_key: StagePoolKey)
spark_sched_sim/components/executor_tracker.py:116
↓ 1 callersMethodpeek_commitment
(self, pool_key: OptPoolKey)
spark_sched_sim/components/executor_tracker.py:175
↓ 1 callersMethodrecord_executor_arrival
(self, stage_pool_key: StagePoolKey)
spark_sched_sim/components/executor_tracker.py:182
↓ 1 callersMethodrecord_stage_completion
increments the count of completed stages
spark_sched_sim/components/job.py:65
↓ 1 callersMethodrecord_task_completion
(self)
spark_sched_sim/components/stage.py:60
↓ 1 callersMethodremove_commitment
(self, executor_id: int, dst_pool_key: PoolKey)
spark_sched_sim/components/executor_tracker.py:156
↓ 1 callersMethodrender_frame
( self, worker_histories, job_completion_times, wall_time: float, avg_
spark_sched_sim/components/renderer.py:45
↓ 1 callersMethodrun
(self)
trainers/rollout_worker.py:97
↓ 1 callersMethodseed
(self)
trainers/rollout_worker.py:119
↓ 1 callersMethodset_seed
(self, seed)
schedulers/heuristics/random_scheduler.py:13
↓ 1 callersFunctionsubgraph
Minimal numpy version of PyG's subgraph utility function Args: edge_links: array of edges of shape (num_edges, 2), follow
spark_sched_sim/utils.py:5
↓ 1 callersMethodtask_duration
(self, job, stage, task, executor)
spark_sched_sim/data_samplers/tpch.py:75
↓ 1 callersMethodtop
(self)
spark_sched_sim/components/event.py:37
↓ 1 callersMethodtrain_on_rollouts
( self, rollout_buffers: Iterable[RolloutBuffer] )
trainers/trainer.py:165
Method__bool__
(self)
spark_sched_sim/components/event.py:31
Method__call__
( self, rank: int, conn: Connection, env_cfg: dict[str, Any], schedule
trainers/rollout_worker.py:53
Method__call__
(self, rewards_list, times_list, resets_list)
trainers/utils/returns_calculator.py:44
Method__call__
(self, ts_list, ys_list)
trainers/utils/baselines.py:9
Method__contains__
(self, item)
spark_sched_sim/data_samplers/tpch.py:273
Method__enter__
(self)
trainers/utils/hidden_prints.py:6
Method__enter__
( self, )
trainers/utils/profiler.py:17
Method__eq__
(self, other)
spark_sched_sim/components/stage.py:23
Method__eq__
(self, other)
spark_sched_sim/components/task.py:22
Method__exit__
(self, exc_type, exc_val, exc_tb)
trainers/utils/hidden_prints.py:10
Method__exit__
(self, exc_type, exc_val, exc_tb)
trainers/utils/profiler.py:23
Method__getitem__
(self, idx)
trainers/ppo.py:26
Method__hash__
(self)
spark_sched_sim/components/stage.py:20
Method__hash__
(self)
spark_sched_sim/components/task.py:19
Method__init__
(self, num_executors, dynamic_partition=True)
schedulers/heuristics/round_robin.py:8
Method__init__
(self, seed=42)
schedulers/heuristics/random_scheduler.py:8
Method__init__
( self, num_executors: int, embed_dim: int, gnn_mlp_kwargs: dict[str, Any],
schedulers/decima/scheduler.py:22
Method__init__
( self, num_node_features: int, embed_dim: int, mlp_kwargs: dict[str, Any] )
schedulers/decima/scheduler.py:143
Method__init__
( self, num_node_features: int, embed_dim: int, mlp_kwargs: dict[str, Any],
schedulers/decima/scheduler.py:174
Method__init__
(self, embed_dim: int, mlp_kwargs: dict[str, Any])
schedulers/decima/scheduler.py:261
Method__init__
( self, num_node_features: int, emb_dims: dict[str, int], mlp_kwargs: dict[str
schedulers/decima/scheduler.py:280
Method__init__
( self, num_executors: int, num_dag_features: int, emb_dims: dict[str, int],
schedulers/decima/scheduler.py:324
Method__init__
(self, env)
schedulers/decima/env_wrapper.py:22
Method__init__
( self, env, num_tasks_scale: int = 200, work_scale: float = 1e5 )
schedulers/decima/env_wrapper.py:40
Method__init__
(self)
trainers/rollout_worker.py:50
Method__init__
(self, rollout_duration: float)
trainers/rollout_worker.py:165
Method__init__
(self, agent_cfg, env_cfg, train_cfg)
trainers/vpg.py:14
Method__init__
(self, obsns, acts, advgs, lgprobs)
trainers/ppo.py:17
Method__init__
(self, agent_cfg, env_cfg, train_cfg)
trainers/ppo.py:42
Method__init__
( self, agent_cfg: CfgType, env_cfg: CfgType, train_cfg: CfgType )
trainers/trainer.py:30
Method__init__
(self, amount=20)
trainers/utils/profiler.py:13
Method__init__
(self, cap, num_cols)
trainers/utils/returns_calculator.py:7
Method__init__
(self, buff_cap=None, beta=None)
trainers/utils/returns_calculator.py:25
Method__init__
(self, num_sequences, num_rollouts)
trainers/utils/baselines.py:5
Method__init__
(self, env_cfg: dict[str, Any])
spark_sched_sim/spark_sched_sim.py:34
Method__init__
( self, id: int, job_id: int, num_tasks: int, rough_task_duration: float )
spark_sched_sim/components/stage.py:5
Method__init__
( self, num_workers: int, num_total_jobs: int | None = None, window_width: int
spark_sched_sim/components/renderer.py:12
Method__init__
(self, id_: int)
spark_sched_sim/components/executor.py:5
Method__init__
(self)
spark_sched_sim/components/event.py:20
Method__init__
( self, id_: int, stages: list[Stage], dag: nx.DiGraph, t_arrival: float )
spark_sched_sim/components/job.py:12
Method__init__
(self, num_executors: int)
spark_sched_sim/components/executor_tracker.py:29
Method__init__
(self, env, mean_time_limit, seed=42)
spark_sched_sim/wrappers/stochastic_time_limit.py:8
Method__init__
job_arrival_rate (float): non-negative number that controls how quickly new jobs arrive into the system. This is the parameter
spark_sched_sim/data_samplers/tpch.py:19
Method__init__
(self)
spark_sched_sim/data_samplers/tpch.py:270
Method__len__
(self)
trainers/rollout_worker.py:45
Method__len__
(self)
trainers/ppo.py:23
Method__unique_id
(self)
spark_sched_sim/components/task.py:16
Method_handle_executor_arrival
performs some bookkeeping when a executor arrives
spark_sched_sim/spark_sched_sim.py:440
Method_handle_task_completion
performs some bookkeeping when a task completes
spark_sched_sim/spark_sched_sim.py:452
Methodaction
(self, act: dict[str, Any])
schedulers/decima/env_wrapper.py:33
Methodall_jobs_complete
(self)
spark_sched_sim/spark_sched_sim.py:228
Methodapprox_remaining_work
(self)
spark_sched_sim/components/stage.py:50
Functionavg_job_duration
(env)
spark_sched_sim/metrics.py:13
Methodclose
(self)
spark_sched_sim/spark_sched_sim.py:223
Functioncollate_obsns
(obsns: Iterable[dict[str, Any]])
schedulers/decima/utils.py:118
Methodcollect_rollout
(self)
trainers/rollout_worker.py:135
Methodcollect_rollout
(self)
trainers/rollout_worker.py:171
← previousnext →101–200 of 248, ranked by callers