MCPcopy Index your code
hub / github.com/pathwaycom/pathway / spawn_program

Function spawn_program

python/pathway/cli.py:206–293  ·  view source on GitHub ↗
(
    *,
    threads,
    processes,
    first_port,
    addresses,
    process_id,
    repository_url,
    branch,
    program,
    arguments,
    env_base,
)

Source from the content-addressed store, hash-verified

204
205
206def spawn_program(
207 *,
208 threads,
209 processes,
210 first_port,
211 addresses,
212 process_id,
213 repository_url,
214 branch,
215 program,
216 arguments,
217 env_base,
218):
219 temp_root_directory = checkout_repository(repository_url, branch)
220 if temp_root_directory is not None:
221 repository_path, venv_path = get_temporary_paths(temp_root_directory)
222 requirements_path = repository_path / "requirements.txt"
223 if program.startswith("python"):
224 program = venv_path / "bin" / program
225 if requirements_path.exists():
226 pip_path = venv_path / "bin" / "pip"
227 command = [
228 os.fspath(pip_path),
229 "install",
230 "-r",
231 os.fspath(requirements_path),
232 ]
233 pip_handle = subprocess.run(
234 command,
235 stderr=subprocess.STDOUT,
236 )
237 if pip_handle.returncode != 0:
238 process_stdout = pip_handle.stdout.decode("utf-8")
239 logging.error(f"Failed to install requirements:\n{process_stdout}")
240 raise RuntimeError("Failed to install dependencies")
241 os.chdir(repository_path)
242
243 run_id = str(uuid.uuid4())
244 process_handles = []
245 try:
246 process_handles = create_process_handles(
247 processes=processes,
248 threads=threads,
249 first_port=first_port,
250 addresses=addresses,
251 process_id=process_id,
252 run_id=run_id,
253 program=program,
254 arguments=arguments,
255 env_base=env_base,
256 )
257 handles_state = ProcessHandlesState()
258 while not handles_state.has_process_with_error:
259 handles_state = wait_for_process_handles(process_handles, timeout=1.0)
260
261 if handles_state.needs_upscaling or handles_state.needs_downscaling:
262 handles_state.has_process_with_error = False
263 terminate_process_handles(process_handles)

Callers 2

spawnFunction · 0.85
replayFunction · 0.85

Calls 9

checkout_repositoryFunction · 0.85
get_temporary_pathsFunction · 0.85
create_process_handlesFunction · 0.85
ProcessHandlesStateClass · 0.85
wait_for_process_handlesFunction · 0.85
maxFunction · 0.85
startswithMethod · 0.80
runMethod · 0.45

Tested by

no test coverage detected