with Manager() as manager: public_prompts = manager.dict() # input text for generator public_outputs = manager.dict() # genrate

if is_value_only, the prompt is used to produce value estimate.

generate as a linked list root -> x -> y -> z

subclass must implement

process output from vllm e.g., outputs = llm.generate(prompts, sampling_params) for output in outputs: step_gener

This one is not for batch inference.

Validate valid python version.