broadcast tensor across mp_group

broadcast tensor across mp_group

r"""Check token ids and texts. encode(text) == token_ids decode(token_ids) == text

r"""Regularizes audios to avoid error. Including reading and resampling.

异步调用LLM

TODO: optimize this. Technically, we only need one broadcast

r"""Get a list of responses of the chat model.

Deprecated. Please use copy_to_local instead.

并发执行异步任务并合并结果 :param task_list: 任务列表，每个元素为元组(func, args, kwargs) func: 异步函数，返回Awaitable[list]

r"""Extract a list of tuples from the response message if using tools. Each tuple consists of function name and function arguments.

end of sentence token can be int or list: 1 or [1, 2] e.g. eos_token=1 response_id: [0, 0, 2, 42, 3, 5, 1, 0, 0] eos_mask: [1, 1,

Get FSDP wrap policy for the module. Args: module: The module to get wrap policy for config: Configuration for wrap policy

r"""Build batched multimodal inputs for VLMs. Arguments: images: a list of image inputs, shape (num_images,) videos:

获取报告文件的完整路径

Check if agent is active and can participate

基于原始数据, 构建DOC对象, 保存到向量库中

基于规则的简单压缩方法 策略： 1. 移除冗余的连接词 2. 合并相似的观点 3. 保留关键推理步骤 4. 简化长句

r"""Get the response token-by-token of the chat model.

模型训练函数

Update token counts

broadcast tensor in tp shards across mp_group

broadcast tensor in tp shards across mp_group

Compute loss with optional sequence parallelism and remove padding features

r"""Convert slots to ollama template.

r"""Escape HTML characters.

r"""Return a list of names of parameters with weight decay. (weights in non-layernorm layers).

r"""Escape single quotes in content.

Add token budget constraint to the question prompt. Args: question: Original question text budget: Token budget limit to

TODO: optimize this. - We can use async ops - We can use only one allgather Args: tensors: size: group:

Create an assistant message

r"""Calculate the number of pixels from the expression.

Properly clean up all sessions and streams using AsyncExitStack

We compute dense reward here so that we can directly train RL without SFT

Connect to an MCP server and add its tools.

Connect to multiple MCP servers running with SSE transport

下载 HTTPS 的 PDF 文件 参数: url: PDF 文件的 HTTPS URL save_path: 保存路径（默认为当前目录，使用 URL 中的文件名）

Estimate the FLOPS based on the number of valid tokens in the current batch and the time taken. Args: batch_seqlens (Lis

从文本中提取\boxed{}中的答案

从文本中提取\boxed{}中的答案

从文本中提取\boxed{}中的答案

The training loop of PPO. The driver process only need to call the compute functions of the worker group through RPC to construct the

获取所有类型板块数据，包括热门板块、概念板块、行业板块和地域板块 Args: sector_types (str, optional): 板块类型，可选值: 'all', 'hot', 'concept', 'regional', 'industry'，默认为No

r"""Return the list of all elements.

获取指数资金流向数据 Args: index_code (str, optional): 指数代码，默认为上证指数 000001 Returns: dict: 资金流向数据

获取所需要的基金与市场相关信息 参数: content (str): 查询内容 返回: str: 获取到的文本信息(json文本)

Get the resource pool of the worker_cls

获取股票资金流向数据，支持获取列表或单只股票数据 参数: page_size: 每页显示数量，默认50 page_num: 页码，默认第1页 stock_code: 股票代码，如果指定，则返回单只股票数据，否则返回列表 返

获取股票的基本介绍信息，包括主营业务、经营范围等。 支持区分A股、港股。 :param symbol: 股票代码（如 '000066'、'00700'） :param market: 市场类型（'A'、'HK'） :return: 返回pandas表格的字符

r"""Get the token ids of thought words.