Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/Open-Bee/DataStudio
/ functions
Functions
1,367 in github.com/Open-Bee/DataStudio
⨍
Functions
1,367
◇
Types & classes
308
↳
Endpoints
27
Method
loader_with_data
Create loader with test data.
tests/datasets/test_data_loader.py:134
Method
logger
Create logger for testing.
tests/utils/test_logging.py:206
Method
logger
Create logger for testing.
tests/utils/test_logging.py:247
Method
logger
Create logger for testing.
tests/utils/test_logging.py:272
Method
manager
Create checkpoint manager for testing.
tests/utils/test_checkpoint.py:49
Method
manager
Create checkpoint manager for testing.
tests/utils/test_checkpoint.py:78
Method
manager
Create checkpoint manager for testing.
tests/utils/test_checkpoint.py:101
Method
manager
Create checkpoint manager for testing.
tests/utils/test_checkpoint.py:144
Method
manager
Create checkpoint manager for testing.
tests/utils/test_checkpoint.py:166
Method
manager
Create checkpoint manager for testing.
tests/utils/test_checkpoint.py:190
Method
manager
Create checkpoint manager for testing.
tests/utils/test_checkpoint.py:303
Method
manager
Create checkpoint manager for testing.
tests/utils/test_checkpoint.py:319
Method
manager
Create checkpoint manager for testing.
tests/utils/test_checkpoint.py:350
Function
mock_image
Mock PIL Image for testing.
tests/conftest.py:165
Method
mock_lmdb
Mock lmdb module.
tests/utils/test_other_utils.py:422
Method
mock_loader
Create a mocked loader for testing.
tests/datasets/test_data_loader.py:77
Function
mock_logger
Mock logger for testing.
tests/conftest.py:139
Function
mock_model
Mock model for MLLM operators.
tests/conftest.py:151
Method
mock_model
Create mock model.
tests/operators/test_mllm.py:144
Method
mock_model
Create mock model.
tests/operators/test_mllm.py:243
Method
mock_model
Create mock model.
tests/operators/test_mllm.py:299
Function
mock_openai_reasoning_response
OpenAI API response with reasoning_content.
tests/models/test_openai_api.py:79
Function
mock_openai_response
Mock OpenAI API response.
tests/conftest.py:159
Function
mock_openai_success_response
Standard successful OpenAI API response.
tests/models/test_openai_api.py:73
Method
mock_pipeline
Create pipeline with mocked sub-pipeline.
tests/pipelines/test_pipeline.py:86
Function
multi_stage_operations
Create multi-stage operations config.
tests/pipelines/test_pipeline.py:33
Method
multi_turn_item
Create multi-turn DataItem for testing.
tests/operators/test_data_item.py:320
Method
multi_turn_item
Create multi-turn DataItem.
tests/operators/test_result.py:271
Method
name
Operator name for records.
datastudio/operators/core/operator.py:36
Method
process
Process a single item (delegates to process_batch).
datastudio/operators/mllm/base.py:56
Method
process
Process by calling check() for each QA pair.
datastudio/operators/core/operator.py:96
Method
process
Process by calling rewrite() for each QA pair.
datastudio/operators/core/operator.py:133
Method
process
Check if any images have excluded extensions (item-level).
datastudio/operators/filters/image_ext.py:47
Method
process
Normalize prompt formatting in questions (modifies question).
datastudio/operators/rewriters/norm_prompt.py:189
Method
process
Remove redundant instruction patterns from multi-turn (item-level).
datastudio/operators/rewriters/norm_multi_turn_prompt.py:64
Method
process
Normalize image tags - move all to first question (item-level).
datastudio/operators/rewriters/norm_image_tag.py:58
Method
process
(self, item: DataItem)
tests/pipelines/test_sub_pipeline.py:91
Method
process
(self, item: DataItem)
tests/pipelines/test_sub_pipeline.py:104
Method
process
(self, item: DataItem)
tests/pipelines/test_sub_pipeline.py:119
Method
process
(self, item)
tests/pipelines/test_sub_pipeline.py:246
Method
process
(self, item)
tests/pipelines/test_sub_pipeline.py:353
Method
process
(self, item)
tests/pipelines/test_sub_pipeline.py:447
Method
process_batch
Process a batch of items. Default implementation calls process() for each item. Override for batch optimization (e.g., MLLM
datastudio/operators/core/operator.py:53
Method
process_batch
Process batch - returns keep result for each item.
datastudio/operators/rewriters/split.py:47
Method
prompt_text
Load and return the prompt text (lazy).
datastudio/operators/mllm/request.py:100
Function
pytest_configure
Configure pytest markers.
tests/conftest.py:270
Method
qa_count
Number of QA pairs.
datastudio/operators/core/data_item.py:68
Method
qa_pairs
Get all QA pairs (cached).
datastudio/operators/core/data_item.py:61
Method
qa_reject_rate
Calculate QA-level reject rate.
datastudio/utils/statistics.py:50
Function
real_image
Create a real PIL Image for testing.
tests/models/test_base_api.py:43
Method
registry
Create clean registry for testing.
tests/utils/test_registry.py:43
Method
registry
Create registry with test module.
tests/utils/test_registry.py:160
Method
registry_with_class
Create a registry with a simple test class registered.
tests/utils/test_registry.py:329
Method
registry_with_modules
Create registry with registered modules.
tests/utils/test_registry.py:102
Method
registry_with_modules
Create registry with registered modules.
tests/utils/test_registry.py:137
Method
reject_rate
Calculate reject rate (item level).
datastudio/utils/statistics.py:36
Method
reject_rate
(self)
datastudio/utils/statistics.py:89
Method
request_builder_cfg
Create request builder config.
tests/operators/test_mllm.py:151
Method
request_builder_cfg
Create request builder config.
tests/operators/test_mllm.py:250
Method
request_builder_cfg
Create request builder config.
tests/operators/test_mllm.py:306
Method
request_builder_cfg
Create request builder config.
tests/operators/test_mllm.py:337
Method
rewrite
Remove Reasoning: prefix from an answer.
datastudio/operators/rewriters/remove_reason.py:36
Method
rewrite
Not used - this rewriter modifies questions.
datastudio/operators/rewriters/norm_prompt.py:204
Method
rewrite
Not used - item-level rewriter.
datastudio/operators/rewriters/norm_multi_turn_prompt.py:89
Method
rewrite
Not used - this rewriter operates at item level.
datastudio/operators/rewriters/norm_image_tag.py:110
Method
rewrite
Normalize think tags in an answer.
datastudio/operators/rewriters/norm_think.py:51
Method
rewrite
Add empty think tags to a response missing them.
datastudio/operators/rewriters/add_nothink.py:35
Method
rewrite
(self, item, qa_idx)
tests/operators/test_operator.py:263
Method
rewrite_rate
Calculate rewrite rate.
datastudio/utils/statistics.py:43
Method
rewriter
Create rewriter instance.
tests/operators/test_rewriters.py:36
Method
rewriter
Create rewriter instance.
tests/operators/test_rewriters.py:109
Method
rewriter
Create rewriter instance.
tests/operators/test_rewriters.py:148
Method
rewriter
Create rewriter instance.
tests/operators/test_rewriters.py:185
Method
rewriter
Create rewriter instance.
tests/operators/test_rewriters.py:229
Method
rewriter
Create rewriter instance.
tests/operators/test_rewriters.py:257
Method
rewriter
Create rewriter instance.
tests/operators/test_rewriters.py:277
Method
rewriter
Create rewriter instance.
tests/operators/test_rewriters.py:321
Method
rewriter
Create rewriter instance.
tests/operators/test_rewriters.py:350
Method
rewriter_answer
Create MLLMRewriter instance with rewrite_type='answer'.
tests/operators/test_mllm.py:518
Method
rewriter_instance
Create MLLMRewriter instance.
tests/operators/test_mllm.py:261
Method
rewriter_question
Create MLLMRewriter instance with rewrite_type='question'.
tests/operators/test_mllm.py:532
Function
sample_config_dict
Sample configuration dictionary.
tests/conftest.py:224
Function
sample_conversation
Sample conversation data for testing.
tests/conftest.py:27
Function
sample_data_item
Sample data item dictionary without image.
tests/conftest.py:47
Function
sample_data_item_with_image
Sample data item with image.
tests/conftest.py:53
Function
sample_data_list
List of sample data items.
tests/conftest.py:63
Function
sample_multi_turn_conversation
Sample multi-turn conversation data.
tests/conftest.py:36
Function
sample_result
Create a sample Result for testing.
tests/conftest.py:243
Method
save
Save data to JSON file. Args: data: List of data dictionaries. file_path: Output file path. indent: JSON
datastudio/datasets/formatters/json_formatter.py:39
Method
save
Save data to file. Args: data: List of data dictionaries. file_path: Output file path. auto_denormalize:
datastudio/datasets/formatters/__init__.py:108
Method
save
Save data to JSONL file. Args: data: List of data dictionaries. file_path: Output file path. ensure_ascii
datastudio/datasets/formatters/jsonl_formatter.py:56
Method
save
Save configuration to YAML file. Args: config: StandardConfig to save. yaml_path: Output file path.
datastudio/datasets/config/config.py:159
Method
save
(self, data, file_path, **kwargs)
tests/datasets/test_formatters.py:347
Method
saver
Create a saver instance for testing.
tests/datasets/test_data_saver.py:70
Method
saver
Create a saver instance for testing.
tests/datasets/test_data_saver.py:109
Method
saver
Create a saver instance for testing.
tests/datasets/test_data_saver.py:392
Method
shutdown
Shutdown and release resources. Workers are ephemeral (created per generate() call), nothing to clean up.
datastudio/models/mp_openai_api.py:325
Method
side_effect
(*args, **kwargs)
tests/models/test_openai_api.py:362
Method
simple_saver
Create SimpleSaver instance.
tests/datasets/test_saver_components.py:25
Method
system_prompt_text
Load and return the system prompt text (lazy).
datastudio/operators/mllm/request.py:107
← previous
next →
401–500 of 1,367, ranked by callers