MCPcopy Create free account

hub / github.com/ScrapeGraphAI/scrapecraft / functions

Functions307 in github.com/ScrapeGraphAI/scrapecraft

⨍ Functions 307◇ Types & classes 99↳ Endpoints 44
Functiondelete_pipeline
Delete a pipeline.
backend/app/api/pipelines.py:72
Methoddisconnect
Remove a WebSocket connection.
backend/app/services/enhanced_websocket.py:157
Functionexecute_pipeline
Execute the generated scraping code.
backend/app/api/execution.py:35
Functionexecute_scraping
Execute a scraping job asynchronously.
backend/app/api/scraping.py:32
Functionexport_pipeline_results
Export pipeline results in various formats.
backend/app/api/pipelines.py:133
Functiongenerate_code
Generate Python code for the scraping pipeline.
backend/app/agents/tools.py:108
Methodgenerate_code
Generate the SmartScraper code.
backend/app/agents/langgraph_agent.py:646
Methodget_active_users
Get list of active users for a pipeline.
backend/app/services/enhanced_websocket.py:517
Functionget_agent_mode
Get current agent mode (tools-based or state-based).
backend/app/api/workflow_v2.py:18
Functionget_api_keys
Get user's API keys (masked).
backend/app/api/auth.py:170
Functionget_available_tools
Get list of available tools in the tool-based agent.
backend/app/api/workflow_v2.py:123
Functionget_chat_history
Get chat history for a specific pipeline.
backend/app/api/chat.py:50
Functionget_current_active_user
( current_user: User = Depends(get_current_user) )
backend/app/api/auth.py:112
Functionget_current_user
(token: str = Depends(oauth2_scheme))
backend/app/api/auth.py:89
Functionget_db
()
backend/app/services/database.py:16
Functionget_enhanced_scraping_service
Get an enhanced scraping service instance.
backend/app/services/scraping_service_enhanced.py:311
Methodget_extraction_tips
Get extraction tips for a specific domain.
backend/app/services/pattern_learner.py:522
Methodget_field_suggestions
Get field suggestions for a domain and data type.
backend/app/services/pattern_learner.py:489
Functionget_manager
Dependency to get workflow manager.
backend/app/api/workflow.py:40
Functionget_phase_options
Get available phase transition options.
backend/app/api/workflow.py:186
Functionget_pipeline
Get a specific pipeline by ID.
backend/app/api/pipelines.py:36
Functionget_pipeline_results
Get the results of a pipeline execution.
backend/app/api/pipelines.py:118
Methodget_pipeline_state
Get current pipeline state.
backend/app/services/enhanced_websocket.py:513
Functionget_pipeline_status
Get the current status of a pipeline.
backend/app/api/pipelines.py:104
Functionget_scraping_results
Get the results of a completed scraping task.
backend/app/api/scraping.py:99
Functionget_scraping_status
Get the status of a scraping task.
backend/app/api/scraping.py:86
Functionget_workflow
Get current workflow state.
backend/app/api/workflow.py:46
Functionget_workflow_history
Get workflow transition history.
backend/app/api/workflow.py:169
Functionget_workflow_summary
Get workflow summary.
backend/app/api/workflow.py:56
FunctionhandleAddField
()
frontend/src/components/Pipeline/SchemaEditor.tsx:33
FunctionhandleApprovalRequest
(event: CustomEvent)
frontend/src/components/Workflow/ApprovalManager.tsx:12
FunctionhandleApprove
()
frontend/src/components/Workflow/ApprovalDialog.tsx:26
FunctionhandleApprove
(approvalId: string, reason?: string)
frontend/src/components/Workflow/ApprovalManager.tsx:22
FunctionhandleClose
()
frontend/src/components/Workflow/ApprovalManager.tsx:50
FunctionhandleCopyCode
()
frontend/src/components/Pipeline/CodeViewer.tsx:16
FunctionhandleDownloadCode
()
frontend/src/components/Pipeline/CodeViewer.tsx:23
FunctionhandleError
(event: CustomEvent)
frontend/src/components/Workflow/WorkflowSidebar.tsx:125
FunctionhandleEsc
(e: KeyboardEvent)
frontend/src/components/Settings/SettingsModal.tsx:18
FunctionhandleExecuteCode
()
frontend/src/components/Pipeline/CodeViewer.tsx:37
FunctionhandleKeyDown
(e: KeyboardEvent<HTMLTextAreaElement>)
frontend/src/components/Chat/InputArea.tsx:42
FunctionhandleKeyPress
(e: React.KeyboardEvent)
frontend/src/components/Pipeline/URLManager.tsx:41
FunctionhandleMessage
(event: MessageEvent)
frontend/src/components/Workflow/WorkflowSidebar.tsx:107
FunctionhandleMouseDown
()
frontend/src/components/Layout/SplitView.tsx:10
FunctionhandleMouseMove
(e: React.MouseEvent)
frontend/src/components/Layout/SplitView.tsx:14
FunctionhandleMouseUp
()
frontend/src/components/Layout/SplitView.tsx:25
FunctionhandleReject
()
frontend/src/components/Workflow/ApprovalDialog.tsx:31
FunctionhandleReject
(approvalId: string, reason?: string)
frontend/src/components/Workflow/ApprovalManager.tsx:36
FunctionhandleSave
()
frontend/src/components/Settings/SettingsModal.tsx:30
Methodhandle_collaborative_edit
Handle collaborative editing with conflict resolution.
backend/app/services/enhanced_websocket.py:312
Methodhandle_error
Handle errors in the workflow.
backend/app/agents/langgraph_agent.py:683
Functionhealth_check
()
backend/app/main.py:57
Methodhuman_feedback
Node for human feedback/approval.
backend/app/agents/langgraph_tools_agent.py:94
Methodinitialize
Initialize async components.
backend/app/agents/unified_agent.py:61
Functionlifespan
(app: FastAPI)
backend/app/main.py:18
Functionlist_pipelines
List all pipelines.
backend/app/api/pipelines.py:44
Functionlogin
Login and get access token.
backend/app/api/auth.py:139
Functionlogout
Logout the current user.
backend/app/api/auth.py:179
Functionmanual_transition
Manually transition to a different phase.
backend/app/api/workflow.py:135
Functionmarkdownify_tool
Convert a webpage to clean, formatted markdown. This tool extracts the main content from a webpage and converts it to markdown forma
backend/app/agents/tools/scraping_tools.py:212
Functionmigrate_workflow
Migrate a workflow from state-based to tool-based agent. This helps users transition their existing workflows to the new agent.
backend/app/api/workflow_v2.py:184
Methodnotify_optimization
Notify users of optimization opportunities.
backend/app/services/enhanced_websocket.py:295
Methodnotify_pattern_detected
Notify users when a pattern is detected.
backend/app/services/enhanced_websocket.py:277
Functionpreview_scraping
Preview what would be scraped from a URL.
backend/app/api/scraping.py:136
Methodprocess_message
Process a message through the workflow.
backend/app/services/workflow_manager.py:39
Methodprocess_message
Process incoming WebSocket message with unified agent.
backend/app/services/enhanced_websocket.py:422
Methodprocess_message
Process incoming WebSocket messages.
backend/app/services/websocket.py:69
Methodprocess_message
Process a message through the LangGraph workflow.
backend/app/agents/langgraph_agent.py:812
Methodprocess_message
Process a user message using the Kimi-k2 model.
backend/app/agents/kimi_agent.py:131
Methodprocess_message
Process a user message and return the response.
backend/app/agents/simple_agent.py:10
Methodprocess_message
Process a user message with full conversational understanding. Args: message: User's message pipelin
backend/app/agents/unified_agent.py:120
Methodprocess_message
Process a user message using the Kimi-k2 model.
backend/app/agents/openrouter_agent.py:130
Functionread_users_me
Get current user information.
backend/app/api/auth.py:156
Functionregister
Register a new user.
backend/app/api/auth.py:120
Functionremove_url
Remove a URL from the scraping pipeline.
backend/app/agents/tools.py:39
Methodrequest_approval
Request user approval and wait for response.
backend/app/services/workflow_manager.py:186
Functionroot
()
backend/app/main.py:49
Methodroute_after_analysis
Route after analyzing the request.
backend/app/agents/langgraph_agent.py:693
Methodroute_after_approval
Route after approval gate.
backend/app/agents/langgraph_agent.py:706
Methodroute_code_approval
Route after code generation approval.
backend/app/agents/langgraph_agent.py:715
Functionrun_pipeline
Execute a scraping pipeline.
backend/app/api/pipelines.py:81
Functionrun_scraping_task
Run the actual scraping task.
backend/app/api/scraping.py:60
Functionsave_api_keys
Save user's API keys (encrypted).
backend/app/api/auth.py:161
Methodsave_pipeline
Save a pipeline for future reuse.
backend/app/agents/unified_agent.py:1025
Methodscrape_multiple
Scrape multiple URLs concurrently with rate limiting. Args: urls: List of URLs to scrape prompt: Ext
backend/app/services/scraping_service_enhanced.py:267
Functionscrape_urls
Scrape multiple URLs with a given prompt. Request body: { "pipeline_id": "string", "urls": ["url1", "url2"],
backend/app/api/workflow_v2.py:83
Methodscrape_with_limit
(url: str)
backend/app/services/scraping_service_enhanced.py:288
Functionsearch_scraper_tool
Search for websites and extract relevant URLs based on a query. This tool performs web searches to find relevant websites for your s
backend/app/agents/tools/scraping_tools.py:139
Functionsearch_urls
Search for URLs using the enhanced workflow manager. This endpoint uses the tool-based agent if enabled for better search capabilities.
backend/app/api/workflow_v2.py:52
Methodsearch_urls
Search for URLs with retry logic. Args: query: Search query max_results: Maximum number of results
backend/app/services/scraping_service_enhanced.py:130
Methodsearch_urls
Search for URLs using the tool-based agent.
backend/app/services/workflow_manager_v2.py:225
Functionsend_message
Send a message to the scraping agent and get a response.
backend/app/api/chat.py:28
Functionsmart_crawler_tool
Crawl and extract data from multiple pages starting from a URL. This tool navigates through a website following links and extracts d
backend/app/agents/tools/scraping_tools.py:90
Functionsmart_scraper_tool
Extract structured data from a single webpage using natural language. This tool uses ScrapeGraphAI's SmartScraper to intelligently extra
backend/app/agents/tools/scraping_tools.py:43
Methodstream_scraping_progress
Stream real-time scraping progress with preview.
backend/app/services/enhanced_websocket.py:228
Methodstream_suggestions
Stream AI suggestions as they're generated.
backend/app/services/enhanced_websocket.py:252
Functionsubmit_feedback
Submit feedback for a specific message.
backend/app/api/chat.py:66
Functionupdate_pipeline
Update an existing pipeline.
backend/app/api/pipelines.py:50
Functionupdate_schema
Manually update schema in the workflow.
backend/app/api/workflow.py:86
Functionupdate_urls
Manually update URLs in the workflow.
backend/app/api/workflow.py:62
Methodvalidate_api_key
Validate if the API key is working.
backend/app/services/scrapegraph.py:165
← previousnext →201–300 of 307, ranked by callers