github.com/666ghj/BettaFish @v3.0.0 sqlite

English | 中文文档

[!IMPORTANT]
Check out our newly released prediction engine: MiroFish - A Simple and Universal Swarm Intelligence Engine for Predicting Everything

The "Data Analysis Three-Step Approach" is now fully connected: We are excited to announce the official release of MiroFish! With the final piece of the puzzle in place, we have built a complete pipeline from BettaFish (data collection and analysis) to MiroFish (panoramic prediction). The closed loop from raw data to intelligent decision-making is now complete, making it possible to foresee the future!

⚡ Project Overview

"BettaFish" is an innovative multi-agent public opinion analysis system built from scratch. It helps break information cocoons, restore the original public sentiment, predict future trends, and assist decision-making. Users only need to raise analysis needs like chatting; the agents automatically analyze 30+ mainstream social platforms at home and abroad and millions of public comments.

Betta is a small yet combative and beautiful fish, symbolizing "small but powerful, fearless of challenges".

See the system-generated research report on "Wuhan University Public Opinion": In-depth Analysis Report on Wuhan University's Brand Reputation

See a complete system run example on "Wuhan University Public Opinion": Video - In-depth Analysis Report on Wuhan University's Brand Reputation

Beyond just report quality, compared to similar products, we have 🚀 six major advantages:

1. AI-Driven Comprehensive Monitoring: AI crawler clusters operate 24/7 non-stop, comprehensively covering 10+ key domestic and international social media platforms including Weibo, Xiaohongshu, TikTok, Kuaishou, etc. Not only capturing trending content in real-time, but also drilling down to massive user comments, letting you hear the most authentic and widespread public voice.

2. Composite Analysis Engine Beyond LLM: We not only rely on 5 types of professionally designed Agents, but also integrate middleware such as fine-tuned models and statistical models. Through multi-model collaborative work, we ensure the depth, accuracy, and multi-dimensional perspective of analysis results.

3. Powerful Multimodal Capabilities: Breaking through text and image limitations, capable of deep analysis of short video content from TikTok, Kuaishou, etc., and precisely extracting structured multimodal information cards such as weather, calendar, stocks from modern search engines, giving you comprehensive control over public opinion dynamics.

4. Agent "Forum" Collaboration Mechanism: Endowing different Agents with unique toolsets and thinking patterns, introducing a debate moderator model, conducting chain-of-thought collision and debate through the "forum" mechanism. This not only avoids the thinking limitations of single models and homogenization caused by communication, but also catalyzes higher-quality collective intelligence and decision support.

5. Seamless Integration of Public and Private Domain Data: The platform not only analyzes public opinion, but also provides high-security interfaces supporting seamless integration of your internal business databases with public opinion data. Breaking through data barriers, providing powerful analysis capabilities of "external trends + internal insights" for vertical businesses.

6. Lightweight and Highly Extensible Framework: Based on pure Python modular design, achieving lightweight, one-click deployment. Clear code structure allows developers to easily integrate custom models and business logic, enabling rapid platform expansion and deep customization.

Starting with public opinion, but not limited to public opinion. The goal of "WeiYu" is to become a simple and universal data analysis engine that drives all business scenarios.

For example, you only need to simply modify the API parameters and prompts of the Agent toolset to transform it into a financial market analysis system.

Here's a relatively active Linux.do project discussion thread: https://linux.do/t/topic/1009280

Check out the comparison by a Linux.do fellow: Open Source Project (BettaFish) vs manus|minimax|ChatGPT Comparison

Say goodbye to traditional data dashboards. In "WeiYu", everything starts with a simple question - you just need to ask your analysis needs like a conversation

🪄 Sponsors

LLM Model API Sponsor:

Provider of core agent capabilities including AI web search, file parsing, and web content scraping:

Anspire Open is a leading infrastructure provider for the agent era. We offer developers the core capability stack needed to build powerful agents. Currently available services include AI web search (multiple versions, highly competitive pricing), file parsing (limited-time free), web content scraping (limited-time free), cloud browser automation (Anspire Browser Agent, in beta), multi-turn rewriting, and more. We continue to provide a solid foundation for agents to connect and operate in complex digital worlds. Seamlessly integrates with mainstream agent platforms such as Dify, Coze, and Yuanqi. Through a transparent credit-based billing system and modular design, we provide enterprises with efficient, low-cost customized support to accelerate intelligent transformation.

Claim 1GB free trial, enterprise-grade global proxy IP and Scraper API solution provider, register now:

Thordata helps enterprises easily access publicly available web data through highly reliable proxy networks and automated scraping solutions, with a commitment to 99.9% uptime and 99.7% success rate.

🏗️ System Architecture

Overall Architecture Diagram

Insight Agent Private Database Mining: AI agent for in-depth analysis of private public opinion databases

Media Agent Multimodal Content Analysis: AI agent with powerful multimodal capabilities

Query Agent Precise Information Search: AI agent with domestic and international web search capabilities

Report Agent Intelligent Report Generation: Multi-round report generation AI agent with built-in templates

A Complete Analysis Workflow

Project Code Structure Tree

``` BettaFish/ ├── QueryEngine/ # Domestic and international news breadth search Agent │ ├── agent.py # Agent main logic, coordinates search and analysis workflow │ ├── llms/ # LLM interface wrapper │ ├── nodes/ # Processing nodes: search, formatting, summarization, etc. │ ├── tools/ # Domestic and international news search toolkit │ ├── utils/ # Utility functions │ ├── state/ # State management │ ├── prompts/ # Prompt templates │ └── ... ├── MediaEngine/ # Powerful multimodal understanding Agent │ ├── agent.py # Agent main logic, handles video/image multimodal content │ ├── llms/ # LLM interface wrapper │ ├── nodes/ # Processing nodes: search, formatting, summarization, etc. │ ├── tools/ # Multimodal search toolkit │ ├── utils/ # Utility functions │ ├── state/ # State management │ ├── prompts/ # Prompt templates │ └── ... ├── InsightEngine/ # Private database mining Agent │ ├── agent.py # Agent main logic, coordinates database queries and analysis │ ├── llms/ # LLM interface wrapper │ │ └── base.py # Unified OpenAI-compatible client │ ├── nodes/ # Processing nodes: search, formatting, summarization, etc. │ │ ├── base_node.py # Base node class │ │ ├── search_node.py # Search node │ │ ├── formatting_node.py # Formatting node │ │ ├── report_structure_node.py # Report structure node │ │ └── summary_node.py # Summary node │ ├── tools/ # Database query and analysis toolkit │ │ ├── keyword_optimizer.py # Qwen keyword optimization middleware │ │ ├── search.py # Database operation toolkit (topic search, comment retrieval, etc.) │ │ └── sentiment_analyzer.py # Sentiment analysis integration tool │ ├── utils/ # Utility functions │ │ ├── config.py # Configuration management │ │ ├── db.py # SQLAlchemy async engine + read-only query wrapper │ │ └── text_processing.py # Text processing utilities │ ├── state/ # State management │ │ └── state.py # Agent state definition │ ├── prompts/ # Prompt templates │ │ └── prom