MCPcopy Index your code
hub / github.com/Datakitpage/Datakit

github.com/Datakitpage/Datakit @main

Chat with this repo
repository ↗ · DeepWiki ↗ · + Follow
2,277 symbols 5,024 edges 427 files 152 documented · 7%
What it actually does AI analysis from the code graph — generated when you open this
loading…
README

DataKit - Private Data Analysis Studio

DataKit

Website Documentation GitHub Discord LinkedIn License

What is DataKit?

DataKit is a browser-based data analysis platform that processes multi-gigabyte files locally. All processing happens in your browser - no data is sent to external servers.

Key Features

Data Import & Processing

  • Local File Support: Import CSV, Excel (XLSX), JSON, and Parquet files directly in your browser
  • Large File Handling: Process files up to several GBs efficiently using WASM
  • Remote Data Sources: Connect to various cloud platforms:
  • Amazon S3 buckets
  • Google Sheets (public)
  • HuggingFace datasets
  • MotherDuck (cloud DuckDB)
  • PostgreSQL databases
  • Custom URLs

Data Preview & Inspection

  • Interactive Grid View: Browse your data with sortable columns and resizable cells
  • Smart Data Detection: Automatic detection of data types and formats
  • Data Quality Analysis: Instant insights about:
  • Missing values and null patterns
  • Data type distributions
  • Column statistics (mean, median, mode, etc.)
  • Outlier detection
  • Data quality scores
  • Quick Overview Panel: Key metrics and patterns
  • Export Options: Export filtered data or specific columns in various formats

SQL Query Engine

  • DuckDB Integration: Full-featured SQL database running entirely in your browser
  • Query Editor: Professional SQL editor with:
  • Syntax highlighting and auto-completion
  • Query history and favorites
  • Smart query optimization suggestions
  • Real-time error detection
  • Schema Browser: Explore your data structure
  • Query Templates: Pre-built queries for common operations
  • Performance Optimization: Automatic query optimization for large datasets
  • Results Management:
  • Paginated results for smooth browsing
  • Export query results to CSV
  • Full-screen mode for focused work

AI Assistant

  • Natural Language Queries: Ask questions in plain English
  • SQL Generation: Automatically generate SQL from your questions
  • Data Insights: Get AI-powered insights and recommendations
  • Multiple AI Providers:
  • DataKit Cloud AI (default)
  • OpenAI GPT models
  • Anthropic Claude
  • Groq
  • Local models via Ollama
  • Context-Aware: Understands your data structure and suggests relevant queries
  • Query Explanation: Plain-English explanations of complex SQL

Python Notebooks

  • Interactive Notebooks: Write and execute Python code in cells
  • DuckDB Bridge: Direct access to your loaded data via SQL
  • Data Science Libraries: Pre-loaded with pandas, numpy, matplotlib, and more
  • Hugging Face Transformers: Built-in support for ML models
  • Variable Inspector: Track your Python variables and their values
  • Package Manager: Install additional Python packages on demand
  • Export Options: Save as Jupyter notebooks or PDF reports
  • Templates: Start with pre-built templates for common analyses

Privacy & Security

Complete Privacy: All data processing happens in your browser. No data is sent to external servers unless you explicitly connect to cloud services.

No Installation Required: Works in any modern browser - Chrome, Firefox, Safari, or Edge.

Getting Started

  1. Open DataKit in your browser
  2. Import your data - drag & drop or click to browse
  3. Choose your workflow:
  4. Preview data in the grid view
  5. Write SQL queries in the Query tab
  6. Ask questions using the AI Assistant
  7. Write Python code in Notebooks

Supported File Formats

  • CSV - Comma-separated values
  • Excel - .xlsx and .xls files
  • JSON - Including nested structures
  • Parquet - Columnar storage format
  • Remote Sources - S3, Google Sheets, PostgreSQL, and more

System Requirements

  • Modern web browser (Chrome 90+, Firefox 88+, Safari 14+, Edge 90+)
  • At least 4GB RAM (8GB+ recommended for large files)
  • Desktop or laptop computer (mobile support coming soon)

📋 License

DataKit is available under a dual licensing model:

Open Source (AGPL-3.0)

  • Free for open source projects
  • Must disclose source code
  • For learning and non-commercial use

Commercial License

  • No source code disclosure required
  • Enterprise self-hosting allowed
  • Priority support included
  • Custom features available

Need a commercial license? Enterprise and commercial users must obtain a commercial license.

Get your license: Contact Us at hello@datakit.page

Contributing

We welcome contributions! Please read our Contributing Guide for details on our code of conduct and the process for submitting pull requests.

Support

Extension points exported contracts — how you extend this code

AnalysisStep (Interface)
* Analysis step configuration
frontend/src/components/tabs/preview/inspector/components/LoadingStates.tsx
CreateRefreshTokenData (Interface)
(no doc)
backend/integration_tests/fixtures/auth.fixtures.ts
JwtPayload (Interface)
(no doc)
backend/api/src/auth/strategies/jwt.strategy.ts
EnhancedLoadingStateProps (Interface)
* Enhanced loading component with step-by-step progress
frontend/src/components/tabs/preview/inspector/components/LoadingStates.tsx
JwtPayload (Interface)
(no doc)
backend/integration_tests/fixtures/auth.fixtures.ts
PasswordStrengthResult (Interface)
(no doc)
backend/api/src/auth/validators/password.validator.ts
NoAnalysisEmptyStateProps (Interface)
* Empty state when no analysis results are available
frontend/src/components/tabs/preview/inspector/components/EmptyStates.tsx
AuthTokens (Interface)
(no doc)
backend/integration_tests/fixtures/auth.fixtures.ts

Core symbols most depended-on inside this repo

get
called by 161
frontend/src/lib/api/apiClient.ts
get
called by 159
backend/api/src/postgres-proxy/utils/cache.util.ts
escapeColumnName
called by 153
frontend/src/lib/duckdb/inspector/utils/filtering.ts
set
called by 140
backend/api/src/postgres-proxy/utils/cache.util.ts
has
called by 120
backend/api/src/postgres-proxy/utils/cache.util.ts
cn
called by 100
frontend/src/lib/utils.ts
safeToNumber
called by 67
frontend/src/lib/duckdb/inspector/utils/bigint.ts
post
called by 48
frontend/src/lib/api/apiClient.ts

Shape

Function 980
Method 631
Interface 429
Class 228
Enum 7
Route 2

Languages

TypeScript99%
Python1%
Rust1%

Modules by API surface

frontend/src/lib/ai/providers/webllm.ts33 symbols
frontend/src/lib/ai/aiService.ts29 symbols
backend/api/src/postgres-proxy/utils/cache.util.ts29 symbols
backend/api/src/postgres-proxy/postgres-proxy.service.ts27 symbols
backend/integration_tests/fixtures/auth.fixtures.ts26 symbols
frontend/src/lib/api/postgresService.ts25 symbols
frontend/src/store/selectors/duckdbSelectors.ts24 symbols
frontend/src/store/selectors/appSelectors.ts24 symbols
frontend/src/lib/ai/modelManager.ts24 symbols
frontend/src/lib/ai/providers/ollama.ts23 symbols
backend/integration_tests/utils/test-helpers.ts23 symbols
frontend/src/types/remoteImport.ts21 symbols

Datastores touched

test-dbDatabase · 1 repos
testdbDatabase · 1 repos

For agents

$ claude mcp add Datakit \
  -- python -m otcore.mcp_server <graph>

⬇ download graph artifact