MCPcopy Create free account
hub / github.com/QQ3221197721/cloudai-fusion

github.com/QQ3221197721/cloudai-fusion @main

Chat with this repo
repository ↗ · DeepWiki ↗ · + Follow
5,841 symbols 18,913 edges 256 files ⚖ Apache-2.0 3,849 documented · 66% updated 7d ago★ 1,1151 open issues

Browse by type

Functions 4,559 Types & classes 1,271 Endpoints 11
What it actually does AI analysis from the code graph — generated when you open this
loading…
README

CloudAI Fusion

Cloud-Native AI Unified Management Platform

Honest by design: it refuses to pretend a fake backend is real.


CloudAI Fusion unifies cloud-native infrastructure management with AI-assisted GPU scheduling across multiple clouds. It is built around one principle that most "platform" projects ignore: a component is either backed by a real dependency, or it says so — and in production it refuses to run on a simulation.

Why this project is different: Run Modes & Capability Transparency

Every external-dependency boundary (cache, messaging, consensus, GitOps, scheduling, cloud, DB, evidence) reports whether it is running on a real backend or a simulated in-memory fallback. A single run_mode setting governs what is allowed:

Run mode Simulated backends Use case
simulation allowed (expected) local dev, unit/integration tests
degraded allowed but surfaced loudly (warnings + /readyz) staging
production forbidden — process refuses to boot production
  • GET /api/v1/capabilities returns, per subsystem, real vs simulated + the active run mode.
  • /readyz reports simulated backends and fails readiness in production.
  • At startup, capability.Enforce() aborts a production boot if any subsystem is simulated.
// GET /api/v1/capabilities  (run_mode=degraded, no infra attached)
{
  "run_mode": "degraded",
  "all_real": false,
  "simulated_count": 1,
  "backends": [
    {"component": "messaging.producer", "mode": "simulated", "driver": "memory",
     "detail": "nats backend requested but server unreachable"}
  ]
}

In production, that same state makes the API server exit 1 at boot instead of serving fake data. This is the core guarantee of the platform.

What is real vs. simulated

Real drivers are used automatically when the dependency is reachable; otherwise the component falls back to an in-memory simulation and reports it (allowed only outside production).

Subsystem Real implementation Library When dependency is absent
Database PostgreSQL (migrations, optimistic locking, transactional events) GORM login/register disabled; server degrades
Cache / Lock / PubSub Redis (SET NX + Lua locks, SCAN, real pub/sub) redis/go-redis in-memory (single-process)
Messaging (durable) NATS (queue groups) / Kafka (consumer groups, acks=all) nats.go / IBM/sarama in-memory (non-durable)
Leader election / HA Kubernetes Lease leader election client-go/leaderelection in-memory single-node
Kubernetes real clusters (in-cluster / kubeconfig / token) client-go scheduler returns no candidates in prod (no fake nodes)
Multi-cloud AWS EKS, Alibaba ACK, Azure AKS, GCP GKE, Huawei CCE, Tencent TKE official cloud SDKs provider registered in stub mode (no creds)
GitOps ArgoCD (REST sync API) + Flux (dynamic-client reconcile-status reads) net/http + client-go/dynamic simulated when neither is reachable
Consensus hashicorp/raft (real leader election + log replication) hashicorp/raft in-memory Raft, reported simulated
Cross-cluster failover client-go API-server health probes + promotion client-go reported simulated without a real DR cluster
Verifiable Control Plane Ed25519-signed, hash-chained, RFC 6962 Merkle transparency log + offline verifier crypto/ed25519, crypto/sha256 always real (no external dependency)
Verifiable AI Red Team scope-gated engagements, evidence-signed actions, LLM planner, web/AD exploit chaining client-go + orchestrated tools tools real-when-installed; LLM real-when-endpoint-set
AISecOps deep wells L1 intel (ClickHouse HTTP + STIX 2.1 feeds), L3 endpoint (/proc EDR, Linux), L8 response (gateway IP-ACL + active NetworkPolicy) net/http, /proc, pkg/security in-memory/static/recording fallback; real when the resp. env var is set
AI / LLM OpenAI / DashScope / Ollama / vLLM; optional PyTorch/SB3 RL OpenAI-compatible + torch/stable-baselines3 rule-based heuristics (honestly reported at /api/v1/models/status)

Real Flux reconcile-status reads, cross-cluster failover, and hashicorp/raft consensus are now implemented and integration-tested against kind (run with -tags integration). What remains gated on external resources (and therefore reported simulated until configured): real multi-cloud SDK calls (credentials), red-team tools (the binaries), a live LLM endpoint, etcd election. Progress is measured objectively by /api/v1/capabilities, not by marketing claims.

Key Features

Feature Description
Run-mode honesty framework simulation/degraded/production + capability registry + fail-fast boot
Multi-Cloud Management Unified API over 6 clouds via official SDKs
GPU Topology-Aware Scheduling NVLink-aware placement, GPU sharing (MPS/MIG), preemption, RL scoring
4 AI Agents Scheduling, security, cost, operations — LLM-enhanced with rule-based fallback
Real messaging & HA NATS/Kafka drivers, Kubernetes Lease leader election
Edge Autonomy (MVP Ready) Offline-first edge decisions with true Delta Sync + conflict resolution (Patent #16-17); real K8s API calls via client-go; full L15 implementation with L2 planning for TEE hardware support
Security & Compliance JWT + RBAC (4 roles), OIDC federation, CIS checks, threat detection, audit log
DevSecOps supply chain SAST, dep/secret/IaC scanning, SBOM, cosign signing, SLSA L3 provenance
Full Observability Prometheus metrics, OpenTelemetry tracing, Grafana, intelligent alerting
Verifiable Control Plane Ed25519-signed, hash-chained, Merkle-transparency-logged receipts for consequential actions; offline-verifiable via cafctl
Verifiable AI Red Team Authorized, evidence-grade security validation: scope-gated engagements, human-in-the-loop approval, web/AD exploit chaining, CVE-Bench harness
AISecOps 16 Deep Wells Intelligence→Operations→Response security fabric (L1-L16): L1 intel (ClickHouse + STIX 2.1), L2 hunting + UEBA, L3-L8 SOC detectors (Sigma) + auto-SOAR, evidence-signed; honest per-well readiness at /api/v1/wells
Plugin Ecosystem 9 contrib plugins across 3 domains: Render Farm (cloud provider + scheduler scoring + metrics), PostgreSQL DR (collector + alerter + failover validation), AI Customer Service (metrics + webhook + threat detection). New: Third-party submission system with Poseidon-based model commitment

Core Advantages (Benchmark-Backed Moats)

Unlike "platform" projects that ship glue code, CloudAI Fusion's differentiation is verifiable in code and reproducible benchmarks. Every number below comes from real go test -bench runs (Intel Core Ultra 9, windows/amd64, Go 1.25.7); reproduce with go test ./pkg/<pkg> -bench=. -benchmem -run='^$'. Where no independent-competitor benchmark exists, we say so instead of inventing one.

Moat What we do that others don't Measured result Reproduce (pkg/)
Honesty-by-design control plane Every backend reports real vs simulated; production refuses to boot on any fake dependency capability.Enforce() aborts prod boot; /api/v1/capabilities + /readyz surface it pkg/capability
Verifiable evidence chain Ed25519 hash-chain + Groth16 ZKP receipts, offline-verifiable (Rekor has no ZKP) ZKP verify ~1.5 ms, prove ~264 ms; append ~37 µs pkg/evidence
Aho-Corasick policy matching Multi-pattern automaton vs regex scan for WAF/policy rules 10k rules ~32 µs, ≈1388× vs Go stdlib regexp linear-scan baseline (our own, not a competitor WAF) pkg/security
Compiled RBAC Compile role graph to O(1) bitmap at build time 10k rules ~160 ns, 0 alloc (~31× vs our naive linear-scan baseline; real Casbin v2 head-to-head via -tags casbin) pkg/auth
Zero-alloc event fabric Arena/sync.Pool router with radix-trie topics ~25M events/sec, 0 alloc/op on hot path pkg/eventbus
GPU topology-aware scheduling dense-k-subgraph (NP-hard) approx vs topology-blind binpack 1.86× NVLink bandwidth vs simulated K8s binpack on SYNTHETIC topology data (not real GPU hardware), p<1e-6 pkg/scheduler
Streaming joint-anomaly detection Online Welford + Ledoit-Wolf shrinkage + rank-1 Cholesky Mahalanobis (O(d²), single-pass) beats sklearn IF on exported data (Python comparison run offline, not in Go test); ~12.8× vs our 3σ baseline pkg/anomaly
Incremental FinOps metrics DGIM log-bucket sliding window + content-addressed delta export O(log W) memory; ≤100 ms incremental; OpenCost 60s cited from its documented ETL cycle, not measured head-to-head pkg/reporting
Bounded-memory exact quantiles TailExact hybrid (exact tail + bounded body) p99 error <0.6% vs our reimplementation of Prometheus histogram_quantile() bucket interpolation (same algo/data; not a running Prometheus) pkg/quantile
Insertion-shift-resistant delta sync FastCDC content-defined chunking + Merkle diff + CRDT merge ~28× vs our naive fixed-block baseline (not rsync or any shipping product) pkg/deltasync
WASM capability security Pure-Go (zero-CGO) sandbox; deny-by-default FS/Net/GPU gates 21 escape vectors defended, gate check sub-µs (FS 146-565 ns) pkg/wasm
Zero-downtime hot-swap State snapshot + migration + rollback with Ed25519 receipt 0 request loss under concurrent load, ~30 ms end-to-end pkg/hotswap
GPU WASI performance moat Zero-copy buffer view (100ns vs memcpy 50µs); Sharded no-lock allocator (<15ns alloc); Token-bucket tenant accounting (<5ns call); NVLink locality placement (greedy + local search) Host dispatch ~15 ns, zero-copy ~45 ns, sharded alloc ~50 ns, token bucket ~36 ns, optimal placement 6ms@8GPUs pkg/wasm (wasi_gpu.go + zerocopy_buffer.go + sharded_allocator.go + gpu_accounting.go + wasi_gpu_locality.go)
Causal alert correlation Tarjan SCC + CausalRank root-cause vs label-equality grouping 58% vs our reimplemented Alertmanager group_by semantics (simplified emulation, not the real Alertmanager); 0% mis-suppression pkg/correlation
FastTracer distributed tracing Zero-alloc span hot path vs OTel SDK SpanStart ~103 ns vs OTel ~657 ns ≈ 6.4× faster (head-to-head: real OTel SDK imported) pkg/tracing
Offline-verifiable learning certs Ed25519 + SHA-256 step hash-chain completion proof (architectural distinction cited from public docs; no perf benchmark vs Katacoda/Qwiklabs) tamper-evident, verifiable with a 32-byte public key, no network pkg/tutorial

Honesty note: some subsystems (messaging drivers, adapters, standard state machines) are solid engineering without a unique algorithmic moat — we label those as such rather than inflating them. Hardware-bound modules (real GPU topology/MIG, CRIU migration, SGX/eBPF capability probing) require physical hardware and are marked accordingly. M53 GPU WASI Extensions is NOT hardware-bound: runs on pure-Go wazero interpreter with simulated GPU service, no physical GPU required.

Baselines labelled "vs X" compare against our reimplementation of X's documented semantics unless marked "(head-to-head)". Only FastTracer vs OpenTelemetry SDK is a true head-to-head (real third-party library imported and benchmarked in-process).

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                          CloudAI Fusion                              │
├─────────────┬──────────────┬─────────────┬─────────────────────────┤
│  API Server │  Scheduler   │    Agent    │      AI Engine          │
│   (Go/Gin)  │ (GPU-aware)  │ (DaemonSet) │   (Python/FastAPI)     │
├─────────────┴──────────────┴─────────────┴─────────────────────────┤
│  runmode + capability registry (real-vs-simulated policy & report)  │
├───────┬───────┬─────────┬──────────┬─────────┬──────┬──────┬───────┤
│  Auth │ Cloud │ Cluster │ Security │ Monitor │ Mesh │ Wasm │ Edge  │
├───────┴───────┴─────────┴──────────┴─────────┴──────┴──────┴───────┤
│  PostgreSQL │ Redis │ Kafka │ NATS │ Kubernetes │ Prometheus         │
└─────────────────────────────────────────────────────────────────────┘

A cross-cutting Verifiable Control Plane (pkg/evidence) signs every consequential action into a hash-chained, Merkle-transparency-logged ledger; the Verifiable AI Red Team (pkg/redteam) runs authorized, evidence-grade security validation on top of it.

Plugin Ecosystem

CloudAI Fusion provides a Kubernetes Scheduler Framework-style plugin system with 9 extension points. The pkg/plugin/contrib/ package ships 9 production-ready plugins across 3 domains:

Domain Plugin Extension Point Description
Render Farm renderfarm.cloud cloud.provider Exposes render clusters (GPU/Spot) as schedulable cloud resources with cost estimation
renderfarm.scheduler.score scheduler.score Scores nodes based on Spot price, interruption rate, and GPU availability
renderfarm.monitor.collector monitor.collector Collects Prometheus metrics (frame rate, Spot interruptions, cost)
PostgreSQL DR dr.monitor.collector monitor.collector Monitors replication lag, RPO/RTO, consistency check status
`dr.monitor.alerter

Extension points exported contracts — how you extend this code

browse all types & interfaces →

Core symbols most depended-on inside this repo

browse all functions →

Shape

Method 2,381
Function 2,178
Struct 1,018
TypeAlias 124
Class 63
Interface 60
Route 11
FuncType 6

Languages

Go93%
Python7%

Modules by API surface

pkg/testutil/mocks.go92 symbols
pkg/cache/cache.go88 symbols
pkg/edge/offline_enhanced.go83 symbols
pkg/controller/controller_test.go74 symbols
pkg/plugin/admission.go73 symbols
pkg/cloud/providers_mock_test.go73 symbols
pkg/store/interfaces.go69 symbols
pkg/api/router.go62 symbols
pkg/scheduler/engine.go56 symbols
pkg/scheduler/engine_test.go54 symbols
pkg/edge/edge_hardware_test.go52 symbols
pkg/edge/edge_cloud_collab.go52 symbols

Datastores touched

insertDatabase · 1 repos
cloudaiDatabase · 1 repos
dbDatabase · 1 repos

For agents

$ claude mcp add cloudai-fusion \
  -- python -m otcore.mcp_server <graph>

⬇ download graph artifact

Ask about this repo answers extend the page