README


CloudOps, without the chaos
CloudQuery is a cloud asset inventory built for platform teams. Sync your cloud infrastructure metadata into your data warehouse, powering insights and automation. Unify data and provide context across AWS, Azure, GCP, and 70+ cloud and SaaS sources such as Wiz, Finout, and GitHub.
Installation
brew install cloudquery/tap/cloudquery
Check out the quickstart guide for install guides for Linux and Windows and for step-by-step instructions on completing your first sync with CloudQuery.
Why CloudQuery?
- Specialized plugin coverage - Support for cloud infrastructure, security, and FinOps sources with normalization, rate limit handling, and more.
- Normalized data -
- Queryable with SQL - No more writing scripts to hit poorly documented APIs: CloudQuery makes it easy to unify and make your cloud asset data accessible.
- Integrate with anything - Turn your cloud data into useful signals by connecting with BI tools, Slack alerts, Jira ticketing, and more.
- Fast, powerful syncs - Move large volumes of data with high performance and fine-grained control, powered by Apache Arrow.
- Runs on your infrastructure - Your cloud data never touches CloudQuery's servers. Full privacy, built for regulated, secure, and performance-critical environments.
- Composable and flexible - Use the languages, destinations, and orchestrators you want. CloudQuery is built to fit into your stack, not the other way around.
- Built for developers - Code-first, extensible plugins, multi-language, open plugin system, no lock-in. Write it, extend it, ship it.
Use Cases
Links
License
By contributing to CloudQuery, you agree that your contributions will be licensed as defined in the LICENSE file.
Contribution
Feel free to open a pull request for small fixes and changes. For bigger changes and new integrations, please open an issue first to prevent duplicated work and to have the relevant discussions first.
Open source
The CloudQuery framework, SDK, CLI, and some integrations are open source - please file an issue before opening a PR.
Any code that was open source and moved to closed source can be found in the git history of this repo, or via the following links:
Extension points exported contracts — how you extend this code
EmbeddingsClient (Interface)
(no doc) [15 implementers]
plugins/destination/bigquery/client/embeddings_client.go
HackernewsClient (Interface)
go:generate mockgen -package=mocks -destination=../mocks/hackernews.go -source=hackernews.go HackernewsClient [2 implementers]
plugins/source/hackernews/client/services/hackernews.go
RecordTransformerOption (FuncType)
(no doc)
cli/internal/transformer/transformer.go
DBPool (Interface)
DBPool is an interface that abstracts the pgxpool.Pool and pgxpool.Conn types for easier fine-grained testing. [1 implementers]
plugins/destination/postgresql/client/client.go
BackendClient (Interface)
go:generate mockgen -package=mocks -destination=../mocks/backend.go -source=backend.go BackendClient
plugins/source/hackernews/client/services/backend.go
OtelReceiverOption (FuncType)
(no doc)
cli/internal/otel/receiver.go
EmbeddingsRequesterOption (FuncType)
EmbeddingsRequesterOption allows customizing the requester.
plugins/destination/postgresql/client/embeddings_requester.go
Option (FuncType)
(no doc)
plugins/source/xkcd/internal/xkcd/xkcd.go
Core symbols most depended-on inside this repo
Errorf
called by 848
cli/internal/analytics/client.go
Append
called by 308
plugins/destination/clickhouse/typeconv/arrow/values/primitive.go
Value
called by 260
plugins/destination/sqlite/typeconv/primitive.go
Close
called by 177
plugins/destination/postgresql/client/client.go
Run
called by 133
cli/internal/otel/opentelemetry.go
Println
called by 121
plugins/destination/kafka/client/logger.go
String
called by 104
plugins/destination/clickhouse/client/spec/engine.go
Printf
called by 82
plugins/destination/kafka/client/logger.go
Shape
Function
1,399
Method
867
Struct
232
Class
50
TypeAlias
21
Interface
15
FuncType
14
Enum
1
Route
1
Languages
Go91%
Python7%
Java1%
TypeScript1%
Modules by API surface
cli/internal/platform/inject_test.go42 symbols
plugins/destination/test/client/client.go29 symbols
cli/internal/publish/plugins.go28 symbols
plugins/transformer/basic/client/recordupdater/record_updater_test.go25 symbols
plugins/transformer/basic/client/recordupdater/record_updater.go25 symbols
cli/internal/otel/receiver.go25 symbols
plugins/source/hackernews/client/mocks/hackernews.go24 symbols
cli/internal/specs/v0/spec_reader.go22 symbols
cli/internal/analytics/client.go22 symbols
cli/internal/platform/inject.go21 symbols
plugins/destination/sqlite-python/plugin/sqlite/migrate.py20 symbols
plugins/destination/postgresql/client/client.go18 symbols
Dependencies from manifests, versioned
cloud.google.com/gov0.123.0 · 1×
cloud.google.com/go/auth/oauth2adaptv0.2.8 · 1×
cloud.google.com/go/bigqueryv1.77.0 · 1×
cloud.google.com/go/compute/metadatav0.9.0 · 1×
cloud.google.com/go/monitoringv1.24.3 · 1×
cloud.google.com/go/storagev1.62.3 · 1×
filippo.io/edwards25519v1.2.0 · 1×
github.com/99designs/go-keychainv0.0.0-2019100805025 · 1×
github.com/99designs/keyringv1.2.2 · 1×
Datastores touched
postgresDatabase · 1 repos
(mongodb)Database · 1 repos
(mysql)Database · 1 repos
dbDatabase · 1 repos
mydbDatabase · 1 repos
myDatabaseDatabase · 1 repos
cloudqueryDatabase · 1 repos
db_nameDatabase · 1 repos