MCPcopy
hub / github.com/zincsearch/zincsearch

github.com/zincsearch/zincsearch @v0.4.10 sqlite

repository ↗ · DeepWiki ↗ · release v0.4.10 ↗
1,044 symbols 4,354 edges 361 files 140 documented · 13%
README

Go Report Card Docs codecov

❗Note: If your use case is of log search (app and security logs) instead of app search (implement search feature in your application or website) then you should check openobserve/openobserve project built in rust that is specifically built for log search use case.

ZincSearch

ZincSearch is a search engine that does full text indexing. It is a lightweight alternative to Elasticsearch and runs using a fraction of the resources. It uses bluge as the underlying indexing library.

It is very simple and easy to operate as opposed to Elasticsearch which requires a couple dozen knobs to understand and tune which you can get up and running in 2 minutes

It is a drop-in replacement for Elasticsearch if you are just ingesting data using APIs and searching using kibana (Kibana is not supported with ZincSearch. ZincSearch provides its own UI).

Check the below video for a quick demo of ZincSearch.

Zinc Youtube

Why ZincSearch

While Elasticsearch is a very good product, it is complex and requires lots of resources and is more than a decade old. I built ZincSearch so it becomes easier for folks to use full text search indexing without doing a lot of work.

Features:

  1. Provides full text indexing capability
  2. Single binary for installation and running. Binaries available under releases for multiple platforms.
  3. Web UI for querying data written in Vue
  4. Compatibility with Elasticsearch APIs for ingestion of data (single record and bulk API)
  5. Out of the box authentication
  6. Schema less - No need to define schema upfront and different documents in the same index can have different fields.
  7. Index storage in disk
  8. aggregation support

Documentation

Documentation is available at https://zincsearch-docs.zinc.dev/

Screenshots

Search screen

Search screen

User management screen

Users screen

Getting started

Quickstart

Check Quickstart

Releases

ZincSearch has hundreds of production installations.

ZincSearch Vs OpenObserve

Feature ZincSearch OpenObserve
Ideal use case App search Logs, metrics, traces (Immutable Data)
Storage Disk Disk, Object (S3), GCS, MinIO, swift and more.
Preferred Use case App search Observability (Logs, metrics, traces)
Max data supported 100s of GBs Petabyte scale
High availability Not available Yes
Open source Yes Yes, OpenObserve
ES API compatibility Yes Yes
GUI Basic Very Advanced, including dashboards
Cost Open source Open source
Get started Open source docs Open source docs or Cloud

Community

  • How to develop and contribute to ZincSearch

Check the contributing guide . Also check the roadmap items

Examples

You can use ZincSearch to index and search any data. Here are some examples that folks have created to index and search enron email dataset using zincsearch:

  1. https://github.com/jorgeloaiza48/Enron-Email-DataSet
  2. https://github.com/jhojanperlaza/email_search_engine
  3. https://github.com/carlosarraes/zinmail
  4. https://github.com/devjopa/golab-search
  5. https://github.com/avaco2312/zincsearch
  6. https://github.com/paolorossig/email-indexer

Extension points exported contracts — how you extend this code

SearchAggregation (Interface)
(no doc) [4 implementers]
pkg/bluge/aggregation/aggregation.go
Storager (Interface)
(no doc) [3 implementers]
pkg/metadata/storage/storage.go
Hasher (Interface)
(no doc) [1 implementers]
pkg/zutils/hash/hash.go
Chainable (Interface)
(no doc)
web/cypress/support/commands.ts
IsTokenRune (FuncType)
(no doc)
pkg/bluge/analysis/tokenizer/char_group.go

Core symbols most depended-on inside this repo

Error
called by 186
pkg/errors/error.go
New
called by 185
pkg/errors/error.go
request
called by 179
test/api/init.go
AuthMiddleware
called by 73
pkg/routes/middleware.go
GinRenderJSON
called by 61
pkg/zutils/gin.go
Contains
called by 53
pkg/zutils/hash/rendezvous/rendezvous.go
GetName
called by 52
pkg/core/index.go
Set
called by 45
pkg/metadata/storage/storage.go

Shape

Function 529
Method 269
Struct 238
Interface 4
TypeAlias 3
FuncType 1

Languages

Go98%
TypeScript2%
Python1%

Modules by API surface

pkg/meta/query_dsl.go44 symbols
pkg/core/index.go23 symbols
pkg/bluge/aggregation/terms.go20 symbols
pkg/core/index_shards.go19 symbols
pkg/bluge/aggregation/auto_date_histogram.go18 symbols
pkg/bluge/aggregation/histogram.go16 symbols
pkg/bluge/aggregation/date_histogram.go15 symbols
pkg/meta/v1/types.go14 symbols
pkg/core/index_shards_wal.go14 symbols
pkg/zutils/hash/rendezvous/rendezvous.go13 symbols
pkg/core/indexlist.go13 symbols
pkg/meta/mappings.go12 symbols

Dependencies from manifests, versioned

github.com/KyleBanks/depthv1.2.1 · 1×
github.com/RoaringBitmap/roaringv0.9.4 · 1×
github.com/axiomhq/hyperloglogv0.0.0-2023020108522 · 1×
github.com/beorn7/perksv1.0.1 · 1×
github.com/bits-and-blooms/bitsetv1.2.2 · 1×
github.com/blevesearch/go-porterstemmerv1.0.3 · 1×
github.com/blevesearch/mmap-gov1.0.4 · 1×
github.com/blevesearch/segmentv0.9.1 · 1×
github.com/blevesearch/snowballstemv0.9.0 · 1×
github.com/blevesearch/vellumv1.0.10 · 1×
github.com/blugelabs/blugev0.1.9 · 1×
github.com/blugelabs/bluge_segment_apiv0.2.0 · 1×

For agents

$ claude mcp add zincsearch \
  -- python -m otcore.mcp_server <graph>

⬇ download graph artifact