generateSessionID creates a unique session ID for this dispatcher instance. Format: {collectorID}_{timestamp_nano}

persistConnection saves a new connection to the collection.

setupCollector creates a complete collector with all services on one gRPC server

Backup creates an online backup of the database to the specified path. This should be implemented in a WAL-friendly way to allow concurrent access.

BackupOnline creates an online backup using incremental page copying. This minimizes lock time by copying pages in small batches. Best for very large

BackupPath generates a backup file path for a collection. Format: {DataDir}/.backup/{namespace}/{collectionname}-{timestampseconds}.db

BackupPathMicro generates a backup file path using microsecond precision. Format: {DataDir}/.backup/{namespace}/{collectionname}-{timestampmicrosecond

CleanupTimedOutOperations scans all collections and clears timed-out operations. Should be called on server startup to recover from crashes.

Clone creates a consistent snapshot of the collection database. Uses SQLite's online backup API for hot backup with minimal locking. This allows concu

CloneRemote clones a collection to a remote collector using streaming.

CollectorID returns the unique identifier for this collector.

CountMessageTypes returns the number of registered message types

DefaultProtoSizeLimits returns default size limits for proto registration

GetCollection retrieves a Collection instance by namespace and name.

GetCollection retrieves a collection by namespace and name.

Load reads content from a file at the given path.

LookupProtoByMessageName searches for a registered proto that contains the given message name. It searches within the specified namespace and returns

Namespace returns the default namespace for this collector.

NewAuditLogger creates a new audit logger backed by the system/audit collection.

NewDynamicJSONConverter creates a converter that looks up the message type from a FileDescriptorProto and uses dynamic protobuf to convert. This is us

NewFetcher creates a new Fetcher with default settings.

NewGRPCRegistryValidator creates a validator using the provided validator implementation

NewRegistryConverterFactory creates a converter factory that looks up types from the registry. It first checks system types, then uses the provided lo

NewSystemLogger creates a new system logger

PullCollection streams a collection to a client.

PushCollection receives a streamed collection from a client and creates it locally.

ReceivePushedCollection handles incoming collection push streams (server-side).

RestoreBackup restores a collection from a backup.

SaveCollection persists a collection to the registry.

SaveDir recursively saves a CollectionDir structure.

SetJSONConverterFactory sets the factory used to create JSON converters for stores. If not set, stores will use fallback behavior (proto_data if valid

SetRegistryValidator sets the registry validator for this dispatcher

SetTypeRegistrar sets the type registrar for this server. This is optional - if not set, types are not registered in the type registry.

SetupCollectionRepoWithValidation demonstrates how to create a CollectionRepo server with registry validation enabled.

SetupDispatcherWithValidation demonstrates how to create a CollectiveDispatcher server with registry validation enabled.

Start starts the server (blocks until shutdown). This is a no-op since the server is already started in New(). Use WaitForShutdown() to block until a

StreamCollectionToPuller handles outgoing collection pull streams (server-side).

UnaryServerInterceptor returns a gRPC interceptor that logs all unary RPCs.

ValidateProtoFileName validates a proto file name (allows dots for .proto extension).

ValidateProtoSizeLimits validates that a FileDescriptorProto is within configured limits

ValidateServiceName validates a service name (rejects slashes which break ID format).

WithValidation returns gRPC server options that add registry validation interceptors for the specified namespace.

addContentToTar adds content from memory to a tar archive.

addFileToTar adds a file from disk to a tar archive.

autoRoute automatically routes a request based on namespace

bootstrapAudit creates the audit logging collection

bootstrapCollectionRegistry creates the self-referential collection registry This is the "chicken and egg" - the collection that tracks all collection

bootstrapConnections creates the connection tracking collection

bootstrapLogs creates the system logs collection for structured logging

bootstrapTypeRegistry creates the type registry collection This will store registered proto types and enable request validation

calculateMaxNestingDepth calculates the maximum nesting depth in a proto file

calculateNestingDepthRecursive recursively calculates nesting depth for a message

checkMessage recursively checks string fields in a protobuf message

checkMessageReflect recursively checks all string fields including nested messages

convertStructpbValue converts a structpb.Value to a native Go type

countMessageTypesRecursive recursively counts message types including nested ones

countPreviousConnections counts how many times a source has connected to us.

countPreviousConnectionsToTarget counts how many times we've connected to a target.

countTotalMessageTypes recursively counts all message types in a proto file

dispatchToTarget sends a request to a specific target collector

escapeJSONPathKey escapes a key for use in SQLite JSON path expressions. Keys with dots, brackets, quotes, or other special characters need to be quot

findSharedNamespaces finds namespaces that are in both lists.

Helper functions

getNamespaceHierarchy returns a namespace and all its parent namespaces For example, "team/project/service" returns ["team/project/service", "team/pro

isWellKnownType checks if a proto file is a well-known type that doesn't require registration

loadFromRegistry loads existing collections from the registry store into the in-memory cache. This is called during initialization to restore state.