newColdCache returns a coldCache with len(bsizes) buckets, each of which can hold ncells cells (or buffers), bsizes is the list of bucket cell sizes.

nextSplit returns the filename indicating the next split considering the following splitting rules: - the part number is the string such '.part-XXX' w

DurationWithTags adds a duration to an histogram and associates that duration with a set of tags.

This function is called when the filter is finished with the memory received through the input channel. Since the memory was allocated by Input, it is

MainCLI provides a handy way to quickly create a command-line interface to Baker by providing the list of components available to build and run a topo

NewStatsDumper creates and initializes a StatsDumper using the given topology and writing stats on standard output. It also exports metrics via the Me

Signal compressedInput that we've finished enqueuing files, and it can exit whenever it has finished processing what was already enqueued. This can be

PublishedMetrics returns a list of strings, each of which represent arguments and method of calls to methods of the baker.MetricsClient interface. Pre

Run starts dumping stats every second on standard output. Call stop() to stop periodically dumping stats, this prints stats one last time.

evaluateURLs evaluates urls using the current configuration..

newBucket instantiates a new bucket with ncells cells, each of which can hold 'cellbytes' bytes long (actually cellbytes - 4 since the buffer length i

putInCold puts buf in the cold cache if its size fits it, otherwise the buffer is put into the hot cache.

putInHot places buf in the hot cache, copying it into a new hot cache buffer if that's the first with that key, or appending to the previous buffer wi

testFileWriterIntegration builds and run a topology reading from testdata/filewriter/input.csv.log.zst and using the FileWriter output, configured wit

CheckRequiredFields checks that all fields that are tagged as required in cfg's type have actually been set to a value other than the field type zero

DeltaCount increments the value of a metric of type counter by delta. delta must be positive.

GaugeWithTags sets the value of a metric of type gauge and associates that value with a set of tags.

Meta returns the metadata having the given specific key, if any.

New returns an io.WriteCloser that writes to fname; when it's closed, fname will be split in multiple files each of which having at most maxsize bytes

NewHook returns a Logrus hook that forwards log entries as events to a statsd client, such as the datadog-agent. Log entries with a level higher than

NewTimestamp creates and configures a Timestamp filter.

NewTimestampRange creates and configures a TimestampRange filter.

PrintHelp prints the help message for the given component, identified by its name. When name is '*' it shows the help messages for all components. Th

RawCountWithTags sets the value of a metric or type counter and associates that value with a set of tags.

RenderHelpMarkdown prints markdown formatted help for a single component or for all of them (with name = '*'), and renders it so that it can be printe

RequiredFields returns the names of the underlying configuration structure fields which are tagged as required. To tag a field as being required, a "r

SetWriter sets the writer into which stats are written. SetWriter must be called before Run().

TempDir is a test helper that creates a temporary directory, returns its name and a function which when called removes that directory. This is useful

assignFieldMapping verifies that field mapping has been set once, but only once (either in cfg or comp). Then if that is the case, assignFieldMapping

assignValidationMapping verifies that validation has been set once either in comp or in cfg. If validation has not been set assignValidationMapping ge

clearBit clears bit i (0-based, starting from LSB)

copyByte copies c at index pos of the internal buffer, growing it if necessary.

countLines returns the number of lines in a file, either compressed (zstd/gz) or uncompressed.

findFirstZero64 returns the index of the first bit at 0, starting from the LSB (least significant bit).

findFreeCell returns the index of the first free cell or false.

flushBlocks decompresses and flushes consecutive blocks in buf, interleaving each of them with a `\n` in the decompressed buffer.

Check if scheme is s3 or s3a or s3n

lineAt returns the line in text starting at offset offs.

makeUnivocal ensure each string in slist is univocal, appending '_2' to duplicates, '_3' to triplicates, and so on. Non-repeated strings are not modif

makeWriteCloser converts an io.Writer and a Close function into a WriteCloser.

moveToHotCache moves a buffer from the cold to the hot cache. loc represents the location of the buffer in the cold cache.

newClient creates a baker.MetricsClient that pushes to the datadog server using the dogstatsd format. All exported metrics will have a name prepended

Open a file and return the io Reader, the size, the last modification, and the path as URL. If the sizeOnly parameter is true no io Reader will be ret