MCPcopy Create free account
hub / github.com/AdRoll/baker / ProcessFile

Method ProcessFile

input/inpututils/compressedstream.go:224–233  ·  view source on GitHub ↗

Enqueue a file for processing by compressedInput. This function must be called by subclasses to schedule processing a (gzip|zstd) logfile. The function just enqueues the file and exits, so it's normally fast, but might block if the backlog is bigger than internal channel size (default: 1024 files)

(fn string)

Source from the content-addressed store, hash-verified

222// but might block if the backlog is bigger than internal channel size
223// (default: 1024 files)
224func (s *CompressedInput) ProcessFile(fn string) error {
225 // Use the Sizer on the file to acquire the length
226 sz, err := s.Sizer(fn)
227 if err != nil {
228 return err
229 }
230 s.stats.NewFile(sz)
231 s.files <- fn
232 return nil
233}
234
235// Signal compressedInput that we've finished enqueuing files, and it can exit
236// whenever it has finished processing what was already enqueued. This can

Callers 5

TestGzipStreamFunction · 0.95
ProcessDirectoryMethod · 0.80
processListMethod · 0.80
processFileOrListMethod · 0.80
ProcessDirectoryMethod · 0.80

Calls 1

NewFileMethod · 0.80

Tested by 1

TestGzipStreamFunction · 0.76