Method decode

arrow-csv/src/reader/mod.rs:636–649

Decode records from `buf` returning the number of bytes read This method returns once `batch_size` objects have been parsed since the last call to [`Self::flush`], or `buf` is exhausted. Any remaining bytes should be included in the next call to [`Self::decode`] There is no requirement that `buf` contains a whole number of records, facilitating integration with arbitrary byte streams, such as th

(&mut self, buf: &[u8])

    /// integration with arbitrary byte streams, such as that yielded by [`BufRead`] or
    /// network sources such as object storage
    pub fn decode(&mut self, buf: &[u8]) -> Result<usize, ArrowError> {
        if self.to_skip != 0 {
            // Skip in units of `to_read` to avoid over-allocating buffers
            let to_skip = self.to_skip.min(self.batch_size);
            let (skipped, bytes) = self.record_decoder.decode(buf, to_skip)?;
            self.to_skip -= skipped;
            self.record_decoder.clear();
            return Ok(bytes);
        }

        let to_read = self.batch_size.min(self.end - self.line_number) - self.record_decoder.len();
        let (_, bytes) = self.record_decoder.decode(buf, to_read)?;
        Ok(bytes)
    }

    /// Flushes the currently buffered data to a [`RecordBatch`]
    ///