MCPcopy Create free account
hub / github.com/DeepRec-AI/DeepRec / ReadChecksummed

Method ReadChecksummed

tensorflow/core/lib/io/record_reader.cc:87–109  ·  view source on GitHub ↗

Read n+4 bytes from file, verify that checksum of first n bytes is stored in the last 4 bytes and store the first n bytes in *result. offset corresponds to the user-provided value to ReadRecord() and is used only in error messages.

Source from the content-addressed store, hash-verified

85// offset corresponds to the user-provided value to ReadRecord()
86// and is used only in error messages.
87Status RecordReader::ReadChecksummed(uint64 offset, size_t n, string* result) {
88 if (n >= SIZE_MAX - sizeof(uint32)) {
89 return errors::DataLoss("record size too large");
90 }
91
92 const size_t expected = n + sizeof(uint32);
93 TF_RETURN_IF_ERROR(input_stream_->ReadNBytes(expected, result));
94
95 if (result->size() != expected) {
96 if (result->empty()) {
97 return errors::OutOfRange("eof");
98 } else {
99 return errors::DataLoss("truncated record at ", offset);
100 }
101 }
102
103 const uint32 masked_crc = core::DecodeFixed32(result->data() + n);
104 if (crc32c::Unmask(masked_crc) != crc32c::Value(result->data(), n)) {
105 return errors::DataLoss("corrupted record at ", offset);
106 }
107 result->resize(n);
108 return Status::OK();
109}
110
111Status RecordReader::GetMetadata(Metadata* md) {
112 if (!md) {

Callers

nothing calls this directly

Calls 8

DecodeFixed32Function · 0.85
UnmaskFunction · 0.85
ValueFunction · 0.50
ReadNBytesMethod · 0.45
sizeMethod · 0.45
emptyMethod · 0.45
dataMethod · 0.45
resizeMethod · 0.45

Tested by

no test coverage detected