MCPcopy Create free account
hub / github.com/NodeDB-Lab/nodedb / read_files

Method read_files

nodedb/src/data/io/uring_reader.rs:100–244  ·  view source on GitHub ↗

Read multiple files in a single batched io_uring submission. Opens each file, submits `IORING_OP_READ` SQEs for all, then waits for all completions. Returns file contents in the same order as `paths`. Files that fail to open are returned as empty `Vec `.

(&mut self, paths: &[&Path])

Source from the content-addressed store, hash-verified

98 ///
99 /// Files that fail to open are returned as empty `Vec<u8>`.
100 pub fn read_files(&mut self, paths: &[&Path]) -> Vec<Vec<u8>> {
101 if paths.is_empty() {
102 return Vec::new();
103 }
104
105 // Phase 1: Open files, determine sizes, assign buffers.
106 let mut reads: Vec<PendingRead> = Vec::with_capacity(paths.len());
107 let mut oversized: Vec<AlignedBuf> = Vec::new();
108
109 for (i, path) in paths.iter().enumerate() {
110 let file = match std::fs::File::open(path) {
111 Ok(f) => f,
112 Err(_) => {
113 reads.push(PendingRead::failed(i));
114 continue;
115 }
116 };
117 let size = file.metadata().map(|m| m.len() as usize).unwrap_or(0);
118 if size == 0 {
119 reads.push(PendingRead::failed(i));
120 continue;
121 }
122
123 let buf_source = if size <= self.buf_size {
124 if let Some(slot) = self.free.pop() {
125 BufSource::Pool(slot)
126 } else {
127 // Pool exhausted — allocate dedicated.
128 match AlignedBuf::new(size) {
129 Ok(buf) => {
130 let idx = oversized.len();
131 oversized.push(buf);
132 BufSource::Oversized(idx)
133 }
134 Err(_) => {
135 reads.push(PendingRead::failed(i));
136 continue;
137 }
138 }
139 }
140 } else {
141 match AlignedBuf::new(size) {
142 Ok(buf) => {
143 let idx = oversized.len();
144 oversized.push(buf);
145 BufSource::Oversized(idx)
146 }
147 Err(_) => {
148 reads.push(PendingRead::failed(i));
149 continue;
150 }
151 }
152 };
153
154 reads.push(PendingRead {
155 index: i,
156 file: Some(file),
157 size,

Callers 8

read_single_fileFunction · 0.80
read_oversized_fileFunction · 0.80
read_empty_pathsFunction · 0.80
pool_buffers_are_reusedFunction · 0.80
read_partition_columnsFunction · 0.80

Calls 15

PoolClass · 0.85
round_up_readFunction · 0.85
is_okMethod · 0.80
openFunction · 0.50
is_emptyMethod · 0.45
lenMethod · 0.45
iterMethod · 0.45
pushMethod · 0.45
as_mut_ptrMethod · 0.45
capacityMethod · 0.45
as_raw_fdMethod · 0.45
buildMethod · 0.45

Tested by 6

read_single_fileFunction · 0.64
read_oversized_fileFunction · 0.64
read_empty_pathsFunction · 0.64
pool_buffers_are_reusedFunction · 0.64