MCPcopy Create free account
hub / github.com/BerriAI/litellm / set_cache

Method set_cache

litellm/caching/in_memory_cache.py:175–195  ·  view source on GitHub ↗
(self, key, value, **kwargs)

Source from the content-addressed store, hash-verified

173 return False
174
175 def set_cache(self, key, value, **kwargs):
176 # Handle the edge case where max_size_in_memory is 0
177 if self.max_size_in_memory == 0:
178 return # Don't cache anything if max size is 0
179
180 with self._write_lock:
181 # Always prune expired/outdated heap roots before inserting.
182 # This keeps expiration_heap bounded even when the live cache stays
183 # below max_size_in_memory and keys are reinserted after TTL expiry.
184 self.evict_cache()
185 if not self.check_value_size(value):
186 return
187
188 self.cache_dict[key] = value
189 if self.allow_ttl_override(key): # if ttl is not set, set it to default ttl
190 if "ttl" in kwargs and kwargs["ttl"] is not None:
191 self.ttl_dict[key] = self._clock() + float(kwargs["ttl"])
192 heapq.heappush(self.expiration_heap, (self.ttl_dict[key], key))
193 else:
194 self.ttl_dict[key] = self._clock() + self.default_ttl
195 heapq.heappush(self.expiration_heap, (self.ttl_dict[key], key))
196
197 async def async_set_cache(self, key, value, **kwargs):
198 self.set_cache(key=key, value=value, **kwargs)

Callers 4

async_set_cacheMethod · 0.95
async_set_cache_saddMethod · 0.95
increment_cacheMethod · 0.95

Calls 4

evict_cacheMethod · 0.95
check_value_sizeMethod · 0.95
allow_ttl_overrideMethod · 0.95
floatFunction · 0.50

Tested by

no test coverage detected