How can i set an expiration time (ttl) for an entry? How can i invalidate the cache?

[msalafia]

As the title suggest, i would like to know how can i invalidate some cache entries because my AI Agent returns live data that change quite often, therefore cached answers are not viable anymore after a certain amount of time.

[SimFG]

If you want to use the ttl function for the cache, you need to use the redis dependency. example: https://github.com/zilliztech/GPTCache/blob/main/examples/eviction/distributed_eviction.py
And there are other choice, like fifio/lru, link:https://github.com/zilliztech/GPTCache/blob/main/gptcache/manager/eviction/__init__.py

[msalafia]

What if I am using the gptcache_server docker image? Is this a valuable solution?

[SimFG]

i suggest you use the memory eviction strategy, because if you want to use the redis strategy, you will deploy the redis server and config more. the memory example(the eviction_manager and eviction_params are added):

storage_config:
    data_dir:
        gptcache_data
    manager:
        sqlite,faiss
    vector_params:
        # Set vector storage related params here
    eviction_manager:
        memory
    eviction_params:
        policy: LRU
        maxsize: 1000
        clean_size: 200

more policy: https://github.com/zilliztech/GPTCache/blob/main/gptcache/manager/eviction/memory_cache.py

[msalafia]

The memory solution doesn't seems feasable, because my AI Agent deals with live data on soccer match (so they change quite often during live matches), so i cannot let the Cache evict freely following some policy (e.g. LRU) when the memory is full.

Redis with a ttl seems something more close to my use cases, so i can say to evict an entry after, for instance, 1 minute. Is there any more detailed guide or example on how to accomplish this using gptcache_server?

Furthermore, i imagine i should probably rebuild the docker image with some custom addition to the Dockerfile, am i correct?

[SimFG]

if you want to use the redis, you can try to:

storage_config:
    data_dir:
        gptcache_data
    manager:
        sqlite,faiss
    vector_params:
        # Set vector storage related params here
    eviction_manager:
        redis
    eviction_params:
        host: xxx #redis host
        port: xxx #redis port
        maxmemory: xxx
        policy: xxx
        global_key_prefix: gptcache_cache
        ttl: xxx
        maxmemory_samples: xxx

param details: https://github.com/zilliztech/GPTCache/blob/main/gptcache/manager/eviction/redis_eviction.py