Persisting node-cache

[horak]

I am using node-cache for a dead simple string key/val lookup & I'd like the cache to persist across server restarts.

Would it be possible, to save the cache to, for example, a JSON file which I read into node-cache on server initialization?

Is this something folks already use node-cache for, or shall I look elsewhere?

[Themis3000]

Seems technically possible. You could read all values with myNodeCache.data, store it in a json, then on server startup read that json file and readd the values with something like this (untested psudo-code. may or may not work, just used as example.):

const success = myNodeCache.mset(Object.entries(loadedJsonObject).forEach(([key, value]) => {
  {key: key, val: value.v, ttl: value.t}
}));

This is most likely something others aren't doing with this library though. node-cache as far as I understand is meant only for caching temporary values. If you'd like long term data storage, I'd recommend something like jsoning as a replacement. It doesn't have the feature of ttl with automatic deletion, but it is persistent. Otherwise, if you're more adventurous, something like mongodb would be more robust for persistent data storage.

If you do still want a extremely performant database with the ability to set ttl values that won't clear out every time you restart your program, you could use redis but it will be more complicated as it's a database ran separated from your script (like monngodb). It stores all data in memory, so it's very fast but the database it's self will clear out between restarts. You do not however need to restart the database every time you restart your program. It would be a solution that would make sense if the data you're storing doesn't need to be persisted, it would just be nice to have if it where between restarts.

Otherwise, if some data needs to be persisted and other data does not need to be persisted, it is common practice to use a caching db like node-cache or redis in combination with a db built for persistence like jsoning or mongodb. That way all your temporary data can be stored in your caching database, and all the data that needs to persist can be stored in your persistent database. Don't feel like it's wrong to use 2 databases for 2 different purposes in that case.

Let me know if you have any questions about that. I'll help you persist data between restarts of node-cache if you'd like, just know that's probably starting to reach outside of the intended use of the package. It will work though, so if that solution makes the most sense to you then go for it. Thought I'd just lay out other common solutions for you if you'd like to go for a more common/intended route

[horak]

I am currently using persistent-cache. It suits my needs well and seems similar to your recommendation, jsoning. +1 for keeping it simple.

Thank you wholeheartedly for the detailed response; especially for the additional offer to help me find a solution!

[Themis3000]

Awesome! Glad you found a good solution that fits your needs perfectly, good luck on your projects!

[matys1]

I use persistent-cache to cache the data when in local development since after every file save nodemon restarts the server and I use node-cache in prod when deployed.

[ve3]

Currently there are warnings after installed persistent-cache

4 moderate severity vulnerabilities

Use with care.

[ve3]

This is my implement persistent cache based on node-cache.

/**
 * Make cache stay alive even the process started and ended and start again.
 * 
 * @link https://github.com/node-cache/node-cache/issues/293 The ideas.
 */


'use strict';


// import node & installed packages.
import fs from 'node:fs';
import NodeCache from 'node-cache';
import path from 'node:path';


export default class Cache {


    /**
     * @property {NodeCache} cacheObj The cache object using Node package.
     */
    #cacheObj = {};


    /**
     * @property {object} options The options.
     */
    #options = {};


    /**
     * 
     * @param {object} options Accept arguments.
     * @param {string} options.cacheJsonPath The cache JSON file full path. This file will be use for persistent cache across process end/start.
     * @param {object} options.cacheOptions The cache options. Currently use node-cache, see https://www.npmjs.com/package/node-cache.
     */
    constructor({} = {}) {
        const defaults = {
            cacheJsonPath: path.resolve(YOUR_APP_ROOT_PATH + '/.cache/cache.json'),
            cacheOptions: {},
        }

        this.#options = {
            ...defaults,
            ...arguments[0]
        }

        this.#prepareCacheFolder();

        this.#cacheObj = new NodeCache(this.#options.cacheOptions);
        this.#restoreCacheFile();
        this.#listenEvents();
    }// constructor


    /**
     * Listen cache class events to work.
     * 
     * @private This method was called from `constructor()`.
     */
    #listenEvents() {
        this.#cacheObj.on('set', (key, value) => {
            this.#writeAllCacheData(this.#cacheObj.data);
        });

        this.#cacheObj.on('del', (key, value) => {
            this.#writeAllCacheData(this.#cacheObj.data);
        });

        this.#cacheObj.on('expired', (key, value) => {
            this.#writeAllCacheData(this.#cacheObj.data);
        });

        this.#cacheObj.on('flush', () => {
            this.#writeAllCacheData(this.#cacheObj.data);
        });
    }// listenEvents


    /**
     * Prepare cache JSON file.
     * 
     * @private This method was called from `#prepareCacheFolder()`.
     */
    #prepareCacheFile() {
        const content = {};

        this.#writeAllCacheData(content);
    }// prepareCacheFile


    /**
     * Prepare cache folder where JSON cache file will be store.
     * 
     * @private This method was called from `constructor()`.
     * @returns {void}
     */
    #prepareCacheFolder() {
        if (fs.existsSync(this.#options.cacheJsonPath)) {
            // if file is already exists.
            // do nothing.
            return ;
        }

        const cacheDir = path.dirname(this.#options.cacheJsonPath);
        if (!fs.existsSync(cacheDir)) {
            fs.mkdirSync(cacheDir, {recursive: true});
            this.#prepareCacheFile();
        }

        // check again.
        if (!fs.existsSync(cacheDir)) {
            // if still not exists.
            throw new Error('The cache folder is not exists and could not be created. (' + cacheDir + ')');
        }
    }// prepareCacheFolder


    /**
     * Restore cache file.
     * 
     * @private This method was called from `constructor()`.
     */
    #restoreCacheFile() {
        const JSONContent = fs.readFileSync(
            this.#options.cacheJsonPath,
            {
                'encoding': 'utf8',
                'flag': 'r',
            }
        );

        this.#cacheObj.data = JSON.parse(JSONContent);
    }// restoreCacheFile


    /**
     * Write all cache data to JSON file.
     * 
     * @private This method was called from `#listenEvents()`, `#prepareCacheFile()`.
     * @param {mixed} data The data to write.
     */
    #writeAllCacheData(data) {
        const dataJSON = JSON.stringify(data, undefined, 4);

        fs.writeFileSync(
            this.#options.cacheJsonPath, 
            dataJSON,
            {
                'encoding': 'utf8',
            }
        );
    }// writeAllCacheData


    /**
     * Returns the cache object on `#cacheObj` property.
     * 
     * @returns {object} Returns the cache object on `#cacheObj` property.
     */
    cacheObj() {
        return this.#cacheObj;
    }// cacheObj


}

Usage:

import Cache from '../Libraries/Cache.mjs';

const cacheObj = new Cache();
const nodeCacheObj = cacheObj.cacheObj();
const windowProcessID = process.ppid;
const cacheKey = 'cache-' + windowProcessID;
const cachedVal = nodeCacheObj.get(cacheKey);

console.log('cached value: ', cachedVal);

if (typeof(cachedVal) === 'undefined') {
    // if no cache.
    nodeCacheObj.set(cacheKey, 'hello world');
    console.log('cache was set.');
}

console.log('Retrieve cache again: ', nodeCacheObj.get(cacheKey));

I hope it is useful for you.