python_setup

MMKV for Python (on POSIX)

MMKV is an efficient, small, easy-to-use mobile key-value storage framework used in the WeChat application. It's currently available on both Android, iOS/macOS, Windows and Python/Golang (POSIX).

Getting Started

Prerequisites

• Python 2.7 and above.
• CMake 3.8.0 and above.
• C++ compiler that supports C++ 17 standard.
• Linux(Ubuntu, Arch Linux, CentOS, Gentoo), Unix(macOS, FreeBSD, OpenBSD) are supported.

Installation Via Source

1. Getting source code from the git repository:

git clone https://github.com/Tencent/MMKV.git

2. Prepare pybind11.
   2.1 Get pybind11 (v2.5.0) source code:

cd MMKV
git submodule update --init --recursive

2.2 On Linux, install python-dev for Python 2.x, or python3-dev for Python 3.x. On macOS, you don't need to install those things. See also pybind11 docs.
For example, let's say we are using Python 3.x on Ubuntu:

sudo apt-get install python3-dev

3. Build & install the MMKV native library for Python (you might need root permission to install it). Assuming we are using Python 3.x:

cd POSIX/Python
python3 setup.py install
# or if you're not login as root
sudo python3 setup.py install

That's it. We are all set.

• Note: If you don't want to install MMKV, you can just build the native library:

cd POSIX/Python
mkdir build
cd build
cmake ..
make -j8

You can find out which Python is used to build MMKV from the logs of cmake. It's something like:

-- Found PythonInterp: /usr/bin/python3.6 (found version "3.6.9") .

And if you find it not correct (let's say you want to use Python 2.7 instead), you can surpass it by setting PYBIND11_PYTHON_VERSION for cmake:

cmake -DPYBIND11_PYTHON_VERSION=2.7 ..

Then you can copy the generated mmkv.so (the exact name is vary on different platforms, for example mmkv.cpython-36m-x86_64-linux-gnu.so) to your Python library directory PYTHONPATH. If you are using Python 3, you can just copy it to your code directory.

4. Test MMKV:

python3 demo.py
python3 unit_test.py

Tutorial

You can use MMKV as you go. All changes are saved immediately, no save, no sync calls are needed.

Configuration

• Setup MMKV on App startup, say in your main() function, add these code:

  import mmkv
  
  if __name__ == '__main__':
      mmkv.MMKV.initializeMMKV('/tmp/mmkv')

CRUD Operations

• MMKV has a global instance, that can be used directly:

  kv = mmkv.MMKV.defaultMMKV()
  
  kv.set(True, 'bool')
  print('bool = ', kv.getBool('bool'))
  
  kv.set(-1 * (2 ** 31), 'int32')
  print('int32 = ', kv.getInt('int32'))
  
  kv.set((2 ** 32) - 1, 'uint32')
  print('uint32 = ', kv.getUInt('uint32'))
  
  kv.set(2 ** 63, 'int64')
  print('int64 = ', kv.getLongInt('int64'))
  
  kv.set((2 ** 64) - 1, 'uint64')
  print('uint64 = ', kv.getLongUInt('uint64'))
  
  kv.set(3.1415926, 'float')
  print('float = ', kv.getFloat('float'))
  
  kv.set('Hello world, MMKV for Python!', 'string')
  print('string = ', kv.getString('string'))
  
  lst = range(0, 10)
  kv.set(bytes(lst), 'bytes')
  bt = kv.getBytes('bytes')
  print('raw bytes = ', bt, ', decoded bytes = ', list(bt))

  As you can see, MMKV is quite easy to use.

• Deleting & Querying:

  kv = mmkv.MMKV.defaultMMKV()
  print('keys before remove:', sorted(kv.keys()))
  
  kv.remove('bool')
  print('"bool" exist after remove: ', ('bool' in kv))
  
  kv.remove(['int32', 'float'])
  print('keys after remove:', sorted(kv.keys()))

• If different modules/logics need isolated storage, you can also create your own MMKV instance separately:

  kv = mmkv.MMKV('test_python')
  kv.set(True, 'bool')

• If multi-process accessing is needed, you can set MMKV_MULTI_PROCESS on MMKV initialization:

  kv = mmkv.MMKV('test_python', mmkv.MMKVMode.MultiProcess)
  kv.set(True, 'bool')

Supported Types

• Primitive Types:

  • Boolean, int, float

  Note: Due to the limitation of protobuf, int32, uint32, int64, uint64 has a different encoding & decoding protocol. You should call different getXXXInt() method according to the value's possible range. For each range of those integer types, checkout MMKV/POSIX/Python/unit_test.py.

• Classes:

  • str, bytes

Logs

• By default, MMKV doesn't print any logs. You can turn on logging in initialization:

  mmkv.MMKV.initializeMMKV('/tmp/mmkv', mmkv.MMKVLogLevel.Info)

• MMKV prints log to stdout, which is not convenient for the interpreter or diagnosing online issues. You can set up MMKV log redirecting on App startup. Implement a callback function with signature (logLevel: mmkv.MMKVLogLevel, file: str, line: int, function: str, message: str) -> None, register it as log handler. Remember to unregister it before exiting, otherwise, your script won't exit properly.

def logger(log_level, file, line, function, message):
    level = {
        mmkv.MMKVLogLevel.NoLog : 'N',
        mmkv.MMKVLogLevel.Debug : 'D',
        mmkv.MMKVLogLevel.Info : 'I',
        mmkv.MMKVLogLevel.Warning : 'W',
        mmkv.MMKVLogLevel.Error : 'E'
    }
    # use your logging tools instead of print()
    print('[{0}] <{1}:{2}:{3}> {4}'.format(level[log_level], file, line, function, message))


if __name__ == '__main__':
    # enable logging & redirect logging
    mmkv.MMKV.initializeMMKV('/tmp/mmkv', mmkv.MMKVLogLevel.Info, logger)
    
    # some logic
    ...

    # unregister before exit, otherwise the script won't exit properly
    mmkv.MMKV.unRegisterLogHandler()

    # or just call onExit() will do the job
    # mmkv.MMKV.onExit()

Backup & Restore

• You can use MMKV's backup & restore API to backup your data to somewhere else, and restore them later.

    root_dir = "/tmp/mmkv_backup"
    // backup one instance
    ret = mmkv.MMKV.backupOneToDirectory(mmap_id, root_dir)
    // backup all instances
    count = mmkv.MMKV.backupAllToDirectory(root_dir)
  
    // restore one instance
    ret = mmkv.MMKV.restoreOneFromDirectory(mmap_id, root_dir)
    // restore all instances
    count = mmkv.MMKV.restoreAllFromDirectory(root_dir)

Auto Expiration

• Starting from v1.3.0, you can upgrade MMKV to auto key expiration. Note that this is a breaking change. Once upgraded to auto key expiration, the file is not valid for any older version of MMKV (<= v1.2.16) to operate correctly.

• Global Expiration. The most simple way to do it is to turn on auto key expiration for all keys in the whole file.

  // expire in a day
  kv.enableAutoKeyExpire(24 * 60 * 60);

  Or, if you prefer, you can enable auto key expiration without setting a global expiration duration. In this case, each key is not expired by default.

  // enable auto key expiration without global duration
  kv.enableAutoKeyExpire(0);

• Individual Expiration. You can set a special expiration duration for a key, regardless with the file has a global duration or not. Note that you have to enable auto key expiration first.

  // enable auto key expiration with an hour duration
  kv.enableAutoKeyExpire(60 * 60);
  
  // set a key with the file's global expiration duration, aka 60 * 60
  kv.set("some value", "key_1");
  
  // set a special key that expires in two hours
  kv.set("some value", "key_2", 2 * 60 * 60);
  
  // set a special key that never expires
  kv.set("some value", "key_3", 0);

  Or, if you prefer, you can enable auto key expiration without setting a global expiration duration. In this case, each key is not expired by default.

  // enable auto key expiration without global duration
  kv.enableAutoKeyExpire(0);
  
  // set a key that never expires
  kv.set("some value", "key_1");
  
  // set a special key that expires in an hour
  kv.set("some value", "key_2", 60 * 60);

• The expire duration is counted in seconds. You can use any other duration you prefer. For example, expiration in a week is 7 * 24 * 60 * 60.