Color Palettes in Movies

by Austin Poor

Using clustering algorithms to create color palettes from movie stills.

Info

The notebook color-palettes.ipynb goes through using both K-Means and Agglomerative clustering to create color palettes from images.

The sample images come from the film Only God Forgives (2013) and the stills came from the site FILMGRAB.

I chose to use KMeans and Agglomerative clustering for their speed and in order to get a definite color palette size after running the algorithm.

In addition to clustering the images' RGB (red-green-blue) values, I applied the same algorithms on the images' HSV (hue-saturation-value) values.

Running the Notebook

You can run the notebook using binder, here.

Or you can download the notebook and run it (requirements.txt).

Note that the algorithms have only been tested on images with 3 color-channels.

Issues

The clustering algorithms tend to pick more commonly occuring colors even if they're similar.

For example, if you look at the above image, the colors red (from the standing boxer and boxing rope) or blue (from the KO-ed boxer or the mat) seem to stand out, even though they don't take up a lot of the frame.

But the palette generated using K-Means clustering didn't include either and instead picked colors that were more similar and more abundant.

When creating palettes using HSV color representations, both clustering algorithms included a blue but not a red in the palette.

Some possible solutions would be increasing the palette size, adjusting other hyperparameters for the clustering algorithms, or using different clustering algorithms altogether.

Update -- Filtering Pixels

I've also added a second notebook, color-palettes-filtered.ipynb, where I try to address the issue of the clustering algorithm choosing colors too close to pure black or pure white by filtering the image before clustering.

You can also open that notebook in binder or nbviewer.

Flask App

In addition to the notebook, I created simple flask-app-API for creating color palettes.

The flask app can be found here: flask-app/app.py

and the color-palette code is in a separate file here: flask-app/make_palette.py

With the flask app running, you can make API calls with the following:

import requests

FLASK_URL = "http://localhost:5000"
IMG_URL = "https://film-grab.com/wp-content/uploads/photo-gallery/Colour_Out_of_Space_046.jpg?bwg=1598860186"

response = requests.get(
    FLASK_URL,
    parameters={"url":IMG_URL}
)
print("Color palette:", response.json())

The API doesn't include options for picking the clustering algorithm or setting the size of the palette, but those changes should be pretty easy to make.