Interactive digital frame with head-tracking using Three.js & TensorFlow.js

Using TensorFlow.js and Three.js, this project is a prototype of an interactive digital art frame using head-tracking to create an effect of head-coupled perspective.

A famous use of this effect can be seen in Ghost Protocol.

I wanted to experiment with making interactive art pieces so this project is a PWA that can be run fullscreen on an iPad.

⚠️ This is a prototype made as a side project so the code is not production-ready. Its performance is not optimised so the first load can take a while. I developed and tested it on a Macbook Pro and iPad Pro so if you try it on another device and it doesn't work or doesn't look super responsive, I don't intend to fix it. With that in mind, here's the live demo that works with both mouse movements and head-tracking. ⚠️

Demo

If you'd like to read more about it, check out the blog post!

Local development

• Clone this repo
• Run npm install to install the dependencies
• Run npm start to start the local server
• Your browser should automatically open a new tab to http://localhost:3000