This is a part of the final project for DES5XX course Digital Audio.
Data Sonification is a a popular paradigm that lets you ’hear’ data. It is the use of non-speech audio to develop a visualisation or perception of the data. This data can be of various kinds - numbers, text, signals or even images. In this project, we provide a novel idea of sonifying images with the question - Can we really hear what we see? This project paves way for several ideas in the future. We can use it to sonify Comic strips, sceneries or even a collection of images like a collage or providing background scores for videos. Since this is our first experiment with Data Sonification, we introduce some key concepts, the tools used and the provide a detailed explanation to why we used them and finally, a taste of the experiments that we have conducted.
As described in "Application of Image Sonification Methods to Music" (Yeo et. Al, 2005), there are two major techniques for organizing time-independent data for auditory purposes, depending on whether they are pre-scheduled and fixed, or arbitrary and freely adjustable.
- Scanning - This refers to when the data is scheduled to be sonified in a fixed, non-modifiable order. The speed of scanning is usually fixed, and not allowed to be changed arbitrarily during sonification.
- Probing - If the speed and the location of sonification pointer can be varied by an operator arbitrarily during sonification, we classify this as probing. One may arbitrarily probe anywhere and anytime within an image. It requires some pre-assessment of the qualities or features of the image.
Till now, we have explored some techniques which are combinations of scanning and probing, and successfully tried to decipher a 'mood' of an image via music.
Please refer to the report for more information.