An iOS application using a Machine Learning Algorithm for classifying music files on Spotify into different human moods in order to recommend new music or any associated to-do list tasks to the user.
- Getting Started
- Introduction
- What Adagio actually means
- Background
- Importance of this project
- Objectives and Scope
- Built Using
- Components of Adagio
- Behind the Curtains of a Spotify Music File
- Thayer's Model of Mood
- Mood Based Task Recommendation
- Authorization Code Flow for Spotify Accounts Login
- Future Scope
- Deployment and Testing
- Conclusion
- References
In order to run this app, you will need to have Xcode installed on your Mac and you can test run the app in your iOS device via Xcode. You will also be required to have a Spotify User Account, Developer Account, and your own Spotify API Key.
NOTE: Please feel free to reach out to ivedantshrivastava@gmail.com with any doubts or suggestions regarding this repository and the Adagio project.
From the beginning of our time, we humans have always been engaged in multifarious activities and have been striving to be more productive and composed so that we can drive better results and outcomes. With the expanding domains of technology and smartphone applications, we propose a revolutionary iOS Application that would analyse the user’s mood in real time and suggest various pieces of music and to-do list tasks accordingly. This would eventually help the user enhance his/her productivity while tending to make him/her more composed and motivated throughout. Alongside this, this app tries to make use of Music Information Retrieval and Mood Analysis Algorithm to drive better results in both music and task recommendation.
With advancements in computational intelligence and machine learning, a very innovative science called Music Information Retrieval (MIR) has emerged. Such a creative intersectional field of Liberal Arts and Technology where information can be retrieved through music has enthralled us to develop solutions for today’s challenges and tomorrow’s possibilities. Using the methodologies involved in MIR, Adagio makes use of inputs such as the user’s musical predilections, tastes, likings, listening history, profile, etc. to be able to map it on our Mood Recognition and Classification Model which is the major component of our Smart Mood Analysis Algorithm. This enables Adagio to not only predict the user’s mood in real time but also recommend him/her other pieces of music and tasks that the user has added in Adagio’s to-do list. Alongside this, Adagio also acts as a pocket motivator for its users since it also has the option of delivering motivational content to its users. All this would eventually help the user change his/her mood as per requirement and thus bring a positive change in their lives and finally enhance their overall productivity.
The planned outcomes for Adagio are that it should efficiently be able to accurately predict the user’s mood in real time and recommend music and tasks accordingly. Since there is no existing iOS application that involves the use of MIR and predicts accurate mood or acts as a highly accurate recommender system, we have devoted ourselves to the challenge of building this application and thus add something truly wonderful and significant to the flow. We hope that Adagio inspires more works further in the field of Music Information Retrieval.
Adagio (Italian - “ad agio” meaning “at ease”) is a piece of music that is to be performed slowly by the musicians in order to produce a calm and soothing sound. Just like musical Adagio, our application tries to produce the same effect and make it a whole new and intensely satisfying experience for the user. We classify Adagio as a Productivity & Music iOS Application that helps the user manage various to-do list tasks in an effective manner. Adagio specifically aims at making you experience your “work musically”.
There are some applications on the App Store that tend to give suggestions as per a user’s music listening history. Music applications tend to enhance a user’s interest and focus on the likes and dislikes of a user. There is no application in the application market right now that tries to analyse and correct the user’s mood and then suggest to-do list tasks as per the user’s real time mood. This is what motivated us to make Adagio - an application that does exactly that. We aim to build Adagio in such a way that it serves as a natural extension to the self. It is what should help the user to get his/her to-do list tasks in order while maintaining a healthy and productive mood. We are positive that Adagio is going to be the revolution in the pocket and it will help all people become better individuals and get their work efficiency increased alongside enhanced productivity.
Our proposed application has an aspiration to promote and evolve a favorable feeling of satisfaction in various sectors such as improve test results, improve learning and analyzing habits and hence reducing errors by increasing the brain’s working credibility and concentration level. Thus by analyzing a user’s behavior and traits, we are going to see an enormous boost in an individual’s productivity. A user can use this application when he/she is feeling low or demotivated and can morph their moods towards a calmer/energetic and happier state. We have focused very finely upon making the app highly user-friendly and intensely secure by implementing some state-of-the-art techniques. Here, we are trying to please each and every mind to experience a lifestyle with a complete different and positive approach with Adagio.
The main objective of the application is to serve as a music and productivity application that would help the user listen to music anytime, anywhere and help him/her in prioritising and completing their tasks added in the Adagio to-do list. The secondary objective of the application is to make use of state-of-the-art authentication and login systems for maximum level of security by implementing login via biometric systems such as Face ID or Touch ID that are available on iPhone 5s or later models. We believe that ensuring the privacy of the user is one very important priority and so, we have implemented some really strong authentication and data storage methods within Adagio for ensuring the same.
Adagio focuses on helping a user complete a few day-to-day tasks along with listening to music of choice. This basically enlightens the mood and also helps to switch from a negative to a really positive vibe. This project has a good scope further. It can also be implemented with facial gestures and thus a user’s mood can be analyzed quickly. Music has always been a prime way in deciding and enhancing mood of an individual.
- Xcode IDE
- Swift Programming Language
- Google Firebase
- Realm Database
- Spotify SDK and API
- Heroku Server
Adagio has three main components:
- Smart Login System
- Enhanced To-Do List
- Advanced Music Player
The Login System of Adagio works with Apple's Face ID and Touch ID features. This enables the users for quick log in into the app.
We register our users and maintain their login credentials database using Google Firebase.
The To-do List in Hourglass is more user-friendly and functional than majority of the other apps of similar category in the App Store. It has 5 major differences than its competition which are as follows:
- Categorize Lists by Color - The user can add multiple lists of tasks inside the To-do list which are then color coded into various categories. For example, the user can assign red color to tasks such as Homework that require immediate action and yellow color to tasks that can be flexibly scheduled.
- Sort Lists by Time - The user can sort these lists as per their specific time priorities. For example, if the list of Algorithms class homework needs more attention than the Structured Programming homework list, the user can simply drag these lists from their placeholders and set them as per their priority order.
- Priority Coloring - Within all the lists, the tasks that have higher priority get a darker color shade within the list so to signify the user of the level of priority and immediate action that they require.
- Search Bar - The Search Bar in the To-do List can be used to quickly find any list or any task saved within the app.
- Reminders - Users can set reminders for each and every task within the to-do list so that they can get the app notifications accordingly.
We built the Local Persistant Storage using Realm Database so that the users can access this feature at all times, even when they are offline.
The Advanced Music Player plays music via Spotify API so the user must be logged into the Spotify App on their devices in order to use this feature. This player tries to observe patterns between songs that the user is listening to and then predicts the mood of the song in order to determine what kind of songs can be added to the Mood playlists of the user. It does so using an algorithm based on Thayer's model of mood (discussed below). After the mood is specified, another algorithm checks if the current mood is associated with any of the to-do list tasks. If yes, then Adagio sends a notification to the user recommending the task that may be completed during that period. The recommended songs as per the mood are added to either of the 4 Mood Zone playlists on the app automatically.
There are certain factors involved that define a music file metadata. Here is an example.
We use some of these factors into our algorithm to specify which mood a song must be associated with.
Thayer came up with a two-dimensional mood model in the late 90’s which was later applied by Microsoft Research Asia. The two dimensions considered in the model are: Stress dimension (happy/anxious) and Energy dimension (calm/energetic). This model effectively helps us divide the music mood into four clusters, each to fit into one of the quadrants as shown in the figure below. In this model, the four clusters refer to the following:
- Contentment: Happy and Calm Music
- Depression: Calm and Anxious Music
- Exuberance: Happy and Energetic Music
- Anxious/Frantic: Anxious and Energetic Music [4].
The Music Mood Response will be analysed by the application using Thayer’s model of mood and will thus work effectively and more accurately as per the user's requirements at that specific time. This model is the music mood taxonomy that we use for better working of the application.
Based on this model and our algorithms, we could observe a somewhat generalized relationship between music genres and mood classifiers. Here is a visualization of the same.
In order to recommend mood related tasks to the user, we came up with a model that predefined the type of mood associated with a particular set of tasks. This limits the current version of Adagio from understanding the associated mood with all types of tasks. We may be able to overcome this in the next versions using Natural Language Processing (NLP) solutions.
Here is a visualization of some of the tasks with the mood classifiers in the predefined Mood Based Task Recommendation model.
Adagio sends a request to the Spotify Accounts service. The reason your application sends this request may vary:
- A step in the initialization of your application.
- A response to a user action, like a button click.
The GET request is sent to the endpoint of the Accounts service: Fig. 4.3.2: Obtaining Authorization. GET https://accounts.spotify.com/authorize
Mood Analysis is based on what kind of music segment the user prefers to listen to at a specific period of time. Also, with a change in the music segment, it is possible that the user’s mood may change as well. That is why, Real Time Mood Analysis is done on a prediction basis in which the user picks up a preferred music segment and thus it helps the application to derive the user's current mood by analysis the genre and type of the music segment. Adagio will drastically change the face of how people accomplish their tasks while still being calm and relaxed throughout that period. This would promote a better standard of living. It would also increase the potential of an individual. It benefits all kinds of users by helping them and connecting them to themselves on a whole new level.
Being an interactive multimedia application, it has great future scopes. Today, music is considered to be a reliable alternative for treatment of impaired body movements viz. Parkinson’s or Alzheimer’s disease. Here, through the application, the imbalanced rotary functions of human body can be given a thrust and a rhythmic pattern to collaborate and carry out chores easily. This in turn will encourage the user to develop a strong physical, emotional, mental, social, aesthetic and spiritual caliber. This opens a door of opportunity to Adagio to explore and extend its reach in medical research and sciences. Further work in musicology represent the possibilities of drone music which would be able to replace the consumption of recreational drugs while creating a similar temporary effect of tranquillisation and brain boosting. This can help rehabilitate a lot of recreational narcotics addicts. There are chances that that music as a result, will radically fetch a new type of language, that is much easier to understand, commute and relate hence bringing together a revolutionary change in the history of mankind forever. Another emerging trend uses the visual details of the skin, as captured in standard digital or scanned images. This technique, called Skin Texture Analysis, turns the unique lines, patterns, and spots apparent in a person’s skin into a mathematical space. Tests have shown that with the addition of skin texture analysis, performance in recognising faces can increase 20 to 25 percent.
Adagio is a preliminary attempt at mood detection and analysis from music meta-data and thus finally helping the user with managing his/her resources well and be more efficient. We hope it can inspire more research works on music analysis.
The first version of Adagio’s Enhanced To-Do List and Smart Login System has been deployed and tested over various iPhone simulators which involve simulators for iPhone 6S, iPhone 7, iPhone 7 Plus, iPhone 8, iPhone 8 Plus, iPhone X, iPhone XR, iPhone XS, and iPhone XS Max. All these simulators ran iOS 10.2 or later. Adagio’s complete first version has been tested on a physical iPhone XS Max running iOS 12.2 which is currently the latest iOS public release. Adagio has been tested exhaustively within the test cases to validate and verify the prototype’s functioning (unit/performance testing) and perform integration testing and system testing. After the successful testing and removal of all the bugs and verifying Adagio’s actual and expected output, Adagio’s first version is ready for deployment as a beta.
Music has been a very significant entity in the sustainment of the human civilisation over the years. Not only is it an art but it is also backed up by promising scientific logic. It brings people belonging from various walks of life and backgrounds closer and makes the world a far better place to live in. With Adagio, we are simply trying to give the everyman a tool that he/she can use in order to understand themselves better as a person and help them become more work efficient and productive.
Adagio’s commitment to putting the user at the centre of the experience and the product is what helps it increase the value of the customer. With its Smart Login System, the user can simply log into Adagio using his/her biometrics. This is Adagio being a step ahead than majority of the applications available which make use of login systems that still require a user to remember and type his/her login credentials every time they want to gain access to their own account data. The Enhanced To-Do List strives to provide a highly user-friendly interface that adds up to the simplicity and elegance of the design of the overall Adagio app. It helps the user to prioritise his/her tasks on basis of shades of colours. The Advanced Music Player not only plays music but also generates playlists as per the appropriate mood of the user based on his/her energy v/s stress levels. All these wonderful features combine together to serve as one true revolution in the pocket. Adagio intends to serve as a natural extension of the self by helping its users gain a better standard of living and approach to various tasks in life.
We strongly believe that Adagio will drastically change the face of how people accomplish their tasks while still being calm and relaxed throughout. This will basically help the users to work efficiently and with complete concentration. If the user works in a good mood, it eventually leads to working wholeheartedly and with better concentration levels. This is what our project specifically focuses upon. It is an attempt to analyze a user’s mood with the help of music meta-data and thus using the information for further music playlist generation and task recommendation. Thus, it helps all kinds of users by helping them connect with themselves on a completely different level. The proposed approach is a preliminary attempt at mood detection, analysis, and correction from music segments alongside making an efficient and accurate music and task recommender system. We hope that our work inspires more research works in Music Information Retrieval (MIR).
- Downie, J.S.: The Scientific Evaluation of Music Information Retrieval Systems: Foundations and Future. Computer Music Journal 28:2 (2004) 12–23
- Zhongzhe Xiao, Emmanuel Dellandrea, Weibei Dou and Liming Chen: What Is The Best Segment Duration For Music Mood Analysis?. 2008 International Workshop on Content-Based Multimedia Indexing (2008) 18-6
- Lu, L., Liu, D., Zhang, H.J., Automatic Mood Detection and Tracking of Music Audio Signals, Special Issue on Statistical and Perceptual Audio Processing, IEEE Trans. on Audio, Speech and Language Processing, Vol. 14, No. 1, pp. 5-18, 2006.
- Thayer, R. E., The biopsychology of mood and arousal, Oxford University Press, 1989
- Jean-Julien Aucouturier and Francois Pachet. Representing musical genre: A state of the art. Journal of New Music Re- search, 32(1):83–93, 2003.
- Geoffroy Peeters. Rhythm classification using spectral rhythm patterns. In Proc. of the International Society for Music Information Retrieval Conference, pages 644–647, 2005.
- George Tzanetakis and Perry Cook. Musical genre classification of audio signals. IEEE Trans. on Audio, Speech and Language Processing, 10(5):293–302, 2002.
- S Jothilakshmi and N Kathiresan. Automatic music genre classification for indian music. In Proc. Int. Conf. Software Computer App, 2012.
- Philippe Hamel, Sean Wood, and Douglas Eck. Automatic identification of instrument classes in polyphonic and poly- instrument audio. In Proc. of the International Society for Music Information Retrieval Conference, pages 399–404, 2009.
- Franco Fabbri.: A theory of musical genres:Two applications. Popular music perspectives, 1:52–81, 1982.
Vedant Shrivastava (GitHub: https://github.com/ivedants ; Email: ivedantshrivastava@gmail.com)