Feature: Integrate Apple Music API

[FoxxMD]

Apple Music implements a recently played api.

• Register for a MusicKit identifier to use for testing
• Find an existing library for authenticating with the service or DIY authentication
• Test polling the api and parsing responses

[FoxxMD]

Won't implement. Musickit requires enrollment in the Developer program which costs $99 a year. Obnoxious 🙄

[hmhrex]

Hear me out, what if we set up a service that let's others use this API. So they would only have to login via the multi-scrobbler app, but it would use an API from an active dev account. I already pay the $99/year for a sub, and think it could benefit others.

[FoxxMD]

I'd be willing to do that. Biggest roadblock with this initially was that I couldn't even begin to develop & test this without the $99 upfront cost -- and I don't use apple music.

If you'd be willing to lend me your credentials/developer token/whatever it is I need to develop this I would do it. Hit me up at multi@foxxmd.dev or on discord at FoxxMD#5986 if you're interested.

[hmhrex]

I added you on Discord. Let's communicate there.

[FoxxMD]

Resources and requirements I've gathered so far...

Requirements

Required: Developer Token

To generate need:

• MusicKit private key
• key identifier (kid) from developer account
• team id (tid) from developer account

Using these credentials a JWT is created with kid/tid and signed with private key (.p8) file. This token is then used in the header as Authorization: Bearer [developer token] to all https://api.music.apple.com/v1/* requests.

apple-music-token-node can be used as a convenience library to help create the token (takes care of all the JWT business, reading private key file, etc.)

node-musickit-api can also do this and has personalization routes. Would be best to use just this library.

Required: Music User Token

To access personalized music content we must retrieve Music User Token (under Create a Music User Token heading)

The only way I've found to do this so far is to use apple's MusicKit JS sdk. Unfortunately use of this library seems to be limited to browser-only. Which makes sense for user authorization but not helpful for post-auth server user.

Some resources for use of musickit js:

• https://areknawo.com/apple-music-javascript-integration-guide/
• https://github.com/KoleMyers/apple-musickit-example

Getting Recently Played Tracks

API

There is an simple API endpoint for recently played tracks

NOTE: This is different from recently played resources which only returns grouped album/playlist/stations (reference)

Header requires Authorization plus Music-User-Token with user token.

MusicKit JS

MusicKit also contains a method for recentPlayed which I think can be accessed like this:

        document.addEventListener('musickitloaded', function() {
            // MusicKit global is now defined
            MusicKit.configure({
                developerToken: 'DEVELOPER-TOKEN',
                app: {
                    name: 'My Cool Web App',
                    build: '1978.4.1'
                }
            });
            let mu = MusicKit.getInstance();
            mu.authorize().then(function(token) {
                // token is music user token we should store for later
                mu.api.recentPlayed()
            });
        });

Not ideal since it would require using pages served up by express in MS to get user token to begin with (see example in apple-musickit-example)

Community

• https://github.com/yujinakayama/apple-music-node (does not include personalized methods)
• https://github.com/wsmd/musickit-typescript (typings)

node-musickit-api contains methods for personalized results but not recent/played -- might be worth forking to add instead of re-inventing the wheel (requested feature here) EDIT: should have the route available soon™️

Implementation

At a low level gathering all the necessary info to get to the point where we can get recently played tracks would require:

1. Parsing credential values and .p8 private key (read file)
2. Validating credentials are present and in the correct format
3. Serving HTML (express) in order for a user to authenticate with apple to get user music token
4. Storing user music tokens for persistence and associating them with scrobble source (apple) and clients (only scrobble User X to client Y)
5. Access control to validate user completing apple auth flow is a user we want to access poll tracks for

Given MS already has 1-3 built and 4-5 will require tight integration with MS to ensure proper access control I don't think it makes sense to create a separate app as an api proxy. Instead this should be integrated in MS and an optional endpoint can be created to access the api flow if an MS operator so desires.

[Exerra]

node-musickit-api contains methods for personalized results but not recent/played

Just published v2.1.0 which includes various personalized functions, but mainly, the ability to fetch recently played resources

My package also handles JWT, so no need to stress about that, however it (yet) cannot generate user tokens as the Apple documentation on it is... well... for Swift (not really practical for NodeJS is it), so you will need to use MusicKit.js for the token.

Documentation can be found on musickit.js.org

[FoxxMD]

Awesome thank you @Exerra ! One question..since node-musickit-api handles JWT can I get direct access to the resolved developerToken by just initializing node-musickit-api? That would help with the userToken step when using musickitjs since it requires a developer token to initialize.

[Exerra]

Hmm, didn't even think of this use case

No, you can't. However I can make it so it gets exported in v2.1.1. Would you like me to do that?

[FoxxMD]

Yes please that would be hugely helpful. Or if possible export the method that creates the JWT so I can just do

createDeveloperToken({
key: "",
teamId: "",
keyId: "",
})

and get the token back without having to initialize the whole thing

[Exerra]

Added it into the code, will validate if everything truly works and push v2.1.1 tommorow

Also will send instructions on how to use the function :)

[FoxxMD]

thank you!

[Exerra]

v2.1.1 is published on NPM

It fixed a major bug and also added the createJWT() function

It needs the same parameters as the non-personalized constructor (key, teamid, keyId)
Usage:

// convert it to import if needed
const { createJWT } = require("node-musickit-api/modules/createJWT")

let token = createJWT({
    key: "shjdfgdshj",
    teamId: "32423",
    keyId: "32g443"
})

[FoxxMD]

perfect! thank you!

[FoxxMD]

I'm still working on this (haven't had much free time) but in the meantime there is a macOS application to scrobble from itunes to maloja, if this helps anyone.

[kelchm]

I've started looking into the /v1/me/recent/played/tracks API a bit, and it appears to have some pretty frustrating limitations:

• There does not appear to be any kind of timestamp that is associated with the returned items.
• Despite there being both songs and library-songs types specified in the docs, I only seem to be able to retrieve plays for songs which are actually added to my library. EDIT: It seems like this issue is actually specific to 'Autoplay' and radio stations.
• Tracks seem to be added immediately when playback starts rather than after completion

[FoxxMD]

@kelchm thanks for looking into this!

No timestamp and added immediately after playback starts

This isn't a huge issue. The subsonic api also has this issue so I implemented a simple state machine-esque source that keeps track of when a track is first seen through polling and doesn't scrobble it until its been seen for 30 seconds, among other things.

Only retrieves plays for songs in library EDIT

I'm not quite sure what you're saying here. So the api won't return results that are played from radio? Or it only returns a specific subset of your library? This is definitely a bigger issue.

---

A general PSA for development requirements on apple music outside of the stuff listed above:

• It's not enough to have an apple ID (account). You must actually subscribe to apple music to login and generate a token
• Depending on your security settings...
  • You may be required to login on a known device already signed with an apple id (macbook, etc..)
  • If you have 2FA enabled (is this the default?) then you must ALSO have an iphone with you to confirm 2FA login

This is all just login using the generated URL from the apple music JS sdk. It's a huge hassle.

I will be pushing a semi-working branch for apple music here soon if anyone wants to give it a try.

The above reasons are why I haven't had much progress as of late. I am not an apple music subscriber so, along with the graciously provided developer credentials from @hmhrex I also need to borrow a friends laptop and phone to do any kind of development. 😩

[kelchm]

Only retrieves plays for songs in library EDIT

I'm not quite sure what you're saying here. So the api won't return results that are played from radio? Or it only returns a specific subset of your library? This is definitely a bigger issue.

Sorry, it seems that my initial understanding was partially incorrect. Here is my current understanding of the limitations:

It appears that the history API does not report any tracks that are played via a radio station or "autoplay" (toggled at the top of the queue list).

Any tracks that a user directly adds to the queue themselves will be reported.

Example: if a user plays a song from an album, that song and all the following songs from that album are added to the queue and will be reported by the history api after they are played, even if they are not added to the user's library. If autoplay is turned on, Apple Music will automatically play similar music after the last song of the album completes and these tracks will not be reported via the history API.

[mariolopjr]

@FoxxMD you mind pushing up the in-progress branch (I'd like to take a stab at getting this implemented)? I subscribe to Apple Music and it would be nice to scrobble from it. A roundabout way I can achieve this is using multi-scrobbler to scrobble lastfm to maloja and use third-party apps on macOS and iOS to scrobble to lastfm. It'd be vastly easier to have multi-scrobbler use the music api.

From what I understand, in order to work on this, I'll not only need an Apple Music sub, but I would need to upgrade my free dev account to a paid one right? ( why Apple??)

[FoxxMD]

@mariolopjr the appleIntegration branch is now on par with develop (current)

I would need to upgrade my free dev account to a paid one right?

I don't know...definitely should give it a try with your free one first though!

The quick start guide for node-musickit-api is a good starting point for what data you'll need and I have more details about those in requirements post above

Using all the info you've gathered you'll create a file for MS in the config folder named apple.json:

[
  {
    "name": "aap",
    "clients": ["myScrobblerClientName"],
    "data": {
      "keyId": "34B45GH",
      "teamId": "123GF45BG",
      "key": "/absolute/path/to/applePrivateKey.p8"
    }
  }
]

After starting MS click on the authenticate link for the Apple source

You'll get a mostly blank screen with one button that triggers auth init popup for the official musickit client

If the login flow is successfully (you'll need to have your second device and 2FA stuff handy for the login) you will receive a user token in the callback at which point you should be able to initialize node-musickit-api and do whatever you want with it API-wise.

Back when I posted in December I was able to get a token but then apple always returned a 500 for every endpoint i used the generated token on. I was not able to determine if there was something wrong with the auth flow I implemented or if it was something else...this is, unfortunately, as far as I could get.

[mariolopjr]

@FoxxMD awesome thanks!! Will take it for a spin and see what happens :D

[FoxxMD]

I no longer have access to an apple account that is paying for apple music so I cannot work on this. It also really does not seem feasible that anyone would want to implement it as it requires a developer account ($$$) to even use.

I have updated the appleIntegration branch to be on par with the big rewrite recently done for MS but it is functionally in the same place as it was in my last comment. If someone wants to dig into this I will re-open this issue but for now I'm closing it.

Workarounds

It's still possible to get scrobbles into MS via apple music! Or even straight into Maloja. However all of these solutions are on a per-device basis:

• OngakuKiroku is an app for macOS that can scrobble directly to Maloja
• There are free macOS app store offerings like Scrobbles for Last.fm that can scrobble from Apple Music to Last.fm => forward to Maloja using multi-scrobbler Last.fm source
• For iphone/ipad there are app store offerings (non-free) like QuietScrob that can scrobble to Last.fm => forward to Maloja using multi-scrobbler Last.fm source
• This script bypasses the need for a macOS app store app by directly monitoring itunes and scrobbling to lastfm => forward to Maloja using multi-scrobbler Last.fm source
  • its just python so if there was interest it could be easily modified to make a POST request to a multi-scrobbler endpoint
• This script also bypasses the API and can scrobble to listenbrainz which can be used as a source

[samip5]

Hi, I would be interested in getting this implemented.

I currently have a developer account sub (will expire in 12 months as I got it in December of 2023), and Apple Music, so it should be doable. How out-of-sync is the appleintegration branch?

[FoxxMD]

@samip5 you can use https://github.com/FoxxMD/multi-scrobbler/tree/appleDev branch which is now up to date with the develop branch. Ive tested that this works up to clicking "login to apple music" but can't do anymore since i don't have a dev account or an apple music subscription.

---

The instructions from this comment #9 (comment) are still mostly accurate.

At the "login to apple music" step from the above comment instead use the URL http://localhost:9078/api/apple. You will also need to manually save the token generated by this step:

• click "login to apple music" and complete the login flow
• open the developer tools for your browser for the "login to apple music" page
  • the developer tools console should show something like: Authorized, music-user-token: MY_TOKEN_IS_HERE
• create a new file in the same directory as your apple.json config file named currentCreds-apple-NAME.json where NAME is the name you gave to the apple source
  • Using the example from the above comment it would be currentCreds-apple-aap.json
  • the file contents should be like this:

{
  "userToken": "MY_TOKEN_IS_HERE"
}

substitute the token you got from the login flow developer console window.

You can now uncomment these lines in src/backend/index.ts and after restarting MS should(?) get recent plays using node-musickit-api using the saved user token file you created.

---

If you can get that to actually work (recently played), the next steps would be:

• AppleSource
  • implement formatPlayObj
  • implement getRecentlyPlayed using node-musickit-api like in the sample from backend/index.ts
• Refactor the apple music login flow to use the CRA front end client (in src/client) instead of express ejs (example project for using apple musickit js in react?) OR add a onClick callback to complete auth flow to MS auth endpoint like /api/callback/apple?token=MY_TOKEN_IS_HERE

[samip5]

@samip5 you can use https://github.com/FoxxMD/multi-scrobbler/tree/appleDev branch which is now up to date with the develop branch. Ive tested that this works up to clicking "login to apple music" but can't do anymore since i don't have a dev account or an apple music subscription.

In the login with Apple Music step, I get greeted with a "problem connecting" from the authorise.music.apple.com pop-up after logging in, any ideas as to what's the issue there?  Problem was wrong keyID. :))))

Next issue is that it doesn't seem to actually use the currentCreds-apple-aaap.json file I created and instead yells about the user interaction is required for auth. I was able to get the token from Apple for it.

[samip5]

Currently it seems to be an issue regarding source auth. How is it supposed to verify the creds?

[FoxxMD]

The path to the credentials file is set here and then if it exists is used to build the api client with all the necessary credentials here.

You'll need to debug startup process and see why it isn't using your file (is the file name correct?).

In Webstorm I debug using the npm script configuration like this

and set a breakpoint on the lines I mentioned above.

[samip5]

The path to the credentials file is set here and then if it exists is used to build the api client with all the necessary credentials here.

You'll need to debug startup process and see why it isn't using your file (is the file name correct?).

In Webstorm I debug using the npm script configuration like this

and set a breakpoint on the lines I mentioned above.

It is trying to use it but something about the authGate is returning undefined but doesn't return the unable to parse error, it just instead doesn't use it instead..

[samip5]

@FoxxMD The problem seems to be relevant at backend/sources/ScrobbleSources, line 529. The newSources.testAuth, has nothing, so it cannot verify the creds and thus fallback to as if it had none.

There doesn't appear to be a way to verify creds:

• Way to check if your API credentials works, aka if logged-in successfully Exerra/node-musickit-api#10

[FoxxMD]

For now you could just check if the api client exists, in AppleSource class add

    protected doAuthentication = async (): Promise<boolean> => {
        return this.apiClient !== undefined;
    }

[samip5]

I did actually just check for the cred file instead. Current progress: samip5@3845cf8

GetRecentPlayes somewhat works:

[nodemon-server] 2024-01-04T22:54:34+02:00 debug   : [Sources] [Apple - aap] Refreshing recently played
[nodemon-server] 2024-01-04T22:54:39+02:00 debug   : [Sources] [Apple - aap] RecentlyPlayedData: [object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
[nodemon-server] 2024-01-04T22:54:41+02:00 warn    : [Sources] [Apple - aap] More than one data/state for Player NoDevice-SingleUser found in incoming data, will only use first found.
[nodemon-server] 2024-01-04T22:54:41+02:00 debug   : [Sources] [Apple - aap] [Player NoDevice-SingleUser] New Play: (1250095906) SANA - Lelupoika
[nodemon-server] 2024-01-04T22:54:41+02:00 debug   : [Sources] [Apple - aap] [Player NoDevice-SingleUser] Started new Player listen range.

How should it be handled that the response from Apple doesn't include any "playedAt" things?

[Exerra]

I would like to quickly jump in and reiterate something that I mentioned in Exerra/node-musickit-api#9 and that is that I no longer have have a paid Apple Dev account, so I cannot continue work on node-musickit-api (sorry!).

The code for the package to be honest isn't that good, mostly wrote it in a rush and it doesn't even have TS types (which probably has caused some pain) so I very much want to rewrite it, however I don't really wish to spend 99 USD just for one small package.

TLDR: No support for node-musickit-api anymore, so if it works, it works. If it doesn't, it doesn't and nothing will change in the foreseeable future.

[samip5]

which probably has caused some pain

More than pain because typing is missing entirely as a result. :(

[FoxxMD]

@samip5 nice work!

I assume the song data looks like the response from this api documentation?

Since there is no "played at" multi-scrobbler will have to guess (assume) that when a new song is found in the response data it was played shortly before it appeared.

You can see it doing basically the same thing in YTMusicSource -- it keeps a list of data found after it starts polling (after initially getting the data) and then checks to see when/if new songs are added. When it finds new songs it sets the playDate for the PlayObject as that time and scrobbles it directly by returning newPlays (do not use this.processRecentPlays)

@Exerra thank you again for your work! I'm aware your library is not being maintained but that's fine because IMO the number of people who will be using this integration will be 1) tiny given the $99/yr barrier and 2) requires so much setup for just one integration. This is and always will be a "beware of dragons" approach.

EDIT: Ideally this integration gets close enough to useable that it entices samip, or someone else stumbling upon it, to improve both your library and this integration. When/if this gets merged into the main branch the documentation will heavily feature "this is not supported and you should not do this if you don't know what you are doing" language.

[samip5]

@samip5 nice work!

I assume the song data looks like the response from this api documentation?

Correct, and I think genres might be something that could be worthy to also track but not sure if it's implemented elsewhere either.

[FoxxMD]

I don't believe any of the scrobblers are designed to track genre but that would be nice.

[FoxxMD]

Hi @cdransf I saw your blog post and was hoping to tempt you into working on this integration. 👀

[cdransf]

Hi @cdransf I saw your blog post and was hoping to tempt you into working on this integration. 👀

@FoxxMD 👋🏻 I'm not sure how much time I could devote to it but I'm not opposed to the idea — the biggest stumbling blocks I hit in that post remain though. Apple adding a timestamp to their response would make this a whole lot simpler (though it sounds like you may have that sorted in MemorySource?).

But yeah, I suppose not strictly opposed to getting brought up to speed and contributing.