play:Sub Music Streamer - test connection failure

[tonydub]

The play:Sub Music Streamer (https://apps.apple.com/fr/app/play-sub-music-streamer/id955329386) failed to connect to Gonic server (eg. https://gonic.example.com) with correct login credentials.

The docker logs command shows:

gonic_1                  | 2019/11/25 13:04:40 response  400  for `/rest/ping.view`

Thank you for your work!

[sentriz]

hi

in the logs for the ping.view, was there a u= and p=? or a t= and s= arguments? and if "u an p", did p= start with "enc:"

thanks!

( sorry I can't test better without an iPhone)

[tonydub]

Hi

The ping.view was called without query parameters by play:Sub.

Which is normal according to the API documentation(http://www.subsonic.org/pages/api.jsp#ping) this endpoint does not take parameters.

Authentication should not be implemented for this endpoint.

What do you think about that?

[sentriz]

hmm that is strange. for both DSub (Android) and jamstash (web), ping.view is called with authentication parameters. it is how they know the username/password provided is correct when adding a new server.

so it makes sense that gonic is giving you a 400 for ping.view since play:Sub isn't giving the parameters.

I can add a fix to gonic, something like
ping.view without params -> 200
ping.view with correct auth params -> 200
ping.view with incorrect auth params -> 400

and that might be okay for all of dsub, jamstash, play:Sub. but I need to find out how airsonic etc does it

edit:
from the docs www.subsonic.org/pages/api.jsp in the "Introdution" section it says "Please note that all methods take the following parameters", then mentions the auth parameters. maybe this is a bug with play:Sub?

[tonydub]

Note that this works with another project in Go https://github.com/hednowley/sound

[sentriz]

interesting, but I think that may be wrong.
here is official airsonic being called without args:

$ curl "http://localhost:4242/rest/ping.view"                                                                                                                  
<?xml version="1.0" encoding="UTF-8"?>
<subsonic-response xmlns="http://subsonic.org/restapi" status="failed" version="1.15.0">
   <error code="10" message="Required parameter is missing."/>
</subsonic-response

so maybe a bug with play:Sub?

[MichaelBechHansen]

This is intentional use of ping.view from play:Sub...

Calling ping.view without the u/p parameters allows play:Sub to learn the server API version and possibly variant without revealing the password.
If play:Sub determines that the server supports token-based authentication, it will proceed with s/t parameters only.
In total this means that the password will never be transmitted if the server supports token-based authentication.

When calling ping.view without u/p parameters, play:Sub expects a HTTP 200 response with a Subsonic-error in the response data.
Subsonic and other return "Required parameter missing":
{ "subsonic-response": { "status": "failed", "version": "1.16.1", "error": { "code": 10, "message": "Required parameter is missing." } } }

As stated above, this reveals server API version, and several servers send a "type" = "my-server-type" as part of the response.

[tonydub]

Thank you for your answer @MichaelBechHansen. The bug encountered can now be explained.

[sentriz]

Ahh I see @michael thank you. basically I think it means gonic could return a 200 instead of 400 when no arguments are passed to the view and the rest should work

[MichaelBechHansen]

@sentriz I also expect returning 200 OK would make everything work.

[tonydub]

By testing locally by making the modifications so that the /rest/ping.view endpoint returns a 200 (OK) response with the XML payload indicated previously, the ios play:sub application still indicates that the Gonic server is offline.
Do you have an @MichaelBechHansen explanation?

[MichaelBechHansen]

I think the next step would be to debug this with the app against a running server...
Do you have a test-server with the fix for the 400-error code available?
I tried with Gonic from the docker image, but that one returns 400 for the pings as discussed previously.

[sentriz]

hi all. I just pushed to docker hub image sentriz/gonic:no400 (instead of :latest. which should include the fix.
thank you for debugging this

EDIT: it's still building
EDIT: built now

[MichaelBechHansen]

Got it!

So there was more to it than the 400 vs 200 error code...
Here's what's happening with ping.view and other requests from play:Sub:

• HTTP requests are sent with a POST method - not GET.
• the request parameters are in the HTTP-body - not as URL-parameters.

It seems gonic does not pick up the request parameters from the HTTP-body, and therefore does not pick up the "f=json" in the http body.
In the end gonic sends back an XML-response and Content-Type: application/xml.
play:Sub rejects the response since it gets application/xml when expecting JSON.

[sentriz]

interesting @MichaelBechHansen

I always presumed query parameters were they way to do subsonic parameters, not body - but not sure where I got that thought.

why does play:Sub choose body instead of query?
also, thinking about it...

if Airsonic supports both play:Sub and DSub, where play:Sub does body params, and DSub does query params - does Airsonic support both? should gonic support both? I can't find anything here about body vs query

[MichaelBechHansen]

The first many versions of play:Sub used URL-parameters and GET, but when running into length limitations on the URL, I switched to POST and parameters in the body.

The URL length limitation is only a problem for a few Subsonic APIs, in particular when creating/editing large playlists, the parameters contain a list of track-ids which can be pretty long.
AFAIR I hit the length limitation with some user playlists that had 20000+ tracks.

All the Subsonic server variants play:Sub has been running against is supporting parameters in both URL and body.

https://stackoverflow.com/a/26717908 has some details on the parameters in body for POST.

EDIT: Sidenote, all servers also seems to support POST with parameters in the URL... Subsonic supports having parameters in both URL and body, and combines them all.

[sentriz]

ok cool, thank you Michael. currently thinking of a nice abstraction my handlers can have for post/query parameters

[sbhal]

@sentriz did you got time to make this work?

[sentriz]

hi @sbhal
I finished my last college exam yesterday :)

I've about two weeks of free time, so working on gonic now

[sentriz]

branch created for this here

https://github.com/sentriz/gonic/tree/param_refactor

just need to check I haven't broken anything now. and might get one of my friends with an iPhone to test play:Sub with it

[MichaelBechHansen]

If you have a public docker image, I can test play:Sub against it if you want.

[sentriz]

that's great @MichaelBechHansen thank you. also could you give an example of a cURL with params in the body, the same way play:Sub does it?

some options might be

curl -d "param1=value1&param2=value2" -X POST <addr>
curl -F "param1=value1" -F "param2=value2" -X POST <addr>

[MichaelBechHansen]

This should be a curl command line simulating what play:Sub sends for a ping:

curl -v -X POST -H 'Content-Type: application/x-www-form-urlencoded' 'http://SERVER/rest/ping.view' --data 'c=test&v=1.9.0&f=json&u=USER&p=PASS'

All parameters in the body, in the same format it would have been in the URL query-string.

[sentriz]

ok thank you. could you try :latest from docker hub if you get a chance? it should support params in the body. and very possibly something else will not work

[MichaelBechHansen]

Success: play:Sub can ping and login against the latest docker image.
I don't see any media in the API or in the sonic web interface... but maybe I need to do something configuration for that to happen.

[MichaelBechHansen]

Update: With a correctly configured path to the music folder, things look pretty good:

• Folders, artists and albums load into play:Sub.
• Music plays.

👍 Well done.

I'll poke a bit around to see what works and what doesn't.

[sentriz]

great to hear :) thanks for bearing with me. please let me know what you find

[MichaelBechHansen]

So I got around to testing a bit more...
Here's a list of things that play:Sub uses from the Subsonic API that seems to not be implemented or working with version 0.5.1:
Unimplemented:

• getRandomSongs.view
• getSongsByGenre.view
• getRandomSongs.view
• getSimilarSongs2.view
• getTopSongs.view
• getGenres.view
• getStarred.view / getStarred2.view
• getAlbumList.view
• getArtistInfo2.view
• getAlbumInfo2.view
• getLyrics.view
• getInternetRadioStations.view + CRUD
• getBookmarks.view + CRUD
• getPodcasts.VIEW

Partial implementation:

• many views do not return all metadata/properties
• getAlbum.view - most metadata is not present.

The above probably just represents what you already know, and where you are in the project.

I have tried to list the unimplemented views in the order of importance from my subjective point of view. A number of these rely on last.fm data (similar/top songs), and some rely on metadata tags (genre), but some are more "low hanging fruit" (getrandomsongs)

One thing in play:Sub that does not work with gonic 0.5.1 is folder browsing: the actual folders are browsable but no songs appear. I will see if I can figure out why over the weekend... most likely it's related to what I have lumped under "partial implementation" above.

[sentriz]

cool thanks for this. some good info here.

// begin common
rout.Handle("/getLicense{_:(?:\\.view)?}", ctrl.H(ctrl.ServeGetLicence))
rout.Handle("/getMusicFolders{_:(?:\\.view)?}", ctrl.H(ctrl.ServeGetMusicFolders))
rout.Handle("/getScanStatus{_:(?:\\.view)?}", ctrl.H(ctrl.ServeGetScanStatus))
rout.Handle("/ping{_:(?:\\.view)?}", ctrl.H(ctrl.ServePing))
rout.Handle("/scrobble{_:(?:\\.view)?}", ctrl.H(ctrl.ServeScrobble))
rout.Handle("/startScan{_:(?:\\.view)?}", ctrl.H(ctrl.ServeStartScan))
rout.Handle("/getUser{_:(?:\\.view)?}", ctrl.H(ctrl.ServeGetUser))
rout.Handle("/getPlaylists{_:(?:\\.view)?}", ctrl.H(ctrl.ServeGetPlaylists))
rout.Handle("/getPlaylist{_:(?:\\.view)?}", ctrl.H(ctrl.ServeGetPlaylist))
rout.Handle("/createPlaylist{_:(?:\\.view)?}", ctrl.H(ctrl.ServeUpdatePlaylist))
rout.Handle("/updatePlaylist{_:(?:\\.view)?}", ctrl.H(ctrl.ServeUpdatePlaylist))
rout.Handle("/deletePlaylist{_:(?:\\.view)?}", ctrl.H(ctrl.ServeDeletePlaylist))
// begin raw
rout.Handle("/download{_:(?:\\.view)?}", ctrl.HR(ctrl.ServeStream))
rout.Handle("/getCoverArt{_:(?:\\.view)?}", ctrl.HR(ctrl.ServeGetCoverArt))
rout.Handle("/stream{_:(?:\\.view)?}", ctrl.HR(ctrl.ServeStream))
// begin browse by tag
rout.Handle("/getAlbum{_:(?:\\.view)?}", ctrl.H(ctrl.ServeGetAlbum))
rout.Handle("/getAlbumList2{_:(?:\\.view)?}", ctrl.H(ctrl.ServeGetAlbumListTwo))
rout.Handle("/getArtist{_:(?:\\.view)?}", ctrl.H(ctrl.ServeGetArtist))
rout.Handle("/getArtists{_:(?:\\.view)?}", ctrl.H(ctrl.ServeGetArtists))
rout.Handle("/search3{_:(?:\\.view)?}", ctrl.H(ctrl.ServeSearchThree))
// begin browse by folder
rout.Handle("/getIndexes{_:(?:\\.view)?}", ctrl.H(ctrl.ServeGetIndexes))
rout.Handle("/getMusicDirectory{_:(?:\\.view)?}", ctrl.H(ctrl.ServeGetMusicDirectory))
rout.Handle("/getAlbumList{_:(?:\\.view)?}", ctrl.H(ctrl.ServeGetAlbumList))
rout.Handle("/search2{_:(?:\\.view)?}", ctrl.H(ctrl.ServeSearchTwo))

here are the views that are currently implemented. so i was suprised to see getAlbumList.view in your list. that one is working for me with DSub / Android

and one more thing. which metadata is missing for example getAlbum.view? some audio tag metadata? looking here i don't see what would be added

thanks!

and probably a well done for play:Sub, never used it but it sounds very feature rich

[MichaelBechHansen]

getAlbum.view for a random album with gonic:

{
"subsonic-response": {
"status": "ok",
"version": "1.9.0",
"type": "gonic",
"gonicVersion": "v0.5.1",
"album": {
"id": 43,
"coverArt": 43,
"created": "2019-11-27T00:48:18.191278992Z"
}
}
}

getAlbum.view for a random album with Subsonic 6.1.4:

{
"subsonic-response": {
"status": "ok",
"version": "1.16.1",
"album": {
"id": "4814",
"name": "Chris Cornell",
"artist": "Chris Cornell",
"artistId": "271",
"coverArt": "al-4814",
"songCount": 16,
"duration": 4130,
"playCount": 0,
"created": "2019-09-09T15:14:20.000Z",
"year": 2018,
"genre": "Rock",
"song": [
{
"id": "64272",
"parent": "64232",
"isDir": false,
"title": "Hunted Down",
"album": "Chris Cornell",
"artist": "Soundgarden",
"track": 1,
"year": 2018,
"genre": "Rock",
"coverArt": "64232",
"size": 19494389,
"contentType": "audio/mp4",
"suffix": "m4a",
"transcodedContentType": "audio/mpeg",
"transcodedSuffix": "mp3",
"duration": 161,
"bitRate": 964,
"path": "Chris Cornell/[2018] Chris Cornell/[2018] Chris Cornell (CD1)/1-01 Hunted Down.m4a",
"playCount": 12,
"discNumber": 1,
"created": "2019-09-09T15:02:23.000Z",
"albumId": "4814",
"artistId": "18",
"type": "music"
},
... a lot og more lines for the remaining track data...

[MichaelBechHansen]

Also: I think the examples are often partial on the Subsonic API site.
Using the xsd's for reference should give more complete info: http://www.subsonic.org/pages/inc/api/schema/subsonic-rest-api-1.16.1.xsd

[MichaelBechHansen]

I don't know if it has anything to do with the partial data reported on getAlbum.view, but it seem my gonic 0.5.1 docker only has around 60% of the tracks it should have.
When starting a new scan, the album, artist and track counts go up for a few minutes, but the stops and reverts to the pre-scan count.

It does not seem to run out of resources:
using 100mb of the 1000 allocated to the container.
using very little CPU, also while scanning.
gonic database folder is one a volume with plenty of free space, and takes < 15mb all files included (db,wal,db-shm).

[sentriz]

possibly related is #26

i will discuss it there

[sentriz]

closing now as the original problem is fixed. again thank you for the help and info folks.