Joining forces with Apprise; Food for Thought

[caronc]

Hi,

I’m the developer of Apprise and I just came across shoutrrr. I think it’s great that you see the value in URL based notification system! Thank you for even linking to my project on your README.md file! 😄

I was honestly just curious why you were re-designing the idea (in its entirety) from scratch when we’d both benefit from combined efforts on working from one? Just food for thought. I just added the 61st notification service a few days ago; i guess I just mean to say that I'm still very actively supporting it and trying to making it better. I would love to provide the service and support for your very active platform with my already active and well tested notification one.

Anyway, I'm not challenging you by any means; I don't want you to stop the complete rewrite of the idea in another language if you feel it's the best route forward for you. I was thinking that we might better benefit each other by joining forces and working together on the same solution? I wrote this Web Based API a while back, I'm not sure if it could be easily tailored to suite your needs or not?

Thoughts?

Chris

[piksel]

Hi!

There are basically two reasons, one was that some URL format (can't remember which one) didn't parse correctly with the go net.URL library, which caused me to alter the syntax somewhat. The other reason is basically my own (probably totally unjustified) sense of "order"; you have examples like this:
xmpp://user:password@hostname
and
twist://password:login/#team:channel (which is not a valid URL btw)
where the username/password parts are swapped. And others where the tokens go at the end etc.
I figured I could take the opportunity to do things more coherently instead of just copying what I figured were mistakes that couldn't be rectified due to backwards compability.

I will take a look at the API, maybe we can add support for it?

Thanks for your great work!

edit: I guess, what you meant was the fact that shoutrrr CLI can be used as an alternative to apprise, but that is not the main goal of this library. The reason for making it in the first place was to abstract notification functionality from the watchtower project in it's own library. The main benefit from this is so that it can be statically compiled in a single go binary that will run on any supported system with no dependencies at all. That also means that it can be used in slim docker containers with no base image, making them really small.

[caronc]

@piksel

one was that some URL format (can't remember which one) didn't parse correctly with the go net.URL library, which caused me to alter the syntax somewhat.

My first argument to that is: you don't need to parse a URL if a underlining system is doing that for you. Some upstream services have both '@' and '/' in their API key, others are more then 64 characters long (which all violate different URL RFC elements); I'm not sure what net.URL ignores and enforces. You may find yourself building awfully strange URLs if you are always going to try to remain compliant.

Secondly, Apprise violates some of the URL convention intentionally in order to make forming them easier for the end user (and documentation). I'll explain this below as your second point allows me to expand on exactly why this is the case.

The other reason is basically my own (probably totally unjustified) sense of "order"; you have examples like this:
twist://password:login/#team:channel (which is not a valid URL btw)

Twist isn't a very good example because you might find yourself doing something similar down the road. The URL in question is twist://{password}:{email}/#channel. Consider that that (in this case) the {email} is what being is used to log into to the account with and it's format is: user@hostname. So really {email} could be seen expanded in the Twist URL as twist://{password}:{user}@{hostname}/#channel. If you change this to a proper format twist://{user}:{password}@{hostname} you will be requiring users to break apart their actual full login credentials just to stick their {password} in the middle of it.

As you expand your Shoutrrr notification library, you will come across this same thing and may find yourself violating the same URL conventions too. 😉

Basically there is a gray area between the URL you choose to represent the service with, and the format you chose within it to make it easy for the end users to construct. That said, I have no doubt that I have inconsistencies in other locations... but that brings me to my next point:

Your 2 arguments could have just been a simply ticket filed against Apprise (more than a year ago - first commit was Apr 12, 2019) to bring it to my attention so that it can be fixed or discussed. Your thought process is awesome; I mean Watchtower is amazing, and you're great with your community! I just personally think that forking an idea or project in an open source world usually happens when a project goes dead, or the developer of a project isn't listening or trying to adapt to it's user base. I personally don't feel i sit in these categories. I'd love to make Apprise more useful rather then watch it be rewritten by a larger community that didn't even reach out to me (not once). 🙂.

The reason for making it in the first place was to abstract notification functionality ... in it's own library.

You described Apprise here; so we see eye to eye here. The only difference is the language of choice.

The main benefit from this is so that it can be statically compiled in a single go binary that will run on any supported system with no dependencies at all.

Python is installed on every Linux distribution out of the box such as Ubuntu, Debian, RedHat, CentOS, Mint, ArchLinux, etc. So Apprise works out of the box as well. The exception to this would be containers.

The Python library is without a doubt larger then a small compiled static go executable. I won't argue this. But if we're to only focusing on containers: If a Python container is say 150MB larger on your user's hard-drive than the equivalent Go based alternative... well that's all it is... 150MB. The containers fantastic feature of using re-usable layers will prevent any more disk space from ever being used thereafter. So even a second Python app chosen will re-use 90% of the same layers as the previous so the additional disk space lost is just in KB (the same as the second Go container would be). You can get a 2TB hard drive for $50USD these days. So users who already have at least one container that has Python on it will see a negligible disk space usage adding Apprise to their fleet.

Also consider that statically compiled libraries come with their faults too. When a fix comes upstream to SSL, or glib, or anything for that matter, you need to re-compile your library again. You have a bug fix? You need to recompile your library again. If you use shared libraries, that's a bit better because things like SSL can be patched independently by the upstream provider. But if you're having your users do this, well then it's not a small static binary any more, it's the entire Go development (headers, compiler, etc) so the code can be re-compiled all the time. So there are pro's and cons with everything.

I will take a look at the API, maybe we can add support for it?

That's totally up to you! 🙂

Apprise supports:

• tagging: associating unique ID's with the URLs that spell out all of their credentials for others to see. With tagging you hide all this and protect the user's private information. Tagging also allows you to just notify some services and not others, but still keeping all your service URLs protected in one location.
• configuration files; even ones stored on remote servers
• YAML configuration files to 'really' customize your notifications while still supporting the very simple text based ones, and comma separated if you want to notify more then one.
• file attachments (for the services that support it); maybe you want to send error log, or other details

My true intentions of this ticket were to try see if we could collaborate in such a way that we could focus efforts on a single solution.

• Maybe I can fix Apprise to better address the net.URL reference you're using? But please consider the reasons for doing the URLs the way I do as well.
• Maybe I can update the Apprise-API to better suite your needs? This would be a true side-car option and detach watchtower from also having to maintain it's own library dedicated to notifications.
• I'd love to hear your suggestions? Perhaps there is something else we could consider?

Don't get me wrong; I realize you stated that you're not trying to rewrite Apprise, but from this vantage point it certainly seems like you're building the exact same thing (literally). Someone on Reddit here said:

shoutrrr is basically the Go version of apprise.

If you still want to to go the re-write route (but please consider what was said above), maybe there is some way we can at least compromise and/or agree on the URL syntax so that we won't need to add a section to our documentation: If coming from Apprise do this:, If coming from Shoutrrr do this...? Maybe there is a way we can share what we learn about different upstream providers; caveats, etc?

Thoughts?

Chris

p.s: sorry to write so much 😲

Edit: Thinking more about it, maybe your suggestion of adding support for the Apprise API in Watchtower would be a cool idea? Worst case, you'll grant your user base access 50+ more services then they already had. You may find yourself saving a lot of precious effort (on new notification services) that could have been focused on outstanding Watchtower issues instead. That all said, I have no doubt you'll eventually get all of these services ported over to Shoutrrr in good time if this is still the route you intend to follow.

[simskij]

I just want to be very clear here that I love the apprise project and by far see it as the number one project in the notifications service space.

Our aim is not to compete with you or the divide the user base in any way, hence always mentioning apprise in all written and spoken content on shoutrrr. The main goal is to provide an embeddable, zero-dependency alternative for go developers. It is not really meant to be run standalone, even if that certainly is possible with our minimal CLI, but rather as a go library.

Taking watchtower as an example, embedding apprise wouldn't be a viable option as it would inflate the image by quite a bit. The reason for this is that go is perfectly runnable on scratch images, and for apprise to work we'd have to switch that to some kind of minimal linux dist, like alpine.

I'd love for us to collaborate on this in some way. For instance, maybe a url-based notification standard/specification would make sense to make sure it's easy to move between the projects? I'd love to speak more about this and see what we can do to make sure there is no competition between the projects.

Sorry for not replying to all of your comments above at this point. Will definitely try to do that once I get some time to sit down and go through it, I just wanted to add my two cents before the discussion got any further. 👍

[caronc]

Our aim is not to compete with you or the divide the user base in any way, hence always mentioning apprise in all written and spoken content on shoutrrr.

You're totally awesome for your transparency; I really do appreciate it (really i do!) and I do recognize that you've credited Apprise. I honestly don't have a problem with what you're doing. I'm just trying to comprehend it and see if there is a way I can work with you and save you all of this effort. 🙂

embedding apprise wouldn't be a viable option as it would inflate the image by quite a bit.

I agree, i wouldn't suggest embedding it. That's why I was throwing the idea of the Apprise API because you wouldn't have to deal with notifications at all. My arguments with disk size (in my last comment) was just relate to the addition of adding a new (Apprise API) container which would be purely optional for those just who want more notification services (that you haven't developed yet in Shoutrrr).

If you adopted support for the Apprise API into Watchtower, your notification code reduces to this (Stateless Example):

// maybe an APPRISE_HOST container environment variable?
// also need APPRISE_URLS variable

// for stateless queries, just add /notify/ to the end of the APPRISE_HOST
url := "http://apprise-api-hostname/notify/"

formData := url.Values{
   // URLs can be comma separated if there is more than one
   "urls": {"pbul://apikey, mailto://user:pass@gmail.com"},
   "title": {"Houston, we have a problem."},
   "body": {"The developer of Apprise won't leave us alone."},
   "type": {"info"},
}

resp, err := http.PostForm("POST", url, formdata)

If the user wants to protect their private information residing in their URLs, you could optionally support the Stateful approach, then you only need to get the {KEY} the data is saved under:

// maybe an APPRISE_HOST container environment variable?
// also need APPRISE_KEY variable

// for stateful queries, just add /notify/{KEY}/ to the end of the APPRISE_HOST
//  the {KEY} which might be something like 'watchtower' (or whatever the end user wants to point to)
url := "http://apprise-api-hostname/notify/{KEY}/"

formData := url.Values{
   "title": {"Houston, we have a problem."},
   "body": {"The developer of Apprise won't leave us alone."},
   "type": {"info"},
}

resp, err := http.PostForm("POST", url, formdata)

The API would allow you to either just send local traffic on the container network, or even post remotely and utilize a WAN as well.

I'd love for us to collaborate on this in some way. For instance, maybe a url-based notification standard/specification would make sense to make sure it's easy to move between the projects? I'd love to speak more about this and see what we can do to make sure there is no competition between the projects.

I agree, it would be great to at least have consistency as i would love to be as compatible as possible.

[iamareebjamal]

Adding support for webhooks (URL) will automatically mean Apprise API and various other services like IFTTT and Zapier can be integrated, which will move the need for supporting a lot of different services from shoutrrr to any service which supports accepting a payload to a URL including custom services handmade for specific usecases by users.

[piksel]

The stateful queries could be added using something like apprise+http://<HOST>/notify/<KEY>.

The generic webhook/POST service could of course be used as well, but that is still just in the design stages (there are still some go template issues that needs sorting).

[taw123]

@piksel and @caronc This thread from 2 years ago still looks open and I am curious if any progress (conceptually or even code-wise) was made in terms of integration possibilities (web hook or otherwise)?

I ask because I find I should move off ouroboros which I was using for years to the re-invisioned containarr/watchtower (ironically I looked at the old version of watchtower before 3 or 4 years ago when I originally chose to deploy ouroboros).

Unfortunately as Ouroboros, is not "actively" under development (and yes works for all my needs save one...)

My choices seem to be as follows:

1. Fork Ouroboros, and try to revise & abstract the Apprise support so that to get the notification updates I am seeking (email issues currently with sending from gmail). Since this would get me back on a current/supported version of Apprise this has many advantages as described in this thread, and also continues to give back to the community (both Apprise & revitalizing Ouroboros (in a similar, though small way). As a former long time Evangelist at Apple and software engineer from a CONCEPT perspective this seems to have to most appeal even though it clearly would required the most effort, for me. To be clear I would NOT be intending to take over the project or replace it which for the past 2+ years has continued function "as is" with no further development planned. I would ideally "just" abstract the version of Aspirre implemented, so that Ouroboros users could in the future simply update their notification support using the already published Aspirre container and gain implemented Aspirre fixes (and potentially any future notification additions- assuming the abstraction is sufficient). Though again to be clear, my PRIMARY goal would be to just get any/all updates to the current/legacy Ouroboros supported notifications.
2. Alternatively I could simply move to containarr/watchtower which IS supported. This of course gets me a supported Docker updater, but means of course I have to revise/recreate my grafana dashboards from scratch (though I haven't looked too deeply to see if there are any good dashboard for Watchtower to adopt or use as a starting point). This option might also help others, this time by potentially avoiding the confusion of yet another Docker update management tool that won't be supported long term. As stated I don't want to take over Ouroboros just get it abstracted from it's static version of Aspirre, publish it and then step aside.

Unfortunately, and I say this without ANY attempt at insult to anyone, my notification options are decreased choosing option 2. And while this might not be particularly impactful to me at present, it LOGICALLY seems to make sense to use a single general purpose notification engine that is focused on JUST notification (pluggable notification) that I can leverage for other purposes than just Docker or Arrg updates. I understand the reasons stated in the thread for why Watchtower uses shoutrrr and the design considerations for shoutrrr but for myself I seem to be leaning toward option 1 knowing it means a lot more work if (as it seems?) the Watchtower devs have little interest I supporting Aspirre as an option in WatchTower or shoutrrr might have an option to pass notification requests to Aspirre.

Any thoughts, insight or corrections? My apologies for the length of post and potential bad form in cross posting but felt it relevant to post to WatchTower, Aspirre, and Ouroboros. Please let me know if I can help.

--T

[caronc]

This thread from 2 years ago still looks open and I am curious if any progress (conceptually or even code-wise) was made in terms of integration possibilities (web hook or otherwise)?

Hi, this is not my project, so I would not be able to contribute much to your solution. I'm also not a Go programmer, so I'm not sure what the effort level involves to get a shoutrrr webhook to Apprise API installed. I believe this comment must be relatively close to the answer (minus any error handling, etc). I'll leave your response in the good hands of the shoutrrr package maintainers. 👍