REST api

[dmarx]

A common request we've received is to support a REST api. For a variety of reasons this is not something we are planning to support at this time. There's clearly a non-negligible segment of the community that has strong feelings about this and I'd like to open the floor to better understand the variety of use-cases here. By all means: please convince us of why we need to expose a REST API!

[blueturtleai]

I'm the author of the GIMP stable diffusion plugin https://github.com/blueturtleai/gimp-stable-diffusion. I already checked, if it would be possible to use the SDK. There are two reasons why I were not able to use it:

• The current GIMP version 2.10 is based on Python 2.7, the SDK needs Python 3. GIMP 3 will be based on Python 3, but a release date is still unknown.

• Even if GIMP 3 will be available, I doubt that it would be possible to use external Python packages in a GIMP plugin. At least I didn't find a way to do that.

It should also be pretty easy to port the plugin to the stable diffusion REST API. It currently already uses a REST API to talk to the colab notebook based backend server.

[imperator-maximus]

Hello,
after implementing two fully-featured clients in GRPC using a least used 4 different GRPC client libs and used it on various client tests and server-side as well (all SD) also used both GRPC and GRPC-Web I know a lot of it. All my clients are running fine so for me personally it does not make any difference anymore. But I think that I am quite an expert of using this interface and can compare - so here is my opinion:

Cons of GRPC:

1. very difficult to find a working client. Improbable for JS - just does not work everywhere. PIP install is also a huge problem (in many environments just not possible). So I had to find a working pure python client and copy manually 20 dependency libs to project for example
2. so implementation time is always at least several days for a very experienced developer for a SD client in GRPC and not a few hours for a REST. most other developers just give up here (I know many). This means that they find alternative servers with REST instead of using the official interface
3. confusion about grpc/grpc-web: both could run on different ports and offer different features. also for end-user port chaos is not easy to understand
4. extremely hard to debug: none of Chrome extensions are working but if you are in a specific environment it is even more difficult
5. different syntax across languages: so in Javascript it is (getter) and setter and in Python kwargs in constructor. So an interface cannot be rewritten in short time because code is very different here (in comparison: JSON is same in REST across all languages)
6. for a beginner never had used it before it is hard to generate stub classes from a proto - one example:
   protoc -I=. engines.proto --js_out=import_style=commonjs:. --grpc-web_out=import_style=commonjs,mode=grpcwebtext:.
   this is nightmare command line syntax. also several different tools had to be installed before to get this one line running
7. if interfaces had been extended and new .pb had not been updated it is nearly impossible to fix this in client without updating pb before and regenerate stub classes (im comparison just modify the JSON for REST)
8. just make a simple test like in REST from browser using URL for get or a test tool (integrated in an online documentation) is not possible/not available. So in REST an online documentation with all endpoints and JSON examples + test function is a common feature. Here in SD there is a Python example client which is by far not that beginner friendly.
9. GRPC is still quite unknown and in enterprise environments but also in small projects just never in use and therefore most developers just do not know it and I have seen many(!) interfaces in last 20+ years.

Pros of GRPC:

1. it has got a clear interface description. so if there is an error (e.g. type conflict) it usually will shown in client side and not after unexpected data had send to server.
2. in theory: binary data transfer and streaming (with cancel feature). but grpc-web does have problems with streaming sometimes (always?) and does not do binary of course. So if images would come in via stream and use could cancel it would have been a cool feature.

Since this is a REST API discussion my personal opinion here:

• I can fully understand the arguments for using GRPC (streaming, better interface definition, binary transfer, "the future") at the beginning and therefore the decision at that time for using it and not REST. It was a professional point of view here. I appreciate this.

• but at the end and after some time people are using it I have to say it was the wrong decision because it feels like CORBA or SOAP and a lightweight interface would have been better. I do not have meet many (zero to be honest) people who had been happy about programming a GRPC client for SD.

So at the end: it was a good idea in beginning but it didn't worked out which can happen - maybe better fully move to REST for next SD release and abandon GRPC and if streaming is needed just use Websockets.

[benrugg]

I wasn't aware of this discussion thread until just now. It looks a bit stale, but I hope that a REST API is still something that is being considered. There are many, many use cases where GRPC is either vastly more difficult to implement, or actually impossible. (A few examples: Software like Krita and Blender; Any web app!)

There are already other services wrapping Stable Diffusion and providing REST API's (one notable example: https://xno.ai/api).

This is certainly doable, and it would allow more creativity and faster community implementation/support. Please consider again :-)

[johnsabath]

The Stability REST API is coming - will share some additional information next week.

[benrugg]

Woohoo! That's great news!

[cruhl]

We'll creating a developer preview/feedback channel in the main Discord under #api when it's ready to share around! 👍

[ryanakann]

A common request we've received is to support a REST api. For a variety of reasons this is not something we are planning to support at this time. There's clearly a non-negligible segment of the community that has strong feelings about this and I'd like to open the floor to better understand the variety of use-cases here. By all means: please convince us of why we need to expose a REST API!

Unity game is my use case. C# or REST API would be of great help!

[johnsabath]

REST API developer preview just dropped:

https://twitter.com/jcsabath/status/1597629332184145920

[JackLangerman]

Hey guys -- thanks for the awesome work on the SDK and REST API. I'm trying to replicate my results from the SDK with the REST API, but am having trouble getting identical images. Just to be sure before I start dropping code here: if I use the same init image, same settings, and same seed with no clip guidance is it correct that I should be able to get identical images with both the SDK and REST API?

[benrugg]

I am not affiliated with Stability, but I believe the answer is no. I think that the SDK is running a different pre-processor of some kind (not the right technical word), and that you won't get identical images between the two.

Hopefully someone else can give a more technical explanation.

[johnsabath]

Would you mind sharing the list of parameters that you're setting a value for? My first reaction is that there may be some variance between the default values that are applied when parameters are omitted.

For instance, it looks like step_schedule_end has a different default value between the SDK and the REST API, so if you're not providing a value explicitly it could explain this behavior.

[benrugg]

Yes, I think you're right that the defaults are different in some cases. I actually haven't done any experiments with attempting the same parameters through both systems, but I just remember seeing someone ask this same question on the api channel on the Stability Discord, and one of the devs said it goes through a different processor (again, not the right technical word).

[benrugg]

Also, I should mention - that discord channel is probably the best place to get your question answered by one of the stability devs!