Add cloud API specs

[kbatuigas]

Deploy preview
Review deadline: 16 May 2024

• Add a new reference page for the cloud API using the OAS file imported from the cloud repo

NOTE: Adding the cloud API reference (i.e. the specs) are done in this PR since it has to be merged in the api branch where our API reference pages live. #411 is open against the main branch, where our regular docs live and where we add non-reference (e.g. conceptual topics, or task-oriented docs) content.

This PR also uses Rapidoc "slots" to inject custom HTML into the generated reference documentation. Currently, this only works for the overview and auth sections. The content added here has some overlap with the supplemental docs (added in #411 ) but is a condensed version, and more relevant to readers who might want to make API requests directly from the reference doc instead of programmatically.

• The overview also contains a "how to use the API reference" and "quickstart" subsections

• The overview lists error responses in one subsection

• The Regions/Throughput tiers section in the overview needs the formatting fixed (@JakeSCahill to investigate).

• There will be a fix to remove the text box next to the "Get Token" button

[netlify]

✅ Deploy Preview for redpanda-docs-preview ready!

Name	Link
🔨 Latest commit	599fb4c
🔍 Latest deploy log	https://app.netlify.com/sites/redpanda-docs-preview/deploys/6647da96d29b120008e533a3
😎 Deploy Preview	https://deploy-preview-283--redpanda-docs-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[JakeSCahill]

Change this back to main before merging. This is just here so we can test including content from that branch.

[Feediver1]

^^ @kbatuigas Reminder - as I see this still reads main

[kbatuigas]

Thank you @mo-redpanda ! Let me address per item:

1. Change from what is the Cloud API to how do I use the Cloud API for common use cases

   • Possibly remove Controlplane and Dataplane as nav items
   • Add common use cases getting started pages e.g. Cluster, Topics, IAM that would give sample code to do common uses
   • cases with links to the API reference for more detailed
   • Add API reference link to the Cloud API section

Cloud API ── Overview ──── Architecture ──── Controlplane ──── Dataplane ── Getting started / Examples ──── Authenticate ──── Clusters ────── Create cluster ────── List clusters ────── Delete clusters ──── Topics ────── Create ... ────── List ... ────── Delete ... ──── IAM ────── Create ... ────── List ... ────── Delete ... ──── API reference

• At this point, we just don't have enough content to organize in this way. I expect this to evolve as the cloud team continues working with the docs team to add more pages, use cases, and examples.
• Note that we are also planning on taking out all Cloud docs into its own nav tree, so when that's complete we may be able to get a bit closer to the structure / information architecture you have in mind.

2. Make it clear which endpoints use the Controlplane and which use the Dataplane

   • Solutions

     • Move CURL example up as it has the URL
     • Denote it in some way
     • Separate pages for Data and Controlplane. I don’t recommend this approach as it puts the burden on the user to understand what is in the controlplane vs the dataplane

• I don't know if we can move the curl example unless with JS. Ideas @JakeSCahill ?
• There was also a conversation regarding adding "tags" to the spec file so that we could do some more organization or denote this somehow on the front end. We'd need a new docs request to work on visually organize/sort the endpoints using tags on the front end.

• We should guide and give users the different options to get the Dataplane URL. Currently only states API, but users could easily get this from the GUI and possibly rpk? It also isn’t clear what response body this is referring to or which API call to get it.

• Is there anyone who can confirm whether the URL can be retrieved via GUI or rpk?
• We currently say:

The base URL of the Data Plane APIs is unique to every cluster. You can retrieve this value by making a Get Cluster request to your target cluster. Use the dataplane_api.url from the response body as the base URL when calling the Data Plane API endpoints.

and also:

For requests to the Data Plane API endpoints:

1. Make a Get Cluster request for your target cluster.
2. The Get Cluster response contains the Data Plane API URL. Copy the value of dataplane_api.url from the response body.
3. Click API Servers in the sidebar and select “{dataplane_api_url} - Data Plane API”.
4. Paste the URL into the dataplane_api_url input field.

@mo-redpanda Can you provide guidance on how to make it clearer which response body to check, or which API call it is?

3. Fill example doesn’t seem to do anything

• If an example value is provided in the spec, I think that value is prepopulated in either the param fields or the request body. See Rapidoc docs. Other caveat to this is we don't have examples for all components. It's doable, and we can work towards it as a milestone, but is a pretty heavy lift between engineering and docs.

4. Visual treatment of having the verbs (GET, PUT, POST, etc) being in a “pill” is distracting and makes the list less scannable.
   Recommend removing as IMO it doesn’t add much value and is repeated in the details for that endpoint

• We have this in our other API reference docs currently (example: Admin API). We can certainly look into removing it. @Deflaimun @JakeSCahill thoughts?

6. Regions and throughput tiers are listed with no context on how this should be used which is when creating a cluster. This should be part of creating a cluster endpoint, or a separate page and linked to from there.

• @micheleRP has an open PR to improve our tiers and regions content. When that's published we can definitely incorporate that here.
• With regards to moving it to either the Create Cluster endpoint, we'd have to do that directly in the spec file. I can ask @birdayz or @sago2k8 if that's possible. As for moving it to a separate page, @towfiqa , @micheleRP , and I met separately to discuss this, and it was decided that we want to keep this in the reference page, since only API users need this exact information and is less relevant to GUI users.

7. API server section is confusing. This should be clarifying the difference between the control and data plane APIs and how to retrieve the dataplane api via gui, api and CLI. We should just give them the curl command to determine the dataplane url

• We can see what we can do in the spec file to improve guidance here.

8. Authentication is confusing

• No API key applied?
• OAuth (auth0) is implementation
• RedpandaCloud could be removed
• What is Implicit flow?
• Why does scopes have nothing there?

• This is all generated from what's in the spec file. I can check with @birdayz , @sago2k8 etc to see what can be done here. But in any case, you do need to first click "Get Token" in the Authentication section. We mention this step in the Quickstart as well.

8. Oauth auth0 seems redundant and unnecessary

• The icon with label is a Rapidoc standard I believe (see examples 1, 2, 3). Not sure how much we can customize here except maybe change the auth0 key in the spec file? Not sure what downstream implications there are for changing that. cc @birdayz

9. Operations header should be left-aligned

• We might be able to change this with JS? @JakeSCahill

10. Header section needs cleaning up

• @mo-redpanda Would you mind specifying what needs to be cleaned up in the header? (Adding screenshot here)

[kbatuigas]

@towfiqa adding questions here -

1. The Requirements section you suggested adding to the reference is already in the Overview page in the main docs. Are you asking for the same content to be shared across both?
2. I made an update so that instructions for the page now has a heading "Use this page to try the cloud API". (See updated preview.) This was meant to provide guidance/navigation to readers who want to test the API from their browser, and might not go out to the main (what we've been referring to as "supplemental") docs. We can remove it entirely, if you're OK with not having to handhold readers through navigating through the reference doc.
3. From the wireframe, it appears that you want the reference doc to link out to separate pages for Errors and Response Codes as well as Regions-Zones-Tiers. Could you confirm if that's the case? Because if these are meant to be inline references to sections that are on the same page, we can try and do that but we would not place it under an "Additional References" header.

Please also note that we can't add any more nav sections on the left hand side on top of "Redpanda Cloud," "API Servers," and "Authentication". This is a Rapidoc limitation.

[birdayz]

Oauth auth0 seems redundant and unnecessary

The icon with label is a Rapidoc standard I believe (see examples 1, 2, 3). Not sure how much we can customize here except maybe change the auth0 key in the spec file? Not sure what downstream implications there are for changing that. cc @birdayz

we must give it some name. it's used in openapi fields .security and components.securitySchemes. the name can be changed no problem, but it will have SOME name.

See https://github.com/redpanda-data/cloudv2/blob/main/proto/gen/openapi/openapi.prod.yaml#L3550 and https://github.com/redpanda-data/cloudv2/blob/main/proto/gen/openapi/openapi.prod.yaml#L6647

Authentication is confusing

No API key applied?
OAuth (auth0) is implementation
RedpandaCloud could be removed
What is Implicit flow?
Why does scopes have nothing there?

• We don't use API Key authentication - instead we use oauth client credentials grant
• Implicit flow is the oauth2 flow used by the docs app. It's not important and can be hidden.
• We do not use token scopes
• These things can be hidden, it's only a presentation thing, e.g. the rapidoc tool should hide it.

Regions and throughput tiers are listed with no context on how this should be used which is when creating a cluster. This should be part of creating a cluster endpoint, or a separate page and linked to from there.

@micheleRP has an #475 to improve our tiers and regions content. When that's published we can definitely incorporate that here.
With regards to moving it to either the Create Cluster endpoint, we'd have to do that directly in the spec file. I can ask @birdayz or @sago2k8 if that's possible. As for moving it to a separate page, @towfiqa , @micheleRP , and I met separately to discuss this, and it was decided that we want to keep this in the reference page, since only API users need this exact information and is less relevant to GUI users.

Technically it can be moved. Is this the correct location? https://github.com/redpanda-data/cloudv2/blob/5461591ab003d593a17afc9e3a7a5f665ba96b08/proto/gen/openapi/openapi.prod.yaml#L5668

I find it problematic though, if we move it to CreateCluster. It's not just specific to the CreateCluster endpoint. Regions, zones apply to Network and Network Peering as well. Also, in the future, it is related to Cloud Storage Bucket resource as well. It's NOT just for clusters.

[Feediver1]

Be sure to make the update at the top that Jake marked. Also make the updates re GUI elements per the Style Guide. Thx!

[kbatuigas]

Opened a PR in cloudv2 to edit the Regions description https://github.com/redpanda-data/cloudv2/pull/15391

[kbatuigas]

Per Joyce's update to PM, we are releasing the cloud API docs so we can test the try-it functionality prior to the beta announcement.