Type-safe, extensible and standardized API clients
NOTE: This repo is fully working but documentation is still work in progress. In the mean time please check the examples folder folder for usage guide and the sdks folder for a list of pre-packaged SDKs.
OpenSDKs is a repository of type-safe and standardized SDKs for all your APIs - powered by a single lightweight and extensible runtime that embraces Web Standards and HTTP, and a cli that can generate a custom SDKs from any OpenAPI spec.
npm install @opensdks/runtime
npm install -D @opensdks/cli
- Quick start
- Why (aka the problem)
- Features
- Links
- List of pre-packaged SDKs
- Usage Examples
- FAQs
- The API I need does not have an OpenAPI spec!
- But I would prefer adhoc ts types because codegen is annoying
- What if an existing / official spec is wrong, incomplete or un-ergonomic?
- How are links executed under the hood?
- Why would I want to use this instead of the native SDK provided by each API vendor?
- This looks great. Can I use this for our internal / private API?
- I don't like types, should I still use OpenSDKs?
- What do you plan to work on next?
- How does this relate to trpc? Does this replace it?
- Community
- Contributors
- Credits
If the API you are looking to work with is already a pre-packaged SDK, then you can import and use directly.
import {initSDK} from '@opensdks/runtime'
import {githubSdkDef} from '@opensdks/sdk-github'
const github = initSDK(githubSdkDef, {
headers: {authorization: `Bearer ${process.env['GITHUB_TOKEN']}`},
})
const res = await github.GET('/repos/{owner}/{repo}/commits', {
params: {path: {owner: 'tonyxiao', repo: 'openSDKs'}},
})
console.log(res.data[0]?.committer)
// ^? (property) committer: {
// name?: string | null | undefined;
// email?: string | null | undefined;
// login: string;
// id: number;
// node_id: string;
// avatar_url: string;
// gravatar_id: string | null;
// url: string;
// html_url: string;
// ... 11 more ...;
// starred_at?: string | undefined;
// } | null | undefinedYou can also work with any API by generating a client using the OpenAPI spec.
npx @opensdks/cli generate $YOUR_OPENAPI_URL --name $YOUR_API_NAME
# e.g.
# npx @opensdks/cli generate https://raw.githubusercontent.com/openai/openai-openapi/master/openapi.yaml --name openAI --output openai.tsimport initOpenAISDK from './openai'
const openai = initOpenAISDK({
headers: {
authorization: `Bearer ${process.env['OPENAI_SECRET_KEY']}`,
},
})
const {data} = await openai.GET('/models')
// ^ data is fully typed! @example https://share.cleanshot.com/Wv69QY1RUnlike SDKs offered by major platforms like iOS, Windows, or Unity that are valuable by themselves, most 3rd party SDKs are merely glorified API clients that supposedly make it easier to interact with the underlying HTTP-based API. The problem is that these glue code often introduce what are known as 'leaky abstractions', obscuring what is happening under the hood while introducing its own nuances and a dual-learning curve.
For example, debugging is much more difficult without easy access to the underlying HTTP request, and often times the HTTP API have more up to date and better documentation than SDK for a specific programming language so you end up having to reference the API anyways.
What's worse is sometimes essential information such as rate limiting details that would have been accessible in HTTP headers are inaccessible when using the SDK (I'm looking at you Hubspot!), forcing you to monkey-patch or completely opt-out of the SDK for specific requests.
SDKs, by design, operate directly within your codebase, and this in-process nature means their technical choices and dependencies can negatively impact your applications.
For instance, for a long time the OpenAI SDK used axios for HTTP requests, which uses APIs that are not available in edge environments. Therefore for their docs Vercel had to recommend a community SDK called openai-edge that worked for their edge runtime. Other times vendors choose to ship platform-specific SDKs (e.g. stripe-node), forcing you to polyfill or implement hacky bundler workarounds if you are building an isomorphic application.
SDKs may also choose to make asynchronous requests for performance, but in a typical serverless environment all async workloads must finish before the HTTP handler completes. Hopefully your SDK of choice provides you a way to await for requests to flush, otherwise you are pretty much guaranteed data loss or subtle bugs.
Logging is another common issue. Your engineers may spend hours coming through your codebase to make sure every log statement is properly instrumented, only to get surprised by a rogue console.log deep inside a SDK that you have no control over messing up your observability setup and eating into your logs storage.
Case in point, as of the time of this writing (Jan 19, 2024), the PagerDuty JS SDK was last updated 2 years ago, even though the API itself was updated just last week. Needless to say, to learn how you use the SDK, you are asked to refer to the API documentation.
Even major APIs like Salesforce don't have their own SDKs, instead you will have to rely on community built jsforce, which unfortunately has a 13 year old untyped JavaScript codebase that still transpiles its code back to ECMAScript 5 (2009), presumably for Internet Explorer support? Good luck keeping your own bundle size lean and startup time fast. Understandable though, as it is hard especially for a team of volunteers to justify a rewrite when it still "technically works".
And sometimes a provider (ahem Apollo) doesn't even have an SDK at all, and you are having to essentially build a wrapper client "SDK" in your own codebase so you can interact with them, which is arguably preferable than using an out of date SDK that introduces confusion and bundle bloat.
It's not their fault really. The core business of API providers is their applications & APIs, and writing well built, extensible SDK in language they may or may not have experience in is not an easy job, no to mention staying on top of changes as programming languages themselves evolve (e.g. introduction of the fetch API). Some of those API providers are large and slow moving companies, others are tiny startups starved of engineering resources. Expecting all them to build great SDKs is unfortunately not realistic.
All of the problems listed above compounds if you work with multiple SDKs / APIs (e.g. if you have lots of integrations within your product). Trying to achieve consistent behavior across things like logging, error handling, and retries become nearly impossible as the number of integrations increase. Some companies that work with lots of APIs (e.g. integrations platform) end up ditching SDKs entirely for this reason and revert to making raw HTTP requests, but then they lose out on type safety and other convenience that SDK provides.
And thats why OpenSDKs exists - to give you a single extensible SDK runtime that lets you work with all APIs in a typesafe and consistent way.
Once initialized, the methods available map one to one to the underlying HTTP API so there is nothing to learn. If you know how to HTTP, you know how to use OpenSDK.
const res = await github.GET('/repos/{owner}/{repo}/commits', {
params: {path: {owner: 'tonyxiao', repo: 'openSDKs'}},
})
console.log(res.data[0]?.committer?.name)
// ^? (property) name?: string | null | undefinedWorking directly with HTTP doesn't mean you have to lose type-safety though. Both the request (including path, params, headers and body) and response are fully typed with auto-complete support, enabling both productivity and correctness.
OpenSDKs use the fetch API by default (swappable if you prefer axios or any other request lib) which is pretty much universally available on all environment that javascript runs on. From node.js, to browser, to deno, to bun, to vercel edge, to even completely bespoke environments such as the coda pack runtime. This also means that caching works out of the box in environments like next.js that extends the fetch API for efficient rendering of server components.
Instead of navigating through thousands of lines of code across multiple files, OpenSDKs provide a streamlined in-editor documentation experience. This means you no longer need to leave your editor to consult API documentation, which is frequently more convenient than going to an API's online documentation.
Inspired by trpc link and Apollo link, OpenSDKs runtime includes a composable set of middlewares called Links that are atomic in scope and can add incredibly powerful functionalities in an easy to understand way. For example
const discord = createSdk(discordSdkDef, {
links: [
rateLimitLink({backend: RedisCache}),
retryLink(),
authorizationLink({storage: AsyncStorage}), // Store credentials in react native environmemnt
oauth2RefreshLink({onChange: () => {}}),
logLink({verbose: true}),
errorHandlingMiddleware(),
axiosLink(), // Use axios instead of fetch
(req, next) => {
// Do something custom here...
return next(req)
}
],
})This interface is one of the best ways to extend the sdk / runtime with your own custom functionalities. See Links section for more details.
Because each SDK is just an adapter that plugs into the same underlying runtime & cli, things like API documentation, error handling and request interface are automaticaly consistent, making it much easier to enforce global patterns like logging and metrics in your codebase and scale your integrations infrastructure.
An added bonus is that you no longer have a dual-learning curve whenever your product needs to use a new API, and can therefore build new integrations and go to production that much faster.
When an SDK is initialized, it receives an array of Links that are chained together to return.
Link is a function that accepts two arguments, a standard web Request and a next parameter, and it must asynchronously return a standard web Response. It may do so by calling the next link in the chain or returning a response directly (in which case it is called a terminating link).
type Link = (
req: Request,
next: (req: Request) => Promise<Response>,
) => Promise<Response>| Link | Description | Accepted options |
|---|---|---|
logLink |
Print request / response to a configurable destination for debugging / monitoring | log?: typeof console.log |
retryLink |
Retry request that have failed due to 5xx error with expotential backup | maxRetries?: number |
throwLink |
Great for simulating error conditions and writing tests. Customize the error that gets thrown in getError or return null to let the request through to the next link in chain. |
`getError: (req) => Error |
fetchTermLink |
Terminating link that uses fetch to send request to the backend server. You may provide a |
fetch?: typeof globalThis.fetch |
axiosTermLink |
Terminating link that uses axios to send request to the backend server. It does not include its |
axios: typeof import('axios').default |
oauthLink |
Adds Authorization: Bearer $accessToken header to the request. Optionally checks for validity of the accessToken and proactive refresh the accessToken before making the actual request. |
tokens: OauthTokensrefreshConfig: OauthConfigonRefresh?: (tokens: OauthTokens) => void |
runtimeValidateLink |
Validate the actual request & response payload using the provided OpenAPI spec at runtime. This is especialy helpful when interacting with flaky 3rd party APIs that don't provide or stick with their spec. It can also catch the inevitable bugs in your own client code because JavaScript is ultimately a dynamic language (Don't tell me you have never used an as any 😈) |
`oas: oas30.OpenAPIObject |
rateLimitLink |
Most APIs have rate limit that needs to be respected. This link can either delay the request to stay below the rate limit or throw a RateLimited error so you can handle it in your client / UI. It would be even better if it could be backed by something like Redis so you don't blow past your limit in a multi-process / serverless environment. |
TBD |
There are plenty of other good ideas for links, for example I would love to see a mockLink that uses the OpenAPI spec to starting a fully fledged mock server, or a remoteLogLink that sends the Request / Response to chromeDevTools or Pulse app inspection in a GUI. Please create issues for new ideas or better yet create a PR and contribute! :)
The simplest link is one that does nothing and simply passes the request along to the next one in the chain.
const noopLink: Link = (req, next) => next(req)If you have used axios before, you may know about requestInterceptor and responseInterceptor. They can be easily expressed as links (kind of like that concept in mathematics where a more general theorem simplifies down to a more specific theorem in specific situations)
const requestInterceptorLink: Link = (req, next) => {
// do something with req, such as setting a header
req.headers.set('x-api-key', 'my-super-secret-key')
return next(req)
}
const responseInterceptorLink: Link = async (req, next) => {
const res = await next(req)
// do something with res, such as checking for errors
if (res.status !== 200) {
throw new Error(`Response failed with status ${res.status}`)
}
return res
}Because Links simply use the web standard Request & Response objects, the terminating fetchLink is also a trivial one liner.
const fetchLink: Link = (req) => fetch(req)To use a custom link, simply pass it into the link array when initializing the SDK. e.g.
initSdk(sdDef, {
links: defaultLinks => [
(req, next) => {
console.log('My custom link just got a request', req.url)
return next(req)
},
...defaultLinks
]
})A pre-packaged SDK is not limited to just a pre-generated API client (i.e. it can be more than just running npx @opensdks/cli generate on an OpenAPI spec). It also handles things like adding credentials the request and providing additional functionality beyond HTTP Endpoint wrapper. For example the QBO SDK contains methds to make it easier to make a QBO query and paginate through all records via an AsyncIterator
Partial list of SDKs
| Package | OpenAPI Spec Source | Version |
|---|---|---|
@opensdks/sdk-github |
github/rest-api-description | 0.0.1 |
@opensdks/sdk-apollo |
apollo.openapi.ts |
0.0.1 |
@opensdks/sdk-slack |
apis.guru:slack.com/1.7.0 | 1.7.0 |
@opensdks/sdk-twilio |
github:twilio/twilio-oai/api_v2010 github:twilio/twilio-oai/messaging_v1 |
0.0.1 |
@opensdks/sdk-venice |
||
@opensdks/sdk-discord |
||
@opensdks/sdk-openai |
||
@opensdks/sdk-plaid |
||
For the most up to date list, see the sdks folder
We are working on tooling to make this much easier, in the meantime you can follow the guidelines below. When in doubt, look at how other SDKs are implemented and follow the pattern.
-
Duplicate an existing folder inside sdks as a template and rename it to be an unique name in the format of
sdk-$name. Depending on the API you are working with, each of the following in would be the best starting point- sdk-github: Simplest case with a single up to date OpenAPI spec
- sdk-twilio: Multiple OpenAPI spec in a single SDK
- sdk-apollo: Poliyfilling OpenAPI spec for API that does not natively have one
- sdk-salesloft: Extending the OpenAPI spec for an api that technically has a spec but is incomplete / out of date
- sdk-qbo: Support a non-RESTful HTTP api a with OpenAPI spec polyfill plus custom methods to handle custom sql-like QUERY for Quickbooks Online
-
Update references to match your API in
package.jsonnamefielddownload*command inscriptsfield. If available, get OpenAPI spec from a url and save into the package as$name.oas.json. This step should depend on the internetgenerate*command inscriptsfield. Optionally generate OpenAPI spec json, then generate the$name.oas.meta.tsand$name.oas.types.d.tsvia the@opensdks/cli. Unlikedownload, this step should NOT depend on the internet.
-
Modify
index.ts- Add in custom initialization parameters (typically auth credentials)
- (Optionally) extend the sdk definition in
index.tsin thecreateClientproperty, for example adding support for custom non-RESTful api endpoints or simplify pagination withAyncIterator. - Add an
index.spec.tsideally to ensure it works with a sample credential
-
Submit a PR 🎆
Lines 1 to 128 in a3281e9
openSDKs/examples/summarize-pr.ts
Lines 1 to 54 in a3281e9
For a full app example, check out the What did we work on app. Demo link is here https://what-did-we-work-on.vercel.app/.
You can polyfill it by create your own spec document and you don't need the API provider's permission to do so. After all a spec is nothing but a standardized description of the inputs expected by and output return by the API. And this is a lot easier than you might think because
- You don't have to describe all the endpoints, only the ones you need to use, identical to how you are already manually adhoc-typing those
axios.getor(o)fetchrequets anyways and - With the help of
zod-openapiit is boilerplate-free and you can use the full power of TS to avoid repeating yourself and auto-complete your way to success.
For an example, see apollo.openapi.ts in sdk-apollo
Agreed codegen is annoying. The good news is that our chosen OpenAPI spec authoring library zod-openapi is buit on top of zod and if you use that you'd already have TypeScript types without any codegen! We would like to create a type utility that can transform the inferred types into the same structure as generated types expected by @opensdks/runtime so you can have a tight codegen-less feedback loop when polyfill-ing APIs. There are already prior art of this in zodios and fets so we know it's possible. Contribute if you'd like to see it sooner!
When publshing an idk-package however, you should always publish the oas.json file along with generated types, this is because 1) having OpenAPI document unlocks interoperatability & powerful functionality (such as the planned runtimeValidateLink) and 2) type inferance can be very taxing for the ts compiler and you wouldn't want to incur this cost unless you are the one authoring the spec. (The github generated types is 100k LOC, now try that with Zodios-style inference and watch your editor go up in flames 🔥) On that note, if you notice your IDE becoming slow you might also consider switching over to codegen pattern to help your compiler (and thus yourself) out. Luckily by building on top of zod-openapi, you can get the best of both worlds.
We can use the source spec as starting point and modify it. For example the Slack OpenAPI spec requires a token header parameter for basically every operation, instead of properly leveraging securitySchemas and make it "generic" so to speak. Therefore we post-process the openapi spec in slack.openapi.ts to remove these token parameters so that the sdk consumer can just pass token once during initialization rather than on every request separately.
It actually a trivial recursive function which would have been a single line of code if not for the fact that we need to check every there is at least one link that actually terminates the request and returns a response to the client
function applyLinks(
request: Request,
links: Link[],
): Promise<Response> {
const [link, ...rest] = links
if (!link) {
throw new Error('Terminating link missing in links chain')
}
return link(request, (req) => applyLinks(req, rest))
}Because links operate on standard web Request / Response objects, you can also use them outside of @opensdks/runtime. For example, here's how you can create a custom fetch function with links
import { applyLinks } from '@opensdks/links'
const myFetch: typeof globalThis.fetch = (url, init) =>
applyLinks(
new Request(url, init),
[
// add your own custom links, just don't forget to terminate
logLink(),
fetchLink()
]
)
await myFetch('https://httpbin.org/get')
// [log] Request: HTTP GET https://httpbin.org/get
// [log] Response: 200 from https://httpbin.org/getIf you are creative, you can even find other uses for links For example, you can use links insde next.js' new app directory route handlers.
// @file: app/my-endpoint/route.ts
import { applyLinks } from '@opensdks/links'
const handler = (req) => {
// do something
return NextResponse.json({some: 'data'})
}
export function GET(req: NextRequest) {
return applyLinks(req, [logLink(), handler])
}I mean did you read the why section? 😅
Ok more seriously, it's easier to answer the reverse question - "when should I use the native SDK provided by API vendor instead of the standardized OpenSDKs"?
-
You only work with a single api and therefore don't need to care about infrastructure consistency across them
-
You prefer a different call style (e.g.
stripe.customers.list()vs.stripe.GET('/customers')) -
You need features that are not yet supported by OpenSDKs
- Examples of this include streaming responses, or support for converting JSON into non-JSON values (e.g. ISO date strings to/from
Dateobjects) - Please do open a github issues though so we can prioiritize those based on demand.
- Examples of this include streaming responses, or support for converting JSON into non-JSON values (e.g. ISO date strings to/from
-
You want your life to be difficult (sorry did I say I was going to be serious? 🙊)
It's worth noting that the best APIs already generate SDKs from OpenAPI specs (e.g. Github, Slack, OpenAI) so an OpenSDK would therefore give you the exact same set of typesafe endpoints as custom SDK, just in more a consistent and extensible way.
Yes as long as you have a RESTful API. OpenSDKs works best if your API already produces an OpenAPI spec, otherwise you can always polyfill one (we have many examples of this, e.g. sdk-apollo).
If you already use tech such as GraphQL / trpc / grpc / direct db access in React server component that comes with type safety out of the box, then OpenSDKs would not be necessary.
There are other protocols that don't have inherent type safety such as json-rpc that could benefit from OpenSDKs. We are looking to support those too, please open an issue to let us know your use case.
🤷 I feel bad for you, but you can still use this library from JS. You will still benefit from the extensibility provided by links and pre-packaged SDKs that expose additional functionalities on top of api calls. For example, see the getAll method inside sdk-qbo that returns an AsyncIterator that uses Quickbooks' custom query language to paginate through all entities of a given type.
You tell us! Check out our roadmap on Github and tell us what's important to you.
No not at all. In all likelihood you will using OpenSDKs together with trpc.
trpc is designed for internal client-server communication in your full-stack TypeScript web app, and it is useless when you don't have access to the server's source code, assuming it is even written in TypeScript to begin with.
On the other hand OpenSDKs is most valuable when you are working with multiple 3rd party APIs that you don't control, where each SDK is either an npm install or npx @opensdks/cli generate away.
The OpenSDKs roadmap can be found on GitHub Projects.
Follow us on Twitter for updates.
To get support from the team and chat with other community members you can join the OpenSDKs Discord.
- drwpow/openapi-typescript - OpenSDKs would not be exist if not for the amazing work done by @drwpow.
@opensdks/runtimedepends onopenapi-fetchand@opensdks/cli'sgeneratecommand usesopenapi-typescriptunder the hood. - APIs-guru/openapi-directory - For creating the largest OpenAPI spec registry in the world. I just wish more people knew about you.
- trpc/trpc - For pioneering such an amazing deveoper experience! We borrowed a lot of ideas from you (ok fine I know you borrowed links from apollo too :P), in particular using ES proxy to allow VSCode to
go to definitionfrom api call site directly to route definition is amazing, and we are shamelessly borrowing that. - apollographql/apollo-client - for being the OG of links (as far as I know!) and certainly for introducing me to them. I am still missing
apollo-link-stateandgraphql-anywhereto this date. Would you ever bring it back? :/