Investigation: do we still need Panoptes JavaScript Client (PJC)?

[shaunanoordin]

Investigation

Question: do we still need the old panoptes-javascript-client (aka PJC) in FEM? If so, why, and where?

Context:

• As of 21 Mar 2024, we're still using PJC (which was previously use for PFE and several custom front ends) in parts of our FEM code.
• However, we also have Panoptes.JS (aka lib-panoptes-js) which was intended to replace PJC.
• So what's still missing from Panoptes.JS?

Additional reading (for nerds):

• ADR 3 explains our impetus to design a new auth client (Panoptes.JS) to replace PJC
• ADR 21 expands on the design of Panoptes.JS, with some references to issue Implement server-side authentication #1306 (as of writing, server-side auth still hasn't been implemented, AFAIK)
• Mark's notes on "Why is lib-user auth agnostic?"

Current PJC Use

The following indicates where PJC is still used in the FEM code, as of 21 Mar 2024.

app-content-pages

• auth.signOut() is used in PageHeader
• (PJC is also used in PageHeader's tests, but I'll ignore any instance of PJC in .spec.js files from here on.)

app-project

• auth.signOut() is used in PageHeader
• auth.checkBearerToken() is used in usePanoptesAuth.js
• auth.checkBearerToken() and auth.checkCurrent() are used in usePanoptesUser.js
• sugarClient.subscribeTo()/unsubscribeFrom() are used in useSugarProject.js
• PJC's auth is passed down to the Classifier, on ClassifierWrapper.js
• auth.checkBearerToken() is used in checkRetiredStatus.js
• auth.checkBearerToken() is used in Collections.js
• auth.checkBearerToken() is used in Recents.js
• auth.checkBearerToken() and sugarClient are used in Notifications.js
• auth.checkBearerToken() is used in UserProjectPreferences.js
• auth.checkBearerToken() is used in YourStats.js

(You know, I think I'm starting to detect a pattern here.)

app-root

• auth is passed down as the "authClient" in the Next.js pages: GroupPage, MyGroupsPage, UserStatsPage.
• auth.signOut() is used in PageHeader

lib-async-states: no PJC here

lib-classifier

• oauth is used in the dev server
• Deep dive:
  • (authClient).checkCurrent() is used in RootStore.js
  • (authClient).checkBearerToken() is used in models/Annotation.js
  • (authClient).checkBearerToken() is used in utils/getBearerToken.js

lib-grommet-theme: no PJC here

lib-panoptes-js: no PJC here, obviously. This is Panoptes.js!

lib-react-components

• auth.signIn() is used in LoginFormContainer.js
• auth.register() is used in RegisterFormContainer.js
• auth.checkBearerToken() is used in fetchPanoptesUser.js
• auth.checkCurrent(), auth.listen('change'), and auth.stopListening('change') are used in usePanoptesUser.js
• auth.checkCurrent() and auth.checkBearerToken() are used in useUnreadMessages.js
• auth.checkCurrent() and auth.checkBearerToken() are used in useUnreadNotifications.js

lib-user

• oauth is used in the dev server.
  • Note that once lib-user: Refactor dev App to use PJC auth client #5992 is merged, this will change to: auth.checkCurrent(), auth.listen('change'), and auth.stopListening('change') are used in the dev server
• Deep dive:
  • (authClient).checkBearerToken() is used in utils/getBearerToken.js.

tools-standard: no PJC here

Analysis

The following auth functions exist in PJC, but don't (yet?) exist in Panoptes.js. See PJC's auth.js for the full code.

• auth.checkCurrent(): ⭐ used in multiple instances
• auth.checkBearerToken(): ⭐ used in multiple instances
• auth.signIn() and auth.signOut(): only used on the sign in/out components (the page headers)
• auth.register(): only used in the register user form
• auth.listen('change') and auth.stopListening('change'): only used in lib-react-components to listen for when the user logs in or out.

Compare with Panoptes.js's auth.js code, which only has...

• verify()
• decodeJWT()

oauth is only directly used as "authClient"s passed down to packages in dev servers. Mark's work with lib-user seems to indicate we may be able to replace PJC's oauth with PJC's auth.

I have no idea what to make of the sugarClient. 🤷‍♂️

❗ We now present Shaun arguing with himself about oAuth:

• If PJC's oAuth is never used in our FEM code, then we won't need to build oAuth functionality in Panoptes.JS.
• Hang on. While FEM doesn't need oAuth, other CFEs (example) DO need oAuth functionality.
• The example you've given is really old, and ASM only needs oAuth because it doesn't have a *.zooniverse.org domain name. Newer CFEs such as Community Catalog do have *.zooniverse.org domain names, so they can use PJC's auth
• Is *.zooniverse.org a domain name or a hostname? Should the sentence be "Community Catalog is a subdomain parked under the .zooniverse.org domain"?
• It doesn't matter, stop being pedantic. Point is, we don't need to build oAuth in Panoptes.JS, because nobody uses it.
• You mean for now.

Status

Main investigation complete. Awaiting discussion, to plan further steps for Panoptes.js.

[eatyourgreens]

Roger wrote some docs for oauth.js, and I extended them to include auth.js (or, at least, the parts of auth.js that handle bearer tokens and refresh tokens.)
https://zooniverse.github.io/panoptes-javascript-client/

The important thing to remember about those clients is that they are stateful. When you import auth from 'panoptes-client/lib/auth.js', you aren't just importing the client, you're also importing the current user and their OAuth tokens (both the refresh token and access/bearer token), which are stored in the client. This gets messy if you already have a user state store, as you now have user state stored in the client and in your own store, and the two can fall out-of-step. That's the source of most monorepo auth bugs.

[eatyourgreens]

Also, if it's useful at all: Panoptes access tokens are good for two hours, so make sure you have a refreshed token before making a credentialled request. AFAIK Panoptes refresh tokens never expire (maybe double-check with the backend team about this) but Panoptes session cookies expire after two weeks of inactivity. You can check the session cookie lifetime in browser dev tools.

Frontend auth for a typical session looks something like this:

1. const user = await auth.checkCurrent(): exchange your Panoptes session cookie for an access token and refresh token. Run this once on page load to set up a Panoptes session in the current tab. The client will store the Panoptes user object, the access token and the refresh token in its internal state. The session cookie is a Secure, HttpOnly cookie, so this is probably the most secure way to authenticate a Zooniverse user from a browser. The client's internal state isn't exposed via any public interfaces, so should be secure from third-party scripts also running on the page (maybe? JS has no real concept of private variables.)
2. auth.signIn(): If you don't have a session cookie, you can sign in with auth.signIn(), using the OAuth password grant flow. However, the password flow is disallowed by OAuth 2.1, as it makes it quite easy to steal passwords.
3. const token = await auth.checkBearerToken(): use the OAuth refresh token flow to get an access token in order to perform a credentialled request eg. making a classification or reading your inbox. This will always return a refreshed access token, so there shouldn't be any need to worry about having expired credentials, unless you hang on to this token and try to reuse it after it's expired. Try to avoid saving these tokens in local component state. The auth client will manage them for you.

Panoptes.js doesn't have an auth client at all, just some helper functions that can decode an access token, which is a JWT, and return the user and permission scopes that are encoded inside. You still need to use an OAuth client, of some sort, in order to get that token from Panoptes OAuth.

[eatyourgreens]

If you're new to OAuth, I highly recommend watching Kim Maida's The Art of Authentication before jumping into any Panoptes auth code. That talk really clarified Panoptes OAuth for me.

Refresh Tokens: What are they and when to use them is also a useful read.

[eatyourgreens]

Point is, we don't need to build oAuth in Panoptes.JS, because nobody uses it.

Zooniverse Classrooms use oauth.js. However, that library uses the implicit grant flow, which is also deprecated and not recommended for use any more.

[eatyourgreens]

Panoptes auth uses access tokens to encode both user info (identity) and scopes (access permissions.) That's not really recommended any more. A modern approach, still using OAuth, would be to put user identities into ID tokens and have the access token only responsible for limiting access to protected resources. Maybe something to bear in mind for things like Zooniverse user groups and Zooniverse user certificates? The whole Panoptes auth implementation in the frontend is something like ten years old now.

What's an ID token?

As the name may suggest, an ID token is an artifact that client applications can use to consume the identity of a user. For example, the ID token can contain information about the name, email, and profile picture of a user. As such, client applications can use the ID token to build a user profile to personalize the user experience.

An authentication server that conforms to the OpenID Connect (OIDC) protocol to implement the authentication process issues its clients an ID token whenever a user logs in. The consumers of ID tokens are mainly client applications such as Single-Page Applications (SPAs) and mobile applications. They are the intended audience.

What's an access token?

When a user logins in, the authorization server issues an access token, which is an artifact that client applications can use to make secure calls to an API server. When a client application needs to access protected resources on a server on behalf of a user, the access token lets the client signal to the server that it has received authorization by the user to perform certain tasks or access certain resources.

https://auth0.com/blog/refresh-tokens-what-are-they-and-when-to-use-them/

[eatyourgreens]

@mcbouslog this issue, from 2018, raises the same issues as you are interested in, I think, so could be worth reading.

• 3rd party cookies disabled can't renew a token - auth code overhaul required. panoptes-javascript-client#76

Specifically, there's a bunch of repeated code in the JS API client for token storage and management (both refresh tokens and access tokens):

This library needs an overhaul in how it manages and re-uses tokens and this code can be shared between the supported authentication flows we currently use:

• Auth.js uses the password credentials flow and gets a refresh token
• Oauth.js uses the implicit flow and does not get a refresh token

We should split out the token retrieval to different strategies patterns and consolidate the token storage and management code where we can. The different token flows above will require different token renewal strategies as well:

• flows with a refresh token can get a new access token directly
• flows without a refresh token will have to re-authenticate or rely on an existing session to gain a new token.

The updated code will also provide management hooks that the including app can use to configure and customize the authentication flows as well. This will allow events like failing to refresh a token to bubble up to the calling app to better manage the experience of our users on sites that use implicit flows.

There's also an open request to support the authorization code flow (for an old PRN server app), which would also be useful now that password grant is disallowed and implicit grant is discouraged.

• Add support for authorization code flow in auth module  panoptes-javascript-client#37

The Python API client already supports authorization codes for server-side authentication eg. I use an authorization code here to build new subjects for the SLSN project. As a user of both the JS and Python API clients, it would be useful to have parity between them.

[mcbouslog]

Thank you @eatyourgreens ! This comment and links, as well as related Slack posts are very helpful. It's looking the panoptes-javascript-client auths at minimum and likely auth in general could use an overhaul.

[eatyourgreens]

Also worth noting that none of the code discussed here uses fetch, which has been the standard in browsers for years and is supported in Node since Node 18.

See #317

[eatyourgreens]

Also noting that the Next.js app router expects data-fetching to use fetch. So data caching might not work with superagent.