New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add the ability to "retain" a dismissed screen, to support iOS Picture In Picture overlays. #11097
base: 6.x
Are you sure you want to change the base?
Conversation
Hey jaredly! Thanks for opening your first pull request in this repo. If you haven't already, make sure to read our contribution guidelines. |
✅ Deploy Preview for react-navigation-example ready!
To edit notification comments on pull requests, go to your Netlify site settings. |
packages/native-stack/package.json
Outdated
@@ -61,7 +61,7 @@ | |||
"react": "*", | |||
"react-native": "*", | |||
"react-native-safe-area-context": ">= 3.0.0", | |||
"react-native-screens": ">= 3.0.0" | |||
"react-native-screens": ">= 3.19.0" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I set this to a fictitious future version, because it relies on a change to react-native-screens
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jaredly the peer dependency shouldn't be bumped, the package is still compatible with react native screens 3, only the new feature needs a newer version so that can be documented separately
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey! Thanks for the PR.
I'm not sure about the approach, this is highly specific to the native stack and native features so I don't think it belongs in the router.
Native Stack should expose this with a different API such as a custom hook and store the additional data regarding screens that need to be retained in its internal state.
It'd also be useful to see some code snippets of how this may be used to understand the APIs needed.
packages/native-stack/package.json
Outdated
@@ -61,7 +61,7 @@ | |||
"react": "*", | |||
"react-native": "*", | |||
"react-native-safe-area-context": ">= 3.0.0", | |||
"react-native-screens": ">= 3.0.0" | |||
"react-native-screens": ">= 3.19.0" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jaredly the peer dependency shouldn't be bumped, the package is still compatible with react native screens 3, only the new feature needs a newer version so that can be documented separately
@satya164 cool, thanks for the feedback! I'll try the custom hook direction. |
@satya164 So in order to implement screen restoration, I will need to be able to specify the |
Since this is very specific to native-stack, there shouldn't be any changes to routers or the core package. Ideally, native stack would expose a method from its hook that you can call which would internally use reset action to add the route back to the state - so your key will be preserved in this case. |
oh cool, yeah that will work great |
/** | ||
* Contexts | ||
*/ | ||
export { RetainContext } from './views/RetainContext'; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let me know if you want this to go into a /contexts/
directory, or somewhere else
return ( | ||
<ScreenStack style={styles.container}> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
best viewed with whitespace ignored
if (!retainedKeys.includes(route.key)) { | ||
setNextDismissedKey(route.key); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this warning fires spuriously when restoring a previously dismissed screen
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jaredly yea this one has some issue, need to address it later
@satya164 ok I've got it working 😅 this repo https://github.com/jaredly/rnsdemo with the pip.mp4 |
Thanks @jaredly for updating the MR! I'll try to review it when I get some free time |
bump? 🙏 |
Hey @jaredly! Thanks for opening your first pull request in this repo. If you haven't already, make sure to read our contribution guidelines. |
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## 6.x #11097 +/- ##
==========================================
- Coverage 74.17% 73.82% -0.35%
==========================================
Files 178 179 +1
Lines 5618 5670 +52
Branches 2212 2216 +4
==========================================
+ Hits 4167 4186 +19
- Misses 1402 1435 +33
Partials 49 49
☔ View full report in Codecov by Sentry. |
@satya164 I'd really like to get this merged so I can stop depending on a fork -- is there anything I can do to help this along? |
@jaredly I'll try to review it this week. would you be able to write a short usage example with the code so it's easier for me to see the API? |
Fantastic, thanks! Here's a little demo repo I made to show how |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jaredly
Thanks for your hard work.
Few things:
- Please, specify what is the public API here. We prefer not to expose the contexts directly.
- you should warn/throw if one wants to use that on Android/web
- Avoid using values from refs in the state update. In general, I cannot see why
latestValue
is needed here. If it's very needed, please elaborate - Avoid shadowing of variables
- Add some examples.
@@ -373,74 +377,86 @@ function NativeStackViewInner({ state, navigation, descriptors }: Props) { | |||
|
|||
useInvalidPreventRemoveError(descriptors); | |||
|
|||
const { retainContext, hiddenRoutes, hiddenDescriptors, retainedKeys } = | |||
useRetainContext(state, navigation, descriptors); | |||
descriptors = { ...hiddenDescriptors, ...descriptors }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd avoid overriding props, as ot may be confusing for someone reading the code. I guess it's a mental model that we don't mutate values as long as possible.
How about mergedDescriptors
and mergedRoutes
.
It's a bit confusing (at least, I can imagine, it can be) that you use create an array and object without memorizing. Of course, it makes sense here as we don't pass them as props, but I'd add a comment, e.g., "no need to memorize as we don't pass them directly as props"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no need to memorize as we don't pass them directly as props
imo the comment isn't necessary since memorizing depends on other factors and not about props. it'd be confusing for me to see such a comment.
); | ||
})} | ||
</ScreenStack> | ||
<RetainContext.Provider value={retainContext}> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about ScreensRetentionContext
? I'd prefer to use nouns if possible.
<RetainContext.Provider value={retainContext}> | ||
<ScreenStack style={styles.container}> | ||
{routes.map((route, offsetIndex) => { | ||
const index = offsetIndex - hiddenRoutes.length; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's confusing. I don't think we have any offset here.
const index = offsetIndex - hiddenRoutes.length; | |
const isHidden = index < hiddenRoutes.length; |
and then hidden={isHidden}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
const isHidden = hiddenRouteKeys.includes(route.key)
maybe more descriptive.
* This context allows you to retain screens in memory, even after | ||
* they have been popped off the stack. This is important for | ||
* supporting native iOS "Picture in Picture" mode, as the | ||
* originating UIViewController must be retained in memory for the | ||
* PiP overlay to continue running. | ||
* Retained screens can also be "restored" to the top of the stack, | ||
* for example when the user taps the button in the PiP overlay | ||
* to return to the app. | ||
*/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
* This context allows you to retain screens in memory, even after | |
* they have been popped off the stack. This is important for | |
* supporting native iOS "Picture in Picture" mode, as the | |
* originating UIViewController must be retained in memory for the | |
* PiP overlay to continue running. | |
* Retained screens can also be "restored" to the top of the stack, | |
* for example when the user taps the button in the PiP overlay | |
* to return to the app. | |
*/ | |
* This context allows you to retain screens in memory, even after | |
* they have been popped off the stack. That is important for | |
* supporting native iOS "Picture in Picture" mode, as the | |
* originating UIViewController must be retained in memory for the | |
* PiP overlay to continue running. | |
* Retained screens can also be restored to the top of the stack, | |
* for example, when the user taps the button in the PiP overlay | |
* to return to the app. | |
*/ |
I'm not native so feel free to ignore. I believe there's no need for a quotation as the description is precise enough.
* to return to the app. | ||
*/ | ||
export const RetainContext = React.createContext<RetainContextT>({ | ||
retain() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd do something even stronger.
retain() { | |
get retain() { |
This will throw an error on access, without waiting for invoking the function.
const { [key]: _, ...rest } = screens; | ||
return rest; | ||
}); | ||
return latestValues.current.retainedScenes[key] != null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you should not rely on the ref here as concurrent mode can make it flaky. Why can't you rely on the previous state?
// for which we still want to keep the memory allocated. | ||
const hiddenRoutes: Route<any>[] = []; | ||
const hiddenDescriptors: NativeStackDescriptorMap = {}; | ||
Object.keys(retainedScenes).forEach((key) => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd prefer
const {
hiddenDescriptors: descriptors,
hiddenRoutes: routes
} = Object.values(retainedScreen).reduce(() => { ... }, { routes: [], descriptors: {} })
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code will be much simpler if there was just a hiddenRouteKeys
array. Seems like the only place that needs it is when we set hidden={true}
, and with such an array, it could just check hidden={hiddenRoutes.includes(route.key)}
without changing descriptor and routes related logic.
const hiddenRoutes: Route<any>[] = []; | ||
const hiddenDescriptors: NativeStackDescriptorMap = {}; | ||
Object.keys(retainedScenes).forEach((key) => { | ||
if (!routeKeys[key]) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would just check the state.routes
here directly, even though it's a higher theoretical complexity. This is not a big deal as this array should be small anyway.
const latestValues = React.useRef({ state, retainedScenes, descriptors }); | ||
React.useEffect(() => { | ||
latestValues.current = { state, retainedScenes, descriptors }; | ||
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm rather against patters like that. The new state should be the result of props and the previous state as react may batch those updates, we should not rely on what is currently in useRef
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe this is not very needed here and it's possible to refactor the code without that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume the intention is to memoize the retain
, release
, restore
functions. In that case define those functions separately with useLatestCallback
and then they can be safely added to dependency array for useMemo
.
With that it's unnecessary to do such a ref.
const retainContext = React.useMemo<RetainContextT>( | ||
() => ({ | ||
retain() { | ||
const { state, descriptors } = latestValues.current; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why not state
? Also, please avoid shadowing variables as it gets harder to understand the code.
Actually, ignore my comment about adding the example. It's not possible now |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think an API like the following would be simpler:
const { retain, release, restore } = useRetain();
// ...
<Video
// ...
onPictureInPictureStatusChanged={isActive => {
if (isActive) {
retain();
} else {
release();
}
}}
>
It'd be simpler not to have to manage a key
. If user wants to retain multiple things (is that even possible?), they can use the hook multiple times. Though looking at the code, the key
seems to be just the route key which will always be the same and can be aquired from useRoute
inside the hook itself.
Also why does restoreUserInterfaceForPictureInPictureStopCompleted
need a timeout? Is it possible to make restore
return a promise so consumer can wait until this function can be called?
@@ -373,74 +377,86 @@ function NativeStackViewInner({ state, navigation, descriptors }: Props) { | |||
|
|||
useInvalidPreventRemoveError(descriptors); | |||
|
|||
const { retainContext, hiddenRoutes, hiddenDescriptors, retainedKeys } = | |||
useRetainContext(state, navigation, descriptors); | |||
descriptors = { ...hiddenDescriptors, ...descriptors }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no need to memorize as we don't pass them directly as props
imo the comment isn't necessary since memorizing depends on other factors and not about props. it'd be confusing for me to see such a comment.
<RetainContext.Provider value={retainContext}> | ||
<ScreenStack style={styles.container}> | ||
{routes.map((route, offsetIndex) => { | ||
const index = offsetIndex - hiddenRoutes.length; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
const isHidden = hiddenRouteKeys.includes(route.key)
maybe more descriptive.
const latestValues = React.useRef({ state, retainedScenes, descriptors }); | ||
React.useEffect(() => { | ||
latestValues.current = { state, retainedScenes, descriptors }; | ||
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume the intention is to memoize the retain
, release
, restore
functions. In that case define those functions separately with useLatestCallback
and then they can be safely added to dependency array for useMemo
.
With that it's unnecessary to do such a ref.
() => ({ | ||
retain() { | ||
const { state, descriptors } = latestValues.current; | ||
const route = state.routes[state.routes.length - 1]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of assuming the last screen, this hook should use the useRoute
hook to get the correct route object
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As currently written, the retain
function isn't a hook. I'll look into changing the API to a useRetain
and see whether that can meet all our needs
[route.key]: { | ||
route, | ||
descriptor: descriptors[route.key], | ||
}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should only store the route key. Duplicating the route
and descriptor
is problematic as those could change after storing here making them out of sync. They should always be read from state.routes
and descriptors
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm it can't be read from state.routes
if the screen has been popped off the stack; that's why I'm storing it here. In the case where the route hasn't been popped, I use the one from state.routes
.
const index = state.routes.findIndex((r) => r.key === key); | ||
const routes = | ||
index !== -1 ? state.routes.slice(0, index) : state.routes.slice(); | ||
routes.push(route); | ||
navigation.dispatch({ | ||
type: 'RESET', | ||
payload: { ...state, index: routes.length - 1, routes }, | ||
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't refer to actions by string. Use the CommonActions.reset
action creator instead. In this case it should be possible to just use navigation.reset
.
Also, I assume it's moving the screen to the end. Is this necessary/desirable? If the screen wasn't the last one, seems strange to change the order after pip restore. I'd remove this, consumers can do this themselves if they want to.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the screen wasn't the last one, seems strange to change the order after pip restore.
I thought that in order for a screen to be visible on a screen stack, it needed to be the last one. Is that not the case? restore()
is supposed to bring the screen back to visibility; either adding it to the end if it had been popped off, or popping off anything on top of it if the screen still exists in the stack.
|
||
return true; | ||
}, | ||
supported: true, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This boolean seems unnecessary if it's always true
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it's true here, but the default value for the RetainContext
has supported: false
// for which we still want to keep the memory allocated. | ||
const hiddenRoutes: Route<any>[] = []; | ||
const hiddenDescriptors: NativeStackDescriptorMap = {}; | ||
Object.keys(retainedScenes).forEach((key) => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code will be much simpler if there was just a hiddenRouteKeys
array. Seems like the only place that needs it is when we set hidden={true}
, and with such an array, it could just check hidden={hiddenRoutes.includes(route.key)}
without changing descriptor and routes related logic.
This allows the native stack router to retain some viewControllers in memory but out of sight, ready to be restored at a later time. This is necessary for a good PictureInPicture experience.
FYI we have added |
@satya164 oh interesting! Unfortunately it looks like that API doesn't align quite well enough:
In order for picture-in-picture to work as users expect, we need to be able to "mark a screen as retained" without also removing it from the active routes, such that when it is removed from the active routes (e.g. by a POP or any number of other events) it does not get detached. |
Responding to your questions (thanks for giving it a thorough review!)
I realize now that my demo repository doesn't make use of the keys, but a full application (such as ours, the Khan Academy app) that needs to support picture-in-picture does need them to function correctly. Specifically, when viewing another screen with a video on it, the PIP overlay should be automatically dismissed, but when returning to the screen that originated the PIP overlay, it should not be automatically dismissed. In order to implement that behavior, we need to keep track of the screen where the video was
In order for the PIP overlay to be restored fluidly (animating smoothly back to the location of the |
I think we can make that change. |
(Feature request fwiw) (corresponding react-native-screens PR)
Thanks for making this wonderful library! I've really enjoyed using it. I'm transitioning a large app from custom native navigation code (written before we adopted react-native) to react-navigation, and the only thing I'm having trouble with is iOS Picture-In-Picture.
If you're unfamiliar, Picture-in-Picture (PIP) refers to popping a video out into an "overlay" that persists as you navigate around the app, until you either close it or "restore" it, which results in the originating screen being pushed back onto the stack, and the video sliding back into its original location.
Here's a little video of that in action, for context:
pip.mp4
There are two things needed to support PIP well:
onRestoreUserInterfaceForPictureInPictureStop
callback for react-native-video).I've made a pull-request to
react-native-screens
adding ahidden
prop to theScreen
component, which allows a screen to stay in the react tree (so the resources are retained & events still propagate), but be ignored by the NavigationStack.On the react-navigation side, we need an additional flag on a given
route
, to allow a screen to be (a) marked for retention (e.g. if a user dismisses this screen, hang on to it but mark it "hidden"), (b) restored from the hidden state, and (c) dropped from retention (which, if the screen is hidden, means it is removed, and if it is still in the visible stack, it just has the flag cleared).Test code and steps to reproduce
Here's a minimal repo illustrating the problem behavior: https://github.com/jaredly/rnsdemo
And a video:
https://user-images.githubusercontent.com/112170/207732805-7750c1e8-7276-4f51-9244-f86ec312092b.mp4