Crash Reporting with Sentry
Firefox Focus for iOS uses Sentry for crash and exception reporting. This kind of reporting gives Mozilla insight as to why Firefox Focus for iOS crashes or incorrectly behaves. It is one of the key methods we use to improve the product in terms of stability.
This page explains how Sentry works, how the various parts interact and what kind of data it sends back to Mozilla.
Sentry is an open source crash reporting and aggregation platform. Both the client SDK, github.com/getsentry/sentry-cocoa, and the server, github.com/getsentry/sentry, are open source.
Since January 2022 we use the Sentry service hosted at sentry.io. Previously Mozilla hosted their own server. This means crash reports are sent and stored on sentry.io.
On the client side Sentry is invisible. There are no parts to interact with. It reports crashes and fatal errors back to Mozilla in the background. Sentry is enabled when the Send usage data switch in the Firefox Focus for iOS settings is enabled by the user. By default this switch is enabled, it is an opt-out mechanism.
On the server side there is a dashboard that the Firefox Focus for iOS team uses to look at incoming crash reports. The dashboard lets us inspect the crash report in detail and for example see where in the application the crash happened, what version of the application was used and what version of iOS was active.
We make sure that no personally identifying information is sent as part of crash reports. For example we make sure we do not attach URLs to crash reports and we also scrub information like your IP address.
Below is an overview of all the attributes that are part of a crash report.
A typical Sentry crash report contains three categories of data: device, application, crash.
Sentry collects basic information about the device the application is running on. Both static (device type) and dynamic (memory in use, boot time).
Example:
"model_id": "simulator",
"family": "iOS",
"arch": "x86",
"storage_size": 250140434432,
"free_memory": 87408640,
"memory_size": 8589934592,
"boot_time": "2017-10-07T22:07:14Z",
"model": "iPhone8,4",
"usable_memory": 6127521792,
"type": "device"
Sentry collects basic information about Firefox Focus for iOS. The device_app_hash is a Sentry generated identifier that allows it to group crash reports for a specific client. This identifier is unique to Sentry and is useless outside of Sentry. It cannot be used to correlate a specific user to a crash. It is also not related to any identifiers that Firefox Focus for iOS uses internally.
Example:
"app_identifier": "org.mozilla.ios.Focus",
"app_name": "Firefox Focus",
"device_app_hash": "f6b338df9308a9c36658d5a07a1b37ce0a43e338",
"executable_path": "/Users/stefan/Library/Developer/CoreSimulator/Devices/54FF4D59-1B27-4A59-9B32-95ACED42A96F/data/Containers/Bundle/Application/86981850-4155-47E2-9CB5-A83D157E47D7/Firefox Focus.app/Firefox Focus",
"app_build": "1",
"app_start_time": "2017-10-13T13:59:47Z",
"app_version": "3.10",
"type": "app",
"build_type": "simulator"
Every crash report contains a reason - why did this crash happen. This field can contain two different values:
- an error message generated by iOS
- an error message generated by Firefox Focus
Both Apple and Mozilla make sure that no personally identifiable information is put in any of these messages. We keep them technical and to the point.
Example of an iOS generated message:
NSInternalInconsistencyException - fatalApplication threw exception NSInternalInconsistencyException:
attempt to delete item 1 from section 0 which only contains 1 items before the update
Example of a Firefox Focus generated message:
"EXC_BAD_INSTRUCTION crash | respondsToSelector: _T013Firefox_Focus17SentryIntegrationC5crashyyF"
Every crash report contains a list of libraries and frameworks that the application links against. These include: Frameworks written by the Firefox Focus for iOS team, Third-Party frameworks that we link against and finally system frameworks.
The UUID fields in these image descriptions are the same for every copy of Firefox Focus for iOS and are not connected to a specific installation, device or user.
Example:
"cpu_subtype": 3,
"name": "/Users/stefan/Library/Developer/CoreSimulator/Devices/54FF4D59-1B27-4A59-9B32-95ACED42A96F/data/Containers/Bundle/Application/86981850-4155-47E2-9CB5-A83D157E47D7/Firefox Focus.app/Firefox Focus",
"revision_version": 0,
"major_version": 0,
"image_vmaddr": "0x100000000",
"image_addr": "0x10b45e000",
"minor_version": 0,
"cpu_type": 16777223,
"image_size": 1191936,
"type": "apple",
"uuid": "F89DC09D-E3B0-30AB-AD5E-558C3DC50A78"
Every crash report contains a stack trace, which shows what functions in the Firefox Focus for iOS code led to this crash. It includes names of iOS system functions and Firefox Focus for iOS functions.
0 Sentry 0x10b7ccd04 -[SentryClient crash]
1 Firefox Focus 0x20b52a2da _T013Firefox_Focus17SentryIntegrationC5crashyyF
2 Firefox Focus 0x20b4b909c _T013Firefox_Focus11AppDelegateC11applicationSbSo13UIApplicationC_s10DictionaryVySC0F16LaunchOptionsKeyVypGSg022didFinishLaunchingWithI0tF
3 Firefox Focus 0x20b4bad5a _T013Firefox_Focus11AppDelegateC11applicationSbSo13UIApplicationC_s10DictionaryVySC0F16LaunchOptionsKeyVypGSg022didFinishLaunchingWithI0tFTo
4 UIKit 0x10e32c35a -[UIApplication _handleDelegateCallbacksWithOptions:isSuspended:restoreState:]
5 UIKit 0x10e32ddb7 -[UIApplication _callInitializationDelegatesForMainScene:transitionContext:]
6 UIKit 0x10e333245 -[UIApplication _runWithMainScene:transitionContext:completion:]
7 UIKit 0x10e6eec6c __111-[__UICanvasLifecycleMonitor_Compatability _scheduleFirstCommitForScene:transition:firstActivation:completion:]_block_invoke
8 UIKit 0x10eabc3ef +[_UICanvas _enqueuePostSettingUpdateTransactionBlock:]
9 UIKit 0x10e6ee865 -[__UICanvasLifecycleMonitor_Compatability _scheduleFirstCommitForScene:transition:firstActivation:completion:]
10 UIKit 0x10e6ef0c3 -[__UICanvasLifecycleMonitor_Compatability activateEventsOnly:withContext:completion:]
11 UIKit 0x10f04cc11 __82-[_UIApplicationCanvas _transitionLifecycleStateWithTransitionContext:completion:]_block_invoke
12 UIKit 0x10f04caca -[_UIApplicationCanvas _transitionLifecycleStateWithTransitionContext:completion:]
13 UIKit 0x10ed3ab9c __125-[_UICanvasLifecycleSettingsDiffAction performActionsForCanvas:withUpdatedScene:settingsDiff:fromSettings:transitionContext:]_block_invoke
14 UIKit 0x10ef35c3e _performActionsWithDelayForTransitionContext
15 UIKit 0x10ed3a998 -[_UICanvasLifecycleSettingsDiffAction performActionsForCanvas:withUpdatedScene:settingsDiff:fromSettings:transitionContext:]
16 UIKit 0x10eabba4c -[_UICanvas scene:didUpdateWithDiff:transitionContext:completion:]
17 UIKit 0x10e331ac6 -[UIApplication workspace:didCreateScene:withTransitionContext:completion:]
18 UIKit 0x10e8fa523 -[UIApplicationSceneClientAgent scene:didInitializeWithEvent:completion:]
19 FrontBoardServices 0x1135cf158 -[FBSSceneImpl _didCreateWithTransitionContext:completion:]
20 FrontBoardServices 0x1135d7c4d __56-[FBSWorkspace client:handleCreateScene:withCompletion:]_block_invoke_2
21 libdispatch.dylib 0x11194e2b5 _dispatch_client_callout
22 libdispatch.dylib 0x111953565 _dispatch_block_invoke_direct
23 FrontBoardServices 0x113603672 __FBSSERIALQUEUE_IS_CALLING_OUT_TO_A_BLOCK__
24 FrontBoardServices 0x113603328 -[FBSSerialQueue _performNext]
25 FrontBoardServices 0x113603897 -[FBSSerialQueue _performNextFromRunLoopSource]
26 CoreFoundation 0x10d38a2b1 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__
27 CoreFoundation 0x10d429d31 __CFRunLoopDoSource0
28 CoreFoundation 0x10d36ec19 __CFRunLoopDoSources0
29 CoreFoundation 0x10d36e1ff __CFRunLoopRun
30 CoreFoundation 0x10d36da89 CFRunLoopRunSpecific
31 GraphicsServices 0x113ef69c6 GSEventRunModal
32 UIKit 0x10e334d30 UIApplicationMain
33 Firefox Focus 0x20b4c0747 main
34 libdyld.dylib 0x1119c2d81 start