SpeedTest

[okdistribute]

A “canary” that an end-user can run to ensure Ditto is working correctly, to rule out an issue with Ditto, or to find a slow device in the mesh.

Problem

Essentially the problem comes from a customer:

“It isn’t working!”
“Oh it’s just really slow”
“What’s slow?”
“Must be ditto!!”

Solution

Having a tool that tests Ditto speed would be helpful to rule out an issue with Ditto transports or the mesh itself and isolate the problem to be something in the developer's application code in the ViewModel or the UI.

1. Open the tool on each device you want to test. Start a subscription on the speed_test for any document created in the past X minutes.
2. Configure the test - add timer and button to start the test. Create a new document or update the existing document in the speed_test collection for the given device
3. Click a button to start the test. Create a new document every 5 seconds with a new created_at time.
4. Show a table of all of the devices in the mesh. Show a number representing how long it took each device to receive the document.
5. If the other devices receive the heartbeat within 2 seconds, then show a blue row.
6. If a device receives the document in over 2 seconds, turn the row red.
7. When the screen closes, stop the subscription and evict the collection.

Additional features

• Tool configuration: Allow the developer to pass the following settings to the component as parameters:

  • baseDocument: object: A document that will be used as the base document. This allows you to change the document size or model your own data as part of the speed test. Add __ditto_internal_lastEdited to that document as a new field

Stretch goal

• exposeConfig: bool: Show the configuration options to the end-user. Defaults to false. Show a toggle for mode. When the mode "new" is selected, the end-user should have a slider to change the document size from 500bytes-10kb.

Additional comments

This is still very early ideation phase, so feedback welcome

[kitplummer]

@ajfabbri @byoung-ditto @anyercastillo - this is something that we could also use/do under Orion (maybe a feature of the TAK plugin?).

[byoung-ditto]

Yes could be helpful for sure. Especially if there was a central way to see if any device was "slow" e.g. the device that initiates the test (or even an external test controller which has access to the test Ditto collection), could list all devices on the mesh along with the status of each device in mesh via a stoplight list: R/Y/G (didnt get it/got it slow/got it fast)

Also really like the configuration options mentioned above (e.g. size of document being sync'ed), and possibly add the amount of time before its considered "slow" e.g. 2 sec vs 1 sec vs 30 seconds)

I could see this type of test being used in automated testing (no human in loop, on physical devices), as well as a quick "network health check" to be run by a human.

[ajfabbri]

Tools like this are a good investment, especially as we scale the company.

One thought: the "healthy" latency threshold likely needs to be adaptive (best) or configurable (good) since it will vary based on mesh scale, traffic, and transports. This graph shows average latency versus number of peers in the mesh, based on a similar type of instrumentation:

[skylerjokiel]

@okdistribute , are you thinking of this as a stand alone app that users can run along side their native app to test the speed of Ditto outside of their app.

The goal is to help diagnose performance issues with Ditto?

[okdistribute]

@skylerjokiel this is part of the DittoAndroidTools and DittoSwiftTools, so it would need to be integrated into the existing app by customers using the package manager (SPM or Maven). This is a very important question because many of these devices are MDM-controlled and you cannot just install external apps on them.

[kitplummer]

I also think this would be a useful feature of the dto CLI that I'm poking at (stuck in implementation of DQL INSERTs rn). But if you could start it in SpeedTest mode, you could plop it on the network under test from a lappy or rpi appliance-y thing too.