What's happening in Shadow 2023-06-30

[sporksmith]

This is part of a series of periodic updates of development in Shadow. This work is sponsored by the NSF. Previous update: 2023-03.

We've merged 175 non-dependabot pull requests and closed 29 issues since our previous update.

Release status

We've released Shadow 3.0.0! This release includes some quality of life improvements, a substantial performance improvement, and as usual some bug fixes and support for additional syscalls. As the major version bump indicates, there are some breaking changes in this release, but most simulation setups can be migrated with only minor changes; see the release notes for details.

Shadow in The Tor Project

We've continued to help The Tor Project use Shadow to test changes to tor. In addition to ongoing simulations to tune network parameters, simulations helped identify a possible segfault in the upcoming proof-of-work feature.

We've also developed some workarounds allowing asan-compiled binaries to run under Shadow, and have used them to check tor for other memory issues. This doesn't work in mainline Shadow yet; there is still some work to address these problems in a "mergeable" way.

Shadow is also in the continuous integration testing for arti. We are in the process of adding testing for arti's upcoming onion-service-client feature: https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/1292.

Current work

Some current development directions are:

• Rewriting Shadow's TCP stack in Rust, while improving its maintainability and correctness.

• Improving robustness of Shadow's "shim" LD_PRELOAD library by migrating it from C to no_std Rust to avoid conflicts with libc state. This is one of the changes needed to support asan-compiled applications.

• Adding support for spawning processes (fork and exec).

• Continuing migration of remaining C code to Rust.

Happy simulating!
The Shadow team

[akumar1512]

Thanks for the constant updates on what's next with Shadow. I have been following the changes planned for spawning processes and was wondering if there is any timeline for the release with the support for fork and exec. Thanks!

[sporksmith]

It's a bit hard to say. It's currently my ~top priority, BUT there's a fair bit of prep work to do, and likely more problems to be solved as I get deeper into it.

Our current funding runs out in September; I'm hoping to have some version of this working before then.

[akumar1512]

Thanks for this info! Looking forward.

[sporksmith]

This is now available @ HEAD. We'll be doing a release in the next week or two :)

[akumar1512]

Thanks @sporksmith for the update!