Google Season of Docs Project 2021
Scylla is the real-time big-data NoSQL database. API-compatible with Apache Cassandra and Amazon DynamoDB. Scylla has both open source and commercial enterprise releases. As well, Scylla has a number of other supporting applications, such as Scylla Monitoring Stack, based on the open-source Grafana and Prometheus projects, as well as other various open-source drivers, connectors, and related utilities. The primary Scylla document repository is associated with the NoSQL database server code, but often other associated projects are updated at or around the same time that would also require documentation changes. For instance, if a new driver is updated or a new monitoring component is required to support a new server feature.
Originally, our documentation was in one repository: scylla-docs. This repository is not public, which made contributions impossible. As well, we couldn’t divide the content by product version so users were getting confused as to which feature was enabled for the version they were running.
To remedy this, we started moving selected content out of the scylla-docs repository. We started with small projects and saw that the benefits were huge. At this point, we are ready to take the “big plunge” and move all of our content from the scylla-docs repository to the matching open-source projects. This will officially make our documentation open-sourced.
The scylla-docs project contains thousands of files, so the migration effort is a big project. Once it's done, the documentation will be better organized, much easier to find, and will be maintainable by the community. The majority of the content will be moved to the main scylla project repository and this is the focus of our task.
We are not planning to implement any changes to our current processes in terms of how work is done and what tools we use. The major change is the location which will be described in more detail in the scope of the project.
The backbone of the docs is written in reStructuredText and the Documentation is compiled with Sphinx. Some of our upstream content is written in Markdown and Sphinx also supports Markdown as well.
We have templates and cheat sheets to aid in document creation and to make sure all documents are organized and the markup is tagged correctly.
We use the following tools for the following purposes:
| Tool | Purpose |
|---|---|
| Sphinx | Used to compile the HTML pages. This tool converts .rst and .md to .html. It also has directives and extensions to add more style and options for formatting the text. The CSS for the project is a sphinx theme. Our theme is not open-sourced. |
| Vale | Used as a linter to check for issues with style, spelling, and grammar |
| PiPy | Hosts the Sphinx theme and allows it to be distributed to the projects |
| GitHub | Hosts all our code and docs |
| GitHub Pages | Static site generator for the docs. |
| GitHub Actions | Builds and deploys the documentation to GitHub Pages every time there is new content. All the repositories deploy the documentation automatically except the repository scylla-docs, which is deployed manually. |
| Markdown | Content written upstream is often written in Markdown. You can use markdown to create content pages as an alternative to RestructuredText. |
| RestructuredText | Most of our content is written in RestructuredText. RestructuredText is a markdown-like language that can be converted to HTML with Sphinx. |
The migration project will:
- Audit the existing documentation, creating issues for any found problems
- Tag all topics with metadata allowing for better SEO
- Incorporate feedback from users/customers
- Move specific files from the scylla-docs repository to the scylla repository without any 404 errors. Not every file will be moved, only those tagged to be moved will be moved.
- Create redirection links so pages are not lost
- Test for broken links and repair them
- Test the contribution documentation and make sure the templates and linters work and that contributing to the project is easy to do and is not a cumbersome effort.
We estimate that this project will take approximately three months to complete. We are allotting for two junior writers. As we want to have time for our work as well, the internship will be part-time. We estimate about 80 hours per month per writer. The entire organization is supporting this effort and help will be available as we see this project as something that has very high priority. Our documentation and product manager have agreed to be actively involved as mentors, and more help is always available if needed.
When the migration is complete, the docs will reside side-by-side with the code that it describes and will allow us to adopt the docs-as-code philosophy. It will also allow us to make sure that any change in code will also require a change in docs as part of the definition of done. Additional changes will allow documentation to be automatically generated when the product deploys. In addition, having the documentation and code sources in the same repository allows us to include code snippets such as YAML configuration files directly from the source code as an include statement with no need to manage or upkeep the file. All of these changes will create documentation that is more useful to our users, while at the same time will create an atmosphere where contribution to the documentation is not only possible but welcome.
Going forward, we will focus our activities to make the contributor’s experience a positive one and they feel they are in a supportive and welcoming environment.
We would consider the project successful if, after the publication of the new documentation:
- The number of issues drops by 30%
- The bounce rate on the pages drops by 10%
- The number of contributors increases by 3%
- Feedback from customers (satisfaction survey) indicates an increase in satisfaction year/year
- The project is deployed and there are less than 5 broken links.
| Budget Item | Amount (USD) |
|---|---|
| Technical Writer #1 240 hours | $6,900 |
| Technical Writer #2 240 hours | $6,900 |
| SwagT-shirts, bag, stickers | $200 |
| Volunteer Stipends (2 @ $500 each) | $1,000 |
| Total | $15,000 |
ScyllaDB participated in Google Summer of Code in 2016, 2018, and 2020. The links describe our projects:
- https://www.scylladb.com/2018/02/21/seastar-google-summer-of-code/
- https://summerofcode.withgoogle.com/archive/2020/projects/5176655267495936/
- https://summerofcode.withgoogle.com/archive/2018/organizations/5132636779446272/
Laura Novich, our documentation manager is an instructor and mentor at a technical writing school and has supervised a large-scale documentation internship for technical writing students with GitLab. She teaches classes on GitHub, Agile, and API documentation. Laura presents at workshops and conferences all over the world.
Tzach Livyatan is our VP of Product who is responsible for the Product Department. The department includes product management, training, and documentation. He is very supportive of the docs and is the maintainer for the scylla-docs repository.
There are other supportive volunteers who will join in for reviews and other help. Read more about the Company and the Management team.
Scylla is the real-time big-data database. API-compatible with Apache Cassandra and Amazon DynamoDB, Scylla embraces a shared-nothing approach that increases throughput and storage capacity by as much as 10X. Comcast, Discord, Disney+ Hotstar, Grab, Medium, Starbucks, Ola Cabs, Samsung, IBM, Investing.com, and many more leading companies have adopted Scylla to realize order-of-magnitude performance improvements and reduce hardware costs. ScyllaDB was founded by the team responsible for the KVM hypervisor and is backed by Bessemer Venture Partners, Eight Roads Ventures, Innovation Endeavors, Wing Venture Capital, Qualcomm Ventures, TLV Partners, Magma Venture Partners, Western Digital Capital, and Samsung Ventures, and more. For more information: ScyllaDB.com .