How To Maintain RubyCI Servers

This page is to explain how to maintain the servers running on RubyCI. We write the content on this page, considering the balance between the security of the servers and scaling the knowledge.

Technical architecture

Each server listed in RubyCI runs ruby/chkbuild program start-rubyci regularly on cron by chkbuild user. The program is to build and test Rubies (ruby-master, and the last 3 stable ruby-X.Y Rubies), then upload the result data to the AWS S3. The Ruby committers can log the servers in via SSH, and debug issues.

Setup a new server

If you have a server, and you want to use it in RubyCI, it's possible to add it to the RubyCI. If you are a Ruby committer registered in the system of the ruby-lang.org, you may ask it on the ruby-lang Slack. Otherwise, you can ask it by opening the issue ticket on Ruby Issue Tracker.

Your server needs to be logged in via SSH.

Decide a subdomain name for your server, discussing with a maintainer in charge of this topic. Then the maintainer

Then they makes all the commits to log the server in via SSH. Then the maintainer would setup the server for Ruby project by the common recipe files. And after that, all the Ruby committers can log the server in via SSH.

Check "chkbuild" user's crontab by crontab -l, and set up it by crontab -e. You can check other servers chkbuild user's crontab as a reference.

Below is a typical crontab setting. The AWS_* environment variables are required to upload the test result data to AWS S3. The chkbuild user uses a Ruby on rbenv. So, you need to set PATH to use Ruby on rbenv on cron. Run the env | grep ^PATH= on chkbuild user, and copy and paste it to the crontab.

$ crontab -l

AWS_*=<masked>
...
RUBYCI_NICKNAME=<same with the subdomain>
PATH=<masked>
0 * * * * cd ~/chkbuild && git pull origin master && start-rubyci && rm -rf tmp/build

$ id -un
chkbuild

$ env | grep ^PATH=

Run chkbuild start-rubyci manually

It's a good practice for you to run the start-rubyci manually to test it before running it on cron. The command is like this. You can use the DISABLE_S3_UPLOAD=1 not to upload the test result data to S3 in the test.

$ id -un
chkbuild

$ cd ~/chkbuild

$ git remote -v
origin	https://github.com/ruby/chkbuild (fetch)
origin	https://github.com/ruby/chkbuild (push)

$ git pull origin master

$ DISABLE_S3_UPLOAD=1 nohup ruby start-rubyci >& /tmp/rubyci.log

The program log is stored to the ~chkbuild/tmp/build/<date-time-directory>/log.

If you see the following error in the log file, the error means the program ruby start-rubyci fails to upload the test result data to AWS S3 due to a lack of the AWS credential information. In this case, you may run with the AWS_* environment variables set in crontab or run with DISABLE_S3_UPLOAD=1.

~chkbuild/tmp/build/<date-time-directory>/log

...
== end2 # 2023-09-12T07:13:05-04:00
uploading '/home/chkbuild/chkbuild/tmp/public_html/ruby-master/log/20230911T130004Z.diff.txt.gz' to ruby01/ruby-master/log/20230911T130004Z.diff.txt.gz...
#<Aws::Errors::MissingCredentialsError: unable to sign request without credentials set>

Make your server's result visible on RubyCI

As a default, your server is invisible on the RubyCI page. Ask the maintainer to make the server visible on the page.