Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LINUXCN-18 Update Linux CN to Debian 12 and LXD 5 #20

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

travispaul
Copy link

@travispaul travispaul commented Nov 13, 2023

Distribution tarballs (Node, ZFS, and Python) and build artifacts (ZFS, Python) are now being cached which speeds up build times.

Debian packages and npm modules can also optionally be retrieved from a cache using either of the following options which makes the build resilient to network or upstream service disruption:

-a URL Tested using apt-cache-ng
-n URL Tested using verdaccio

Testing requirements

The following requirements will be available in the next Triton release ("E.D.I.T.H.") until then one needs to meet the following:

  • If booting via UEFI, the CN must be booted with the updated iPXE bits from LINUXCN-21 a USB key with this headnode build will work.
  • The headnode (specifically the joysetup.sh) needs the changes from LINUXCN-25
  • A Linux CN platform image with the above changes such as one of these: platform, usb, iso

See LINUXCN-18 for more testing notes.

Copy link

@danmcd danmcd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good testing notes!

Also print size and location of build artifacts
@Smithx10
Copy link

I was able to build LINUXCN-18, and add it to our Triton DC cleanly after updating the appropriate SDC services / agents.

tools/debian-live Show resolved Hide resolved
Comment on lines 421 to 452
function build_python2 {

fetch_extract_distfile "$python_distfile"

if [[ -d "$buildcache/Python-$python_ver" ]]; then
cp -a "$buildcache/Python-$python_ver" "$root/tmp"
chroot "$root" env PYTHON_VER="$python_ver" bash -c '
cd "/tmp/Python-$PYTHON_VER"
make install
'
return 0
fi

rm -f /tmp/zfs-modules-*.deb
chroot "$root" env PYTHON_VER=$python_ver bash -c '
cd "/tmp/Python-$PYTHON_VER"
./configure
make install
'
cp -a "$root/tmp/Python-$python_ver" "$buildcache"
}

function remove_python2 {
# There is no uninstall target in the Makefile
rm -rf "$root/tmp/Python-$python_ver" \
"$root/usr/local/bin/python2" \
"$root/usr/local/lib/python2.7" \
"$root/usr/local/bin/smtpd.py" \
"$root/usr/local/bin/idle" \
"$root/usr/local/bin/pydoc" \
"$root/usr/local/bin/2to3" \
"$root/usr/local/share/man/man1/python2"
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with this approach for now, but I wonder how much time this adds to the build. We might want to look at a way to cache py2.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has a noticeable impact on build times, but it is currently being cached locally (along with ZFS builds.) Caching both of those cut the build time from ~35mins to ~8mins for me.

tools/debian-live Show resolved Hide resolved
Comment on lines +795 to +800
apt-get install -y systemd-resolved
sed -i -e "s/^#DNSSEC=.*/DNSSEC=false/" /etc/systemd/resolved.conf

# This service will attempt to parse the ip parameter on the kernel
# command line and fail, it is not needed for Triton
rm -f /usr/lib/systemd/system/systemd-network-generator.service
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we get by without this?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm open to other options if you have any, I've tried quite a few things this was the only working solution I found.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean systemd-resolved itself. Can it operate without that package?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the moment the PI cannot operate without it.

I tested removing it (to remind myself) and I can build a PI without it but the CN will be unable to resolve triton service names, the first symptom is that triton-vm-agent.service fails to start, unable to resolve vmapi.

Also, if I attempt to install it with along with the other $packages in the PI, the build will fail early as the chroot becomes unable to resolve DNS names and install packages which is why I added the install step in the postinstall step.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants