Skip to content

DLPX-79631 Decide on a way to retain more of the Ubuntu revision information so that we can more easily determine what's in the package #269

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
sumedhbala-delphix merged 2 commits into from
Jun 28, 2023
Merged

DLPX-79631 Decide on a way to retain more of the Ubuntu revision information so that we can more easily determine what's in the package #269

sumedhbala-delphix merged 2 commits into from
Jun 28, 2023

Conversation

@sumedhbala-delphix
Copy link
Contributor

@sumedhbala-delphix sumedhbala-delphix commented Dec 13, 2022 •
edited
Loading

Modifying the package versioning to a format that security scans will recognize the upstream version.
I had to modify the version in the META file to satisfy zfs builds which seem to use it to create a directory.
I had to add --allow-downgrades since some of our packages were considered downgrades due to the versioning change.

Sample of packages before the changes:
delphix@ip-10-110-202-33:~ apt list |grep delphix
....
bcc-tools/now 0.22.0-1-delphix-2022.01.15.05 all [installed,local]
bpftrace-dbgsym/now 1.0.0-delphix-2022.01.08.17 amd64 [installed,local]
bpftrace/now 1.0.0-delphix-2022.01.08.17 amd64 [installed,local]
cloud-init/now 21.4-delphix-2022.01.08.17 all [installed,local]
....
After
delphix@ip-10-110-207-180:~$ apt list |grep delphix

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

bcc-tools/now 0.22.0-1+delphix.2023.06.27.16.13 all [installed,local]
bpftrace/now 1.0.0-1delphix.2023.06.27.16.49 amd64 [installed,local]
cloud-init/now 23.1.2-0ubuntu0~20.04.2+delphix.2023.06.27.16.13 all [installed,local]
connstat-module-5.15.0-1031-dx2023062716-24f86c0a3-aws/now 1.0.0-1delphix.2023.06.27.19.25 amd64 [installed,local]
connstat-util/now 1.0.0-1delphix.2023.06.27.19.25 amd64 [installed,local]
crash-python/now 1.0.0-1delphix.2023.06.27.16.13 amd64 [installed,local]
delphix-build-info/now 1.0.0-delphix-2023.06.27.22.00.57-0ba01512 all [installed,local]
delphix-entire-aws/now 13.0.0.0-snapshot.20230627214434603+jenkins-selfservice-appliance-build-develop-pre-push-597 all [installed,local]
delphix-kernel-5.15.0-1031-dx2023062716-24f86c0a3-aws/now 1.0.0-1delphix.2023.06.27.19.25 amd64 [installed,local]
delphix-masking/now 1.0.0-20230627T210409 amd64 [installed,local]
delphix-platform-aws/now 1.0.0-delphix.2023.06.27.16.13 amd64 [installed,local]
delphix-rust/now 1.0.0-1delphix.2023.06.27.16.13 amd64 [installed,local]
delphix-sso-app/now 2023.06.27.21 all [installed,local]
delphix-virtualization/now 2023.06.27.21 amd64 [installed,local]
delphix-zfs/now 2.1.99-1delphix.2023.06.27.19.25 amd64 [installed,local]
docker-python-image/now 1.0.0-1delphix.2023.06.27.16.13 all [installed,local]
drgn/now 1.0.0-1delphix.2023.06.27.16.49 amd64 [installed,local]
fluentd-gems/now 1.0.0-1delphix.2023.06.27.16.14 amd64 [installed,local]
gdb-python/now 1.0.0-1delphix.2023.06.27.16.13 amd64 [installed,local]
grub-common/now 2.04-1ubuntu26.17+delphix.2023.06.27.16.14 amd64 [installed,local]
grub-pc-bin/now 2.04-1ubuntu26.17+delphix.2023.06.27.16.14 amd64 [installed,local]
grub-pc/now 2.04-1ubuntu26.17+delphix.2023.06.27.16.14 amd64 [installed,local]
grub2-common/now 2.04-1ubuntu26.17+delphix.2023.06.27.16.14 amd64 [installed,local]
host-jdks/now 1.0.0-1delphix.2023.06.27.16.13 amd64 [installed,local]
kdump-tools/now 1:1.6.7-1ubuntu2.4+delphix.2023.06.27.16.14 amd64 [installed,local]
libbcc/now 0.22.0-1+delphix.2023.06.27.16.13 amd64 [installed,local]
libkdumpfile/now 0.3.0-1delphix.2023.06.27.16.14 amd64 [installed,local]
libnvpair1linux/now 2.1.99-1delphix.2023.06.27.19.25 amd64 [installed,local]
libuutil1linux/now 2.1.99-1delphix.2023.06.27.19.25 amd64 [installed,local]
libzfs2linux/now 2.1.99-1delphix.2023.06.27.19.25 amd64 [installed,local]
libzfslinux-dev/now 2.1.99-1delphix.2023.06.27.19.25 amd64 [installed,local]
libzpool2linux/now 2.1.99-1delphix.2023.06.27.19.25 amd64 [installed,local]
makedumpfile/now 1:1.6.7-1ubuntu2.4+delphix.2023.06.27.16.14 amd64 [installed,local]
nfs-common/now 1:1.3.4-2.5ubuntu3.4+delphix.2023.06.27.16.14 amd64 [installed,local]
nfs-kernel-server/now 1:1.3.4-2.5ubuntu3.4+delphix.2023.06.27.16.14 amd64 [installed,local]
performance-diagnostics/now 1.0.0-1delphix.2023.06.27.16.13 amd64 [installed,local]
ptools/now 0.2.0-1delphix.2023.06.27.16.13 amd64 [installed,local]
python3-bcc/now 0.22.0-1+delphix.2023.06.27.16.13 all [installed,local]
python3-rtslib-fb/now 2.1.71-0ubuntu1.1+delphix.2023.06.27.16.14 all [installed,local]
recovery-environment/now 1.0-1+delphix.2023.06.27.21.01 amd64 [installed,local]
savedump/now 0.1.0-1delphix.2023.06.27.16.13 amd64 [installed,local]
sdb/now 1.0.0-1delphix.2023.06.27.16.14 amd64 [installed,local]
targetcli-fb/now 1:2.1.51-0ubuntu1+delphix.2023.06.27.16.49 all [installed,local]
zfs-dbg/now 2.1.99-1delphix.2023.06.27.19.25 amd64 [installed,local]
zfs-headers-5.15.0-1031-dx2023062716-24f86c0a3-aws/now 2.1.99-1delphix.2023.06.27.19.25 amd64 [installed,local]
zfs-initramfs/now 2.1.99-1delphix.2023.06.27.19.25 all [installed,local]
zfs-modules-5.15.0-1031-dx2023062716-24f86c0a3-aws-dbg/now 2.1.99-1delphix.2023.06.27.19.25 amd64 [installed,local]
zfs-modules-5.15.0-1031-dx2023062716-24f86c0a3-aws/now 2.1.99-1delphix.2023.06.27.19.25 amd64 [installed,local]
zfs-test/now 2.1.99-1delphix.2023.06.27.19.25 amd64 [installed,local]
zfs-zed/now 2.1.99-1delphix.2023.06.27.19.25 amd64 [installed,local]
zfsutils-linux/now 2.1.99-1delphix.2023.06.27.19.25 amd64 [installed,local]

....

Example package: 2.04-1ubuntu26.16+delphix.2023.04.10.21.43
The version numbers in Debian are of the form
[epoch:]upstream_version[-debian_revision]
where

  • epoch is a single (generally small) unsigned integer, which is included to allow mistakes in the version numbers of older versions of a package. If omitted, the epoch is assumed to be zero.
  • upstream_version is usually the version number of the original source package from which the .deb file has been made. It is usually kept the same as the format used for the upstream source.
  • debian_revision specifies the version of the Debian package based on the upstream version. It is optional and is omitted in cases where a piece of software was written specifically to be a Debian package.

Example package: 1:2.1.51-0ubuntu1+delphix.2023.06.27.16.49
1 is epoch
2.1.51 is upstream_version
0ubuntu1+delphix.2023.06.27.16.49 is debian_revision

Testing:
Building all packages:http://selfservice.jenkins.delphix.com/job/appliance-build-orchestrator-pre-push/5946/ passed in ADDITIONAL_PACKAGES_TO_BUILD
QA confirmed that security scans recognize the package version.
Ran upgrade during a object storage stress run http://selfservice.jenkins.delphix.com/job/blackbox-self-service/61733/console + http://selfservice.jenkins.delphix.com/job/blackbox-self-service/61937/consoleFull and the stress run did not notice any traffic going down or change in VDB state. Only the stack went down for a few mins as part of the upgrade.

@sumedhbala-delphix sumedhbala-delphix force-pushed the dlpx/pr/sumedhbala-delphix/b51d83cd-f80e-406a-9c6e-ba755bb59090 branch from 6f6d3c6 to 8734e2b Compare December 13, 2022 20:02
@prakashsurya
Copy link
Contributor

prakashsurya commented Dec 15, 2022

All the packages get built now: http://selfservice.jenkins.delphix.com/job/linux-pkg/job/6.0/job/stage/job/build-kernel/job/pre-push/375/console

this is only building kernel related packages.. not all of the packages.. is this the correct link? I think you want a "build-package" run, with the parameter set to "all"

@sumedhbala-delphix
Copy link
Contributor Author

sumedhbala-delphix commented Dec 21, 2022

All the packages get built now: http://selfservice.jenkins.delphix.com/job/linux-pkg/job/6.0/job/stage/job/build-kernel/job/pre-push/375/console

this is only building kernel related packages.. not all of the packages.. is this the correct link? I think you want a "build-package" run, with the parameter set to "all"

Building all packages: http://selfservice.jenkins.delphix.com/job/appliance-build-orchestrator-pre-push/4009/. passed in ADDITIONAL_PACKAGES_TO_BUILD

@sumedhbala-delphix sumedhbala-delphix marked this pull request as ready for review December 21, 2022 21:04
@sebroy
Copy link
Contributor

sebroy commented Dec 21, 2022

@sumedhbala-delphix can you clarify (with examples) what the version format will look like with this PR in the PR description?

sebroy
sebroy requested changes Dec 21, 2022
View reviewed changes
@sumedhbala-delphix sumedhbala-delphix force-pushed the dlpx/pr/sumedhbala-delphix/b51d83cd-f80e-406a-9c6e-ba755bb59090 branch 4 times, most recently from 0e98cbf to d59b9b6 Compare April 14, 2023 18:41
@sumedhbala-delphix
Copy link
Contributor Author

sumedhbala-delphix commented Apr 14, 2023

@sumedhbala-delphix can you clarify (with examples) what the version format will look like with this PR in the PR description?

I have added a before and after in the description, have added what the version means giving an example

@grodr grodr changed the base branch from 6.0/stage to develop April 17, 2023 18:19
prakashsurya
prakashsurya approved these changes Apr 17, 2023
View reviewed changes
@sumedhbala-delphix
DLPX-79631 Decide on a way to retain more of the Ubuntu revision info…
b723ace 
…rmation so that we can more easily determine what's in the package
@sumedhbala-delphix
DLPX-79631 Decide on a way to retain more of the Ubuntu revision info…
147f03f 
…rmation so that we can more easily determine what's in the package

PR URL: https://www.github.com/delphix/linux-pkg/pull/269
@sumedhbala-delphix sumedhbala-delphix force-pushed the dlpx/pr/sumedhbala-delphix/b51d83cd-f80e-406a-9c6e-ba755bb59090 branch 2 times, most recently from 914402a to 147f03f Compare June 28, 2023 16:25
@sumedhbala-delphix sumedhbala-delphix merged commit 203e593 into develop Jun 28, 2023
@sumedhbala-delphix sumedhbala-delphix deleted the dlpx/pr/sumedhbala-delphix/b51d83cd-f80e-406a-9c6e-ba755bb59090 branch June 28, 2023 20:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@prakashsurya prakashsurya prakashsurya approved these changes

@sebroy sebroy sebroy approved these changes

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@sumedhbala-delphix @prakashsurya @sebroy