Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store roles in database #5134

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Store roles in database #5134

wants to merge 10 commits into from

Conversation

Al2Klimov
Copy link
Member

@Al2Klimov Al2Klimov commented Oct 4, 2023

TODO

  • upgrading SQL
  • Postgres
  • ORM
  • CRUD
    • pagination
    • search
  • actual usage
    • audit ok?
  • management via Ansible, Puppet, ... and via environment variables (Docker)
  • migration
  • installer?

@Al2Klimov Al2Klimov self-assigned this Oct 4, 2023
@cla-bot cla-bot bot added the cla/signed label Oct 4, 2023
Copy link
Member Author

@Al2Klimov Al2Klimov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lippserd Was letzte Schema? :)

schema/mysql.schema.sql Show resolved Hide resolved
schema/mysql.schema.sql Show resolved Hide resolved
schema/mysql.schema.sql Show resolved Hide resolved
schema/mysql.schema.sql Show resolved Hide resolved
schema/mysql.schema.sql Show resolved Hide resolved
schema/mysql.schema.sql Outdated Show resolved Hide resolved
schema/mysql.schema.sql Outdated Show resolved Hide resolved
schema/mysql.schema.sql Outdated Show resolved Hide resolved
schema/mysql.schema.sql Outdated Show resolved Hide resolved
schema/mysql.schema.sql Show resolved Hide resolved
Copy link
Member

@lippserd lippserd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please show sample queries of how roles of a sample user would be loaded. Also note that CTEs can be used.

I'm unsure of the icingaweb_permission and icingaweb_restriction tables. They seem to have zero to no benefit and only make things more complicated. What was your idea behind these tables?

Is ROW_FORMAT=DYNAMIC really needed everywhere? I don't think so.

Please increase all name columns to 256. The SAM-Account-Name attribute can be that long.

@Al2Klimov
Copy link
Member Author

Please show sample queries of how roles of a sample user would be loaded. Also note that CTEs can be used.

WITH RECURSIVE rl(id, parent_id, unrestricted) AS (
  SELECT id, parent_id, unrestricted FROM icingaweb_role
  WHERE all_users OR all_groups
-- I guess an index over (all_users, all_groups) would be nice, wouldn't it?
  OR id IN (SELECT role_id FROM icingaweb_role_user WHERE user_name=?)
-- Shall I swap icingaweb_role_user's PK's columns for this query by user_name?
  OR id IN (SELECT role_id FROM icingaweb_role_group WHERE group_name IN (?, ?, ?))
-- Shall I swap icingaweb_role_group's PK's columns for this query by group_name?
UNION
-- This even handles relationship cycles. 😎
  SELECT id, parent_id, unrestricted FROM icingaweb_role
  WHERE id IN (SELECT parent_id FROM rl)
)
SELECT id, unrestricted FROM rl;
SELECT name FROM icingaweb_permission WHERE id IN (
  SELECT permission_id FROM icingaweb_role_permission WHERE role_id IN (?, ?, ?)
)
SELECT (SELECT name FROM icingaweb_restriction WHERE id=irr.restriction_id) name, filter
FROM icingaweb_role_restriction irr WHERE role_id IN (?, ?, ?)

I'm unsure of the icingaweb_permission and icingaweb_restriction tables. They seem to have zero to no benefit and only make things more complicated. What was your idea behind these tables?

The same idea as behind Icinga DB's action_url and notes_url: deduplication of probably frequently used strings.

Is ROW_FORMAT=DYNAMIC really needed everywhere? I don't think so.

I thought this is our new guideline as Icinga DB uses this literally everywhere. (grep ENGINE=InnoDB schema/mysql/schema.sql |grep -v ROW_FORMAT=DYNAMIC)

Please increase all name columns to 256. The SAM-Account-Name attribute can be that long.

It's length is actually limited to 20:

And my length limit was just inspired from the existing username columns which are btw. 254 long to contain eMail addresses.

@Al2Klimov
Copy link
Member Author

Btw.:

Distros supporting CTEs

via MariaDB 10.2.1+

  • SUSE 12.5 👍
  • Debian 10 👍
  • Ubuntu 20.04 👍
  • RHEL 8 👎 (RHEL 7 EOL: 2024-06-30)

@Al2Klimov
Copy link
Member Author

Is ROW_FORMAT=DYNAMIC really needed everywhere? I don't think so.

Now as I see it: It IS literally everywhere, even the damn simple table icingaweb_user of this schema. Let's keep it for consistency.

@lippserd
Copy link
Member

lippserd commented Apr 9, 2024

Let's keep it for consistency.

No. It is not needed. And consistency is no reason to add it. New code should reflect how it's done properly and not how it was done ages ago.

@lippserd
Copy link
Member

lippserd commented Apr 9, 2024

Distros supporting CTEs

via MariaDB 10.2.1+

...

  • RHEL 8 👎 (RHEL 7 EOL: 2024-06-30)

Red Hat states something else.

@Al2Klimov
Copy link
Member Author

That's what I wanted to express, sorry for the misunderstanding. Btw. Ubuntu 20.04 also has MySQL 8 and RHEL 7 has PostgreSQL (almost forgot!) 9.2. v8.4 has CTEs.

@Al2Klimov
Copy link
Member Author

Is ROW_FORMAT=DYNAMIC really needed everywhere? I don't think so.

It seems the default, so no – it's indeed not needed.

@Al2Klimov
Copy link
Member Author

WITH RECURSIVE rl(id, parent_id, unrestricted) AS (
  SELECT id, parent_id, unrestricted FROM icingaweb_role
  WHERE id IN (SELECT role_id FROM icingaweb_role_user WHERE user_name IN ('*', ?))
  OR id IN (SELECT role_id FROM icingaweb_role_group WHERE group_name IN ('*', ?, ?, ?))
UNION -- This even handles relationship cycles. 😎
  SELECT id, parent_id, unrestricted FROM icingaweb_role
  WHERE id IN (SELECT parent_id FROM rl)
)
SELECT id, unrestricted FROM rl;

@Al2Klimov Al2Klimov force-pushed the roles2db branch 6 times, most recently from 279e6b7 to ae976a1 Compare April 12, 2024 13:12
@Al2Klimov
Copy link
Member Author

If this is an old schema,

CREATE TABLE `icingaweb_role`(
  `name`    varchar(64) COLLATE utf8_unicode_ci NOT NULL,
  `public`  tinyint(1) NOT NULL DEFAULT 0,
  `ctime`   timestamp NULL DEFAULT NULL,
  `mtime`   timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE `icingaweb_role_permission`(
  `role_name`   varchar(64) COLLATE utf8_unicode_ci NOT NULL,
  `name`        varchar(255) COLLATE utf8_unicode_ci NOT NULL,
  `ctime`       timestamp NULL DEFAULT NULL,
  `mtime`       timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`role_name`,`name`),
  CONSTRAINT `fk_icingaweb_role_permission_icingaweb_role` FOREIGN KEY (`role_name`) REFERENCES `icingaweb_role` (`name`)
    ON UPDATE CASCADE ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE `icingaweb_role_restriction`(
  `role_name`   varchar(64) COLLATE utf8_unicode_ci NOT NULL,
  `name`        varchar(255) COLLATE utf8_unicode_ci NOT NULL,
  `restriction` text NOT NULL,
  `ctime`       timestamp NULL DEFAULT NULL,
  `mtime`       timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`role_name`,`name`),
  CONSTRAINT `fk_icingaweb_role_restriction_icingaweb_role` FOREIGN KEY (`role_name`) REFERENCES `icingaweb_role` (`name`)
    ON UPDATE CASCADE ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE `icingaweb_user_role`(
  `username`  varchar(64) COLLATE utf8_unicode_ci NOT NULL,
  `role_name` varchar(64) COLLATE utf8_unicode_ci NOT NULL,
  `ctime`     timestamp NULL DEFAULT NULL,
  `mtime`     timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`username`,`role_name`),
  CONSTRAINT `fk_icingaweb_user_role_icingaweb_role` FOREIGN KEY (`role_name`) REFERENCES `icingaweb_role` (`name`)
    ON UPDATE CASCADE ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE `icingaweb_group_role`(
  `group_name`  varchar(64) COLLATE utf8_unicode_ci NOT NULL,
  `role_name`   varchar(64) COLLATE utf8_unicode_ci NOT NULL,
  `ctime`       timestamp NULL DEFAULT NULL,
  `mtime`       timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`group_name`,`role_name`),
  CONSTRAINT `fk_icingaweb_group_role_icingaweb_role` FOREIGN KEY (`role_name`) REFERENCES `icingaweb_role` (`name`)
    ON UPDATE CASCADE ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

we can

  1. Make room for the new tables
    • alter table icingaweb_role rename to icingaweb_role1;
    • alter table icingaweb_role_permission rename to icingaweb_role_permission1;
    • alter table icingaweb_role_restriction rename to icingaweb_role_restriction1;
  2. Upgrade the schema as usual
  3. Migrate the data
    • insert into icingaweb_role(name, ctime, mtime) select name, unix_timestamp(ctime)*1000, unix_timestamp(mtime)*1000 from icingaweb_role1;
    • insert into icingaweb_role_user(role_id, user_name) select r.id, ur.username from icingaweb_user_role ur inner join icingaweb_role r on r.name = ur.role_name;
    • insert into icingaweb_role_group(role_id, group_name) select r.id , gr.group_name from icingaweb_group_role gr inner join icingaweb_role r on r.name = gr.role_name;
    • insert into icingaweb_role_permission(role_id, permission, allowed) select r.id, p1.name, 'y' from icingaweb_role_permission1 p1 inner join icingaweb_role r on r.name = p1.role_name;
    • insert into icingaweb_role_restriction(role_id, restriction, filter) select r.id, r1.name, r1.restriction from icingaweb_role_restriction1 r1 inner join icingaweb_role r on r.name = r1.role_name;

@Al2Klimov Al2Klimov requested a review from lippserd April 18, 2024 09:13
@Al2Klimov
Copy link
Member Author

Al2Klimov commented Apr 18, 2024

  • management via Ansible, Puppet, ... and via environment variables (Docker)

Our Docker entry point already handles users in DB, so roles wouldn't be a problem. But the others already can't manage users or groups. Roles would be just yet another blind point (unless you let Ansible do INSERT). I think we need a more general strategy. Like a bunch of CLI commands (in the setup mod? or even a new mod "cfgmgmt"?) which manage stuff stored in the IW2 DB.

@Al2Klimov Al2Klimov marked this pull request as ready for review April 18, 2024 09:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants