Aquilon doc: add documentation on configuring cluster #259

jouvin · 2018-10-30T15:30:08Z

No description provided.

jouvin · 2018-10-30T16:25:01Z

Failing test is caused by the fix of an invalid link in an old post... This post is then checked and full of "spelling mistakes"...

_aquilon/technical_details.md

_aquilon/management.md

ned21 · 2018-10-31T16:37:38Z

_aquilon/technical_details.md

@@ -124,6 +124,10 @@ thresholds associated with the cluster, like the minimum or maximum number of ho
 any time. Clusters can also be used to describe a HA cluster. Note that Aquilon is not a replacement for the
 cluster middleware: it just allows to represent a group of machines managed by such a middleware.

+Clusters, like hosts, have an archetype and personality attached. It allows to ensure that all hosts that are


I would say:

Clusters, like hosts, have an archetype and personality attached. It allows for cluster-wide configuration to be applied by including a new file after the host personality but before archetype/final. That file is $host-archetype/$cluster-archetype/$cluster-personality/config (relative to the LOADPATH).

Typically relative to the root of your template-king domain the file is:

$host-archetype/features/$host-archetype/$cluster-archetype/$cluster-personality/config

@ned21 are you sure you want to give these details. These templates are plenary templates. An Aquilon user should not try (and probably will not be able) to access them directly. He/she should use aq cat. At the end it works line for any other personality... What about the new wording?

I like the start of the paragraph that Nathan proposed:

Clusters, like hosts, have an archetype and personality attached. It allows for cluster-wide configuration to be applied by including a new file after the host personality

Not sure for the rest of the paragraph. If we decide to have it also, I would recommend clarifying the variable names (perhaps like in bash with ${cluster-personality} for instance, as when reading it for now I was not sure at first glance if it was ${cluster}-personality or the above)

@XaF I still don't understand... The last version of this PR has this sentence copy/pasted. The rest of the § proposed by @ned21 IMO is not appropriate and I phrased it differently.

Sorry, checking from my phone and github mobile version is not that clean to navigate. I'm good with the current version of that sentence.

@ned21 are you sure you want to give these details. These templates are plenary templates. An Aquilon user should not try (and probably will not be able) to access them directly. He/she should use aq cat.

These are not plenaries, they are files that must appear in template-king if you want the cluster personality to influence the host profile.

At the end it works line for any other personality...

Not really. I think this is the confusing thing about clusters: the cluster object is a separate PAN-generated object profile. The personality is the configuration of that cluster object. You can use PAN's $object:/path/ syntax to access attributes of that object from within the Host profile but the cluster object is an independent entity. The inclusion of a template file named after the cluster $archetype/$personality is what allows the cluster object to add configuration to the member hosts, but that's just a standard include hook and has no relation to the Cluster object.

(The development of this data model predates my employment at MS. :-)

What about the new wording?

The new wording is good but I am concerned that if the path I referenced isn't documented here, it's not documented anywhere.

@ned21 The inclusion logic you mention for cluster personalities is not what I'm seeing... at the end of a host template you have the cluster/$cluster_name/client' included (just before final.pan`) and it includes the cluster personality template if it exists. There is no direct reference to the cluster object.

hmm.. This may be LOADPATH dependent. I've just spotted that my original filename reference version is perhaps slightly incorrect. I believe what gets included is: features/$cluster-archetype/$cluster-personality/config which traditionally means that must appear under the host's archetype because the cluster archetype is not added to the host's LOADPATH. i.e. the full path to that is:

$host-archetype/features/$host-archetype/$cluster-archetype/$cluster-personality/config

This is not the same file as:

$cluster-archetype/personality/$cluster-personality/config

jouvin · 2018-11-02T12:09:21Z

As for me, ready for merging. Should cover the main operations related to the creation of a cluster.

ned21 · 2018-11-02T22:43:56Z

_aquilon/technical_details.md

@@ -124,6 +124,10 @@ thresholds associated with the cluster, like the minimum or maximum number of ho
 any time. Clusters can also be used to describe a HA cluster. Note that Aquilon is not a replacement for the
 cluster middleware: it just allows to represent a group of machines managed by such a middleware.

+Clusters, like hosts, have an archetype and personality attached. It allows to ensure that all hosts that are


@ned21 are you sure you want to give these details. These templates are plenary templates. An Aquilon user should not try (and probably will not be able) to access them directly. He/she should use aq cat.

These are not plenaries, they are files that must appear in template-king if you want the cluster personality to influence the host profile.

At the end it works line for any other personality...

Not really. I think this is the confusing thing about clusters: the cluster object is a separate PAN-generated object profile. The personality is the configuration of that cluster object. You can use PAN's $object:/path/ syntax to access attributes of that object from within the Host profile but the cluster object is an independent entity. The inclusion of a template file named after the cluster $archetype/$personality is what allows the cluster object to add configuration to the member hosts, but that's just a standard include hook and has no relation to the Cluster object.

(The development of this data model predates my employment at MS. :-)

What about the new wording?

The new wording is good but I am concerned that if the path I referenced isn't documented here, it's not documented anywhere.

ned21 · 2018-11-02T22:43:59Z

_aquilon/technical_details.md

 thresholds associated with the cluster, like the minimum or maximum number of hosts that must be running at
 any time. Clusters can also be used to describe a HA cluster. Note that Aquilon is not a replacement for the
 cluster middleware: it just allows to represent a group of machines managed by such a middleware.

+Clusters, like hosts, have an archetype and personality attached. It allows to ensure that all hosts that are
+part of the cluster receive the same configuration. The cluster personality is added to the host, in addition


The cluster personality is added to the host, in addition to the host personality.

To me this wording implies the cluster personality is "merged to" the host personality which is not strictly true. I am not sure if I am just nitpicking the wording here or if the model is just so complex I haven't managed to explain it properly yet. :-(

This was the reason I thought it was good to reference the file being include ($host-archetype/$cluster-archetype/$cluster-personality/config) because if that file doesn't exist compilation will continue, and so if you don't know that's the path you have to create, it's hard to figure out how to add cluster-driven configuration to the host profile--I doubt that file path is properly documented anywhere else.

jouvin · 2018-11-12T07:55:14Z

I update the text related to cluster archetypes and reconfiguration, based on quattor/aquilon#124 (comment).

I still need to check and figure out what needs to be said about LOADPATH-related issues mentionned by @ned21 in his last comments.

jouvin · 2018-11-13T07:58:36Z

@ned21 after looking more carefully at the inclusion code for the cluster personality, I see your point. I didn't check carefully enough the resulting profile to realise that the feature was not included (I haven't tried yet to deploy the cluster members). And, as the include is with if_exists, the missing template is silently ignored... I'll improve the doc on this point.

BTW, is there a good reason (probably yes!) to use if_exists here as it clearly makes easy to miss the point without any easy way to get an information about the error... What does mean exactly saying that a cluster member must have a given feature configured as part of the cluster personality and having some nodes where configuring this feature is just ignored (because the corresponding template is missing). If you think we should rediscuss this, I'll open an issue in aquilon repo which is more appropriate.

ned21 · 2018-11-17T12:42:13Z

BTW, is there a good reason (probably yes!) to use if_exists here as it clearly makes easy to miss the point without any easy way to get an information about the error.

I don't think so. A lot of the older PAN and Aquilon code from pre-2010 seems to favour if_exists and in practice it has led to a hard to debug errors and oddities. I don't believe it would be a problem for our site to replace it with a normal include.

jouvin · 2018-11-17T13:36:46Z

Good then! I'll submit a PR along these lines when I find some quiet time...

jouvin · 2022-03-23T22:24:13Z

I should try to deal with my very old backlog and finish this one... I'll try!

jouvin requested review from ned21, jrha, XaF and gombasg October 30, 2018 15:30

jouvin force-pushed the aq_clusters branch 2 times, most recently from 2f13253 to 46820f3 Compare October 30, 2018 16:08

jrha approved these changes Oct 30, 2018

View reviewed changes

This was referenced Oct 30, 2018

Add notes from UGent workshop #260

Merged

Doc menu: create an AII-related submenu #261

Merged

XaF reviewed Oct 31, 2018

View reviewed changes

_aquilon/technical_details.md Outdated Show resolved Hide resolved

jouvin force-pushed the aq_clusters branch 3 times, most recently from 08f4d06 to 262b832 Compare October 31, 2018 08:42

jouvin changed the title ~~Aquilon doc: add documentation on configuring cluster~~ [WIP] Aquilon doc: add documentation on configuring cluster Oct 31, 2018

jouvin force-pushed the aq_clusters branch from 262b832 to 15789e7 Compare October 31, 2018 10:24

jouvin mentioned this pull request Oct 31, 2018

Fix a wrong link to Quattor ReadTheDocs in an old post #262

Merged

jouvin force-pushed the aq_clusters branch from 15789e7 to 48cb4f0 Compare October 31, 2018 10:27

ned21 reviewed Oct 31, 2018

View reviewed changes

jouvin force-pushed the aq_clusters branch 4 times, most recently from 5f3e9d0 to 80ac303 Compare November 2, 2018 12:08

jouvin changed the title ~~[WIP] Aquilon doc: add documentation on configuring cluster~~ Aquilon doc: add documentation on configuring cluster Nov 2, 2018

jouvin changed the title ~~Aquilon doc: add documentation on configuring cluster~~ [WIP] Aquilon doc: add documentation on configuring cluster Nov 2, 2018

jouvin force-pushed the aq_clusters branch from 80ac303 to a466f8f Compare November 2, 2018 16:59

jouvin changed the title ~~[WIP] Aquilon doc: add documentation on configuring cluster~~ Aquilon doc: add documentation on configuring cluster Nov 2, 2018

jouvin force-pushed the aq_clusters branch from a466f8f to e13c6c9 Compare November 2, 2018 17:36

ned21 reviewed Nov 2, 2018

View reviewed changes

jouvin mentioned this pull request Nov 5, 2018

1.12.65: make the cluster object profile optional? quattor/aquilon#124

Open

Aquilon doc: add documentation on configuring cluster

f9dc0b2

jouvin force-pushed the aq_clusters branch from e13c6c9 to f9dc0b2 Compare November 12, 2018 07:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Aquilon doc: add documentation on configuring cluster #259

Aquilon doc: add documentation on configuring cluster #259

jouvin commented Oct 30, 2018

jouvin commented Oct 30, 2018

ned21 Oct 31, 2018

jouvin Nov 2, 2018 •

edited

XaF Nov 2, 2018 •

edited

jouvin Nov 2, 2018

XaF Nov 2, 2018

ned21 Nov 2, 2018

jouvin Nov 4, 2018

ned21 Nov 4, 2018

jouvin commented Nov 2, 2018

ned21 Nov 2, 2018

ned21 Nov 2, 2018

jouvin commented Nov 12, 2018

jouvin commented Nov 13, 2018

ned21 commented Nov 17, 2018 •

edited

jouvin commented Nov 17, 2018

jouvin commented Mar 23, 2022

Aquilon doc: add documentation on configuring cluster #259

Are you sure you want to change the base?

Aquilon doc: add documentation on configuring cluster #259

Conversation

jouvin commented Oct 30, 2018

jouvin commented Oct 30, 2018

Choose a reason for hiding this comment

jouvin Nov 2, 2018 • edited

Choose a reason for hiding this comment

XaF Nov 2, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jouvin commented Nov 2, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jouvin commented Nov 12, 2018

jouvin commented Nov 13, 2018

ned21 commented Nov 17, 2018 • edited

jouvin commented Nov 17, 2018

jouvin commented Mar 23, 2022

jouvin Nov 2, 2018 •

edited

XaF Nov 2, 2018 •

edited

ned21 commented Nov 17, 2018 •

edited