This documentation comes in addition to two documents delivered in source tree:
- documentation/Puppet_Automate_update_in_NIM_environment_v0.9.pdf
- documentation/Puppet_Automate_vios_update_in_NIM_environment_v0.9.pdf
- Description
- Setup - The basics of getting started with aixautomation
- Usage - Configuration options and additional functionality
- Reference - An under-the-hood peek at what the module is doing and how
- Limitations - OS compatibility, etc.
- Development - Guide for contributing to the module
This 'Puppet AIX and VIOS Automation' module has been developed against Puppet 5.3.3, and
has been tested against Puppet 5.3.3 and Puppet 5.3.5.
This 'Puppet AIX and VIOS Automation' module runs by using the 'puppet apply' command,
therefore this is not the canonical way of using Puppet: no Puppet master,
no node executing Puppet agent. With 'Puppet AIX and VIOS Automation' module,
a unique Puppet Agent needs to run on the host where the NIM server runs.
This 'Puppet AIX and VIOS Automation' module enables automation of software maintenance
operations (updates of AIX or VIOS levels, updates of eFix) on a list of
AIX LPARs and AIX VIOS, by running this module on a NIM server managing
these LPARs and VIOS.
Operations desired by the user are declared into a manifest file, using Puppet
configuration language called DSL (Domain Specific Language), and using
ad hoc syntax explained below. This syntax uses specific custom types that
have been implemented (with their related providers) for 'Puppet AIX and VIOS
Automation' module, to control necessary operations to automatically perform
AIX updates and patch management, and VIOS updates. These new custom types
enable to declare resources in manifest file. These types 'download',
'patchmngt', 'fix', and 'vios' are described hereafter in detail.
AIX updates and patch management operations performed by this 'AIX and VIOS
Automation Puppet module' are shortly described now.
Firstly as far as AIX LPARs are concerned:
- Necessary AIX level updates are automatically downloaded from FixCentral
using 'suma' features. Downloaded updates are locally kept and shared
between LPARs if possible.
- Downloaded updates can then be automatically applied on a list of LPARs,
through 'NIM push' operations, therefore everything (i.e. all
AIX Automation logic) runs from the NIM server, and nothing needs to be
installed on LPARs themselves.
- List of necessary eFix is computed (for each LPAR) and retrieved using
'flrtvc', downloaded, kept locally (in a local repository of the
system where this Puppet module runs), and they are kept in a way
they can be shared between LPARs. Each LPAR has its own list of eFix
to be applied, depending on its level and depending on the current list
of eFix already applied.
These eFix are then applied on each LPAR of the list (after being sorted by 'PACKAGING DATE': so that most recent ones are applied first). Moreover, before attempting the eFix installations, a 'fileset interlock' check removes from the list of eFix to be installed any eFix presenting a foreseen fileset interlock with any other more recent eFix.
This eFix apply operation is done by using 'NIM push' operations (therefore as already said without performing any operation on the LPARs themselves: without installing anything on these LPARs).
Secondly as far as AIX VIOS are concerned:
- VIOS updates can be automatically applied on a list of VIOS,
by using 'NIM push' operations. As for LPARs updates, everything (i.e. all
AIX Automation logic) runs from the NIM server, and nothing needs to be
installed on VIOS themselves.
To bring trust in VIOS update operation, a mandatory copy of rootvg is taken before performing update, and AIXAutomation logic takes into account the potential mirroring of rootvg, and possible SSP cluster membership. See more explanations below.
This 'AIX and VIOS Automation' module needs to run a the NIM server which manages LPARs
or VIOS to be updated, therefore Puppet needs to be installed and run on this NIM server.
This 'AIX and VIOS Automation' module requires download access to FixCentral, and
http/https/ftp access to eFix download server.
First of all, you'll need to install Puppet on your NIM server.
To run this 'AIX and VIOS Automation' module, you only need Puppet running
in a standalone way.
Download Puppet 5.3 from https://puppet.com/download-puppet-enterprise:
- for AIX 7.2, 7.1: https://s3.amazonaws.com/puppet-agents/2017.3/puppet-agent/5.3.5/repos/aix/7.1/PC1/ppc/puppet-agent-5.3.5-1.aix7.1.ppc.rpm
- for AIX 6.1: https://s3.amazonaws.com/puppet-agents/2017.3/puppet-agent/5.3.5/repos/aix/6.1/PC1/ppc/puppet-agent-5.3.5-1.aix6.1.ppc.rpm
Puppet rpm is installed using rpm -i command line.
After Puppet installation, path to puppet is /opt/puppetlabs/bin/puppet.
Please note that Puppet comes with its own ruby, you'll find it here after Puppet installation:
"/opt/puppetlabs/puppet/bin/ruby -version" returns "ruby 2.4.2p198 (2017-09-14 revision 59899)"
'AIX and VIOS Automation' module needs to be installed into
/etc/puppetlabs/code/environments/production/modules, which is the install
directory referred as INST_DIR below.
You can do this by performing:
- git clone https://github.com/aixoss/aix-puppet.git
into INST_DIR repository. This creates INST_DIR/aixautomation directory, which is referred as AIX_AUTOMATION_DIR below.
All relative paths below are related to this AIX_AUTOMATION_DIR directory.
All 'AIX and VIOS Automation' module setups are done through the './manifests/init.pp' file
using DSL.
As a prerequisites of 'Puppet AIX and VIOS Automation' module, NIM configuration between
NIM server (on which this Puppet module runs), and the LPARs (on which
software maintenance operations are performed) needs to be properly set:
all LPARs which can either not be accessible through a simple
'ping -c1 -w5 ' command or through a simple 'c_rsh' command will be
excluded from the list of targets on which './manifests/init.pp' will be
applied.
List of LPARs on which rules can be applied is retrieved through NIM server by
getting list of standalones. As a matter of fact, only standalones really used
in './manifests/init.pp' are tested (see 'Facters' section below).
This module requires available disk space to store updates downloaded from
FixCentral, and to store downloaded eFix. By default downloads are performed
into '/tmp', but a more appropriate file system definitely needs to be chosen
and then set into './manifests/init.pp' ('root' attribute of 'download' custom
type for AIX updates, and 'root' attribute of 'fix' custom types for eFix).
File system on which downloads are performed is automatically increased
(100 MB each time) if necessary (if system allows).
This module will perform AIX software updates of your systems, and install
(or remove) of eFix.
Manifest './manifests/init.pp' delivered contains one suma-download
As far as 'update' operations are concerned:
You can perform download operations from FixCentral (suma-downloads) and
create a NIM lpp_source resource with results of downloads in a first separate
step, then in a second step you can update LPARs using the NIM lpp_source
resource which was downloaded.
As far as 'eFix' operations are concerned:
You can perform all preparation steps without applying the eFix and see
the results.
You can choose to apply eFix later on.
You can remove all eFix installed if necessary.
Some commented samples are provided in './examples/init.pp', and should be used as a starting point, to customize './manifests/init.pp' which is the manifest file used by the 'puppet apply' command.
The "Puppet.debug('message')" method is used in Puppet framework and as well in
'AIX and VIOS Automation' module and debug messages can only be seen with
the use of Puppet "--debug" flag on the command line.
All other levels ("info", "warning", "error") are displayed without
condition.
These Puppet log messages are displayed on console output.
You can redirect Puppet framework outputs to one file with the use of
this option:
"--logdest=/etc/puppetlabs/code/environments/production/modules/output/logs/logfile.txt"
on the command line.
AIX Automation logs (and only AIX Automation log, and not Puppet log) are
generated into ./output/logs/PuppetAIX Automation.log.
Up to 12 rotation log files of one 1 MB are kept:
./output/logs/PuppetAIX Automation.log.0 to
./output/logs/PuppetAIX Automation.log.12
These logs does not depend from Puppet --debug flag on the command line, and
you'll get them in any case.
As said already, 'AIX and VIOS Automation' module is installed into
/etc/puppetlabs/code/environments/production/modules (called INST_DIR):
/etc/puppetlabs/code/environments/production/modules/aixautomation
(called AIXAUTOMATION_DIR),
and all relative paths in this README are relative to this AIXAUTOMATION_DIR
directory.
The 'AIX and VIOS Automation' module generates all outputs under this directory
(under ./output), except downloads of updates and eFixes which are performed
under 'root' directories mentioned into './manifests/init.pp' file,
(ou have one 'root' attribute for 'download' and one 'root' directory for
'fix').
You can test './manifests/init.pp' manifest by using following command lines:
/opt/puppetlabs/bin/puppet apply \
--noop --modulepath=/etc/puppetlabs/code/environments/production/modules
-e "include aixautomation"
or apply it without debug messages:
/opt/puppetlabs/bin/puppet apply
--modulepath=/etc/puppetlabs/code/environments/production/modules
-e "include aixautomation"
or apply it with debug messages:
/opt/puppetlabs/bin/puppet apply --debug \
--modulepath=/etc/puppetlabs/code/environments/production/modules
-e "include aixautomation"
or apply it with debug message and Puppet logs into a file:
/opt/puppetlabs/bin/puppet apply \
--logdest=/etc/puppetlabs/code/environments/production/modules/output/logs/PuppetApply.log
--debug --modulepath=/etc/puppetlabs/code/environments/production/modules
-e "include aixautomation"
Please note that if you use "--logdest" parameter, you won't see any
output on the command line as everything is redirected to log file.
Specific AIX Automation facters collect the necessary data enabling
'AIX and VIOS Automation' module to run:
- props: to have shared configuration attributes.
- applied_manifest: to read, parse and display manifests/init.pp, so that
we can have into log file the exact contents of manifests/init.pp used at
runtime: this is an help for debugguing.
Moreover the manifests/init.pp is parsed so that:
-- the set of 'targets' really used in manifests/init.pp is computed,
this is used when running 'standalones' facter (see below
'standalones' facter) to restrict computing of this facter to the
sole list of standalones used in manifests/init.pp.
-- the set of 'vios' really used in manifests/init.pp is computed,
this is used when running 'vios' facter (see below
'vios' facter) to restrict computing of this facter to the
sole list of vios used in manifests/init.pp.
- servicepacks: you'll find results on this factor into
./output/facter/sp_per_tl.yml file. If this file does not exist when
"Puppet apply" is launched, it is computed by automatically downloading
Suma metadata files, all Suma metadata files are temporarily downloaded
under ./output/facter/suma, but are removed at the end, when new
./output/facter/sp_per_tl.yml file is generated.
Please note that as part of this computing, metadata downloads are
attempted for SP of each TL until repeatedly errors occur, meaning that
the last SP do not exist. Therefore it is normal to have errors when we
reach non-existing SP, this is the way which is used to go until last
existing SP.
- standalones: to gather data on all standalones managed by the NIM server.
As a matter of fact list of standalones is restricted to the ones really
used in manifests/init.pp as explained above (refer to 'applied_manifest'
facter). Results can be found into ./output/facter/standalones_kept.yml
file, and into ./output/facter/standalones_skipped.yml file. This
'standalones' facter takes some time at the beginning of AIX Automation
module, but is necessary to know the usable standalones.
- vios: to gather data on all VIOS managed by the NIM server.
As a matter of fact list of VIOS is restricted to the ones really
used in manifests/init.pp as explained above (refer to 'applied_manifest'
facter). Results can be found into ./output/facter/vios_skipped.yml file
and ./output/facter/vios_kept.yml file.
- hmc: to gather data on all HMC managed by the NIM server.
Four custom type and their providers constitute the 'AIX and VIOS Automation' module.
All custom types utilizations are documented into './examples/init.pp'
These four custom types constitute the vocabulary which can be used as part of
'Puppet AIX and VIOS Automation' module DSL into manifest file, to perform resources
declaration.
Associated provider is suma. Used for AIX LPARs software maintenance operations.
The aim of this provider is to provide download services using suma
functionality.
Suma requests are generated so that updates are downloaded from Fix Central.
"root" attribute is used as download directory: it should be an ad-hoc file
system dedicated to download updates, keep this file system separated from the
system to prevent saturation.
As a preliminary step, suma metadata are downloaded if ever they are not
locally described into './output/facter/sp_per_tl.yml' file: if this file does
not exist, it is generated by parsing suma metadata downloaded.
As a result, this file gives for each possible technical level the list of
available service packs.
As a reference, the './output/facter/sp_per_tl.yml.Oct_2018' delivered
file is the file computed from Metadata in October 19th 2018.
It is a good practice to regularly consider that the
'./output/facter/sp_per_tl.yml' is maybe not up-to-date, and therefore let
'suma' provider download metadata 'in live' and compute a more recent version
of './output/facter/sp_per_tl.yml'. To perform this, you can rename existing
'./output/facter/sp_per_tl.yml' to './output/facter/sp_per_tl.yml.saved' so
that this './output/facter/sp_per_tl.yml' is computed again. You should
perform this operation once in a while (every month or so).
As a normal step, various types of suma-downloads can be performed:
either "SP", or "TL", or "Latest":
- "SP" contains everything to update a system to a new Service Pack in a given
Technical Level.
- "TL" contains everything to update a system from a Technical Level to another
Technical Level.
- "Latest" contains everything to update a system to the last Service Pack of
a given Technical Level.
As a result of suma-download a lpp_source NIM resource is created. You can choose either to name it (with your own way of naming) and in that case you need to provide a 'lpp_source' attribute. Or you can let 'suma' provider name the lpp_source, in that case naming convention uses "PAA" as a prefix to identify automatically lpp_source NIM resource ("PAA" as a shortcut for "PuppetAixAutomation"'), then the of suma-download is used: "SP", "TL", "Latest", then the and the fields. As a example, you would have: "PAA_SP_7100-03_7100-03-07-1614" indicating a NIM resource to update from 7.3.0 to 7.3.7.1614 SP.
It is possible to perform only the suma 'preview' (and therefore not the suma 'download') by setting the 'to_step' attribute of the 'download' custom type to 'preview' (you'll find an example of this into the delivered './manifests/init.pp'). By default, this 'to_step' attribute is set to 'download', meaning that the suma-download is performed.
One new attribute 'force' enables to force downloads of everything from scratch, even if the results of downloads are already available. Set this 'force' attribute to 'yes' to force downloads again. In that case, 'preview' is done only if explicitly required.
By default 'force is set to 'no', meaning all previous downloads are kept and reused, this can spare a lot of time.
You'll find samples of suma-downloads into './examples/init.pp'.
- provider: this attribute is not mandatory, if mentioned it needs to contain name of the provider implementing the 'download' custom type, therefore the value needs to be: suma.
- ensure: to control flow of execution, this attribute can take two
values, either present or absent. By default present
is assumed.
If set the present, suma command will be attempted so that what is set
into attributes is performed: suma-downloads according to attributes
values.
If set to absent, result of previous present run is clean: NIM lpp_source resource is cleaned, and results of suma-downloads are removed from disks. - name: not a mandatory attribute if you have only one 'download' custom type in './manifests/init.pp'. Otherwise (multiple 'download' custom types), this attribute is necessary to uniquely identify the suma-download to be performed: you can have several 'download' custom types in './manifests/init.pp', each of them being uniquely identified.
- root: root directory to store results of suma-download. This root
directory should reside on a file system separated from the one which hosts
the system itself, and preferably different from /tmp. This root directory
needs to be large enough to contain system updates, and should be an
exportable file system, so that NIM can create NIM lpp_source resource and
perform a mount from the remote LPARs.
Note: it appears that this root directory needs to be either of type jfs or jfs2 (some other types of file system may work too). - type: type of suma-downloads. Three possible values: SP to suma-download a service pack, i.e. what allows you to update from a XXXX-YY technical level to any service packs of this technical level. TL to suma-download a technical level, so that you system gets updated to a new technical level. Latest to suma-download the last service pack of a given technical level. A last value is possible: Meta but is reserved for internal use.
- from: attribute used to launch suma-download command, indicating the starting level of desired updates. For example by indicating 7100-01, it means download contains everything to update from this 7100-01 technical level. Or by indicating 7100-01-02-1150, it means download contains everything to update from this 7100-01-02-1150 service pack, but specifying a SP into the from is equivalent to specifying the closest TL : 7100-01-02-1150 SP is equivalent to 7100-01 TL.
- to: attribute used to launch suma-download command, indicating the level of updates desired. For example by indicating 7100-03, it means download contains everything to update up to this 7100-03 technical level, or by indicating 7100-01-08-1642, it means download contains everything to update up to this 7100-01-08-1642 service pack.
- lpp_source: name of the NIM lpp_source resource created containing results of suma-downloads. If ever this lpp_source name is not provided, a naming convention is used to generate this name, by using a prefix ('PAA'), and by concatenating 'type', 'from', and 'to' attributes. The name of this lpp_source needs to be reused when performing updates of system (using 'nimpush' custom type).
- to_step: to control flow of execution up to a given step. Two possible steps: preview and download. By default, download is assumed. You can perform only preview suma-download, by setting this attribute to preview, in that case download is not done.
- force: to force suma-download again. Two possible values for this
attribute: yes and no. By default, no is assumed, meaning
that suma download won't be done again if expected results of suma-download
can be found under root directory.
If set to yes, this attribute can force a new suma-download (even if this suma-download was already done, and if it can be found under root directory) and can force the creation of a new NIM lpp_source resource (even if this NIM lpp_source resource already exists).
Associated provider is nimpush. Used for AIX LPARs software maintenance operations.
The aim of this provider is to provide software maintenance operations on a
list of LPARs using NIM push mode. Everything is performed from the NIM server
on which this 'Puppet AIX and VIOS Automation' module runs.
This NIM push mode can use suma-downloads performed by suma provider, as
preliminary step, by using into 'lpp_source' attribute the lpp_source which
was created by 'suma' provider.
Software maintenance operations include: install and updates.
You'll find samples of install and update into './examples/init.pp'.
You can perform operation synchronously or asynchronously depending on 'sync'
attribute.
You can perform preview only depending on 'preview' attribute.
Several modes of updates exist: apply, commit, reject. See several
documented examples in './examples/init.pp'.
- provider: this attribute is not mandatory, if mentioned it needs to contain name of the provider implementing the 'patchmngt' custom type, therefore the value needs to be: nimpush.
- ensure: to control flow of execution, this attribute can take two
values, either present or absent. By default present
is assumed. If set the present, NIM push operation will be attempted
so that what is set into attributes is performed.
If set to absent, missing . - name: not a mandatory attribute if you have only one 'patchmngt'
custom type in './manifests/init.pp'. Otherwise (multiple 'patchmngt'
custom types), this attribute is necessary to uniquely identify the NIM
push operation to be performed: you can have several 'patchmngt' custom
types in './manifests/init.pp', each of them being uniquely identified.
- action: action to be performed. This attribute can take several
values: install, update, reboot, status.
By default, status is assumed. install action enables
installation (or un-installation depending on the ensure
value) of a lpp_source, update action enables update of a system
(installation of a service pack or installation of a technical level),
reboot action enables to launch reboot of LPARs.
status action displays version level and eFix information
related to LPARs.
- mode: mode of update. By default update is performed in apply mode, meaning the updates are only applied, mode is set to apply. It is possible to commit the updates, by setting this mode attribute to commit. Or you can set mode to reject, (but in that case ensure needs to be set to absent) to reject all updates.
- lpp_source: name of the NIM lpp_source resource to be installed/un-installed or which needs to be used to performed system update. In case of update, this lpp_source is the one which was created by a previous 'download' custom type (results of suma-downloads).
- targets: names of the LPARs on which to perform action.
- sync: if action needs to be done synchronously or asynchronously. Two possible values for this attribute: yes and no. By default, no is assumed.
- preview: if only preview must be done. Two possible values for this attribute: yes and no. By default, no is assumed.
Associated provider is flrtvc. Used for AIX LPARs software maintenance operations.
The aim of this provider is to provide appropriate eFix installations using
flrtvc functionality to compute list of eFix to be installable, and NIM push
functionality to install this list of eFix.
"root" attribute is used as download directory: it should be an ad-hoc file
system dedicated to download eFix, keep this file system separated from the
system to prevent saturation.
List of appropriate eFix to be installed on a system is firstly computed by
'flrtvc', then checked against constraints and a short list of eFix of
installable eFix is computed, a NIM resource is then created and then applied,
so that eFix are installed.
These several steps necessary to achieve this eFix installation task, are
performed following this order: "installFlrtvc", "runFlrtvc", "parseFlrtvc",
"downloadFixes", "checkFixes", "buildResource", "installResource".
Step "runFlrtvc" launches '/usr/bin/flrtvc.ksh' command which in turn downloads
an 'apar.csv' file into the directory used to launch the command.
This file remains at the end, and should cause no worry.
Step "buildResource" creates a NIM lpp_source resource whose name follows these
naming conventions: prefix is 'PAA_FLRTVC_' and suffix is name of the target,
for example 'PAA_FLRTVC_quimby01' for NIM resource used to perform eFix
installation on quimby01 LPAR.
Executions can be stopped after any step, and this is controlled through the
'to_step' attribute into './manifests/init.pp'.
Each step persists its results into a yaml file, which can be found into
./output/flrtvc directory.
All yaml files can be reused between two executions, to spare time if ever the
external conditions have not changed. The use of yaml files is controlled
through the 'force' attribute which needs to be set to 'no' if you want to use
these yaml files and spare time. By default it is set to 'yes', meaning the
previously computed yaml files are not used.
eFix are sorted by 'Packaging Date' before being applied, i.e. most recent
first. It could occur that one particular eFix prevents another one (less
recent) from being installed if both eFixes touch the same file.
At the end of execution of this provider, you'll find into:
- ./output/flrtvc/[target]_StatusBeforeEfixInstall.yml: how were the [target]
LPAR before eFix installation.
- ./output/flrtvc/[target]_StatusAfterEfixInstall.yml: how are the [target]
LPARs after eFix installation.
- ./output/flrtvc/[target]_StatusBeforeEfixRemoval.yml: how were the [target]
LPAR before eFix de-installation.
- ./output/flrtvc/[target]_StatusAfterEfixRemoval.yml: how are the [target]
LPARs after eFix de-installation.
- provider: this attribute is not mandatory, if mentioned it needs to contain name of the provider implementing the 'fix' custom type, therefore the value needs to be: flrtvc.
- ensure: to control flow of execution, this attribute can take two values, either present or absent. By default present is assumed. If set the present, eFix are installed by running all steps. If set to absent, eFix are removed.
- name: not a mandatory attribute if you have only one 'fix'
custom type in './manifests/init.pp'. Otherwise (multiple 'fix' custom
types), this attribute is necessary to uniquely identify the flrtvc operation
to be performed: you can have several 'fix' custom types in
'./manifests/init.pp', each of them being uniquely identified.
- to_step: to control flow of execution up to a given step.
Possible values: installFlrtvc, runFlrtvc, parseFlrtvc,
downloadFixes, checkFixes, buildResource,
installResource. All these steps are performed in the given order above,
but you can interrupt execution after any given step, by using this
to_step attribute.
By default, installResource is assumed, meaning all steps are done. - targets: names of the LPARs on which to perform flrtvc steps.
- force : to force all flrtvc steps to be done again without exploiting any persisted information. Two possible values for this attribute: yes and no. By default, yes is assumed, meaning all previous results of any flrtvc runs (which are persisted into yaml files) are not kept, and computed again. If ever you want to spare time, and reuse previous flrtvc results, or if you want or run flrtvc step by step (refer to to_step attribute), you can keep previous results by setting this attribute to no.
- root: root directory to store results of flrtvc download. This root directory should reside on a file system separated from the one which hosts the system itself. This root directory needs to be large enough to contain eFix updates, and should be a exportable file system (jfs, jfs2, ...), so that NIM can create NIM lpp_source resource and perform a mount from the remote LPARs. By default /tmp is assumed.
- type: type of desired eFix. Possible values: hiper, sec, all. By default, all is assumed, meaning all possible eFix are installed.
Associated provider is viosmngt. Used for AIX VIOS software maintenance operations.
The aim of this provider is to provide software maintenance operations on a
list of VIOS (associated by pairs) using 'NIM push' operations,
and using 'c_rsh' operations.
Everything is performed from the NIM server on which this Puppet AIX Automation
module runs.
To be able to perform VIOS updates, this NIM push mode needs NIM lppsource
resources to exist prior from using AIX Automation. These NIM lppsource resources
have to be manually built before using AIXAutomation Puppet module
(if we compare with LPAR updates, in which building NIM lppsource resources is
one Puppet Aix Automation step, and can be automatically done).
Software maintenance operations only means in this case VIOS updates, no suma
download, no efix operations.
You'll find samples of manifest declarations in Puppet AIX Automation DSL language
for VIOS updates into './examples/init.pp'.
The VIOS update operation can be performed in 'preview' mode only depending on
'options' attribute.
Several modes of updates may exist in the future, but so far only one
is coded: "commit", using the 'update_options' attribute.
Puppet AIX Automation brings logic to verify VIOS update will be safely
performed. Rollback operation (i.e. coming back on previous VIOS version) is
always possible in case of any issue during update, as rootvg is copied before
performing VIOS operation.
- For each pair of VIOS, a health check should be performed to verify that both
VIOS of the pair share the same view of mappings on all types of storage and
virtual adapters: vSCSI, vFC, SEA, VNIC, etc.
This health check should always be run as a preliminary step, and in case of unsuccessful result, VIOS update cannot be done on any VIOS of the pair, but
this health check is not mandatory, and could be skipped, by omitting the 'health' keyword from the 'actions' attribute. Health check requires a preliminary step to run: 'health init' to retrieve VIOS UUID and CEC UUID, these info are persisted into ./aixautomation/output/vios/vios_kept_and_init.yml file. - Before performing a VIOS update, Puppet AIX Automation insures a copy of rootvg exists or is taken, to always offer possibility to come back on previous VIOS version if anything proves to be wrong during or before the end of VIOS update. The copy of rootvg is taken using 'alt_disk_copy' command: the use of this command results into an 'altinst_rootvg' vg on a dedicated disk. No VIOS update is run without an existing 'altinst_rootvg'. USer should carefully read the options controlling the 'altinst_rootvg' copy.
- If ever rootvg is mirrored, as it is not possible to perform 'alt_disk_copy' on a mirrored disk, the un-mirror operation of the rootvg disk is launched before performing it, and this mirroring is re-established after performing it. But these 'un-mirror' and 'mirror' operations won't be done without an explicit parameter into manifest file, so that this is an explicit choice from the user.
- Moreover as it is not possible to update a VIOS with is a UP SSP cluster node, if ever the VIOS proves to be part of a SSP cluster, this SSP cluster node is stopped just before performing update and restarted just after. But if VIOS are part of a SSP cluster, both nodes need to be UP or both nodes needs to be DOWN when AI Automation is launched. If ever both SSP cluster nodes are UP before launching the VIOS updates, each SSP node is stopped just before performing VIOS update operation on this very node, and restarted when this VIOS update operation is finished.
- VIOS update logs for each VIOS can be found into ./output/vios Each VIOS has its own log file: NIM_UPDATEIOS_output_update_<vios_name>.log
- VIOS update journal for each VIOS can be found into ./output/vios Each VIOS has its own journal file: journal_<vios_name>.log
- provider: this attribute is not mandatory, if mentioned it needs to contain name of the provider implementing the 'vios' custom type, therefore the value needs to be: viosmngt.
- ensure: either present or absent. Not used to control
flow of execution. Both values are possible to performed VIOS update.
By default it is set to present therefore this attribute does not
need to be set.
- name: not a mandatory attribute if you have only one 'vios'
custom type in './manifests/init.pp'. Otherwise (multiple 'vios'
custom types), this attribute is necessary to uniquely identify the VIOS
update operation to be performed: you can have several 'vios' custom types in './manifests/init.pp', each of them being uniquely identified. - actions: actions to be performed. This attribute is a
multi-valuated attribute. Several values can be chosen among the various
values: health, check, gc, unmirror,
save, autocommit, update. By default, no value
is assumed.
-- health action enables VIOS health check prior from performing update. If this health check is negative, VIOS update operation won't be pursued. This action should always be run, but is independent and can be skipped by omitting the 'health' keyword from 'actions' attribute.
-- check action enables mirror and SSP cluster checks.
-- gc action enables to garbagge-collect "old_rootvg" VG while looking for the best disk to host new altinst_rootvg. If ever gc is set, an existing "old_rootvg" VG wil be cleaned, and its disk made available to host the new copy of rootvg. By default, gc is not set. User should use this option cautiously.
-- save action enables to ensure that either a save copy of the rootvg is taken or that a save copy of rootvg already exists prior from performing update.
As already said, an 'altinst_rootvg' needs to exist on a given VIOS before VIOS update can be launched on this specific VIOS. Several ways of providing an 'altinst_rootvg' or insuring that this 'altinst_rootvg' exists are brought by this 'save' action. Carefully read below vios_altinst_rootvg and altinst_rootvg_force attributes.
-- autocommit action enables autocommit of 'applied only' updates to be done if necessary before performing VIOS update. Omitting this -- autocommit action from the list often prevents VIOS update from being done, as a VIOS update requires that all previous installations are committed. By setting this autocommit action, a first pass of NIM updateios is launched to commit all previous filesets, before the second pass of NIM updateios to perform the VIOS update itself.
-- unmirror action is necessary to un-mirror rootvg before performing the save operation (taking an 'altinst_rootvg'), and if un-mirror is done, mirror will be retablished afterwards. No 'un-mirror' and 'mirror' operations will be done
without this explicit keyword into 'actions' attribute.
-- update action to perform VIOS update operation itself. If ever this action is not set, Puppet AIX Automation performs only all other preparation steps which are listed into 'actions' attribute. - vios_pairs: pairs of VIOS on which to perform action.
- update_options: only possible value is commit. Potential future
values are install, commit, reject, cleanup,
remove.
- options: possible values are accept_licenses and preview.
options=>"accept_licenses" is necessary when actions
contains update otherwise update fails on 'accept_licences'
condition. Explicit agreement is needed to accept licences.
options=>"preview" can be set to perform VIOS update in preview mode only. Without options=>"preview", VIOS update is really performed. - vios_altinst_rootvg: this attribute enables to designate a disk to be used
to host 'altinst_rootvg'. A disk can be designated per VIOS following this given syntax :
"vios1=hdisk1, vios2=hdisk2" etc. The given disk will be used as a first choice,
if this disk fits the necessary conditions to host a copy of the rootvg. If
this disk cannot be used, the best disk possible will be chosen : a free disk large
enough to host rootvg, but the smallest disk possible (between all large enough) will
be taken to spare resources.
- altinst_rootvg_force: possible values are
"no", "yes", "reuse". Carefully read what follows:
altinst_rootvg_force=>"no" means that if ever it already exists an 'altinst_rootvg', this one won't be overriden. This is the safest option to preserve any existing 'altinst_rootvg'. The presence of an existing 'altinst_rootvg' will prevent a new 'altinst_rootvg' from being taken, and will prevent the overall process of VIOS update from continuing. With altinst_rootvg_force=>"no", if no 'altinst_rootvg' exists, a new one is created just before VIOS update.
altinst_rootvg_force=>"yes" means that if ever it already exists an 'altinst_rootvg', this one is going to be cleaned and removed, allowing a new one to be taken just before VIOS update. Use this option cautiously as an existing 'altinst_rootvg' will be overridden. If no 'altinst_rootvg' already exists, one 'altinst_rootvg' will be taken just before VIOS update.
altinst_rootvg_force="reuse" means that if ever it already exists an 'altinst_rootvg', this one is going to be kept and used, therefore no fresh 'altinst_rootvg' is taken as we consider the existing one as valuable. If no 'altinst_rootvg' exists, one 'altinst_rootvg' will be taken just before VIOS update. This option is safe as an existing 'altinst_rootvg' is not overridden. - vios_lpp_sources: this attribute enables to designate the NIM lppsource
resource to be used to perform VIOS update operation. A NIM lppsource resource
can be designated per vios following this given syntax :
"vios1=lpp_source1, vios2=lpp_source2" etc.
These given lppsources need to exist before launching Puppet AIXAutomation.
Building the NIM lppsource resource is necessary to update VIOS through NIM. This preliminary step needs to be manually done before launching Puppet AIX and VIOS automation module. Below is described one method to build such NIM resource, this method is provided as a sample, other ways of doing may exist.
- From FixCentral website https://www-945.ibm.com/support/fixcentral/, fill in form to indicate: Product Selector : PowerVM I/O Server Installed version: 2.2.6.23 (this value is provided as an example)
- You’ll be invited to download your fix, which can be downloaded using web interface. You may as well choose to download it using wget, for example through something like: o wget --no-check-certificate https://delivery04.dhe.ibm.com/hsb/iso/H86201476.iso
- When iso file is downloaded on your NIM server (often a large file of 4 or 5 GB), you have to build your NIM lppsource resource using something like: nim -o define -t lpp_source -a source=/export/vios/download/H19819269.iso -a server=master -a location=/export/vios/NIM/22631_iso -a packages=all vios_22623_22631 Preparing to copy install images (this will take several minutes)... /export/vios/NIM/22631_iso/bos.terminfo.dec.data.6.1.1.0.U ... /export/vios/NIM/22631_iso/wio.fcp.6.1.8.0.I Now checking for missing install images... All required install images have been found. This lpp_source is now ready.
Refer to TODO.md
Refer to TODO.md
Shell environments may cause errors with underlying (shell) system commands.
So, it may require an unset of the terminal environment before use.
(For example, if ever you have in your environment this variable set: 'ENV=/.kshrc',
this can lead to abnormal behaviour, not well explained. In that case, perform
'unset ENV' before retrying.)
Error log outputs are displayed in red. Some of these red output cause no incidence, and 'Puppet AIX and VIOS Automation' module*__* can live with without problem. Some of them are listed below.
The following output is frequently displayed when "/usr/sbin/emgr -dXv3 -e ABCD.epkg.Z"
command is run, but this does not prevent the parsing of this command output to be done.
Error: emgr: 0645-007 ATTENTION: /usr/bin/lslpp returned an unexpected result.
Error: emgr: 0645-044 Error processing installp fileset data.
Error: emgr: 0645-060 Unable to determine owning fileset
for /opt/pconsole/bin/pconsole_config.
Error: emgr: 0645-097 Error processing inventory data for file number 1.
Error: emgr: 0645-140 ATTENTION: emgr has issued 1 attention notice(s).
Error: Such notices may not indicate an immediate failure, but may require
Error: further attention. Please see the output above or the log for more details.
The following output is always displayed when "/usr/sbin/suma -x -a Action=Preview"
command is run, but you can pay no attention to this output.
Error: ****************************************
Error: Performing preview download.
Error: ****************************************
Refer to TODO.md file