Skip to content

Commit

Permalink
Pick up latest mq-golang fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
ibmmqmet committed Nov 8, 2023
1 parent bf576da commit d43cd78
Show file tree
Hide file tree
Showing 16 changed files with 66 additions and 557 deletions.
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# Changelog
Newest updates are at the top of this file.

### Nov 08 2023 (v5.5.2)
* Pick up latest mq-golang fixes

### Oct 19 2023 (v5.5.1)
* Update to MQ 9.3.4
* Fix Prometheus label cardinality when reporting disconnect (#245)
Expand Down
12 changes: 6 additions & 6 deletions TUNING.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

If you have a large queue manager - perhaps several thousands of queues - then a lot of data could be produced for
monitoring those queues. Some default configuration options might need tuning to get acceptable performance. Reducing
the frequency of generation and/or collection may be appropriate. There may be several places where tuning might be
the frequency of generation and/or collection may be appropriate. There are several places where tuning might be
done: in this collector, in the database configuration, and in the queue manager.

The following sections describe different pieces that you might want to look at.
Expand All @@ -20,20 +20,20 @@ If you cannot avoid running as a client (for example, you are trying to monitor
network latency between the queue manager and collector as low as possible. For z/OS, you might consider running the
collector in a zLinux LPAR on the same machine. Or perhaps in a zCX container.

Also configure the client to take advantage of readahead when getting publications. This is done by setting
`DEFREADA(YES)` on the nominated ReplyQueue(s).
If you are running as a client, then configure it to take advantage of readahead when getting publications. This is done by
setting `DEFREADA(YES)` on the nominated ReplyQueue(s).

## Collection processing time
The collector reports on how long it takes to collect and process the data on each interval. You can see this in a debug
log. The Prometheus collector also has a `ibmmq_qmgr_exporter_collection_time` metric. Note that this time is the value
as seen by the main collection thread; the real total time as seen by Prometheus is usually longer. There is likely
still work going on in the background to send metrics to the database, and for it to be successfully ingested.
as seen by the main collection thread; the real total time as seen by Prometheus is usually longer. This is because there
is likely still work going on in the background to send metrics to the database, and for it to be successfully ingested.

The first time that the collection time exceeds the Prometheus default `scrape_timeout` value, a warning message is
emitted. This can be ignored if you are expecting a scrape to take a longer period. But it can be helpful if you didn't
know that you might need to do some tuning.

The true total time taken for a scrape can be seen in Prometheus directly. For example, you can use the admininistrative
The true total time taken for a scrape can be seen in Prometheus directly. For example, you can use the administrative
interface at `http://<server>:9090/targets?search=` and find the target corresponding to your queue manager.

For other collectors, there is no specific metric. But the timestamps on each collection block allow you to deduce the
Expand Down
39 changes: 19 additions & 20 deletions cp4i/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,18 @@

## Using sample yamls

This directory contains sample files that show how to deploy the Prometheus
monitor to expose queue metrics for a queue manager running in a Cloud Pak for
Integration deployment on Red Hat OpenShift. The samples are accompanied by the blog post
[Monitoring IBM MQ queue depth in Cloud Pak for Integration](https://community.ibm.com/community/user/integration/blogs/matt-roberts1/2021/05/03/monitoring-mq-qdepth-cp4i) which provides a worked tutorial
illustrating how to implement monitoring scenarios in a Cloud Pak for Integration deployment.

The MQ Certified Container that comes in the Cloud Pak for Integration
emits a range of [queue manager (server) scope metrics](https://www.ibm.com/docs/en/ibm-mq/9.2?topic=containers-metrics-published-by-mq-certified) but does not currently offer an option to
publish information about queues or topics, so use of an additional Prometheus monitor pod as
shown below allows queue (and topic) metrics to be published to Prometheus for consumption
by OpenShift monitoring tools.
This directory contains sample files that show how to deploy the Prometheus monitor to expose queue metrics for a queue
manager running in a Cloud Pak for Integration deployment on Red Hat OpenShift. The samples are accompanied by the blog
post
[Monitoring IBM MQ queue depth in Cloud Pak for Integration](https://community.ibm.com/community/user/integration/blogs/matt-roberts1/2021/05/03/monitoring-mq-qdepth-cp4i)
which provides a worked tutorial illustrating how to implement monitoring scenarios in a Cloud Pak for Integration
deployment.

The MQ Certified Container that comes in the Cloud Pak for Integration emits a range of
[queue manager (server) scope metrics](https://www.ibm.com/docs/en/ibm-mq/latest?topic=operator-metrics-published-when-using-mq)
but does not currently offer an option to publish information about queues or topics, so use of an additional Prometheus
monitor pod as shown below allows queue (and topic) metrics to be published to Prometheus for consumption by OpenShift
monitoring tools.

The steps to do this are as follows;

Expand All @@ -22,8 +23,8 @@ cd mq-metric-samples/scripts
./buildRuntime.sh mq_prometheus
# Tag and push the Docker image to the container registry used by your OpenShift cluster
docker tag mq-metric-prometheus:5.2.0 your.repo/your-namespace/mq-metric-prometheus:1.0
docker push your.repo/your-namespace/mq-metric-prometheus:1.0
docker tag mq-metric-prometheus:5.2.0 your.repo/your-namespace/mq-metric-prometheus:1.0 docker push
your.repo/your-namespace/mq-metric-prometheus:1.0
# Create a ConfigMap and Secret to configure the settings that you wish to apply to your monitor.
# Update these to suit your requirements
Expand Down Expand Up @@ -68,9 +69,8 @@ oc apply -f ./metrics-service.yaml
```


Once the metrics pod is deployed you can see it in action by looking at the logs
for the pod, and the "IBMMQ Collect" statements showing that the metrics are being
scraped by the Prometheus agent running in your OpenShift cluster.
Once the metrics pod is deployed you can see it in action by looking at the logs for the pod, and the "IBMMQ Collect"
statements showing that the metrics are being scraped by the Prometheus agent running in your OpenShift cluster.
```
oc logs mq-metric-prometheus
Expand All @@ -93,8 +93,7 @@ time="2021-04-13T20:13:55Z" level=info msg="Collection time = 0 secs"

## Helm Charts

The `chart` directory contains all the helm charts needed to deploy the set of resources
on an OpenShift cluster.
The `chart` directory contains all the helm charts needed to deploy the set of resources on an OpenShift cluster.

In this case the pod is wrapped in a deployment resource and an OpenShift route is added
to provide access to the metrics application.
In this case the pod is wrapped in a deployment resource and an OpenShift route is added to provide access to the
metrics application.
2 changes: 1 addition & 1 deletion dspmqrtj/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Other dependencies are included in the `vendor` tree or can be refreshed with th

## How to use it

It is similar to **dspmqrte** - [here](https://www.ibm.com/docs/en/ibm-mq/9.2?topic=reference-dspmqrte-display-route-information)
It is similar to **dspmqrte** - [here](https://www.ibm.com/docs/en/ibm-mq/latest?topic=reference-dspmqrte-display-route-information)
is that program's documentation - but without all of the options. I wanted to just get to the basics of displaying the
route. At its simplest, you can
run `dspmqrtj -m QM1 -q LOOP.ALIAS` where the designated queue happens to be the start of a chain.
Expand Down
2 changes: 1 addition & 1 deletion dspmqrtj/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ module dspmqrtj/v5
go 1.17

require (
github.com/ibm-messaging/mq-golang/v5 v5.5.1
github.com/ibm-messaging/mq-golang/v5 v5.5.2
github.com/sirupsen/logrus v1.9.3
golang.org/x/crypto v0.14.0
)
Expand Down
39 changes: 2 additions & 37 deletions dspmqrtj/go.sum
Original file line number Diff line number Diff line change
@@ -1,57 +1,22 @@
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/ibm-messaging/mq-golang/v5 v5.5.1 h1:osHsjgUk0RwGJaSZlAAl0lLy/roUvjgP5Vtsv1bx0/g=
github.com/ibm-messaging/mq-golang/v5 v5.5.1/go.mod h1:NELV8CYOIIH1QmY6pPvulRKjwuih9YAIL9rg8+UNzNw=
github.com/ibm-messaging/mq-golang/v5 v5.5.2 h1:wigp/rO7j975R79rR8geyAtYJFktdexFsTuzbQ4FgY0=
github.com/ibm-messaging/mq-golang/v5 v5.5.2/go.mod h1:NELV8CYOIIH1QmY6pPvulRKjwuih9YAIL9rg8+UNzNw=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.14.0 h1:wBqGXzWJW6m1XrIKlAH0Hs1JJ7+9KBwnIO8v66Q9cHc=
golang.org/x/crypto v0.14.0/go.mod h1:MVFd36DqK4CsrnJYDkBA3VC4m2GkXAM0PvzMCn4JQf4=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.13.0 h1:Af8nKPmuFypiUBjVoU9V20FiaFXOcuZI21p0ycVYYGE=
golang.org/x/sys v0.13.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
golang.org/x/term v0.5.0/go.mod h1:jMB1sMXY+tzblOD4FWmEbocvup2/aLOaQEp7JmGp78k=
golang.org/x/term v0.8.0/go.mod h1:xPskH00ivmX89bAKVGSKKtLOWNx2+17Eiy94tnKShWo=
golang.org/x/term v0.13.0 h1:bb+I9cTfFazGW51MZqBVmZy7+JEJMouUHTUSKVQLBek=
golang.org/x/term v0.13.0/go.mod h1:LTmsnFJwVN6bCy1rVCoS+qHT1HhALEFxKncY3WNNh4U=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion dspmqrtj/vendor/modules.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# github.com/ibm-messaging/mq-golang/v5 v5.5.1
# github.com/ibm-messaging/mq-golang/v5 v5.5.2
## explicit; go 1.16
github.com/ibm-messaging/mq-golang/v5/ibmmq
# github.com/sirupsen/logrus v1.9.3
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ go 1.17

require (
github.com/aws/aws-sdk-go v1.45.19
github.com/ibm-messaging/mq-golang/v5 v5.5.1
github.com/ibm-messaging/mq-golang/v5 v5.5.2
github.com/influxdata/influxdb-client-go/v2 v2.12.3
github.com/prometheus/client_golang v1.17.0
github.com/sirupsen/logrus v1.9.3
Expand Down

0 comments on commit d43cd78

Please sign in to comment.