Get statistics when mongocOnly activated on Orion-LD

[fabiocabrini]

Issue
When Orion-LD is running on Kubernetes and utilizing the clustered MongoDB, the mongocOnly environment variable is used to ensure everything works. However, activating mongocOnly prevents obtaining internal statistics from the clustered Orion-LD. Attempts to retrieve these statistics result in an HTTP message stating that 'Non NGSI-LD requests are not supported when -mongocOnly is set'.

Describe the solution you'd like
I would like to obtain these internal performance indicator information from Orion-LD even when mongocOnly mode is activated. Is there any estimate for this feature to be implemented?

Describe why you need this feature
To analyze these statistics to observe the behavior of each broker present in the cluster.

Do you have the intention to implement the solution
No, I do not have the skills.

[kzangeli]

I assume these statistics you talk about is the metrics from NGSIv2.
NGSI-LD requests don't support the old metrics and thus the request to "GET metrics" makes zero sense when running in mongoc only mode (which means, NGSI-LD only, no support for NGSIv2).

What we do have, quite rudimentary still, is support for Prometheus metrics (on port 8000).

[kzangeli]

That said, if you need some specific metrics, I'd be happy to implement that. It's quite easy.

[fabiocabrini]

Hi Ken, I need the internal metrics from Orion-LD. When the mongocOnly mode is disabled, meaning when Orion-LD is connected to a stand-alone MongoDB, I can access the performance indicators normally. The issue arises when I clusterize MongoDB, requiring the use of mongocOnly, which disables access to these parameters. I included the following parameters in the command: "-statCounters", "-statSemWait", "-statTiming", "-statNotifQueue".

curl --location 'http://{{orion}}:1026/statistics'

Response:

{ "orionError": { "code": "501", "reasonPhrase": "Not Implemented", "details": "Non NGSI-LD requests are not supported with -mongocOnly is set" } }

[fabiocabrini]

Hi Ken, do you have any updates on this issue?

[kzangeli]

Ah, I thought I had answered already.
The metrics you're asking for are for NGSIv2 requests.
NGSi-LD has no metrics, never been added to the spec.
However, I did implement something, on request from a colleague in the FIWARE Foundation, using Prometheus.
It's very basic, but if you tell me what kind of metrics you need, I can add to what little I already have.

[fabiocabrini]

Ah, I thought I had answered already. The metrics you're asking for are for NGSIv2 requests. NGSi-LD has no metrics, never been added to the spec. However, I did implement something, on request from a colleague in the FIWARE Foundation, using Prometheus. It's very basic, but if you tell me what kind of metrics you need, I can add to what little I already have.

Hi Ken, thanks for answering my question. So, I need the following metrics: incomingTransactions, incomingTransactionRequestSize, incomingTransactionResponseSize, incomingTransactionErrors, serviceTime, outgoingTransactions, outgoingTransactionRequestSize, outgoingTransactionResponseSize, and outgoingTransactionErrors. It would be desirable to have access to Orion-LD's internal statistics.

For example:

curl --location 'http://<ip>:1026/admin/metrics'

Response:

{
    "services": {
        "default-service": {
            "subservs": {
                "root-subserv": {
                    "incomingTransactionResponseSize": 1499,
                    "serviceTime": 0.001252000,
                    "incomingTransactions": 4
                }
            },
            "sum": {
                "incomingTransactionResponseSize": 1499,
                "serviceTime": 0.001252000,
                "incomingTransactions": 4
            }
        }
    },
    "sum": {
        "subservs": {
            "root-subserv": {
                "incomingTransactionResponseSize": 1499,
                "serviceTime": 0.001252000,
                "incomingTransactions": 4
            }
        },
        "sum": {
            "incomingTransactionResponseSize": 1499,
            "serviceTime": 0.001252000,
            "incomingTransactions": 4
        }
    }
}

When I connect Orion-LD to MongoDB Community Operator setup with 'mongoConly', this request doesn't work!

Return:

{
    "error": "NotImplemented",
    "description": "Non NGSI-LD requests are not supported with -mongocOnly is set"
}

The same behavior happens in /statistics requests!

For example:

curl --location 'http://<ip>:1026/statistics'

Return:

{
    "counters": {
        "versionRequests": 18,
        "statisticsRequests": 2
    },
    "semWait": {
        "request": 0,
        "dbConnectionPool": 0.000000858,
        "transaction": 0.000004597,
        "subCache": 0,
        "connectionContext": 0,
        "timeStat": 0.000004451,
        "metrics": 0.000003000
    },
    "timing": {
        "accumulated": {
            "total": 0.010224796
        },
        "last": {
            "total": 0.000244606
        }
    },
    "uptime_in_secs": 531,
    "measuring_interval_in_secs": 531
}

When I connect Orion-LD to MongoDB Community Operator setup with 'mongoConly', this request doesn't work!

{
    "orionError": {
        "code": "501",
        "reasonPhrase": "Not Implemented",
        "details": "Non NGSI-LD requests are not supported with -mongocOnly is set"
    }
}

Your help will be of great importance!