Lost Monitored Item in Subscription

[mozerlou]

Type of issue

• Bug
• Enhancement
• Compliance
• Question
• Help wanted

Current Behavior

I am currently facing an issue with my OPC UA client where, after a while, some monitored items in my subscription are not being reinstated. We stop receiving notifications for them/ and or miss the notification. This usually happens during a period of high activity on the machine.

It seems to be consistently happening to the same nodes.

Here is one specific instance when this happened:
I observed the machine logs and saw that a subscription was reinstated , adding 15 nodes back to the subscription (SubscriptionChangeMask.ItemsAdded). Everything seemed fine for the next 30 minutes, with all nodes receiving notifications correctly. However, after that, one node/monitored item missed a critical notification, which caused the entire machine to freeze.
Looking at the server itself, the Boolean node we were monitoring was set to the true value, which we had not received on our end.

More details:
-We have three subscriptions each with about 10-15 items. Some of the monitored items are linked to sensors so it can happen that they can change very quickly from true to false. Others are just bools that are used to determine whats happening.
-I have noticed the subscription.displayName can change quite frequently where for example {MySubcription} will be come -{MySubcription3} {MySubcription4} {MySubcription5} and so on.
-We do run a publish request about every hour or so to keep our subscriptions alive during periods of inactivity.

I know this problem would be a hard one to trace down. So in the meantime would you happen to have a recommendation to make sure our application work?
Should we switch to polling or would a republish such as this might work:

`
//Verifies that we have processed all notification from the server
            uint currentSequenceNumber = item.LastMessage.SequenceNumber;
            uint subscriptionID = item.Subscription.Id;
            if (notificationDictionary.ContainsKey(subscriptionID))
            {
                uint lastSequenceNumber = notificationDictionary[subscriptionID];
                if (currentSequenceNumber - lastSequenceNumber > 0)
                    FrameApp.FileLogger.Info($"OPCUA Client: Detecting a republish is necessary {item.DisplayName}. About {currentSequenceNumber - lastSequenceNumber} notifications have been skipped on {item.Subscription.DisplayName}.");

                Subscription item_subscription = item.Subscription;

                for (uint i = lastSequenceNumber + 1; i < currentSequenceNumber; i++)
                {
                    item_subscription.RepublishAsync(i)
                        .ContinueWith(task =>
                        {
                            if (task.IsFaulted)
                            {
                                Exception exception = task.Exception?.InnerException ?? task.Exception;
                                FrameApp.FileLogger.Error($"OPCUA Client: Failed to republish notification {i} on {item_subscription.DisplayName}. Exception: {exception.Message}");
                            }
                            else if (task.IsCanceled)
                            {
                                FrameApp.FileLogger.Warn($"OPCUA Client: Republish operation canceled for notification {i} on {item_subscription.DisplayName}.");
                            }
                            else
                            {
                                FrameApp.FileLogger.Info($"OPCUA Client: Successfully republished notification {i} on {item_subscription.DisplayName}.");
                            }
                        }, TaskScheduler.Default);
                }
`

Expected Behavior

Receive all notifications

Steps To Reproduce

1. running a server with a good amount of notifications

Environment

- OS: Windows 11,
- Environment:Visual Studio Professional 2019 16.11.33
- Runtime: net462
- Nuget Version: 1.4.367.42
- Component: Opc.Ua.Client
- Server: OPC UA Standard Server 
- Client: C# Application

Anything else?

 m_subscription = new Subscription(m_session.DefaultSubscription)
 {
     DisplayName = subscriptionName,
     PublishingEnabled = true,
     PublishingInterval = 50,
     KeepAliveCount = uint.MaxValue,
     LifetimeCount = uint.MaxValue,
     MaxNotificationsPerPublish = 100,
     Priority = 100
 };

Opc.Ua.Client.MonitoredItem monitoredItem = new Opc.Ua.Client.MonitoredItem(m_subscription.DefaultItem)
{
    DisplayName = item.Name,
    StartNodeId = item.Node,
    AttributeId = Attributes.Value,
    MonitoringMode = MonitoringMode.Reporting,
    SamplingInterval = 200,
    QueueSize = 1,
    DiscardOldest = true
};

private void OnNotification(Opc.Ua.Client.MonitoredItem item, MonitoredItemNotificationEventArgs e)
{
    foreach (var value in item.DequeueValues())
    {
        SubscriptionChangeArg arg = new SubscriptionChangeArg();

        arg.SubscriptionName = item.Subscription.DisplayName;
        arg.Name = item.DisplayName;
        arg.Node = item.StartNodeId.ToString();
        arg.value = value.Value;
        OnSubscriptionValueChangedEvent(arg);
    }
}
            

[mregen]

Hi @mozerlou,
I notice this is an older SDK, I would suggest to update to the latest. Unfortunately .NET 4.6.2 support has been deprecated, I'd suggest to switch to .NET Framework 4.8 on VS2019.

From your description it sounds as if the subscriptions are recreated by the client, maybe because they are timing out.

The publish request/response should always be active, the client/server need it to keep the session and subscription alive.

-We do run a publish request about every hour or so to keep our subscriptions alive during periods of inactivity.

keep alive should be handled on the session and subscription level.
Keep alive settings here should be reasonable to produce low traffic:

KeepAliveCount = uint.MaxValue, --> calculate a count with publishInterval which produces keep alive every 5 or 10 seconds
LifetimeCount = uint.MaxValue, --> Calculate a count which leaves the subscription alive > the maximum keep alive.

Hope this helps..

[mozerlou]

Unfortunately, at this moment, upgrading our SDK isn't feasible. Although our subscription remains active and notifications are being received, we're encountering a shortfall in receiving some notifications that should be reaching us. This issue occurs usually with the same monitored items.

Currently we have scrapped the idea of republishing missing notification and have just restarted the subscriptions.

Like I have said this issue seems similar to #2529

[mozerlou]

Wanted to comment on this thread, again my problem is that I stop receiving notifications for a select few nodes in my subscription. Not all of them. I keep receiving notifications for all other nodes.

To mitigate this issue, I attempted to delete and recreate the subscription and found the following bugs

I found that having this function:

            await existingSubscription.DeleteAsync(false)

When the connection to the machine has been lost or for whatever reason the subscription delete failed . it will create 100s of subscriptions on my server. Just wanted to let you guys know

Here is the code

        {
            //Find the subscription in dictionary, this is to get the list of monitored items 
            if (!m_subscriptionDictionary.TryGetValue(subscriptionToRestart, out Subscription existingSubscription))
            {
                FrameApp.FileLogger.Error($"OPCServer: Subscription {subscriptionToRestart} not found. Unable to restart subscription.");
                return false; // Subscription not found
            }

            //Delete the subscription
            bool deletionResult = await DeleteSubscription(subscriptionToRestart);
            if (!deletionResult)
            {
                FrameApp.FileLogger.Error($"OPCServer: Failed to recreate subscription {subscriptionToRestart}. Unable to delete subscription. ");
                return false; // Unable to delete subscription
            }

            //Recreate the subscription 
            bool creationResult = await CreateSubscription(subscriptionToRestart, null, existingSubscription.MonitoredItems);

            if (!creationResult)
            {
                FrameApp.FileLogger.Error($"OPCServer: Failed to recreate subscription {subscriptionToRestart}. Unable to recreate subscription. ");
                return false; // Unable to create the subscription
            }

            FrameApp.FileLogger.Info($"OPCServer: Subscription {subscriptionToRestart} restarted successfully.");
            return true;
        }```
        
        
I can also share the session reconnect code or anything else that might help you guys figure out whats going wrong