Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE #4735] SubscriptionManager enhancement #4736

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

karsonto
Copy link
Contributor

@karsonto karsonto commented Jan 10, 2024

The main changes are as follows:
SubscriptionManager add a reentrant lock ensure thread safety
SubscriptionManager adds url client mapping
Optimize registerClient logic and remove the inefficient method of finding objects in for loop
SubscribeProcessor and LocalSubscribeEventProcessor remove the synchronized method calling SubscriptionManager

Fixes #4735

Motivation

Explain the content here.
Explain why you want to make the changes and what problem you're trying to solve.

Modifications

Describe the modifications you've done.

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
  • If a feature is not applicable for documentation, explain why?
  • If a feature is not documented yet in this PR, please create a followup issue for adding the documentation

Copy link

codecov bot commented Jan 10, 2024

Codecov Report

Attention: 95 lines in your changes are missing coverage. Please review.

Comparison is base (9a3912a) 16.37% compared to head (65f244a) 17.55%.

❗ Current head 65f244a differs from pull request most recent head 0d9e9d6. Consider uploading reports for the commit 0d9e9d6 to get more accurate results

Files Patch % Lines
...esh/runtime/core/consumer/SubscriptionManager.java 0.00% 50 Missing ⚠️
...re/protocol/http/processor/SubscribeProcessor.java 0.00% 28 Missing ⚠️
...l/http/processor/LocalSubscribeEventProcessor.java 0.00% 17 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #4736      +/-   ##
============================================
+ Coverage     16.37%   17.55%   +1.17%     
- Complexity     1734     1778      +44     
============================================
  Files           856      797      -59     
  Lines         31265    29866    -1399     
  Branches       2701     2581     -120     
============================================
+ Hits           5120     5243     +123     
+ Misses        25665    24140    -1525     
- Partials        480      483       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Pil0tXia
Copy link
Member

Pil0tXia commented Feb 1, 2024

This PR mainly focuses on the thread safety of SubscriptionManager.

@xwm1992
Copy link
Contributor

xwm1992 commented Feb 2, 2024

@karsonto please fix the conflicts thanks.

@karsonto karsonto force-pushed the develop-subscriptionManager-enhancement branch from 2976b25 to 1278952 Compare February 2, 2024 03:13
@karsonto
Copy link
Contributor Author

karsonto commented Feb 2, 2024

@karsonto please fix the conflicts thanks.

Please help to review again,thank you.

xwm1992
xwm1992 previously approved these changes Feb 18, 2024
@Pil0tXia
Copy link
Member

plz resolve conflicts

isContains = true;
localClient.setLastUpTime(new Date());
break;
lock.lock();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thread-safety issue with the method is brought about by the global variable. Could you explain where there is a thread safety issue with the way the original method uses ConcurrentHashMap?

该方法的线程安全问题是由全局变量带来的。能否解释下原方法对ConcurrentHashMap的使用方式哪里存在线程安全问题?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个改动主要消除了SubscribeProcessor 使用 SubscriptionManager synchronized 方法,保证了SubscriptionManager方法调用是线程安全的,是线程安全的类

@@ -56,6 +55,10 @@ public class SubscriptionManager {
*/
private final ConcurrentHashMap<String, List<Client>> localClientInfoMapping = new ConcurrentHashMap<>(64);

private final ConcurrentHashMap<String, Client> urlClientInfoMapping = new ConcurrentHashMap<>(64);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you clarify what the role of this global variable is? I only see one role for it: to store the newly created Client in the registerClient() method. and that role doesn't seem to have an advantage compared to the way the original method handles it, but rather adds overhead to the global variable.

您能否说明下该全局变量的作用是什么?我只看到它的一个作用:在registerClient()方法存储新创建的Client。而该作用相比较原方法的处理方式好像并无优势,反而增加了全局变量的开销。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

添加全局变量保存url 以及client对应关系,消除了每次注册client时的遍历,提高了注册效率

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean eliminating the second for(){} in registerClient() method? If so, it doesn't actually eliminate the traversal. List#contains() is also a traversal operation internally. And there are problems with that traversal, as I explained to you.

您的意思是消除了registerClient()方法中第二个的for(){}吗?如果是的话,实际上并没有消除该遍历。List#contains()内部也是遍历操作。而且该遍历存在的问题,我也向您说明了。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

那用Set 方式代替List 你觉得可行吗?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@karsonto
I think switching to Set might improve iterating speed. But the way it compares elements when iterating has the same problem as List#contains().

我认为改用Set也许能提高遍历速度。但是它遍历中比较元素的方式也存在和List#contains()一样的问题。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

如果改用Set集合,重写Client url hash code方法,加入到集合里就不需要再遍历。

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the value in localClientInfoMapping is changed to Set, does the equals method need to be rewritten in addition to the hashCode method?

如果将localClientInfoMapping中的value改成Set,除了hashCode方法,equals方法是否也需要重写?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

是的,使用HashSet 代替ArrayList 需要重写Client hash code 以及equals 方法,用单一url属性判断防重复

return client;
});
clientInner.reNewLastUpTime();
if (!localClients.contains(clientInner)) {
Copy link
Contributor

@pandaapo pandaapo Feb 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This type of judgment is inconsistent with the original method logic. Your way will determine if they are equal or not by Client#equals(), i.e. memory address. Whereas the original method logic is comparing the content of the Client, i.e. the url attribute (at least).

这种判断方式和原方法逻辑是不一致的。您的方式将通过Client#equals()判断是否相等,即内存地址。而原方法逻辑是比较Client的内容,即url属性(至少)。

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@karsonto It seems that this conversation is waiting for your reply.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Enhancement] Optimize the SubscriptionManager thread safety
4 participants