Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat : add script routing functionality #2669

Merged
merged 21 commits into from
May 18, 2024
Merged

feat : add script routing functionality #2669

merged 21 commits into from
May 18, 2024

Conversation

YarBor
Copy link
Contributor

@YarBor YarBor commented May 10, 2024

JS Standard

  • The expect to recv js-script data is like:
(function route(invokers, invocation, context) {
    var result = [];
    for (var i = 0; i < invokers.length; i++) {
        if ("127.0.0.1" === invokers[i].GetURL().Ip) {
            if (invokers[i].GetURL().Port !== "20000") {
                invokers[i].GetURL().Ip = "anotherIP"
                result.push(invokers[i]);
            }
        }
    }
    return result;
}(invokers, invocation, context));
  • The expected way to return value is like:
    var result = [];
    result.push(invokers[i]);
    return result;

  • Supports method calling.
    Parameter methods are mapped by the passed in type.
    The called method first letter is capitalized (does not comply with js specifications).

e.g. invokers[i].GetURL().SetParam("testKey","testValue")


  • Like the Go language, it supports direct access to exportable variables within parameters.

e.g. invokers[i].GetURL().Port


  • publish config like :
scriptRouteConfig := data, err := os.ReadFile(pathToConfigfile)
conf, _ := config.GetRootConfig().ConfigCenter.GetDynamicConfiguration()
_ = conf.PublishConfig("dubbo.io"+constant.ScriptRouterRuleSuffix, "dubbo", scriptRouteConfig)
  • config file like :
configVersion: v3.0
key: dubbo.io
type: javascript
enabled: true
script: |
  (function route(invokers,invocation,context) {
  	var result = [];
  	for (var i = 0; i < invokers.length; i++) {
  	    if ("10.30.0.218" === invokers[i].GetURL().Ip) {
  			if (invokers[i].GetURL().Port === "20000"){
  				invokers[i].GetURL().Port = "20001" 
  				result.push(invokers[i]);
  			}
  	    }
  	}
  	return result;
  }(invokers,invocation,context));

Important

In the js-support.

  1. Input args are invokers, invocation, context,
    The following methods are not allowed.
    • invokers.Invoke(...)
    • invokers.Destroy()
    • invokers.IsAvailable() will always return true
  2. Wrong input is program acceptable, it will not return an error or go panic , it will make an undefined error.
    for example:
(function route(invokers, invocation, context) {
    var result = [];
    for (var i = 0; i < invokers.length; i++) {
        if ("127.0.0.1" === invokers[i].GetURL().Ip) {
            if (invokers[i].GetURL(" Err Here ").Port !== "20000") {
//---------------------------------^  here wont go err , it will trans to ()
                invokers[i].GetURL().Ip = "anotherIP"
                result.push(invokers[i]);
            }
        }
    }
    return result;
}(invokers, invocation, context));
(function route(invokers, invocation, context) {
    var result = [];
    for (var i = 0; i < invokers.length; i++) {
        if ("127.0.0.1" === invokers[i].GetURL().Ip) {
            if (invokers[i].GetURL(" Err Here ").Port !== "20000") {
                invokers[i].GetURL().AddParam( null , "key string", "value string")
//---------------------------------------------^  here wont make err , it will trans to ("" , "key string" , "value string")
                result.push(invokers[i]);
            }
        }
    }
    return result;
}(invokers, invocation, context));
(function route(invokers, invocation, context) {
    var result = [];
    for (var i = 0; i < invokers.length; i++) {
        if ("127.0.0.1" === invokers[i].GetURL().Ip) {
            if (invokers[i].GetURL(" Err Here ").Port !== "20000") {
                invokers[i].GetURL().AddParam( invocation , "key string", "value string")
//---------------------------------------------^  here wont make err , it will trans to ("[object Object]" , "key string" , "value string")
                result.push(invokers[i]);
            }
        }
    }
    return result;
}(invokers, invocation, context));
  1. Wrong methods-call will go error.
    for example:
(function route(invokers, invocation, context) {
    var result = [];
    for (var i = 0; i < invokers.length; i++) {
        if ("127.0.0.1" === invokers[i].GetURL().Ip) {
            if (invokers[i].GetURLS(" Err Here ").Port !== "20000") {
//--------------------------^ // here will make err 
                invokers[i].GetURLS().Ip = "anotherIP"
                result.push(invokers[i]);
            }
        }
    }
    return result;
}(invokers, invocation, context));

YarBor added 4 commits May 9, 2024 22:52
Signed-off-by: YarBor <yarbor.ww@gmail.com>
Signed-off-by: YarBor <yarbor.ww@gmail.com>
Signed-off-by: YarBor <yarbor.ww@gmail.com>
Signed-off-by: YarBor <yarbor.ww@gmail.com>
if err != nil {
return nil, err
}
result := make([]protocol.Invoker, 0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

map add cap

YarBor and others added 3 commits May 10, 2024 19:46
Signed-off-by: YarBor <yarbor.ww@gmail.com>
add comment to jsInstance

Signed-off-by: YarBor <yarbor.ww@gmail.com>
@YarBor YarBor changed the title Add script routing functionality feat : add script routing functionality May 11, 2024
package instance

import (
"dubbo.apache.org/dubbo-go/v3/protocol"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reformat

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand what the problem is, I ran go fmt and it doesn't work, looks the same.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand what the problem is, I ran go fmt and it doesn't work, looks the same.

Go fmt will not change the order of the import, it will only format locks and spaces. You can use goimports instead,https://pkg.go.dev/golang.org/x/tools/cmd/goimports
Generally speaking, the standard import format is:
import (
go standard packages (sorted alphabetically)
(space)
Three-party packages (sorted alphabetically)
)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我们 dubbogo 有自己的 import format 工具 https://github.com/dubbogo/tools/tree/master/cmd/imports-formatter

编译出二进制,在项目根目录下执行一次,就可以 format 整个工程的 import 顺序。比你们找的这个工具高级多了。

用好前人积累的工具。

)

type ScriptInstances interface {
RunScript(rawScript string, invokers []protocol.Invoker, invocation protocol.Invocation) ([]protocol.Invoker, error)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ScriptInstances struct 里面已经有 Script 了,Run() 这个函数名就 ok 了

factory[tpName] = instance
}

func init() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

init 函数挪到 struct 定义下面

package instance

import (
"dubbo.apache.org/dubbo-go/v3/protocol"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reformat

}
}

func (i *jsInstances) RunScript(_ string, invokers []protocol.Invoker, invocation protocol.Invocation) ([]protocol.Invoker, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

改成 Run

package script

import (
ins "dubbo.apache.org/dubbo-go/v3/cluster/router/script/instance"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reformat

The current problem is that the parameter passed in is a reference. When the URL is modified, the original copy will be modified, which will lead to undefined behavior.

Signed-off-by: YarBor <yarbor.ww@gmail.com>
@codecov-commenter
Copy link

codecov-commenter commented May 11, 2024

Codecov Report

Attention: Patch coverage is 21.18644% with 93 lines in your changes are missing coverage. Please review.

Project coverage is 47.21%. Comparing base (2f5143a) to head (5b51cc0).
Report is 17 commits behind head on main.

Current head 5b51cc0 differs from pull request most recent head a650f8a

Please upload reports for the commit a650f8a to get more accurate results.

Files Patch % Lines
cluster/router/script/instance/js_instance.go 2.94% 65 Missing and 1 partial ⚠️
cluster/router/script/instance/instances_pool.go 47.36% 20 Missing ⚠️
common/url.go 41.66% 5 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2669      +/-   ##
==========================================
- Coverage   47.38%   47.21%   -0.17%     
==========================================
  Files         341      343       +2     
  Lines       25122    25286     +164     
==========================================
+ Hits        11904    11940      +36     
- Misses      12074    12188     +114     
- Partials     1144     1158      +14     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

YarBor added 7 commits May 11, 2024 22:36
Signed-off-by: YarBor <yarbor.ww@gmail.com>
Signed-off-by: YarBor <yarbor.ww@gmail.com>
Signed-off-by: YarBor <yarbor.ww@gmail.com>
Signed-off-by: YarBor <yarbor.ww@gmail.com>
Signed-off-by: YarBor <yarbor.ww@gmail.com>
Signed-off-by: YarBor <yarbor.ww@gmail.com>
Signed-off-by: YarBor <yarbor.ww@gmail.com>
@baerwang baerwang requested a review from justxuewei May 15, 2024 14:25
@chickenlj
Copy link
Contributor

  1. 监听的规则 key 应该是 provider 应用名,而不是当前进程自身应用名。可参考 tagrouter 的 notify 方法
  2. 第1个逻辑改了后,Script Instance 实例的对应关系也要改一下,当前是全局共享的
  3. 要确认下 dynamicconfuguration.AddListener() 的逻辑对不对(zookeeper/nacos),因为有多个 ScriptRouter 实例可能对应到一个 key (provider应用名),dynamicconfuguration.AddListener()应该支持调用多次且记录多个listener --- zookeeper实现看起来没问题,nacos实现好像有问题。
  4. instance.Compile 并发调用有没有问题

YarBor added 3 commits May 16, 2024 12:07
…nces.

Signed-off-by: YarBor <yarbor.ww@gmail.com>
Signed-off-by: YarBor <yarbor.ww@gmail.com>
Signed-off-by: YarBor <yarbor.ww@gmail.com>
}
i.pgLock.Unlock()
return nil
}
Copy link
Contributor

@chickenlj chickenlj May 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lock used here is a bit strange, try the following steps:

  1. check without lock
  2. if exists, return
  3. if not exists, get the lock
    3.1 check again
    3.2 if exists return
    3.3 if not exists compile

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I previously thought that "compiling" was a time-consuming operation that was not suitable to be performed while occupying a lock.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i will rewite it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. I thought the following lock was not necessary, it seems to be necessary in this case.

i.pgLock.RLock()
pg, ok = i.program[rawScript]
i.pgLock.RUnlock()

YarBor added 3 commits May 17, 2024 15:08
Signed-off-by: YarBor <yarbor.ww@gmail.com>
Signed-off-by: YarBor <yarbor.ww@gmail.com>
Signed-off-by: YarBor <yarbor.ww@gmail.com>
Copy link

sonarcloud bot commented May 17, 2024

Quality Gate Passed Quality Gate passed

Issues
9 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
1.4% Duplication on New Code

See analysis details on SonarCloud

Copy link
Contributor

@chickenlj chickenlj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@chickenlj chickenlj merged commit 955c6d6 into apache:main May 18, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants