Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

组件依赖上游模型时,如果依赖组件同属一个Module,则通过job_conf对存在模型依赖的组件设置任何参数都无效(包括need_run) #436

Open
jsuper opened this issue Jun 20, 2023 · 0 comments

Comments

@jsuper
Copy link
Contributor

jsuper commented Jun 20, 2023

训练的DSL组件如图,其中read_0读取的是训练集,read_1读取的是测试集。
8b7f53d41dc6df043aa2783bb6bc841

hetero_lr_1组件时使用hetero_lr_0组件训练好的模型去对测试集数据进行打分,该组件能够正常运行,但是用户输入的参数会无效,例如控制need_run=True|False,无法对hetero_lr_1组件生效。

在对代码进行跟踪后,发现在dsl_parser.py(_init_component_setting)解析组件的参数时, 如果一个组件存在上游模型依赖,则会将cur_component的值修改为上游组件。这就导致获provider_manager.get_component_parameters取到的参数(parse_component_parameters, parse_user_specified_component_parameters)都是上游组件的参数,对于该打分组件(hetero_lr_0)设置的任何参数都无效了(实际上预测执行的时候输入参数是通过load_model去加载的)。
image

    @classmethod
    def get_component_parameters(cls, dsl_parser, component_name, role, party_id, provider: ComponentProvider = None, previous_components_parameters: dict = None):
        if not provider:
            provider = cls.get_component_provider(dsl_parser=dsl_parser,
                                                  component_name=component_name)
        parameters = dsl_parser.parse_component_parameters(component_name,
                                                           ComponentRegistry.REGISTRY,
                                                           provider.name,
                                                           provider.version,
                                                           local_role=role,
                                                           local_party_id=int(party_id),
                                                           previous_parameters=previous_components_parameters)
        user_specified_parameters = dsl_parser.parse_user_specified_component_parameters(component_name,
                                                                                         ComponentRegistry.REGISTRY,
                                                                                         provider.name,
                                                                                         provider.version,
                                                                                         local_role=role,
                                                                                         local_party_id=int(party_id),
                                                                                         previous_parameters=previous_components_parameters)
        return parameters, user_specified_parameters

因此,所有带有模型输入的组件,都无法通过配置文件控制该组件自身的参数。请问这里是否是一个bug?cur_component变量的值是否不应该被修改?又或者
在解析运行时配置的时候,应该传入component 而不是cur_component

        role_parameters = RuntimeConfParserUtil.get_component_parameters(provider,
                                                                         runtime_conf,
                                                                         module,
                                                                         cur_component,
                                                                         redundant_param_check=redundant_param_check,
                                                                         local_role=local_role,
                                                                         local_party_id=local_party_id,
                                                                         parse_user_specified_only=parse_user_specified_only,
                                                                         pre_parameters=pre_parameters)

请问该问题是否是一个bug呢?
@jat001 @dylan-fan @zhihuiwan @mgqa34

@jsuper jsuper changed the title 组件依赖上游模型时,如果以来组件同属一个Module,则通过job_conf对存在模型以来的组件设置任何参数都无效(包括need_run) 组件依赖上游模型时,如果依赖组件同属一个Module,则通过job_conf对存在模型以来的组件设置任何参数都无效(包括need_run) Jun 20, 2023
@jsuper jsuper changed the title 组件依赖上游模型时,如果依赖组件同属一个Module,则通过job_conf对存在模型以来的组件设置任何参数都无效(包括need_run) 组件依赖上游模型时,如果依赖组件同属一个Module,则通过job_conf对存在模型依赖的组件设置任何参数都无效(包括need_run) Jun 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant