Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linq中的where Context.[table].Any(...)未能使用分表 #245

Open
NuxYoung opened this issue May 17, 2023 · 12 comments
Open

Linq中的where Context.[table].Any(...)未能使用分表 #245

NuxYoung opened this issue May 17, 2023 · 12 comments

Comments

@NuxYoung
Copy link

有表A和表B,其中表B采用了按创建时间的月份分表,执行以下查询的时候未能从分表中查询数据:

var test = from a in Context.A
           where Context.B.Any(
               b => b.foreignId == a.Id
               && b.column1 == "1" || a.column2 > 0)

有好的办法解决这个问题吗?

@xuejmnet
Copy link
Collaborator

是生成的sql里面b表没有分片,还是分片了但是没有按规则过滤 @NuxYoung

@NuxYoung
Copy link
Author

是生成的sql里面b表没有分片,还是分片了但是没有按规则过滤 @NuxYoung

是生成的sql里b表没有分片

@xuejmnet
Copy link
Collaborator

@NuxYoung 什么版本的sharding-core

@xuejmnet
Copy link
Collaborator

@NuxYoung 最新版本我测试是可以的我不知道你是什么版本你发一下我看

@NuxYoung
Copy link
Author

@NuxYoung 最新版本我测试是可以的我不知道你是什么版本你发一下我看

我用的也是最新版本(7.7.1.9),但生成的查询sql、查询结果都是没有分片的。
再补充一下,我TableB在进行EF迁移的时候,生成了一个不带月份的表,但我数据都在带月份的表里(如TableB_202305)。然后我进行ToQueryString()的时候就发现生成的sql是查询不带月份的表
我附上分表的代码吧

    public class TableBVirtualTableRoute : AbstractSimpleShardingMonthKeyDateTimeVirtualTableRoute<TableB>
    {
        public override bool AutoCreateTableByTime()
        {
            return true;
        }

        public override void Configure(EntityMetadataTableBuilder<BoxCode> builder)
        {
            builder.ShardingProperty(b => b.BTime);
            builder.AutoCreateTable(true);
        }

        /// <summary>
        /// 使用固定值,否则重启服务的时候会重置
        /// </summary>
        /// <returns></returns>
        public override DateTime GetBeginTime()
        {
            return new DateTime(2023, 3, 1);
        }

        // 若要查看ShadingCore的异常,取消注释
        //public override bool DoLogError => true;
    }

然后目前我是用了INNER JOIN的形式去解决这个问题。

@xuejmnet
Copy link
Collaborator

@NuxYoung ToQueryString()是没办法获取分片后的表的,但是迁移如果存在不带月份的表就说明你的配置在迁移的时候有问题,原则上是不会创建这个表的,前提是迁移配置正确

                    o.UseShardingMigrationConfigure(b =>
                    {
                        b.ReplaceService<IMigrationsSqlGenerator, ShardingMySqlMigrationsSqlGenerator>();
                    });

这个是我执行的结果确实是有的,你可以把log改成debug然后发出来看看
image

@xuejmnet
Copy link
Collaborator

如果日志级别改成debug应该会有如下日志打印

ShardingCore.Sharding.ShardingExecutors.QueryCompilerContextFactory[0]
      queryable combine:ShardingCore.Sharding.ShardingExecutors.QueryableCombines.EnumerableQueryableCombine
dbug: ShardingCore.Sharding.ShardingExecutors.QueryCompilerContextFactory[0]
      queryable combine before:DbSet<SysTest>()
          .Where(ut => WeatherForecastController._defaultTableDbContext.Set<SysUserLogByMonth>()
              .Any(x => x.Id == ut.Id))
dbug: ShardingCore.Sharding.ShardingExecutors.QueryCompilerContextFactory[0]
      queryable combine after:DbSet<SysTest>()
          .Where(ut => WeatherForecastController._defaultTableDbContext.Set<SysUserLogByMonth>()
              .Any(x => x.Id == ut.Id))
dbug: ShardingCore.Sharding.ShardingExecutors.QueryCompilerContextFactory[0]
      data source route result:ds0
AfterShardingRouteUnitFilter:29
dbug: ShardingCore.Sharding.ShardingExecutors.QueryCompilerContextFactory[0]
      table route results:DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202101.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202102.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202103.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202104.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202105.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202106.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202107.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202108.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202109.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202110.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202111.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202112.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202201.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202202.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202203.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202204.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202205.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202206.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202207.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202208.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202209.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202210.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202211.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202212.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202301.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202302.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202303.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202304.Sample.MySql.Domain.Entities.SysUserLogByMonth]),DataSourceName:ds0,TableRouteResult:(has different tail:False,current table:[ds0.202305.Sample.MySql.Domain.Entities.SysUserLogByMonth])
dbug: ShardingCore.Sharding.MergeContexts.QueryableRewriteEngine[0]
      rewrite queryable pagination context:[Skip: ,  Take: ]
dbug: ShardingCore.Sharding.MergeContexts.QueryableRewriteEngine[0]
      rewrite queryable order by context:[]
dbug: ShardingCore.Sharding.MergeContexts.QueryableRewriteEngine[0]
      rewrite queryable group by context:[]
dbug: ShardingCore.Sharding.MergeContexts.QueryableRewriteEngine[0]
      rewrite queryable select context:[]

@NuxYoung
Copy link
Author

@NuxYoung ToQueryString()是没办法获取分片后的表的,但是迁移如果存在不带月份的表就说明你的配置在迁移的时候有问题,原则上是不会创建这个表的,前提是迁移配置正确

                    o.UseShardingMigrationConfigure(b =>
                    {
                        b.ReplaceService<IMigrationsSqlGenerator, ShardingMySqlMigrationsSqlGenerator>();
                    });

这个是我执行的结果确实是有的,你可以把log改成debug然后发出来看看 image

日志输出应该是有异常。其中FactoryOrder对应的是TableABoxCode对应的是TableBFactoryOrder没有做分页。但是在INNER JOIN查询里,是ok的。我应该怎样操作呢?
image

@xuejmnet
Copy link
Collaborator

@NuxYoung
image

@xuejmnet
Copy link
Collaborator

@NuxYoung 还有一种办法你可以吧变量先提取出来

var tableb=Context.B;
var test = from a in Context.A
           where tableb.Any(
               b => b.foreignId == a.Id
               && b.column1 == "1" || a.column2 > 0)

@NuxYoung
Copy link
Author

@NuxYoung image

相关代码如下:
Context:

    public class BoxStickerContext : AbstractShardingDbContext, IShardingTableDbContext
    {
        public BoxStickerContext(DbContextOptions<BoxStickerContext> options) : base(options)
        {
        }

        public DbSet<BoxCode> BoxCode { get; set; }
        public DbSet<FactoryOrder> FactoryOrder { get; set; }
        public IRouteTail RouteTail { get; set; }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            modelBuilder.Entity<BoxCode>();
            modelBuilder.Entity<FactoryOrder>();

            base.OnModelCreating(modelBuilder);
        }
    }

Startup.cs:

        public void ConfigureServices(IServiceCollection services)
        {
                services.AddShardingDbContext<BoxStickerContext>()
                .UseRouteConfig(op =>
                {
                    // 此处配置需要分表的表名
                    op.AddShardingTableRoute<BoxCodeVirtualTableRoute>();
                }).UseConfig((sp, op) =>
                {
                    op.UseShardingQuery((conn, builder) =>
                    {
                        builder.UseSqlServer(conn).UseLoggerFactory(efLogger);
                    });
                    op.UseShardingTransaction((conn, builder) =>
                    {
                        builder.UseSqlServer(conn).UseLoggerFactory(efLogger);
                    });
                    op.AddDefaultDataSource("BoxStickerContext", configuration.GetConnectionString("BoxStickerConnection"));
                    op.UseShardingMigrationConfigure(op =>
                    {
                        op.ReplaceService<IMigrationsSqlGenerator, ShardingMigrationsSqlGenerator<BoxStickerContext>>();
                    });
                }).AddShardingCore();
        }

        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            app.ApplicationServices.UseAutoTryCompensateTable();
        }

分片路由:

    public class BoxCodeVirtualTableRoute : AbstractSimpleShardingMonthKeyDateTimeVirtualTableRoute<BoxCode>
    {
        public override bool AutoCreateTableByTime()
        {
            return true;
        }

        public override void Configure(EntityMetadataTableBuilder<BoxCode> builder)
        {
            builder.ShardingProperty(b => b.BTime);
            builder.AutoCreateTable(true);
        }

        /// <summary>
        /// 使用固定值,否则重启服务的时候会重置
        /// </summary>
        /// <returns></returns>
        public override DateTime GetBeginTime()
        {
            return new DateTime(2023, 3, 1);
        }

        // 若要查看ShadingCore的异常,取消注释
        //public override bool DoLogError => true;
    }

@xuejmnet
Copy link
Collaborator

@NuxYoung 实在不好意思我想问下您具体执行的queryable表达式是什么

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants