Autotest
/
sharding
mirror of https://gitee.com/dotnetchina/sharding-core.git
1
0
Fork 0
Code Issues Releases Wiki Activity

11111

This commit is contained in:
xuejiaming 2021-03-08 14:59:32 +08:00
parent 8039d5b27b
commit 06d9e8f7a7
30 changed files with 665 additions and 286 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

341
readmes/README_5.0.4.md Normal file
View File

@ -0,0 +1,341 @@
<h1 align="center"> ShardingCore </h1>
`ShardingCore` 是一个支持efcore 2.x 3.x 5.x的一个对于数据库分表的一个简易扩展,.Net下并没有类似mycat或者sharding-jdbc之类的开源组件或者说有但是并没有非常适用的或者说个人在用过后有一些地方因为限制没法很好使用所以决定自己开发这个库,目前该库暂未支持分库(未来会支持),仅支持分表,该项目的理念是让你可以已最少的代码量来实现自动分表的实现,经过多个开源项目的摸索参考目前正式开源本项目
项目地址 [github](https://github.com/xuejmnet/sharding-core) 喜欢的朋友可以点下star Thanks♪(･ω･)ﾉ
### 依赖
Release | EF Core | .NET Standard | .NET (Core) | Sql Server | Pomelo.EntityFrameworkCore.MySql
--- | --- | --- | --- | --- | ---
[5.x.x.x](https://www.nuget.org/packages/ShardingCore/5.0.0.4) | >= 5.0.x | 2.1 | 3.0+ | >= 2012 | 5.0.0-alpha.2
[3.x.x.x](https://www.nuget.org/packages/ShardingCore/3.0.0.4) | 3.1.10 | 2.0 | 2.0+ | >= 2012 | 3.2.4
[2.x.x.x](https://www.nuget.org/packages/ShardingCore/2.0.0.4) | 2.2.6 | 2.0 | 2.0+ | >= 2008 | 2.2.6
- [开始](#开始)
- [简介](#简介)
- [概念](#概念)
- [优点](#优点)
- [缺点](#缺点)
- [安装](#安装)
- [配置](#配置)
- [使用](#使用)
- [默认路由](#默认路由)
- [Api](#Api)
- [高级配置](#高级配置)
- [手动路由](#手动路由)
- [自动建表](#自动建表)
- [事务](#事务)
- [批量操作](#批量操作)
- [注意事项](#注意事项)
- [计划(Future)](#计划)
- [最后](#最后)
# 开始
以下所有例子都以Sql Server为例 MySql亦如此
## 简介
目前该库处于初期阶段,有很多bug也希望各位多多理解,一起努力为.net生态做出一份微薄之力,目前该库支持的分表可以进行完全的自定义,基本上可以满足95%以上的
业务需求，唯一的限制就是分表规则必须满足 x+y+z,x表示固定的表名,y表示固定的表名和表后缀之间的联系(可以为空),z表示表后缀,可以按照你自己的任意业务逻辑进行切分,
如:user_0,user_1或者user202101,user202102...当然该库同样适用于多租户模式下的隔离,该库为了支持之后的分库已经重写了之前的union all查询模式,并且支持多种api,
支持多种查询包括```join,group by,max,count,min,avg,sum``` ...等一系列查询,之后可能会添加更多支持,目前该库的使用非常简单,基本上就是针对IQueryable的扩展，为了保证
该库的简介目前仅使用该库无法或者说难以实现自动建表,但是只需要配合定时任务该库即可完成24小时无人看管自动管理。该库提供了 [IShardingTableCreator](https://github.com/xuejmnet/sharding-core/blob/main/src/ShardingCore/TableCreator/IShardingTableCreator.cs)
作为建表的依赖,如果需要可以参考 [按天自动建表](https://github.com/xuejmnet/sharding-core/tree/main/samples/Samples.AutoByDate.SqlServer)
## 概念
本库的几个简单的核心概念:
- [Tail]
尾巴、后缀物理表的后缀
- [TailPrefix]
尾巴前缀虚拟表和物理表的后缀中间的字符
- [物理表]
顾名思义就是数据库对应的实际表信息,表名(tablename+ tailprefix+ tail) [IPhysicTable](https://github.com/xuejmnet/sharding-core/blob/main/src/ShardingCore/Core/PhysicTables/IPhysicTable.cs)
- [虚拟表]
虚拟表就是系统将所有的物理表在系统里面进行抽象的一个总表对应到程序就是一个entity[IVirtualTable](https://github.com/xuejmnet/sharding-core/blob/main/src/ShardingCore/Core/VirtualTables/IVirtualTable.cs)
- [虚拟路由]
虚拟路由就是联系虚拟表和物理表的中间介质,虚拟表在整个程序中只有一份,那么程序如何知道要查询系统哪一张表呢,最简单的方式就是通过虚拟表对应的路由[IVirtualRoute](https://github.com/xuejmnet/sharding-core/blob/main/src/ShardingCore/Core/VirtualRoutes/IVirtualRoute.cs)
,由于基本上所有的路由都是和业务逻辑相关的所以虚拟路由由用户自己实现,该框架提供一个高级抽象
## 优点
- [支持自定义分表规则]
- [支持任意类型分表key]
- [针对iqueryable的扩展方便使用]
- [支持分表下的连表] ```join```
- [支持针对批处理的使用] ``` BulkInsert、BulkUpdate、BulkDelete```
- [提供多种默认分表规则路由] 按时间按取模
- [针对分页进行优化] 大页数跳转支持低内存流式处理
## 缺点
- [暂不支持分库(不久后会支持)]
- [消耗连接]出现分表与分表对象进行join如果条件没法索引到具体表会生成```笛卡尔积```导致连接数爆炸,后期会进行针对该情况的配置
- [该库比较年轻] 可能会有一系列bug或者单元测试不到位的情况,但是只要你在群里或者提了issues我会尽快解决
## 安装
```xml
<PackageReference Include="ShardingCore.SqlServer" Version="5.0.0.4" />
```
## 配置
配置entity 推荐 [fluent api](https://docs.microsoft.com/en-us/ef/core/modeling/) 可以实现自动建表功能
`IShardingEntity`数据库对象必须继承该接口
`ShardingKey`分表字段需要使用该特性
```c#
public class SysUserMod:IShardingEntity
{
/// <summary>
/// 用户Id用于分表
/// </summary>
[ShardingKey]
public string Id { get; set; }
/// <summary>
/// 用户名称
/// </summary>
public string Name { get; set; }
/// <summary>
/// 用户姓名
/// </summary>
public int Age { get; set; }
}
public class SysUserModMap:IEntityTypeConfiguration<SysUserMod>
{
public void Configure(EntityTypeBuilder<SysUserMod> builder)
{
builder.HasKey(o => o.Id);
builder.Property(o => o.Id).IsRequired().HasMaxLength(128);
builder.Property(o => o.Name).HasMaxLength(128);
builder.ToTable(nameof(SysUserMod));
}
}
```
创建virtual route
实现 `AbstractShardingOperatorVirtualRoute<T, TKey>`
抽象,或者实现系统默认的虚拟路由
框架默认有提供几个简单的路由 [默认路由](#默认路由)
```c#
public class SysUserModVirtualRoute : AbstractSimpleShardingModKeyStringVirtualRoute<SysUserMod>
{
public SysUserModVirtualRoute() : base(3)
{
}
}
```
`Startup.cs` 下的 `ConfigureServices(IServiceCollection services)`
```c#
services.AddShardingSqlServer(o =>
{
o.ConnectionString = "";
o.AddSharding<SysUserModVirtualRoute>();
o.UseShardingCoreConfig((provider, config) =>
{
//如果是development就判断并且新建数据库如果不存在的话(ishardingentity不会被创建)
config.EnsureCreated = provider.GetService<IHostEnvironment>().IsDevelopment();
//ishardingentity表是否需要在启动时创建(如果已创建可以选择不创建)
config.CreateShardingTableOnStart = true;
});
});
```
`Startup.cs` 下的 ` Configure(IApplicationBuilder app, IWebHostEnvironment env)` 你也可以自行封装[app.UseShardingCore()](https://github.com/xuejmnet/sharding-core/blob/main/samples/Sample.SqlServer/DIExtension.cs)
```c#
var shardingBootstrapper = app.ApplicationServices.GetRequiredService<IShardingBootstrapper>();
shardingBootstrapper.Start();
```
## 使用
```c#
private readonly IVirtualDbContext _virtualDbContext;
public ctor(IVirtualDbContext virtualDbContext)
{
_virtualDbContext = virtualDbContext;
}
public async Task ToList_All()
{
//查询list集合
var all=await _virtualDbContext.Set<SysUserMod>().ToShardingListAsync();
//链接查询
var list = await (from u in _virtualDbContext.Set<SysUserMod>()
join salary in _virtualDbContext.Set<SysUserSalary>()
on u.Id equals salary.UserId
select new
{
Salary = salary.Salary,
DateOfMonth = salary.DateOfMonth,
Name = u.Name
}).ToShardingListAsync();
//聚合查询
var ids = new[] {"200", "300"};
var dateOfMonths = new[] {202111, 202110};
var group = await (from u in _virtualDbContext.Set<SysUserSalary>()
.Where(o => ids.Contains(o.UserId) && dateOfMonths.Contains(o.DateOfMonth))
group u by new
{
UId = u.UserId
}
into g
select new
{
GroupUserId = g.Key.UId,
Count = g.Count(),
TotalSalary = g.Sum(o => o.Salary),
AvgSalary = g.Average(o => o.Salary),
MinSalary = g.Min(o => o.Salary),
MaxSalary = g.Max(o => o.Salary)
}).ToShardingListAsync();
}
```
更多操作可以参考单元测试
## Api
方法 | Method | [SqlServer Unit Test](https://github.com/xuejmnet/sharding-core/blob/main/test/ShardingCore.Test50/ShardingTest.cs) | [MySql Unit Test](https://github.com/xuejmnet/sharding-core/blob/main/test/ShardingCore.Test50.MySql/ShardingTest.cs)
--- |--- |--- |---
获取集合 |ToShardingListAsync |yes |yes
第一条 |ShardingFirstOrDefaultAsync |yes |yes
最大 |ShardingMaxAsync |yes |yes
最小 |ShardingMinAsync |yes |yes
是否存在 |ShardingAnyAsync |yes |yes
分页 |ToShardingPageResultAsync |yes |yes
数目 |ShardingCountAsync |yes |yes
求和 |ShardingSumAsync |yes |yes
分组 |ShardingGroupByAsync |yes |yes
## 默认路由
抽象abstract | 路由规则 | tail | 索引
--- |--- |--- |---
AbstractSimpleShardingModKeyIntVirtualRoute |取模 |0,1,2... | `=`
AbstractSimpleShardingModKeyStringVirtualRoute |取模 |0,1,2... | `=`
AbstractSimpleShardingDayKeyDateTimeVirtualRoute |按时间 |yyyyMMdd | `>,>=,<,<=,=,contains`
AbstractSimpleShardingDayKeyLongVirtualRoute |按时间戳 |yyyyMMdd | `>,>=,<,<=,=,contains`
AbstractSimpleShardingWeekKeyDateTimeVirtualRoute |按时间 |yyyyMMdd_dd | `>,>=,<,<=,=,contains`
AbstractSimpleShardingWeekKeyLongVirtualRoute |按时间戳 |yyyyMMdd_dd | `>,>=,<,<=,=,contains`
AbstractSimpleShardingMonthKeyDateTimeVirtualRoute |按时间 |yyyyMM | `>,>=,<,<=,=,contains`
AbstractSimpleShardingMonthKeyLongVirtualRoute |按时间戳 |yyyyMM | `>,>=,<,<=,=,contains`
AbstractSimpleShardingYearKeyDateTimeVirtualRoute |按时间 |yyyy | `>,>=,<,<=,=,contains`
AbstractSimpleShardingYearKeyLongVirtualRoute |按时间戳 |yyyy | `>,>=,<,<=,=,contains`
注:`contains`表示为`o=>ids.contains(o.shardingkey)`
#高级
## 批量操作
批量操作将对应的dbcontext和数据进行分离由用户自己选择第三方框架比如zzz进行批量操作或者batchextension
```c#
virtualDbContext.BulkInsert<SysUserMod>(new List<SysUserMod>())
.BatchGroups.ForEach(pair =>
{
///zzz or other
pair.Key.BlukInsert(pair.Value);
});
var shardingBatchUpdateEntry = virtualDbContext.BulkUpdate<SysUserMod>(o => o.Id == "1", o => new SysUserMod()
{
Name = "name_01"
});
shardingBatchUpdateEntry.DbContexts.ForEach(context =>
{
//zzz or other
context.Where(shardingBatchUpdateEntry.Where).Update(shardingBatchUpdateEntry.UpdateExp);
});
```
## 手动路由
```c#
var shardingQueryable = _virtualDbContext.Set<SysUserMod>().AsSharding();
//禁用自动路由
shardingQueryable.DisableAutoRouteParse();
//添加路由直接查询尾巴0的表
shardingQueryable.AddManualRoute<SysUserMod>("0");
//添加路由针对该条件的路由
shardingQueryable.AddManualRoute<SysUserMod>(o=>o.Id=="100");
var list=await shardingQueryable.ToListAsync();
```
## 自动建表
[参考](https://github.com/xuejmnet/sharding-core/tree/main/samples/Samples.AutoByDate.SqlServer)
## 事务
默认savechanges支持事务如果需要where.update需要手动开启事务
```c#
_virtualDbContext.BeginTransaction();
var shardingBatchUpdateEntry = _virtualDbContext.BulkUpdate<SysUserMod>(o=>o.Id=="123",o=>new SysUserMod()
{
Name = "name_modify"
});
foreach (var dbContext in shardingBatchUpdateEntry.DbContexts)
{
//zzz or other batch
}
await _virtualDbContext.SaveChangesAsync();
```
# 注意事项
该库的IVirtualDbContext.Set<T>使用asnotracking所以基本不支持跟踪,目前框架采用AppDomain.CurrentDomain.GetAssemblies();
可能会导致程序集未被加载所以尽可能在api层加载所需要的dll
使用时需要注意
- 实体对象是否继承`IShardingEntity`
- 实体对象是否有`ShardingKey`
- 实体对象是否映射配置已实现`IEntityTypeConfiguration<TEntity>`
- 实体对象是否已经实现了一个虚拟路由
- startup是否已经添加虚拟路由
```c#
services.AddShardingSqlServer(o =>
{
o.ConnectionString = "";
o.AddSharding<SysUserModVirtualRoute>();
o.UseShardingCoreConfig((provider, config) =>
{
//如果是development就判断并且新建数据库如果不存在的话(ishardingentity不会被创建)
config.EnsureCreated = provider.GetService<IHostEnvironment>().IsDevelopment();
//ishardingentity表是否需要在启动时创建(如果已创建可以选择不创建)
config.CreateShardingTableOnStart = true;
});
});
```
- startup
```c#
var shardingBootstrapper = app.ApplicationServices.GetRequiredService<IShardingBootstrapper>();
shardingBootstrapper.Start();
```
# 计划
- [提供官网如果该项目比较成功的话]
- [开发更完善的文档]
- [支持分库]
- [支持更多数据库查询]
# 最后
该框架借鉴了大部分分表组件的思路,目前提供的接口都已经实现,并且支持跨表查询,基于分页查询该框架也使用了流式查询保证不会再skip大数据的时候内存会爆炸,目前这个库只是一个刚刚成型的库还有很多不完善的地方希望大家多多包涵,如果喜欢的话也希望大家给个star.
该文档是我晚上赶工赶出来的也想趁热打铁希望更多的人关注,也希望更多的人可以交流。
凭借各大开源生态圈提供的优秀代码和思路才有的这个框架,希望可以为.Net生态提供一份微薄之力,该框架本人会一直长期维护,有大神技术支持可以联系下方方式欢迎star :)
[博客](https://www.cnblogs.com/xuejiaming)
QQ群:771630778
个人QQ:326308290(欢迎技术支持提供您宝贵的意见)
个人邮箱:326308290@qq.com

4
samples/Sample.SqlServer/Controllers/ValuesController.cs
View File

@ -24,7 +24,9 @@ namespace Sample.SqlServer.Controllers
[HttpGet]
public async Task<IActionResult> Get()
{
return Ok(await _virtualDbContext.Set<SysUserMod>().ToShardingListAsync());
var result = await _virtualDbContext.Set<SysUserMod>().ToShardingListAsync();
var result1 = await _virtualDbContext.Set<SysUserMod>().Where(o=>o.Id=="2").ToShardingListAsync();
return Ok(result1);
}
}
}

12
samples/Sample.SqlServer/DbContexts/DefaultDbContext.cs
View File

@ -3,10 +3,22 @@ using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.EntityFrameworkCore;
using Sample.SqlServer.Domain.Maps;
namespace Sample.SqlServer.DbContexts
{
public class DefaultDbContext: DbContext
{
public DefaultDbContext(DbContextOptions<DefaultDbContext> options):base(options)
{
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.ApplyConfiguration(new SysUserModMap());
modelBuilder.ApplyConfiguration(new SysTestMap());
}
}
}

3
samples/Sample.SqlServer/Domain/Entities/SysTest.cs
View File

@ -2,6 +2,7 @@
{
public class SysTest
{
public string Id { get; set; }
public string UserId { get; set; }
}
}

16
samples/Sample.SqlServer/Domain/Maps/SysTestMap.cs
View File

@ -1,7 +1,17 @@
namespace Sample.SqlServer.Domain.Maps
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Metadata.Builders;
using Sample.SqlServer.Domain.Entities;
namespace Sample.SqlServer.Domain.Maps
{
public class SysTestMap
public class SysTestMap:IEntityTypeConfiguration<SysTest>
{
public void Configure(EntityTypeBuilder<SysTest> builder)
{
builder.HasKey(o => o.Id);
builder.Property(o => o.Id).IsRequired().HasMaxLength(128);
builder.Property(o => o.UserId).IsRequired().HasMaxLength(128);
builder.ToTable(nameof(SysTest));
}
}
}

1
samples/Sample.SqlServer/Properties/launchSettings.json
View File

@ -19,6 +19,7 @@
"commandName": "Project",
"dotnetRunMessages": "true",
"launchBrowser": true,
"launchUrl": "values/get",
"applicationUrl": "https://localhost:5001;http://localhost:5000",
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"

6
samples/Sample.SqlServer/Startup.cs
View File

@ -22,7 +22,9 @@ namespace Sample.SqlServer
services.AddControllers();
services.AddShardingSqlServer(o =>
{
o.AddShardingDbContext<DefaultDbContext>("conn1","123", dbConfig =>
o.EnsureCreatedWithOutShardingTable = true;
o.CreateShardingTableOnStart = true;
o.AddShardingDbContext<DefaultDbContext>("conn1", "Data Source=localhost;Initial Catalog=ShardingCoreDB123;Integrated Security=True", dbConfig =>
{
dbConfig.AddShardingTableRoute<SysUserModVirtualRoute>();
});
@ -43,7 +45,7 @@ namespace Sample.SqlServer
app.UseRouting();
app.UseEndpoints(endpoints => { endpoints.MapGet("/", async context => { await context.Response.WriteAsync("Hello World!"); }); });
app.UseEndpoints(endpoints => { endpoints.MapControllers(); });
}
}
}

10
src/ShardingCore.MySql/DIExtension.cs
View File

@ -42,11 +42,6 @@ namespace ShardingCore.MySql
services.AddScoped<IDbContextOptionsProvider, MySqlDbContextOptionsProvider>();
services.AddSingleton<IShardingParallelDbContextFactory, ShardingMySqlParallelDbContextFactory>();
services.AddSingleton(sp =>
{
var shardingCoreConfig = new ShardingCoreConfig();
return shardingCoreConfig;
});
services.AddSingleton<IShardingBootstrapper,ShardingBootstrapper>();
return services;
}
@ -56,5 +51,10 @@ namespace ShardingCore.MySql
optionsBuilder.ReplaceService<IQuerySqlGeneratorFactory, ShardingMySqlQuerySqlGeneratorFactory>();
return optionsBuilder;
}
public static DbContextOptionsBuilder<TContext> UseShardingSqlServerQuerySqlGenerator<TContext>(this DbContextOptionsBuilder<TContext> optionsBuilder) where TContext:DbContext
{
optionsBuilder.ReplaceService<IQuerySqlGeneratorFactory, ShardingMySqlQuerySqlGeneratorFactory>();
return optionsBuilder;
}
}
}

15
src/ShardingCore.MySql/MySqlDbContextOptionsProvider.cs
View File

@ -1,9 +1,10 @@
using System;
using System.Collections.Generic;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Query.Internal;
using Microsoft.Extensions.Logging;
using ShardingCore.Core.VirtualTables;
#if !EFCORE5
using MySql.Data.MySqlClient;
#endif
@ -44,7 +45,7 @@ namespace ShardingCore.MySql
{
var connectionString = _shardingCoreOptions.GetShardingConfig(connectKey).ConnectionString;
var connection = new MySqlConnection(connectionString);
var dbContextOptions= new DbContextOptionsBuilder()
var dbContextOptions= CreateDbContextOptionBuilder(connectKey)
#if EFCORE5
.UseMySql(connection, _mySqlOptions.ServerVersion, _mySqlOptions.MySqlOptionsAction)
#endif
@ -61,7 +62,17 @@ namespace ShardingCore.MySql
}
return _contextWrapItems[connectKey].ContextOptions;
}
private DbContextOptionsBuilder CreateDbContextOptionBuilder(string connectKey)
{
var shardingConfigEntry = _shardingCoreOptions.GetShardingConfig(connectKey);
Type type = typeof(DbContextOptionsBuilder<>);
type = type.MakeGenericType(shardingConfigEntry.DbContextType);
return (DbContextOptionsBuilder)Activator.CreateInstance(type);
}
/// <summary>
/// ÊÍ·Å×ÊÔ´
/// </summary>
public void Dispose()
{
_contextWrapItems.ForEach(o=>o.Value.Connection?.Dispose());

105
src/ShardingCore.MySql/MySqlOptions.cs
View File

@ -20,7 +20,7 @@ namespace ShardingCore.MySql
* @Date: 2020年4月7日 8:34:04
* @Email: 326308290@qq.com
*/
public class MySqlOptions
public class MySqlOptions: AbstractShardingCoreOptions
{
#if EFCORE5
public MySqlServerVersion ServerVersion { get; set; }
@ -32,108 +32,5 @@ namespace ShardingCore.MySql
{
MySqlOptionsAction = action;
}
private readonly IDictionary<string, ShardingConfigEntry> _shardingConfigs = new Dictionary<string, ShardingConfigEntry>();
public void AddShardingDbContext<TContext>(string connectKey, string connectString, Action<ShardingDbConfigOptions> func) where TContext : DbContext
{
if (_shardingConfigs.ContainsKey(connectKey))
{
throw new ArgumentException($"same connect key:[{connectKey}]");
}
CheckContextConstructors<TContext>();
var creator = CreateActivator<TContext>();
var config = new ShardingConfigEntry(connectKey, connectString, creator, typeof(TContext), func);
_shardingConfigs.Add(connectKey, config);
}
private static void CheckContextConstructors<TContext>()
where TContext : DbContext
{
var contextType = typeof(TContext);
var declaredConstructors = contextType.GetTypeInfo().DeclaredConstructors.ToList();
if (!contextType.IsShardingDbContext())
{
throw new ArgumentException($"dbcontext : {contextType} is assignable from {typeof(AbstractShardingDbContext)} ");
}
if (declaredConstructors.Count != 1)
{
throw new ArgumentException($"dbcontext : {contextType} declared constructor count more {contextType}");
}
if (declaredConstructors[0].GetParameters().Length != 1)
{
throw new ArgumentException($"dbcontext : {contextType} declared constructor parameters more ");
}
if (declaredConstructors[0].GetParameters()[0].ParameterType != typeof(ShardingDbContextOptions))
{
throw new ArgumentException($"dbcontext : {contextType} is assignable from {typeof(AbstractShardingDbContext)} declared constructor parameters should use {typeof(ShardingDbContextOptions)} ");
}
}
private static Func<ShardingDbContextOptions, DbContext> CreateActivator<TContext>() where TContext : DbContext
{
var constructors
= typeof(TContext).GetTypeInfo().DeclaredConstructors
.Where(c => !c.IsStatic && c.IsPublic)
.ToArray();
var parameters = constructors[0].GetParameters();
var parameterType = parameters[0].ParameterType;
var po = Expression.Parameter(parameterType, "o");
var new1 = Expression.New(constructors[0], po);
var inner = Expression.Lambda(new1, po);
var args = Expression.Parameter(typeof(ShardingDbContextOptions), "args");
var body = Expression.Invoke(inner, Expression.Convert(args, po.Type));
var outer = Expression.Lambda<Func<ShardingDbContextOptions, TContext>>(body, args);
var func = outer.Compile();
return func;
}
public void AddShardingDbContext<T>(string connectKey, string connectString) where T : DbContext
{
AddShardingDbContext<T>(connectKey, connectString, null);
}
private readonly Dictionary<Type, Type> _virtualRoutes = new Dictionary<Type, Type>();
public void AddDataSourceVirtualRoute<TRoute>() where TRoute : IDataSourceVirtualRoute
{
var virtualRouteType = typeof(TRoute);
//获取类型
var shardingEntityType = virtualRouteType.GetGenericArguments()[0];
if (shardingEntityType == null)
throw new ArgumentException("add sharding table route type error not assignable from IDataSourceVirtualRoute<>");
if (!_virtualRoutes.ContainsKey(shardingEntityType))
{
_virtualRoutes.Add(shardingEntityType, virtualRouteType);
}
}
public ISet<ShardingConfigEntry> GetShardingConfigs()
{
return _shardingConfigs.Select(o => o.Value).ToHashSet();
}
public ShardingConfigEntry GetShardingConfig(string connectKey)
{
if (!_shardingConfigs.ContainsKey(connectKey))
throw new ShardingConfigNotFoundException(connectKey);
return _shardingConfigs[connectKey];
}
public ISet<Type> GetVirtualRoutes()
{
return _virtualRoutes.Select(o => o.Value).ToHashSet();
}
public Type GetVirtualRoute(Type entityType)
{
if (!_virtualRoutes.ContainsKey(entityType))
throw new ArgumentException("not found IDataSourceVirtualRoute");
return _virtualRoutes[entityType];
}
}
}

14
src/ShardingCore.MySql/ShardingMySqlParallelDbContextFactory.cs
View File

@ -1,3 +1,4 @@
using System;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Query.Internal;
@ -34,13 +35,13 @@ namespace ShardingCore.MySql
{
var shardingConfigEntry = _shardingCoreOptions.GetShardingConfig(connectKey);
var virtualTableConfigs = _virtualTableManager.GetAllVirtualTables(connectKey).GetVirtualTableDbContextConfigs();
var shardingDbContextOptions = new ShardingDbContextOptions(CreateOptions(shardingConfigEntry.ConnectionString), tail, virtualTableConfigs);
var shardingDbContextOptions = new ShardingDbContextOptions(CreateOptions(connectKey,shardingConfigEntry.ConnectionString), tail, virtualTableConfigs);
return _shardingDbContextFactory.Create(connectKey, shardingDbContextOptions);
}
private DbContextOptions CreateOptions(string connectionString)
private DbContextOptions CreateOptions(string connectKey, string connectionString)
{
return new DbContextOptionsBuilder()
return CreateDbContextOptionBuilder(connectKey)
#if EFCORE5
.UseMySql(connectionString,_mySqlOptions.ServerVersion,_mySqlOptions.MySqlOptionsAction)
#endif
@ -54,5 +55,12 @@ namespace ShardingCore.MySql
.UseShardingSqlServerQuerySqlGenerator()
.Options;
}
private DbContextOptionsBuilder CreateDbContextOptionBuilder(string connectKey)
{
var shardingConfigEntry = _shardingCoreOptions.GetShardingConfig(connectKey);
Type type = typeof(DbContextOptionsBuilder<>);
type = type.MakeGenericType(shardingConfigEntry.DbContextType);
return (DbContextOptionsBuilder)Activator.CreateInstance(type);
}
}
}

10
src/ShardingCore.SqlServer/DIExtension.cs
View File

@ -40,11 +40,6 @@ namespace ShardingCore.SqlServer
services.AddScoped<IDbContextOptionsProvider, SqlServerDbContextOptionsProvider>();
services.AddSingleton<IShardingParallelDbContextFactory, ShardingSqlServerParallelDbContextFactory>();
services.AddSingleton(sp =>
{
var shardingCoreConfig = new ShardingCoreConfig();
return shardingCoreConfig;
});
services.AddSingleton<IShardingBootstrapper,ShardingBootstrapper>();
return services;
}
@ -54,5 +49,10 @@ namespace ShardingCore.SqlServer
optionsBuilder.ReplaceService<IQuerySqlGeneratorFactory, ShardingSqlServerQuerySqlGeneratorFactory>();
return optionsBuilder;
}
public static DbContextOptionsBuilder<TContext> UseShardingSqlServerQuerySqlGenerator<TContext>(this DbContextOptionsBuilder<TContext> optionsBuilder) where TContext : DbContext
{
optionsBuilder.ReplaceService<IQuerySqlGeneratorFactory, ShardingSqlServerQuerySqlGeneratorFactory>();
return optionsBuilder;
}
}
}

13
src/ShardingCore.SqlServer/ShardingSqlServerParallelDbContextFactory.cs
View File

@ -36,13 +36,13 @@ namespace ShardingCore.SqlServer
{
var shardingConfigEntry = _shardingCoreOptions.GetShardingConfig(connectKey);
var virtualTableConfigs = _virtualTableManager.GetAllVirtualTables(connectKey).GetVirtualTableDbContextConfigs();
var shardingDbContextOptions = new ShardingDbContextOptions(CreateOptions(shardingConfigEntry.ConnectionString), tail, virtualTableConfigs);
var shardingDbContextOptions = new ShardingDbContextOptions(CreateOptions(connectKey, shardingConfigEntry.ConnectionString), tail, virtualTableConfigs);
return _shardingDbContextFactory.Create(connectKey, shardingDbContextOptions);
}
private DbContextOptions CreateOptions(string connectString)
private DbContextOptions CreateOptions(string connectKey, string connectString)
{
return new DbContextOptionsBuilder()
return CreateDbContextOptionBuilder(connectKey)
.UseSqlServer(connectString)
.UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking)
.ReplaceService<IQueryCompiler, ShardingQueryCompiler>()
@ -50,5 +50,12 @@ namespace ShardingCore.SqlServer
.UseShardingSqlServerQuerySqlGenerator()
.Options;
}
private DbContextOptionsBuilder CreateDbContextOptionBuilder(string connectKey)
{
var shardingConfigEntry = _shardingCoreOptions.GetShardingConfig(connectKey);
Type type = typeof(DbContextOptionsBuilder<>);
type = type.MakeGenericType(shardingConfigEntry.DbContextType);
return (DbContextOptionsBuilder)Activator.CreateInstance(type);
}
}
}

16
src/ShardingCore.SqlServer/SqlServerDbContextOptionsProvider.cs
View File

@ -1,3 +1,4 @@
using System;
using System.Collections.Generic;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
@ -37,11 +38,11 @@ namespace ShardingCore.SqlServer
}
public DbContextOptions GetDbContextOptions(string connectKey)
{
if (_contextWrapItems.ContainsKey(connectKey))
if (!_contextWrapItems.ContainsKey(connectKey))
{
var connectionString = _shardingCoreOptions.GetShardingConfig(connectKey).ConnectionString;
var connection = new SqlConnection(connectionString);
var dbContextOptions = new DbContextOptionsBuilder()
var dbContextOptions = CreateDbContextOptionBuilder(connectKey)
.UseSqlServer(connection)
.UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking)
.UseLoggerFactory(_loggerFactory)
@ -53,7 +54,16 @@ namespace ShardingCore.SqlServer
}
return _contextWrapItems[connectKey].ContextOptions;
}
private DbContextOptionsBuilder CreateDbContextOptionBuilder(string connectKey)
{
var shardingConfigEntry = _shardingCoreOptions.GetShardingConfig(connectKey);
Type type = typeof(DbContextOptionsBuilder<>);
type = type.MakeGenericType(shardingConfigEntry.DbContextType);
return (DbContextOptionsBuilder)Activator.CreateInstance(type);
}
/// <summary>
/// ÊÍ·Å×ÊÔ´
/// </summary>
public void Dispose()
{
_contextWrapItems.ForEach(o => o.Value.Connection?.Dispose());

105
src/ShardingCore.SqlServer/SqlServerOptions.cs
View File

@ -22,110 +22,7 @@ namespace ShardingCore.SqlServer
* @Date: 2020年4月7日 8:34:04
* @Email: 326308290@qq.com
*/
public class SqlServerOptions: IShardingCoreOptions
public class SqlServerOptions: AbstractShardingCoreOptions
{
private readonly IDictionary<string,ShardingConfigEntry> _shardingConfigs = new Dictionary<string, ShardingConfigEntry>();
public void AddShardingDbContext<TContext>(string connectKey,string connectString,Action<ShardingDbConfigOptions> func) where TContext : DbContext
{
if (_shardingConfigs.ContainsKey(connectKey))
{
throw new ArgumentException($"same connect key:[{connectKey}]");
}
CheckContextConstructors<TContext>();
var creator = CreateActivator<TContext>();
var config = new ShardingConfigEntry(connectKey, connectString, creator, typeof(TContext), func);
_shardingConfigs.Add(connectKey, config);
}
private static void CheckContextConstructors<TContext>()
where TContext : DbContext
{
var contextType = typeof(TContext);
var declaredConstructors = contextType.GetTypeInfo().DeclaredConstructors.ToList();
if (!contextType.IsShardingDbContext())
{
throw new ArgumentException($"dbcontext : {contextType} is assignable from {typeof(AbstractShardingDbContext)} ");
}
if (declaredConstructors.Count != 1)
{
throw new ArgumentException($"dbcontext : {contextType} declared constructor count more {contextType}");
}
if (declaredConstructors[0].GetParameters().Length != 1)
{
throw new ArgumentException($"dbcontext : {contextType} declared constructor parameters more ");
}
if (declaredConstructors[0].GetParameters()[0].ParameterType != typeof(ShardingDbContextOptions))
{
throw new ArgumentException($"dbcontext : {contextType} is assignable from {typeof(AbstractShardingDbContext)} declared constructor parameters should use {typeof(ShardingDbContextOptions)} ");
}
}
private static Func<ShardingDbContextOptions, DbContext> CreateActivator<TContext>() where TContext:DbContext
{
var constructors
= typeof(TContext).GetTypeInfo().DeclaredConstructors
.Where(c => !c.IsStatic && c.IsPublic)
.ToArray();
var parameters = constructors[0].GetParameters();
var parameterType=parameters[0].ParameterType;
var po = Expression.Parameter(parameterType, "o");
var new1 = Expression.New(constructors[0], po);
var inner = Expression.Lambda(new1, po);
var args = Expression.Parameter(typeof(ShardingDbContextOptions), "args");
var body = Expression.Invoke(inner, Expression.Convert(args, po.Type));
var outer = Expression.Lambda<Func<ShardingDbContextOptions, TContext>>(body, args);
var func = outer.Compile();
return func;
}
public void AddShardingDbContext<T>(string connectKey, string connectString) where T : DbContext
{
AddShardingDbContext<T>(connectKey, connectString,null);
}
private readonly Dictionary<Type,Type> _virtualRoutes = new Dictionary<Type, Type>();
public void AddDataSourceVirtualRoute<TRoute>() where TRoute : IDataSourceVirtualRoute
{
var virtualRouteType = typeof(TRoute);
//获取类型
var shardingEntityType = virtualRouteType.GetGenericArguments()[0];
if (shardingEntityType == null)
throw new ArgumentException("add sharding table route type error not assignable from IDataSourceVirtualRoute<>");
if (!_virtualRoutes.ContainsKey(shardingEntityType))
{
_virtualRoutes.Add(shardingEntityType,virtualRouteType);
}
}
public ISet<ShardingConfigEntry> GetShardingConfigs()
{
return _shardingConfigs.Select(o=>o.Value).ToHashSet();
}
public ShardingConfigEntry GetShardingConfig(string connectKey)
{
if (!_shardingConfigs.ContainsKey(connectKey))
throw new ShardingConfigNotFoundException(connectKey);
return _shardingConfigs[connectKey];
}
public ISet<Type> GetVirtualRoutes()
{
return _virtualRoutes.Select(o=>o.Value).ToHashSet();
}
public Type GetVirtualRoute(Type entityType)
{
if (!_virtualRoutes.ContainsKey(entityType))
throw new ArgumentException("not found IDataSourceVirtualRoute");
return _virtualRoutes[entityType];
}
}
}

6
src/ShardingCore/Core/Internal/StreamMerge/GenericMerges/GenericStreamMergeEngine.cs
View File

@ -56,9 +56,11 @@ namespace ShardingCore.Core.Internal.StreamMerge.GenericMerges
public async Task<IStreamMergeAsyncEnumerator<T>> GetStreamEnumerator()
{
var enumeratorTasks = _mergeContext.GetDataSourceRoutingResult().IntersectConfigs.SelectMany(connectKey =>
var dataSourceResult = _mergeContext.GetDataSourceRoutingResult();
var enumeratorTasks = dataSourceResult.IntersectConfigs.SelectMany(connectKey =>
{
return _mergeContext.GetRouteResults(connectKey).Select(routeResult =>
var tableResult = _mergeContext.GetRouteResults(connectKey);
return tableResult.Select(routeResult =>
{
return Task.Run(async () =>
{

2
src/ShardingCore/Core/VirtualRoutes/DataSourceRoutes/RoutingRuleEngine/DataSourceRoutingRuleEngine.cs
View File

@ -36,6 +36,8 @@ namespace ShardingCore.Core.VirtualRoutes.DataSourceRoutes.RoutingRuleEngine
var dataSourceMaps = new Dictionary<Type, ISet<string>>();
foreach (var queryEntity in queryEntities)
{
if(!_virtualDataSourceManager.HasVirtualShardingDataSourceRoute(queryEntity))
continue;
var virtualDataSource = _virtualDataSourceManager.GetVirtualDataSource(queryEntity);
var dataSourceConfigs = virtualDataSource.RouteTo(new VirutalDataSourceRouteConfig(routingRuleContext.Queryable));
if (!dataSourceMaps.ContainsKey(queryEntity))

20
src/ShardingCore/DIExtension.cs
View File

@ -3,6 +3,8 @@ using Microsoft.Extensions.DependencyInjection;
using ShardingCore.Core.Internal.RoutingRuleEngines;
using ShardingCore.Core.Internal.StreamMerge;
using ShardingCore.Core.ShardingAccessors;
using ShardingCore.Core.VirtualDataSources;
using ShardingCore.Core.VirtualRoutes.DataSourceRoutes.RoutingRuleEngine;
using ShardingCore.Core.VirtualTables;
using ShardingCore.DbContexts;
using ShardingCore.DbContexts.VirtualDbContexts;
@ -21,14 +23,24 @@ namespace ShardingCore
public static IServiceCollection AddShardingCore(this IServiceCollection services)
{
services.AddScoped<IStreamMergeContextFactory, StreamMergeContextFactory>();
services.AddScoped<IRouteRuleEngine, QueryRouteRuleEngines>();
services.AddScoped<IRoutingRuleEngineFactory, RoutingRuleEngineFactory>();
services.AddSingleton<IStreamMergeContextFactory, StreamMergeContextFactory>();
services.AddScoped<IVirtualDbContext, VirtualDbContext>();
services.AddSingleton<IShardingDbContextFactory, ShardingDbContextFactory>();
services.AddSingleton<IShardingTableCreator, ShardingTableCreator>();
//分库
services.AddSingleton<IVirtualDataSourceManager, VirtualDataSourceManager>();
//分库路由引擎工厂
services.AddSingleton<IDataSourceRoutingRuleEngineFactory, DataSourceRoutingRuleEngineFactory>();
//分库引擎
services.AddSingleton<IDataSourceRoutingRuleEngine, DataSourceRoutingRuleEngine>();
//分表
services.AddSingleton<IVirtualTableManager, OneDbVirtualTableManager>();
services.AddSingleton(typeof(IVirtualTable<>), typeof(OneDbVirtualTable<>));
//分表引擎工程
services.AddSingleton<IRoutingRuleEngineFactory, RoutingRuleEngineFactory>();
//分表引擎
services.AddSingleton<IRouteRuleEngine, QueryRouteRuleEngines>();
//services.AddSingleton(typeof(IVirtualTable<>), typeof(OneDbVirtualTable<>));
services.AddSingleton<IShardingAccessor, ShardingAccessor>();
services.AddSingleton<IShardingScopeFactory, ShardingScopeFactory>();
return services;

2
src/ShardingCore/DbContexts/IShardingDbContextFactory.cs
View File

@ -1,5 +1,6 @@
using Microsoft.EntityFrameworkCore;
using ShardingCore.DbContexts.ShardingDbContexts;
using ShardingCore.DbContexts.VirtualDbContexts;
namespace ShardingCore.DbContexts
{
@ -12,5 +13,6 @@ namespace ShardingCore.DbContexts
public interface IShardingDbContextFactory
{
DbContext Create(string connectKey,ShardingDbContextOptions shardingDbContextOptions);
DbContext Create(string connectKey,string tail, IDbContextOptionsProvider dbContextOptionsProvider);
}
}

14
src/ShardingCore/DbContexts/ShardingDbContextFactory.cs
View File

@ -1,5 +1,8 @@
using Microsoft.EntityFrameworkCore;
using ShardingCore.Core.VirtualTables;
using ShardingCore.DbContexts.ShardingDbContexts;
using ShardingCore.DbContexts.VirtualDbContexts;
using ShardingCore.Extensions;
namespace ShardingCore.DbContexts
{
@ -12,15 +15,24 @@ namespace ShardingCore.DbContexts
public class ShardingDbContextFactory:IShardingDbContextFactory
{
private readonly IShardingCoreOptions _shardingCoreOptions;
private readonly IVirtualTableManager _virtualTableManager;
public ShardingDbContextFactory(IShardingCoreOptions shardingCoreOptions)
public ShardingDbContextFactory(IShardingCoreOptions shardingCoreOptions,IVirtualTableManager virtualTableManager)
{
_shardingCoreOptions = shardingCoreOptions;
_virtualTableManager = virtualTableManager;
}
public DbContext Create(string connectKey, ShardingDbContextOptions shardingDbContextOptions)
{
var shardingConfigEntry = _shardingCoreOptions.GetShardingConfig(connectKey);
return shardingConfigEntry.Creator(shardingDbContextOptions);
}
public DbContext Create(string connectKey, string tail,IDbContextOptionsProvider dbContextOptionsProvider)
{
var virtualTableConfigs = _virtualTableManager.GetAllVirtualTables(connectKey).GetVirtualTableDbContextConfigs();
var shardingConfigEntry = _shardingCoreOptions.GetShardingConfig(connectKey);
return shardingConfigEntry.Creator(new ShardingDbContextOptions(dbContextOptionsProvider.GetDbContextOptions(connectKey), tail, virtualTableConfigs));
}
}
}

5
src/ShardingCore/DbContexts/VirtualDbContexts/IDbContextOptionsProvider.cs
View File

@ -11,6 +11,11 @@ namespace ShardingCore.DbContexts.VirtualDbContexts
*/
public interface IDbContextOptionsProvider:IDisposable
{
/// <summary>
/// 创建数据库链接配置
/// </summary>
/// <param name="connectKey"></param>
/// <returns></returns>
DbContextOptions GetDbContextOptions(string connectKey);
}
}

3
src/ShardingCore/DbContexts/VirtualDbContexts/VirtualDbContext.cs
View File

@ -382,8 +382,7 @@ namespace ShardingCore.DbContexts.VirtualDbContexts
}
if(!dbContexts.TryGetValue(tail,out var dbContext))
{
var virtualTableConfigs = _virtualTableManager.GetAllVirtualTables(connectKey).GetVirtualTableDbContextConfigs();
dbContext = _shardingDbContextFactory.Create(connectKey, new ShardingDbContextOptions(DbContextOptionsProvider.GetDbContextOptions(connectKey), tail == EMPTY_SHARDING_TAIL_ID ? string.Empty : tail, virtualTableConfigs));
dbContext = _shardingDbContextFactory.Create(connectKey, tail == EMPTY_SHARDING_TAIL_ID ? string.Empty : tail, DbContextOptionsProvider);
dbContexts.TryAdd(tail, dbContext);
}

27
src/ShardingCore/EFCores/ShardingModelCustomizer.cs Normal file
View File

@ -0,0 +1,27 @@
using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
namespace ShardingCore.EFCores
{
/*
* @Author: xjm
* @Description:
* @Date: 2021/3/8 14:54:51
* @Ver: 1.0
* @Email: 326308290@qq.com
*/
public class ShardingModelCustomizer: ModelCustomizer
{
public ShardingModelCustomizer(ModelCustomizerDependencies dependencies) : base(dependencies)
{
}
public override void Customize(ModelBuilder modelBuilder, DbContext context)
{
base.Customize(modelBuilder, context);
}
}
}

120
src/ShardingCore/Helpers/ShardingCoreHelper.cs
View File

@ -1,4 +1,11 @@
using System;
using System.Linq;
using System.Linq.Expressions;
using System.Reflection;
using Microsoft.EntityFrameworkCore;
using ShardingCore.DbContexts.ShardingDbContexts;
using ShardingCore.Exceptions;
using ShardingCore.Extensions;
namespace ShardingCore.Helpers
{
@ -63,5 +70,118 @@ namespace ShardingCore.Helpers
{
return GetCurrentMonday(time).AddDays(6);
}
public static void CheckContextConstructors<TContext>()
where TContext : DbContext
{
var contextType = typeof(TContext);
var declaredConstructors = contextType.GetTypeInfo().DeclaredConstructors.ToList();
if (declaredConstructors.Count != 1)
{
throw new ArgumentException($"dbcontext : {contextType} declared constructor count more {contextType}");
}
if (declaredConstructors[0].GetParameters().Length != 1)
{
throw new ArgumentException($"dbcontext : {contextType} declared constructor parameters more ");
}
var paramType = declaredConstructors[0].GetParameters()[0].ParameterType;
if (paramType != typeof(ShardingDbContextOptions) && paramType != typeof(DbContextOptions) && paramType!= typeof(DbContextOptions<TContext>))
{
throw new ArgumentException($"dbcontext : {contextType} declared constructor parameters should use {typeof(ShardingDbContextOptions)} or {typeof(DbContextOptions)} or {typeof(DbContextOptions<TContext>)} ");
}
//if (!contextType.IsShardingDbContext())
//{
// throw new ArgumentException($"dbcontext : {contextType} is assignable from {typeof(AbstractShardingDbContext)} ");
//}
//if (declaredConstructors[0].GetParameters()[0].ParameterType != typeof(ShardingDbContextOptions))
//{
// throw new ArgumentException($"dbcontext : {contextType} is assignable from {typeof(AbstractShardingDbContext)} declared constructor parameters should use {typeof(ShardingDbContextOptions)} ");
//}
}
public static Func<ShardingDbContextOptions, DbContext> CreateActivator<TContext>() where TContext : DbContext
{
var constructors
= typeof(TContext).GetTypeInfo().DeclaredConstructors
.Where(c => !c.IsStatic && c.IsPublic)
.ToArray();
var parameters = constructors[0].GetParameters();
var parameterType = parameters[0].ParameterType;
if (parameterType == typeof(ShardingDbContextOptions))
{
return CreateShardingDbContextOptionsActivator<TContext>(constructors[0],parameterType);
}
else if (typeof(DbContextOptions).IsAssignableFrom(parameterType))
{
if ((parameters[0].ParameterType != typeof(DbContextOptions)
&& parameters[0].ParameterType != typeof(DbContextOptions<TContext>)))
{
throw new ShardingCoreException("cant create activator");
}
return CreateDbContextOptionsGenericActivator<TContext>(constructors[0], parameterType);
}
var po = Expression.Parameter(parameterType, "o");
var new1 = Expression.New(constructors[0], po);
var inner = Expression.Lambda(new1, po);
var args = Expression.Parameter(typeof(ShardingDbContextOptions), "args");
var body = Expression.Invoke(inner, Expression.Convert(args, po.Type));
var outer = Expression.Lambda<Func<ShardingDbContextOptions, TContext>>(body, args);
var func = outer.Compile();
return func;
}
/// <summary>
/// {args => Invoke(x => new DbContext(x), Convert(args, ShardingDbContextOptions))}
/// </summary>
/// <typeparam name="TContext"></typeparam>
/// <param name="constructor"></param>
/// <param name="paramType"></param>
/// <returns></returns>
private static Func<ShardingDbContextOptions, DbContext> CreateShardingDbContextOptionsActivator<TContext>(ConstructorInfo constructor,Type paramType) where TContext : DbContext
{
var po = Expression.Parameter(paramType, "o");
var newExpression = Expression.New(constructor, po);
var inner = Expression.Lambda(newExpression, po);
var args = Expression.Parameter(typeof(ShardingDbContextOptions), "args");
var body = Expression.Invoke(inner, Expression.Convert(args, po.Type));
var outer = Expression.Lambda<Func<ShardingDbContextOptions, TContext>>(body, args);
var func = outer.Compile();
return func;
}
/// <summary>
/// {args => Invoke(o => new DefaultDbContext(Convert(o.DbContextOptions, DbContextOptions`1)), Convert(args, ShardingDbContextOptions))}
/// </summary>
/// <typeparam name="TContext"></typeparam>
/// <param name="constructor"></param>
/// <param name="paramType"></param>
/// <returns></returns>
private static Func<ShardingDbContextOptions, DbContext> CreateDbContextOptionsGenericActivator<TContext>(ConstructorInfo constructor,Type paramType) where TContext : DbContext
{
var parameterExpression = Expression.Parameter(typeof(ShardingDbContextOptions), "o");
//o.DbContextOptions
var paramMemberExpression = Expression.Property(parameterExpression, nameof(ShardingDbContextOptions.DbContextOptions));
var newExpression = Expression.New(constructor, Expression.Convert(paramMemberExpression, paramType));
var inner = Expression.Lambda(newExpression, parameterExpression);
var args = Expression.Parameter(typeof(ShardingDbContextOptions), "args");
var body = Expression.Invoke(inner, Expression.Convert(args, parameterExpression.Type));
var outer = Expression.Lambda<Func<ShardingDbContextOptions, TContext>>(body, args);
var func = outer.Compile();
return func;
}
}
}

11
src/ShardingCore/IShardingCoreOptions.cs
View File

@ -28,5 +28,16 @@ namespace ShardingCore
ISet<Type> GetVirtualRoutes();
Type GetVirtualRoute(Type entityType);
/// <summary>
/// 如果数据库不存在就创建并且创建表除了分表的
/// </summary>
bool EnsureCreatedWithOutShardingTable { get; set; }
/// <summary>
/// 是否需要在启动时创建分表
/// </summary>
bool? CreateShardingTableOnStart { get; set; }
}
}

37
src/ShardingCore/ShardingBootstrapper.cs
View File

@ -1,8 +1,10 @@
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Metadata.Internal;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using ShardingCore.Core.PhysicTables;
@ -34,11 +36,10 @@ namespace ShardingCore
private readonly IShardingTableCreator _tableCreator;
private readonly ILogger<ShardingBootstrapper> _logger;
private readonly IShardingDbContextFactory _shardingDbContextFactory;
private readonly ShardingCoreConfig _shardingCoreConfig;
public ShardingBootstrapper(IServiceProvider serviceProvider, IShardingCoreOptions shardingCoreOptions, IVirtualDataSourceManager virtualDataSourceManager, IVirtualTableManager virtualTableManager
, IShardingTableCreator tableCreator, ILogger<ShardingBootstrapper> logger,
IShardingDbContextFactory shardingDbContextFactory, ShardingCoreConfig shardingCoreConfig)
IShardingDbContextFactory shardingDbContextFactory)
{
ShardingContainer.SetServices(serviceProvider);
_serviceProvider = serviceProvider;
@ -48,12 +49,11 @@ namespace ShardingCore
_tableCreator = tableCreator;
_logger = logger;
_shardingDbContextFactory = shardingDbContextFactory;
_shardingCoreConfig = shardingCoreConfig;
}
public void Start()
{
//EnsureCreated();
EnsureCreated();
//_shardingCoreOptions.GetShardingConfigs().Select(o=>o.ConnectKey).ForEach(connectKey=> _virtualDataSourceManager.AddShardingConnectKey(connectKey));
var isShardingDataSource = _shardingCoreOptions.GetShardingConfigs().Count > 1;
foreach (var virtualRouteType in _shardingCoreOptions.GetVirtualRoutes())
@ -74,7 +74,7 @@ namespace ShardingCore
{
_virtualDataSourceManager.AddConnectEntities(connectKey,entity.ClrType);
if (entity.IsShardingTable())
if (entity.ClrType.IsShardingTable())
{
var routeType = shardingConfig.DbConfigOptions.GetVirtualRoute(entity.ClrType);
var virtualRoute = CreateVirtualRoute(routeType);
@ -157,16 +157,21 @@ namespace ShardingCore
return (IVirtualTable)o;
}
//private void EnsureCreated()
//{
// if (_shardingCoreConfig.EnsureCreated)
// {
// using var scope = _serviceProvider.CreateScope();
// var dbContextOptionsProvider = scope.ServiceProvider.GetService<IDbContextOptionsProvider>();
// using var context = _shardingDbContextFactory.Create(new ShardingTableDbContextOptions(dbContextOptionsProvider.GetDbContextOptions(), string.Empty, new List<VirtualTableDbContextConfig>(), true));
// context.Database.EnsureCreated();
// }
//}
private void EnsureCreated()
{
if (_shardingCoreOptions.EnsureCreatedWithOutShardingTable)
{
foreach (var shardingConfig in _shardingCoreOptions.GetShardingConfigs())
{
using var scope = _serviceProvider.CreateScope();
var dbContextOptionsProvider = scope.ServiceProvider.GetService<IDbContextOptionsProvider>();
using var context = _shardingDbContextFactory.Create(shardingConfig.ConnectKey,new ShardingDbContextOptions(dbContextOptionsProvider.GetDbContextOptions(shardingConfig.ConnectKey), string.Empty, new List<VirtualTableDbContextConfig>()));
var contextModel = context.Model as Model;
context.Database.EnsureCreated();
}
}
}
private bool NeedCreateTable(ShardingEntityConfig config)
{
@ -174,7 +179,7 @@ namespace ShardingCore
{
return config.AutoCreateTable.Value;
}
return _shardingCoreConfig.CreateShardingTableOnStart.GetValueOrDefault();
return _shardingCoreOptions.CreateShardingTableOnStart.GetValueOrDefault();
}
private void CreateDataTable(string connectKey,IVirtualTable virtualTable)
{

22
src/ShardingCore/ShardingCoreConfig.cs
View File

@ -1,22 +0,0 @@
namespace ShardingCore
{
/*
* @Author: xjm
* @Description:
* @Date: Thursday, 14 January 2021 15:50:46
* @Email: 326308290@qq.com
*/
public class ShardingCoreConfig
{
/// <summary>
/// 如果数据库不存在就创建并且创建表
/// </summary>
public bool EnsureCreated { get; set; }
/// <summary>
/// 是否需要在启动时创建分表
/// </summary>
public bool? CreateShardingTableOnStart { get; set; }
}
}

7
src/ShardingCore/ShardingDbConfigOptions.cs
View File

@ -24,7 +24,12 @@ namespace ShardingCore
{
var routeType = typeof(TRoute);
//获取类型
var shardingEntityType = routeType.GetGenericArguments()[0];
var genericVirtualRoute = routeType.GetInterfaces().FirstOrDefault(it => it.IsInterface && it.IsGenericType && it.GetGenericTypeDefinition() == typeof(IVirtualRoute<>)
&& it.GetGenericArguments().Any());
if (genericVirtualRoute == null)
throw new ArgumentException("add sharding route type error not assignable from IVirtualRoute<>.");
var shardingEntityType = genericVirtualRoute.GetGenericArguments()[0];
if (shardingEntityType == null)
throw new ArgumentException("add sharding table route type error not assignable from IVirtualRoute<>");
if (!_virtualRoutes.ContainsKey(shardingEntityType))

2
src/ShardingCore/VirtualRoutes/Mods/AbstractSimpleShardingModKeyIntVirtualRoute.cs
View File

@ -48,7 +48,7 @@ namespace ShardingCore.VirtualRoutes.Mods
public override List<string> GetAllTails()
{
return Enumerable.Range(0, Mod).Select(o => o.ToString()).ToList();
return Enumerable.Range(0, Mod).Select(o => o.ToString().PadLeft(TailLength, PaddingChar)).ToList();
}
protected override Expression<Func<string, bool>> GetRouteToFilter(int shardingKey, ShardingOperatorEnum shardingOperator)

2
src/ShardingCore/VirtualRoutes/Mods/AbstractSimpleShardingModKeyStringVirtualRoute.cs
View File

@ -52,7 +52,7 @@ namespace ShardingCore.VirtualRoutes.Mods
}
public override List<string> GetAllTails()
{
return Enumerable.Range(0, Mod).Select(o => o.ToString()).ToList();
return Enumerable.Range(0, Mod).Select(o => o.ToString().PadLeft(TailLength, PaddingChar)).ToList();
}
protected override Expression<Func<string, bool>> GetRouteToFilter(string shardingKey, ShardingOperatorEnum shardingOperator)