Autotest
/
sharding
mirror of https://gitee.com/dotnetchina/sharding-core.git
1
0
Fork 0
Code Issues Releases Wiki Activity

完成readme

This commit is contained in:
xuejiaming 2022-07-06 13:34:53 +08:00
parent fb1c2cfdd7
commit 3c5f9c13fd
3 changed files with 89 additions and 92 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

97
README-zh.md
View File

@ -166,10 +166,11 @@ dbcontext `AbstractShardingDbContext`和`IShardingTableDbContext`如果你是普
//额外添加分片配置
services.AddShardingDbContext<MyDbContext>()
.AddEntityConfig(op =>
.UseRouteConfig(op =>
{
op.AddShardingTableRoute<OrderVirtualTableRoute>();
}).UseConfig(op =>
{
op.CreateShardingTableOnStart = true;
op.EnsureCreatedWithOutShardingTable = true;
op.UseShardingQuery((conn, builder) =>
{
builder.UseSqlServer(conn);
@ -178,10 +179,6 @@ dbcontext `AbstractShardingDbContext`和`IShardingTableDbContext`如果你是普
{
builder.UseSqlServer(conn);
});
op.AddShardingTableRoute<OrderVirtualTableRoute>();
}).AddConfig(op =>
{
op.ConfigId = "c1";
op.AddDefaultDataSource(Guid.NewGuid().ToString("n"),
"Data Source=localhost;Initial Catalog=EFCoreShardingTableDB;Integrated Security=True;");
}).EnsureConfig();
@ -193,8 +190,10 @@ dbcontext `AbstractShardingDbContext`和`IShardingTableDbContext`如果你是普
{
app.UseDeveloperExceptionPage();
}
// it's importment don't forget it
app.ApplicationServices.GetRequiredService<IShardingBootstrapper>().Start();
//not required, enable job auto create table 非必须 启用自动创建表的任务
app.ApplicationServices.UseAutoShardingCreate();
//not required, enable check table missing and auto create,非必须 启动检查缺少的表并且创建
app.ApplicationServices.UseAutoTryCompensateTable();
// other configure....
}
```
@ -224,13 +223,14 @@ public class ValuesController : Controller
## 性能
以下所有数据均在开启了表达式编译缓存的情况下测试，并且电脑处于长时间未关机并且开着很多vs和idea的情况下仅供参考,所有测试都是基于ShardingCore x.3.1.63+ version
Test
- on expression compile cache
- ShardingCore x.3.1.63+ version
- efcore 6.0 version
- order id is string, sharding mod(hashcode%5)
- N mean execute count
以下所有数据均在[源码中有案例](https://github.com/xuejmnet/sharding-core/blob/main/benchmarks/ShardingCoreBenchmark/EFCoreCrud.cs)
efcore版本均为6.0 表结构为string型id的订单取模分成5张表
N代表执行次数
[Benchmark Demo](https://github.com/xuejmnet/sharding-core/blob/main/benchmarks/ShardingCoreBenchmark/EFCoreCrud.cs)
### 性能损耗 sql server 2012,data rows 7734363 =773w
@ -239,8 +239,8 @@ N代表执行次数
BenchmarkDotNet=v0.13.1, OS=Windows 10.0.18363.1500 (1909/November2019Update/19H2)
AMD Ryzen 9 3900X, 1 CPU, 24 logical and 12 physical cores
.NET SDK=6.0.100
[Host] : .NET 6.0.0 (6.0.21.52210), X64 RyuJIT
DefaultJob : .NET 6.0.0 (6.0.21.52210), X64 RyuJIT
[Host] : .NET 6.0.0 (6.0.21.52210), X64 RyuJIT
DefaultJob : .NET 6.0.0 (6.0.21.52210), X64 RyuJIT
| Method | N | Mean | Error | StdDev |
@ -261,24 +261,23 @@ AMD Ryzen 9 3900X, 1 CPU, 24 logical and 12 physical cores
BenchmarkDotNet=v0.13.1, OS=Windows 10.0.18363.1500 (1909/November2019Update/19H2)
AMD Ryzen 9 3900X, 1 CPU, 24 logical and 12 physical cores
.NET SDK=6.0.100
[Host] : .NET 6.0.0 (6.0.21.52210), X64 RyuJIT
DefaultJob : .NET 6.0.0 (6.0.21.52210), X64 RyuJIT
.NET SDK=6.0.101
[Host] : .NET 6.0.1 (6.0.121.56705), X64 RyuJIT
DefaultJob : .NET 6.0.1 (6.0.121.56705), X64 RyuJIT
| Method | N | Mean | Error | StdDev | Median |
|------------------------------------- |--- |-------------:|------------:|------------:|-------------:|
| NoShardingIndexFirstOrDefaultAsync | 10 | 1.739 ms | 0.0340 ms | 0.0540 ms | 1.739 ms |
| ShardingIndexFirstOrDefaultAsync | 10 | 2.373 ms | 0.0460 ms | 0.0452 ms | 2.379 ms |
| NoShardingNoIndexFirstOrDefaultAsync | 10 | 579.584 ms | 15.7983 ms | 46.5816 ms | 564.566 ms |
| ShardingNoIndexFirstOrDefaultAsync | 10 | 628.567 ms | 12.5324 ms | 35.3478 ms | 615.352 ms |
| NoShardingNoIndexCountAsync | 10 | 521.954 ms | 9.7644 ms | 18.5778 ms | 523.128 ms |
| ShardingNoIndexCountAsync | 10 | 622.595 ms | 11.8567 ms | 10.5107 ms | 619.452 ms |
| NoShardingNoIndexLikeToListAsync | 10 | 6,352.417 ms | 123.3931 ms | 115.4220 ms | 6,360.908 ms |
| ShardingNoIndexLikeToListAsync | 10 | 6,260.610 ms | 122.6605 ms | 108.7353 ms | 6,236.577 ms |
| NoShardingNoIndexToListAsync | 10 | 491.013 ms | 4.0199 ms | 3.5635 ms | 490.473 ms |
| ShardingNoIndexToListAsync | 10 | 620.591 ms | 6.8447 ms | 5.7156 ms | 620.634 ms |
| Method | N | Mean | Error | StdDev |
|------------------------------------- |--- |-----------:|-----------:|-----------:|
| NoShardingIndexFirstOrDefaultAsync | 10 | 1.678 ms | 0.0323 ms | 0.0359 ms |
| ShardingIndexFirstOrDefaultAsync | 10 | 2.005 ms | 0.0161 ms | 0.0143 ms |
| NoShardingNoIndexFirstOrDefaultAsync | 10 | 495.933 ms | 9.4911 ms | 10.5494 ms |
| ShardingNoIndexFirstOrDefaultAsync | 10 | 596.112 ms | 11.8907 ms | 13.2165 ms |
| NoShardingNoIndexCountAsync | 10 | 477.537 ms | 1.4817 ms | 1.2373 ms |
| ShardingNoIndexCountAsync | 10 | 594.833 ms | 7.4057 ms | 5.7819 ms |
| NoShardingNoIndexLikeToListAsync | 10 | 665.277 ms | 1.3382 ms | 1.1174 ms |
| ShardingNoIndexLikeToListAsync | 10 | 840.865 ms | 16.1917 ms | 17.3249 ms |
| NoShardingNoIndexToListAsync | 10 | 480.368 ms | 1.3688 ms | 1.2134 ms |
| ShardingNoIndexToListAsync | 10 | 604.850 ms | 8.6204 ms | 8.0635 ms |
#### mysql 5.7,data rows 7553790=755w innerdb_buffer_size=3G
@ -288,25 +287,25 @@ AMD Ryzen 9 3900X, 1 CPU, 24 logical and 12 physical cores
BenchmarkDotNet=v0.13.1, OS=Windows 10.0.18363.1500 (1909/November2019Update/19H2)
AMD Ryzen 9 3900X, 1 CPU, 24 logical and 12 physical cores
.NET SDK=6.0.100
[Host] : .NET 6.0.0 (6.0.21.52210), X64 RyuJIT
DefaultJob : .NET 6.0.0 (6.0.21.52210), X64 RyuJIT
.NET SDK=6.0.101
[Host] : .NET 6.0.1 (6.0.121.56705), X64 RyuJIT
DefaultJob : .NET 6.0.1 (6.0.121.56705), X64 RyuJIT
| Method | N | Mean | Error | StdDev | Median |
|------------------------------------- |--- |--------------:|------------:|--------------:|--------------:|
| NoShardingIndexFirstOrDefaultAsync | 10 | 4.911 ms | 0.0952 ms | 0.1133 ms | 4.923 ms |
| ShardingIndexFirstOrDefaultAsync | 10 | 5.736 ms | 0.1139 ms | 0.3020 ms | 5.630 ms |
| NoShardingNoIndexFirstOrDefaultAsync | 10 | 11,630.109 ms | 774.0088 ms | 2,282.1824 ms | 11,585.457 ms |
| ShardingNoIndexFirstOrDefaultAsync | 10 | 5,388.529 ms | 39.1442 ms | 36.6155 ms | 5,391.835 ms |
| NoShardingNoIndexCountAsync | 10 | 14,245.844 ms | 74.1221 ms | 69.3339 ms | 14,242.815 ms |
| ShardingNoIndexCountAsync | 10 | 3,007.845 ms | 24.6299 ms | 23.0388 ms | 3,007.830 ms |
| NoShardingNoIndexLikeToListAsync | 10 | 27,026.048 ms | 145.6814 ms | 121.6505 ms | 27,032.112 ms |
| ShardingNoIndexLikeToListAsync | 10 | 5,650.041 ms | 94.9405 ms | 88.8074 ms | 5,622.049 ms |
| NoShardingNoIndexToListAsync | 10 | 26,068.783 ms | 103.7831 ms | 97.0788 ms | 26,094.834 ms |
| ShardingNoIndexToListAsync | 10 | 5,414.644 ms | 71.2123 ms | 59.4655 ms | 5,395.306 ms |
| Method | N | Mean | Error | StdDev |
|------------------------------------- |--- |--------------:|------------:|------------:|
| NoShardingIndexFirstOrDefaultAsync | 10 | 5.646 ms | 0.0164 ms | 0.0145 ms |
| ShardingIndexFirstOrDefaultAsync | 10 | 5.679 ms | 0.0359 ms | 0.0319 ms |
| NoShardingNoIndexFirstOrDefaultAsync | 10 | 5,212.736 ms | 230.0841 ms | 678.4080 ms |
| ShardingNoIndexFirstOrDefaultAsync | 10 | 2,013.107 ms | 10.4256 ms | 9.2420 ms |
| NoShardingNoIndexCountAsync | 10 | 9,483.988 ms | 42.0931 ms | 39.3739 ms |
| ShardingNoIndexCountAsync | 10 | 2,029.698 ms | 12.4008 ms | 10.9929 ms |
| NoShardingNoIndexLikeToListAsync | 10 | 10,569.283 ms | 20.9163 ms | 16.3301 ms |
| ShardingNoIndexLikeToListAsync | 10 | 2,208.804 ms | 11.0483 ms | 10.3346 ms |
| NoShardingNoIndexToListAsync | 10 | 9,485.263 ms | 21.2558 ms | 17.7496 ms |
| ShardingNoIndexToListAsync | 10 | 2,012.086 ms | 39.2986 ms | 45.2563 ms |
具体可以通过first前两次结果来计算得出结论单次查询的的损耗为0.06-0.08毫秒之间， sqlserver的各项数据在分表和未分表的情况下都几乎差不多可以得出在770w数据集情况下数据库还并未是数据瓶颈的关键，但是mysql可以看到在分表和未分表的情况下如果涉及到没有索引的全表扫描那么性能的差距将是分表后的表数目之多，测试中为5-6倍，也就是分表数目
具体可以通过first前两次结果来计算得出结论单次查询的的损耗为0.04毫秒上下， sqlserver的各项数据在分表和未分表的情况下都几乎差不多可以得出在770w数据集情况下数据库还并未是数据瓶颈的关键，但是mysql可以看到在分表和未分表的情况下如果涉及到没有索引的全表扫描那么性能的差距将是分表后的表数目之多，测试中为5-6倍，也就是分表数目
- [使用介绍](#使用介绍)

82
README.md
View File

@ -175,10 +175,11 @@ u need modify `AddDefaultDataSource` method second param(connection string),don'
//sharding config
services.AddShardingDbContext<MyDbContext>()
.AddEntityConfig(op =>
.UseRouteConfig(op =>
{
op.AddShardingTableRoute<OrderVirtualTableRoute>();
}).UseConfig(op =>
{
op.CreateShardingTableOnStart = true;
op.EnsureCreatedWithOutShardingTable = true;
op.UseShardingQuery((connStr, builder) =>
{
//connStr is delegate input param
@ -189,14 +190,10 @@ u need modify `AddDefaultDataSource` method second param(connection string),don'
//connection is delegate input param
builder.UseSqlServer(connection);
});
op.AddShardingTableRoute<OrderVirtualTableRoute>();
}).AddConfig(op =>
{
op.ConfigId = "c1";
//use your data base connection string
op.AddDefaultDataSource(Guid.NewGuid().ToString("n"),
"Data Source=localhost;Initial Catalog=EFCoreShardingTableDB;Integrated Security=True;");
}).EnsureConfig();
}).AddShardingCore();
}
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
@ -205,8 +202,10 @@ u need modify `AddDefaultDataSource` method second param(connection string),don'
{
app.UseDeveloperExceptionPage();
}
// it's importment don't forget it
app.ApplicationServices.GetRequiredService<IShardingBootstrapper>().Start();
//not required, enable job auto create table 非必须 启用自动创建表的任务
app.ApplicationServices.UseAutoShardingCreate();
//not required, enable check table missing and auto create,非必须 启动检查缺少的表并且创建
app.ApplicationServices.UseAutoTryCompensateTable();
// other configure....
}
```
@ -278,24 +277,23 @@ AMD Ryzen 9 3900X, 1 CPU, 24 logical and 12 physical cores
BenchmarkDotNet=v0.13.1, OS=Windows 10.0.18363.1500 (1909/November2019Update/19H2)
AMD Ryzen 9 3900X, 1 CPU, 24 logical and 12 physical cores
.NET SDK=6.0.100
[Host] : .NET 6.0.0 (6.0.21.52210), X64 RyuJIT
DefaultJob : .NET 6.0.0 (6.0.21.52210), X64 RyuJIT
.NET SDK=6.0.101
[Host] : .NET 6.0.1 (6.0.121.56705), X64 RyuJIT
DefaultJob : .NET 6.0.1 (6.0.121.56705), X64 RyuJIT
| Method | N | Mean | Error | StdDev | Median |
|------------------------------------- |--- |-------------:|------------:|------------:|-------------:|
| NoShardingIndexFirstOrDefaultAsync | 10 | 1.739 ms | 0.0340 ms | 0.0540 ms | 1.739 ms |
| ShardingIndexFirstOrDefaultAsync | 10 | 2.373 ms | 0.0460 ms | 0.0452 ms | 2.379 ms |
| NoShardingNoIndexFirstOrDefaultAsync | 10 | 579.584 ms | 15.7983 ms | 46.5816 ms | 564.566 ms |
| ShardingNoIndexFirstOrDefaultAsync | 10 | 628.567 ms | 12.5324 ms | 35.3478 ms | 615.352 ms |
| NoShardingNoIndexCountAsync | 10 | 521.954 ms | 9.7644 ms | 18.5778 ms | 523.128 ms |
| ShardingNoIndexCountAsync | 10 | 622.595 ms | 11.8567 ms | 10.5107 ms | 619.452 ms |
| NoShardingNoIndexLikeToListAsync | 10 | 6,352.417 ms | 123.3931 ms | 115.4220 ms | 6,360.908 ms |
| ShardingNoIndexLikeToListAsync | 10 | 6,260.610 ms | 122.6605 ms | 108.7353 ms | 6,236.577 ms |
| NoShardingNoIndexToListAsync | 10 | 491.013 ms | 4.0199 ms | 3.5635 ms | 490.473 ms |
| ShardingNoIndexToListAsync | 10 | 620.591 ms | 6.8447 ms | 5.7156 ms | 620.634 ms |
| Method | N | Mean | Error | StdDev |
|------------------------------------- |--- |-----------:|-----------:|-----------:|
| NoShardingIndexFirstOrDefaultAsync | 10 | 1.678 ms | 0.0323 ms | 0.0359 ms |
| ShardingIndexFirstOrDefaultAsync | 10 | 2.005 ms | 0.0161 ms | 0.0143 ms |
| NoShardingNoIndexFirstOrDefaultAsync | 10 | 495.933 ms | 9.4911 ms | 10.5494 ms |
| ShardingNoIndexFirstOrDefaultAsync | 10 | 596.112 ms | 11.8907 ms | 13.2165 ms |
| NoShardingNoIndexCountAsync | 10 | 477.537 ms | 1.4817 ms | 1.2373 ms |
| ShardingNoIndexCountAsync | 10 | 594.833 ms | 7.4057 ms | 5.7819 ms |
| NoShardingNoIndexLikeToListAsync | 10 | 665.277 ms | 1.3382 ms | 1.1174 ms |
| ShardingNoIndexLikeToListAsync | 10 | 840.865 ms | 16.1917 ms | 17.3249 ms |
| NoShardingNoIndexToListAsync | 10 | 480.368 ms | 1.3688 ms | 1.2134 ms |
| ShardingNoIndexToListAsync | 10 | 604.850 ms | 8.6204 ms | 8.0635 ms |
#### mysql 5.7,data rows 7553790=755w innerdb_buffer_size=3G
@ -305,25 +303,25 @@ AMD Ryzen 9 3900X, 1 CPU, 24 logical and 12 physical cores
BenchmarkDotNet=v0.13.1, OS=Windows 10.0.18363.1500 (1909/November2019Update/19H2)
AMD Ryzen 9 3900X, 1 CPU, 24 logical and 12 physical cores
.NET SDK=6.0.100
[Host] : .NET 6.0.0 (6.0.21.52210), X64 RyuJIT
DefaultJob : .NET 6.0.0 (6.0.21.52210), X64 RyuJIT
.NET SDK=6.0.101
[Host] : .NET 6.0.1 (6.0.121.56705), X64 RyuJIT
DefaultJob : .NET 6.0.1 (6.0.121.56705), X64 RyuJIT
| Method | N | Mean | Error | StdDev | Median |
|------------------------------------- |--- |--------------:|------------:|--------------:|--------------:|
| NoShardingIndexFirstOrDefaultAsync | 10 | 4.911 ms | 0.0952 ms | 0.1133 ms | 4.923 ms |
| ShardingIndexFirstOrDefaultAsync | 10 | 5.736 ms | 0.1139 ms | 0.3020 ms | 5.630 ms |
| NoShardingNoIndexFirstOrDefaultAsync | 10 | 11,630.109 ms | 774.0088 ms | 2,282.1824 ms | 11,585.457 ms |
| ShardingNoIndexFirstOrDefaultAsync | 10 | 5,388.529 ms | 39.1442 ms | 36.6155 ms | 5,391.835 ms |
| NoShardingNoIndexCountAsync | 10 | 14,245.844 ms | 74.1221 ms | 69.3339 ms | 14,242.815 ms |
| ShardingNoIndexCountAsync | 10 | 3,007.845 ms | 24.6299 ms | 23.0388 ms | 3,007.830 ms |
| NoShardingNoIndexLikeToListAsync | 10 | 27,026.048 ms | 145.6814 ms | 121.6505 ms | 27,032.112 ms |
| ShardingNoIndexLikeToListAsync | 10 | 5,650.041 ms | 94.9405 ms | 88.8074 ms | 5,622.049 ms |
| NoShardingNoIndexToListAsync | 10 | 26,068.783 ms | 103.7831 ms | 97.0788 ms | 26,094.834 ms |
| ShardingNoIndexToListAsync | 10 | 5,414.644 ms | 71.2123 ms | 59.4655 ms | 5,395.306 ms |
| Method | N | Mean | Error | StdDev |
|------------------------------------- |--- |--------------:|------------:|------------:|
| NoShardingIndexFirstOrDefaultAsync | 10 | 5.646 ms | 0.0164 ms | 0.0145 ms |
| ShardingIndexFirstOrDefaultAsync | 10 | 5.679 ms | 0.0359 ms | 0.0319 ms |
| NoShardingNoIndexFirstOrDefaultAsync | 10 | 5,212.736 ms | 230.0841 ms | 678.4080 ms |
| ShardingNoIndexFirstOrDefaultAsync | 10 | 2,013.107 ms | 10.4256 ms | 9.2420 ms |
| NoShardingNoIndexCountAsync | 10 | 9,483.988 ms | 42.0931 ms | 39.3739 ms |
| ShardingNoIndexCountAsync | 10 | 2,029.698 ms | 12.4008 ms | 10.9929 ms |
| NoShardingNoIndexLikeToListAsync | 10 | 10,569.283 ms | 20.9163 ms | 16.3301 ms |
| ShardingNoIndexLikeToListAsync | 10 | 2,208.804 ms | 11.0483 ms | 10.3346 ms |
| NoShardingNoIndexToListAsync | 10 | 9,485.263 ms | 21.2558 ms | 17.7496 ms |
| ShardingNoIndexToListAsync | 10 | 2,012.086 ms | 39.2986 ms | 45.2563 ms |
具体可以通过first前两次结果来计算得出结论单次查询的的损耗为0.06-0.08毫秒之间， sqlserver的各项数据在分表和未分表的情况下都几乎差不多可以得出在770w数据集情况下数据库还并未是数据瓶颈的关键，但是mysql可以看到在分表和未分表的情况下如果涉及到没有索引的全表扫描那么性能的差距将是分表后的表数目之多，测试中为5-6倍，也就是分表数目
具体可以通过first前两次结果来计算得出结论单次查询的的损耗为0.04毫秒上下， sqlserver的各项数据在分表和未分表的情况下都几乎差不多可以得出在770w数据集情况下数据库还并未是数据瓶颈的关键，但是mysql可以看到在分表和未分表的情况下如果涉及到没有索引的全表扫描那么性能的差距将是分表后的表数目之多，测试中为5-6倍，也就是分表数目
- [使用介绍](#使用介绍)

2
samples/Samples.AbpSharding/Samples.AbpSharding.csproj
View File

@ -9,7 +9,7 @@
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\src5x\ShardingCore.5x\ShardingCore.5x.csproj" />
<ProjectReference Include="..\..\test\ShardingCore.Test5x\ShardingCore.Test5x.csproj" />
</ItemGroup>
</Project>