.Net The sub table and sub database of lower limit productivity are fully automated Migrations Code-First
## Introduce this paper ShardinfCore edition x.6.x.x+ Protagonist of this issue : - [`ShardingCore`](https://github.com/dotnetcore/sharding-core) a ef-core High performance 、 Lightweight solution for reading and writing separation of tables and Libraries , With zero dependency 、 Zero learning cost 、 Zero business code intrusion adaptationCatalog
Start
Our theme this time is ultimate productivity , The sub table and sub library are fully automated, which are beyond the reach of other languages Migrations Code-First Add efcore Senseless development of sub table and sub database
I still remember the last time I posted my blog , Last released how compatible WTM After the framework, there are also many small partners to ask me how to be compatible and how to migrate , Through the compatibility of so many frameworks, I have also realized some problems , For example, in ShardingCore Use before initialization ( After all efcore) The initialization of is that there is no need to call initialization manually during dependency injection , such as efcore.tool The problem of migration , This project cannot be migrated , because efcore.tool When using the command, it will not call Configure Which makes it impossible to initialize bug, You must create a new console program to migrate , It cannot be migrated within this project , Or code-first and ShardingCore The startup parameter conflict of requires trivial modification , And does not support sub databases , There was a little friend who divided before 300 If a library cannot be automatically migrated, it is really a headache , Although these problems are actually small things for sub database and sub table , But once the tables and databases reach a certain level, it will be difficult to maintain . therefore ShardingCore A new version has been launched in the last three weeks , The new version mainly solves the above pain points and makes the code more standard
Developing software is generally first to use , Then easy to use , Finally, standardize ,ShardingCore So it is with , Because you need to expand efcore So sometimes in unfamiliar efcore Only static classes can be used for injection access , Static classes are actually a very nonstandard usage , Unless you have to . So the new version x.6.x.x ShardingCore What did you bring? Please look down
Remove static container
The use of static containers leads to ShardingCore There is only one piece of data in the whole application declaration cycle , Then the data is shared, which is quite detrimental to the subsequent test maintenance and expansion , Not as isolated as a single case , So... Was removed ShardingContainer, By providing IShardingRuntimeContext To ensure access to the previous parameter structure , The same DbContext Types are using different IShardingRuntimeContext After that, it can show different characteristics of table and database .
Native efcore
First, we focus on native efcore Expand to separate databases and tables +code-first Automatic migration development
Add dependency ShardingCore 6.6.0.3 MySql
// Please install the latest version currently x.6.0.3+, First version number 6 representative efcore Version number of
Install-Package ShardingCore -Version 6.6.0.3
Install-Package Pomelo.EntityFrameworkCore.MySql -Version 6.0.1
Install-Package Microsoft.EntityFrameworkCore.Tools -Version 6.0.6
Create a todo Entity
public class TodoItem
{
public string Id { get; set; }
public string Text { get; set; }
}
establish dbcontext
Simply map the object to the database, of course DbSet+Attribute It's OK, too
public class MyDbContext:AbstractShardingDbContext,IShardingTableDbContext
{
public MyDbContext(DbContextOptions<MyDbContext> options) : base(options)
{
}
public IRouteTail RouteTail { get; set; }
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.Entity<TodoItem>(mb =>
{
mb.HasKey(o => o.Id);
mb.Property(o => o.Id).IsRequired().HasMaxLength(50).HasComment("id");
mb.Property(o => o.Text).IsRequired().HasMaxLength(256).HasComment(" Thing ");
mb.ToTable(nameof(TodoItem));
});
}
}
Create a new sub database and sub table route
Sub database routing
public class TodoItemDataSourceRoute:AbstractShardingOperatorVirtualDataSourceRoute<TodoItem,string>
{
/// <summary>
/// id Of hashcode Comodule taking 3 sub-treasury
/// </summary>
/// <param name="shardingKey"></param>
/// <returns></returns>
/// <exception cref="InvalidOperationException"></exception>
public override string ShardingKeyToDataSourceName(object shardingKey)
{
if (shardingKey == null) throw new InvalidOperationException("sharding key cant null");
var stringHashCode = ShardingCoreHelper.GetStringHashCode(shardingKey.ToString());
return $"ds{(Math.Abs(stringHashCode) % 3)}";//ds0,ds1,ds2
}
private readonly List<string> _dataSources = new List<string>() { "ds0", "ds1", "ds2" };
public override List<string> GetAllDataSourceNames()
{
return _dataSources;
}
public override bool AddDataSourceName(string dataSourceName)
{
throw new NotImplementedException();
}
/// <summary>
/// id sub-treasury
/// </summary>
/// <param name="builder"></param>
public override void Configure(EntityMetadataDataSourceBuilder<TodoItem> builder)
{
builder.ShardingProperty(o => o.Id);
}
public override Func<string, bool> GetRouteToFilter(string shardingKey, ShardingOperatorEnum shardingOperator)
{
var t = ShardingKeyToDataSourceName(shardingKey);
switch (shardingOperator)
{
case ShardingOperatorEnum.Equal: return tail => tail == t;
default:
{
return tail => true;
}
}
}
}
Table routing
public class TodoItemTableRoute:AbstractSimpleShardingModKeyStringVirtualTableRoute<TodoItem>
{
public TodoItemTableRoute() : base(2, 3)
{
}
/// <summary>
/// Under normal circumstances, the content will not be used as the fragment key, because as the fragment key, there is a premise that it will not be modified
/// </summary>
/// <param name="builder"></param>
public override void Configure(EntityMetadataTableBuilder<TodoItem> builder)
{
builder.ShardingProperty(o => o.Text);
}
}
New migration database script generation
public class ShardingMySqlMigrationsSqlGenerator:MySqlMigrationsSqlGenerator
{
private readonly IShardingRuntimeContext _shardingRuntimeContext;
public ShardingMySqlMigrationsSqlGenerator(MigrationsSqlGeneratorDependencies dependencies, IRelationalAnnotationProvider annotationProvider, IMySqlOptions options,IShardingRuntimeContext shardingRuntimeContext) : base(dependencies, annotationProvider, options)
{
_shardingRuntimeContext = shardingRuntimeContext;
}
protected override void Generate(
MigrationOperation operation,
IModel model,
MigrationCommandListBuilder builder)
{
var oldCmds = builder.GetCommandList().ToList();
base.Generate(operation, model, builder);
var newCmds = builder.GetCommandList().ToList();
var addCmds = newCmds.Where(x => !oldCmds.Contains(x)).ToList();
MigrationHelper.Generate(_shardingRuntimeContext,operation, builder, Dependencies.SqlGenerationHelper, addCmds);
}
}
Configure dependency injection
ILoggerFactory efLogger = LoggerFactory.Create(builder =>
{
builder.AddFilter((category, level) => category == DbLoggerCategory.Database.Command.Name && level == LogLevel.Information).AddConsole();
});
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddControllers();
builder.Services.AddShardingDbContext<MyDbContext>()
.UseRouteConfig(op =>
{
op.AddShardingTableRoute<TodoItemTableRoute>();
op.AddShardingDataSourceRoute<TodoItemDataSourceRoute>();
})
.UseConfig((sp,op) =>
{
op.UseShardingQuery((con, b) =>
{
b.UseMySql(con, new MySqlServerVersion(new Version()))
.UseLoggerFactory(efLogger);
});
op.UseShardingTransaction((con, b) =>
{
b.UseMySql(con, new MySqlServerVersion(new Version()))
.UseLoggerFactory(efLogger);
});
op.AddDefaultDataSource("ds0", "server=127.0.0.1;port=3306;database=mydb0;userid=root;password=root;");
op.AddExtraDataSource(sp=>new Dictionary<string, string>()
{
{"ds1", "server=127.0.0.1;port=3306;database=mydb1;userid=root;password=root;"},
{"ds2", "server=127.0.0.1;port=3306;database=mydb2;userid=root;password=root;"}
});
op.UseShardingMigrationConfigure(b =>
{
b.ReplaceService<IMigrationsSqlGenerator, ShardingMySqlMigrationsSqlGenerator>();
});
}).AddShardingCore();
var app = builder.Build();
// Configure the HTTP request pipeline.
// If there is a need to add timed tasks according to the time slice, otherwise you can not add
app.Services.UseAutoShardingCreate();
using (var scope = app.Services.CreateScope())
{
var defaultShardingDbContext = scope.ServiceProvider.GetRequiredService<MyDbContext>();
if (defaultShardingDbContext.Database.GetPendingMigrations().Any())
{
defaultShardingDbContext.Database.Migrate();
}
}
// If you need to scan whether there are tables after startup, you can add this
//app.Services.UseAutoTryCompensateTable();
//......
app.Run();
Add migration files
Add-Migration Init
Start the program
Automatic migration of tables and databases 
crud
add to todo Fields and migrate
Next we will focus on TodoItem Add one name Field and add a table without database or table, and then migrate
public class TodoItem
{
public string Id { get; set; }
public string Text { get; set; }
public string Name { get; set; }
}
public class TodoTest
{
public string Id { get; set; }
public string Test { get; set; }
}
//docontext
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.Entity<TodoItem>(mb =>
{
mb.HasKey(o => o.Id);
mb.Property(o => o.Id).IsRequired().HasMaxLength(50).HasComment("id");
mb.Property(o => o.Text).IsRequired().HasMaxLength(256).HasComment(" Thing ");
mb.Property(o => o.Name).HasMaxLength(256).HasComment(" full name ");
mb.ToTable(nameof(TodoItem));
});
modelBuilder.Entity<TodoTest>(mb =>
{
mb.HasKey(o => o.Id);
mb.Property(o => o.Id).IsRequired().HasMaxLength(50).HasComment("id");
mb.Property(o => o.Test).IsRequired().HasMaxLength(256).HasComment(" test ");
mb.ToTable(nameof(TodoTest));
});
}
Not surprisingly, we succeeded and started again 
After starting the program, we were surprised to find that not only the original table added a name Field , And tables that are not divided into slices have also been added
Only this and nothing more efcore Native sub database and sub table + Fully automated migration Code-First It's all done , This not only greatly improves the performance of the program, but also greatly facilitates the maintenance of developers .
Integrate AbpVNext
It's done efcore We will migrate the original tables and databases abp Next operation
Let's go first github Under the abp-samples Download the corresponding demo test , Choice here todo-mvc
Then we open the installation dependency locally , Just install ·ShardingCore· 6.6.0.3.
Create two new interfaces to assign the creation time and guid
because ShardingCore need add,update,remove When shardingkey Do not be empty , You can assign your own values , But this way efcore Some features cannot be used , So let's do compatibility
// stay TodoApp.Domain.Shared Add two interfaces （ Not necessary ）
public interface IShardingKeyIsCreationTime
{
}
public class IShardingKeyIsGuId
{
}
AbpDbContext abstract class
because Abp Need to inherit AbpDbContext So here is a modification because ShardingCore Only the interface is needed, so it can meet any situation
// In order to remove most of the code, the rest can be at the end of the text demo Check out
public abstract class AbstractShardingAbpDbContext<TDbContext> : AbpDbContext<TDbContext>, IShardingDbContext, ISupportShardingReadWrite
where TDbContext : DbContext
{
private readonly IShardingDbContextExecutor _shardingDbContextExecutor;
protected AbstractShardingAbpDbContext(DbContextOptions<TDbContext> options) : base(options)
{
var wrapOptionsExtension = options.FindExtension<ShardingWrapOptionsExtension>();
if (wrapOptionsExtension != null)
{
_shardingDbContextExecutor = new ShardingDbContextExecutor(this);
}
}
public DbContext GetDbContext(string dataSourceName, CreateDbContextStrategyEnum strategy, IRouteTail routeTail)
{
var dbContext = _shardingDbContextExecutor.CreateDbContext(strategy, dataSourceName, routeTail);
if (dbContext is AbpDbContext<TDbContext> abpDbContext && abpDbContext.LazyServiceProvider == null)
{
abpDbContext.LazyServiceProvider = this.LazyServiceProvider;
}
return dbContext;
}
}
Add sub database and sub table routes
todoitem id Take the mold branch library
public class TodoDataSourceRoute:AbstractShardingOperatorVirtualDataSourceRoute<TodoItem,string>
{
public override string ShardingKeyToDataSourceName(object shardingKey)
{
if (shardingKey == null) throw new InvalidOperationException("sharding key cant null");
var stringHashCode = ShardingCoreHelper.GetStringHashCode(shardingKey.ToString());
return $"ds{(Math.Abs(stringHashCode) % 3)}";//ds0,ds1,ds2
}
public override List<string> GetAllDataSourceNames()
{
return new List<string>()
{
"ds0", "ds1", "ds2"
};
}
public override bool AddDataSourceName(string dataSourceName)
{
throw new NotImplementedException();
}
public override void Configure(EntityMetadataDataSourceBuilder<TodoItem> builder)
{
builder.ShardingProperty(o => o.Id);
}
public override Func<string, bool> GetRouteToFilter(string shardingKey, ShardingOperatorEnum shardingOperator)
{
var t = ShardingKeyToDataSourceName(shardingKey);
switch (shardingOperator)
{
case ShardingOperatorEnum.Equal: return tail => tail == t;
default:
{
return tail => true;
}
}
}
}
todoitem text Die taking table
public class TodoTableRoute:AbstractSimpleShardingModKeyStringVirtualTableRoute<TodoItem>
{
public TodoTableRoute() : base(2, 5)
{
}
public override void Configure(EntityMetadataTableBuilder<TodoItem> builder)
{
builder.ShardingProperty(o => o.Text);
}
}
To write sqlserver Fragment migration script generation
public class ShardingSqlServerMigrationsSqlGenerator: SqlServerMigrationsSqlGenerator
{
private readonly IShardingRuntimeContext _shardingRuntimeContext;
public ShardingSqlServerMigrationsSqlGenerator(IShardingRuntimeContext shardingRuntimeContext,[NotNull] MigrationsSqlGeneratorDependencies dependencies, [NotNull] IRelationalAnnotationProvider migrationsAnnotations) : base(dependencies, migrationsAnnotations)
{
_shardingRuntimeContext = shardingRuntimeContext;
}
protected override void Generate(
MigrationOperation operation,
IModel model,
MigrationCommandListBuilder builder)
{
var oldCmds = builder.GetCommandList().ToList();
base.Generate(operation, model, builder);
var newCmds = builder.GetCommandList().ToList();
var addCmds = newCmds.Where(x => !oldCmds.Contains(x)).ToList();
MigrationHelper.Generate(_shardingRuntimeContext,operation, builder, Dependencies.SqlGenerationHelper, addCmds);
}
}
abp Of efcore Module Injection
TodoAppEntityFrameworkCoreModule Write injection
public class TodoAppEntityFrameworkCoreModule : AbpModule
{
public static readonly ILoggerFactory efLogger = LoggerFactory.Create(builder =>
{
builder.AddFilter((category, level) => category == DbLoggerCategory.Database.Command.Name && level == LogLevel.Information).AddConsole();
});
public override void PreConfigureServices(ServiceConfigurationContext context)
{
TodoAppEfCoreEntityExtensionMappings.Configure();
}
public override void ConfigureServices(ServiceConfigurationContext context)
{
context.Services.AddAbpDbContext<TodoAppDbContext>(options =>
{
/* Remove "includeAllEntities: true" to create
* default repositories only for aggregate roots */
options.AddDefaultRepositories(includeAllEntities: true);
});
Configure<AbpDbContextOptions>(options =>
{
/* The main point to change your DBMS.
* See also TodoAppDbContextFactory for EF Core tooling. */
options.UseSqlServer();
options.Configure<TodoAppDbContext>(innerContext =>
{
ShardingCoreExtension.UseDefaultSharding<TodoAppDbContext>(innerContext.ServiceProvider, innerContext.DbContextOptions);
});
});
context.Services.AddShardingConfigure<TodoAppDbContext>()
.UseRouteConfig(op =>
{
op.AddShardingDataSourceRoute<TodoDataSourceRoute>();
op.AddShardingTableRoute<TodoTableRoute>();
})
.UseConfig((sp, op) =>
{
//var loggerFactory = sp.GetRequiredService<ILoggerFactory>();
op.UseShardingQuery((conStr, builder) =>
{
builder.UseSqlServer(conStr).UseLoggerFactory(efLogger);
});
op.UseShardingTransaction((connection, builder) =>
{
builder.UseSqlServer(connection).UseLoggerFactory(efLogger);
});
op.UseShardingMigrationConfigure(builder =>
{
builder.ReplaceService<IMigrationsSqlGenerator, ShardingSqlServerMigrationsSqlGenerator>();
});
op.AddDefaultDataSource("ds0", "Server=.;Database=TodoApp;Trusted_Connection=True");
op.AddExtraDataSource(sp =>
{
return new Dictionary<string, string>()
{
{ "ds1", "Server=.;Database=TodoApp1;Trusted_Connection=True" },
{ "ds2", "Server=.;Database=TodoApp2;Trusted_Connection=True" }
};
});
})
.AddShardingCore();
}
public override void OnPostApplicationInitialization(ApplicationInitializationContext context)
{
base.OnPostApplicationInitialization(context);
// If the scheduled task of creating a table needs to be routed according to the system default on month, year and day, remember to open it
context.ServiceProvider.UseAutoShardingCreate();
// Compensation table // Automatic migration does not need
//context.ServiceProvider.UseAutoTryCompensateTable();
}
}
start-up abp Migration project
start-up 
Waiting for output
Insert todoitem
Inquire about
verification 
So far, we have completed the target abpvnext Sub table sub database of + Automate migration
Integrate Furion
Next, let's start integrating Furion The operation of
First, install dependencies
Add dependency ShardingCore 6.6.0.3 MySql
Install-Package Furion -Version 3.7.5
// Please install the latest version currently x.6.0.5+, First version number 6 representative efcore Version number of
Install-Package ShardingCore -Version 6.6.0.5
Install-Package Pomelo.EntityFrameworkCore.MySql -Version 6.0.1
Install-Package Microsoft.EntityFrameworkCore.Tools -Version 6.0.6
newly added todoitem
public class TodoItem:IEntity, IEntityTypeBuilder<TodoItem>
{
public string Id { get; set; }
public string Text { get; set; }
public void Configure(EntityTypeBuilder<TodoItem> entityBuilder, DbContext dbContext, Type dbContextLocator)
{
entityBuilder.HasKey(o => o.Id);
entityBuilder.Property(o => o.Id).IsRequired().HasMaxLength(50).HasComment("id");
entityBuilder.Property(o => o.Text).IsRequired().HasMaxLength(256).HasComment(" Thing ");
entityBuilder.ToTable(nameof(TodoItem));
}
}
Add a new DbContext and Abp equally
Do abstract objects see far directly , Add a new one directly here dbcontext
public class MyDbContext : AppShardingDbContext<MyDbContext>,IShardingTableDbContext
{
public MyDbContext(DbContextOptions<MyDbContext> options) : base(options)
{
}
public IRouteTail RouteTail { get; set; }
}
Add sub table and sub database routes
Add sub database route
public class TodoItemDataSourceRoute:AbstractShardingOperatorVirtualDataSourceRoute<TodoItem,string>
{
/// <summary>
/// id Of hashcode Comodule taking 3 sub-treasury
/// </summary>
/// <param name="shardingKey"></param>
/// <returns></returns>
/// <exception cref="InvalidOperationException"></exception>
public override string ShardingKeyToDataSourceName(object shardingKey)
{
if (shardingKey == null) throw new InvalidOperationException("sharding key cant null");
var stringHashCode = ShardingCoreHelper.GetStringHashCode(shardingKey.ToString());
return $"ds{(Math.Abs(stringHashCode) % 3)}";//ds0,ds1,ds2
}
private readonly List<string> _dataSources = new List<string>() { "ds0", "ds1", "ds2" };
public override List<string> GetAllDataSourceNames()
{
return _dataSources;
}
public override bool AddDataSourceName(string dataSourceName)
{
throw new NotImplementedException();
}
/// <summary>
/// id sub-treasury
/// </summary>
/// <param name="builder"></param>
public override void Configure(EntityMetadataDataSourceBuilder<TodoItem> builder)
{
builder.ShardingProperty(o => o.Id);
}
public override Func<string, bool> GetRouteToFilter(string shardingKey, ShardingOperatorEnum shardingOperator)
{
var t = ShardingKeyToDataSourceName(shardingKey);
switch (shardingOperator)
{
case ShardingOperatorEnum.Equal: return tail => tail == t;
default:
{
return tail => true;
}
}
}
}
Add a sub table route
public class TodoItemTableRoute:AbstractSimpleShardingModKeyStringVirtualTableRoute<TodoItem>
{
public TodoItemTableRoute() : base(2, 3)
{
}
/// <summary>
/// Under normal circumstances, the content will not be used as the fragment key, because as the fragment key, there is a premise that it will not be modified
/// </summary>
/// <param name="builder"></param>
public override void Configure(EntityMetadataTableBuilder<TodoItem> builder)
{
builder.ShardingProperty(o => o.Text);
}
}
Write migration files
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Migrations.Operations;
using Pomelo.EntityFrameworkCore.MySql.Infrastructure.Internal;
using Pomelo.EntityFrameworkCore.MySql.Migrations;
using ShardingCore.Core.RuntimeContexts;
using ShardingCore.Helpers;
namespace TodoApp;
public class ShardingMySqlMigrationsSqlGenerator:MySqlMigrationsSqlGenerator
{
private readonly IShardingRuntimeContext _shardingRuntimeContext;
public ShardingMySqlMigrationsSqlGenerator(MigrationsSqlGeneratorDependencies dependencies, IRelationalAnnotationProvider annotationProvider, IMySqlOptions options,IShardingRuntimeContext shardingRuntimeContext) : base(dependencies, annotationProvider, options)
{
_shardingRuntimeContext = shardingRuntimeContext;
}
protected override void Generate(
MigrationOperation operation,
IModel model,
MigrationCommandListBuilder builder)
{
var oldCmds = builder.GetCommandList().ToList();
base.Generate(operation, model, builder);
var newCmds = builder.GetCommandList().ToList();
var addCmds = newCmds.Where(x => !oldCmds.Contains(x)).ToList();
MigrationHelper.Generate(_shardingRuntimeContext,operation, builder, Dependencies.SqlGenerationHelper, addCmds);
}
}
Start Injection
Here is a brief look furion It seems that no Func<IServiceProvider,DbContextOptionBuilder> Of efcore Injection mode, so we have to adopt static mode ,
If you use a static method, you need to implement an interface IDbContextCreator
// Static creation IShardingRuntimeContext
public class ShardingCoreProvider
{
private static ILoggerFactory efLogger = LoggerFactory.Create(builder =>
{
builder.AddFilter((category, level) => category == DbLoggerCategory.Database.Command.Name && level == LogLevel.Information).AddConsole();
});
private static readonly IShardingRuntimeContext instance;
public static IShardingRuntimeContext ShardingRuntimeContext => instance;
static ShardingCoreProvider()
{
instance=new ShardingRuntimeBuilder<MyDbContext>().UseRouteConfig(op =>
{
op.AddShardingTableRoute<TodoItemTableRoute>();
op.AddShardingDataSourceRoute<TodoItemDataSourceRoute>();
})
.UseConfig((sp,op) =>
{
op.UseShardingQuery((con, b) =>
{
b.UseMySql(con, new MySqlServerVersion(new Version()))
.UseLoggerFactory(efLogger);
});
op.UseShardingTransaction((con, b) =>
{
b.UseMySql(con, new MySqlServerVersion(new Version()))
.UseLoggerFactory(efLogger);
});
op.AddDefaultDataSource("ds0", "server=127.0.0.1;port=3306;database=furion0;userid=root;password=root;");
op.AddExtraDataSource(sp=>new Dictionary<string, string>()
{
{"ds1", "server=127.0.0.1;port=3306;database=furion1;userid=root;password=root;"},
{"ds2", "server=127.0.0.1;port=3306;database=furion2;userid=root;password=root;"}
});
op.UseShardingMigrationConfigure(b =>
{
b.ReplaceService<IMigrationsSqlGenerator, ShardingMySqlMigrationsSqlGenerator>();
});
}).ReplaceService<IDbContextCreator, CustomerDbContextCreator>(ServiceLifetime.Singleton).Build();
}
}
// Start the service
public class ShardingCoreComponent:IServiceComponent
{
public void Load(IServiceCollection services, ComponentContext componentContext)
{
services.AddControllers();
services.AddEndpointsApiExplorer();
services.AddSwaggerGen();
services.AddDatabaseAccessor(options =>
{
// Configure the default database
options.AddDb<MyDbContext>(o =>
{
o.UseDefaultSharding<MyDbContext>(ShardingCoreProvider.ShardingRuntimeContext);
});
});
// Dependency injection
services.AddSingleton<IShardingRuntimeContext>(sp => ShardingCoreProvider.ShardingRuntimeContext);
}
}
public class CustomerDbContextCreator:ActivatorDbContextCreator<MyDbContext>
{
public override DbContext GetShellDbContext(IShardingProvider shardingProvider)
{
var dbContextOptionsBuilder = new DbContextOptionsBuilder<MyDbContext>();
dbContextOptionsBuilder.UseDefaultSharding<MyDbContext>(ShardingCoreProvider.ShardingRuntimeContext);
return new MyDbContext(dbContextOptionsBuilder.Options);
}
}
public class UseShardingCoreComponent:IApplicationComponent
{
public void Load(IApplicationBuilder app, IWebHostEnvironment env, ComponentContext componentContext)
{
//......
app.ApplicationServices.UseAutoShardingCreate();
var serviceProvider = app.ApplicationServices;
using (var scope = app.ApplicationServices.CreateScope())
{
var defaultShardingDbContext = scope.ServiceProvider.GetRequiredService<MyDbContext>();
if (defaultShardingDbContext.Database.GetPendingMigrations().Any())
{
defaultShardingDbContext.Database.Migrate();
}
}
// app.Services.UseAutoTryCompensateTable();
}
}
//Program
using TodoApp;
Serve.Run(RunOptions.Default
.AddComponent<ShardingCoreComponent>()
.UseComponent<UseShardingCoreComponent>());
Add migration files
start-up
Additions and deletions 
Integrate WTM
I inherited it once before, and after that, it was too troublesome to migrate, so here ShardingCore A more perfect migration scheme has been developed and used code-first More insensitive
Add dependency
Add dependency ShardingCore 6.6.0.3 MySql
// Please install the latest version currently x.6.0.5+, First version number 6 representative efcore Version number of
Install-Package ShardingCore -Version 6.6.0.5
Install-Package Microsoft.EntityFrameworkCore.Tools -Version 6.0.6
Add sub table and sub database routes
// Sub database routing
public class TodoDataSourceRoute:AbstractShardingOperatorVirtualDataSourceRoute<Todo,string>
{
/// <summary>
/// id Of hashcode Comodule taking 3 sub-treasury
/// </summary>
/// <param name="shardingKey"></param>
/// <returns></returns>
/// <exception cref="InvalidOperationException"></exception>
public override string ShardingKeyToDataSourceName(object shardingKey)
{
if (shardingKey == null) throw new InvalidOperationException("sharding key cant null");
var stringHashCode = ShardingCoreHelper.GetStringHashCode(shardingKey.ToString());
return $"ds{(Math.Abs(stringHashCode) % 3)}";//ds0,ds1,ds2
}
private readonly List<string> _dataSources = new List<string>() { "ds0", "ds1", "ds2" };
public override List<string> GetAllDataSourceNames()
{
return _dataSources;
}
public override bool AddDataSourceName(string dataSourceName)
{
throw new NotImplementedException();
}
/// <summary>
/// id sub-treasury
/// </summary>
/// <param name="builder"></param>
public override void Configure(EntityMetadataDataSourceBuilder<Todo> builder)
{
builder.ShardingProperty(o => o.Id);
}
public override Func<string, bool> GetRouteToFilter(string shardingKey, ShardingOperatorEnum shardingOperator)
{
var t = ShardingKeyToDataSourceName(shardingKey);
switch (shardingOperator)
{
case ShardingOperatorEnum.Equal: return tail => tail == t;
default:
{
return tail => true;
}
}
}
}
// Table routing
public class TodoTableRoute:AbstractSimpleShardingModKeyStringVirtualTableRoute<Todo>
{
public TodoTableRoute() : base(2, 3)
{
}
/// <summary>
/// Under normal circumstances, the content will not be used as the fragment key, because as the fragment key, there is a premise that it will not be modified
/// </summary>
/// <param name="builder"></param>
public override void Configure(EntityMetadataTableBuilder<Todo> builder)
{
builder.ShardingProperty(o => o.Name);
}
}
establish DbContextCreator
public class WTMDbContextCreator:IDbContextCreator
{
public DbContext CreateDbContext(DbContext shellDbContext, ShardingDbContextOptions shardingDbContextOptions)
{
var context = new DataContext((DbContextOptions<DataContext>)shardingDbContextOptions.DbContextOptions);
context.RouteTail = shardingDbContextOptions.RouteTail;
return context;
}
public DbContext GetShellDbContext(IShardingProvider shardingProvider)
{
var dbContextOptionsBuilder = new DbContextOptionsBuilder<DataContext>();
dbContextOptionsBuilder.UseDefaultSharding<DataContext>(ShardingCoreProvider.ShardingRuntimeContext);
return new DataContext(dbContextOptionsBuilder.Options);
}
}
Migration scripts
public class ShardingMySqlMigrationsSqlGenerator:MySqlMigrationsSqlGenerator
{
private readonly IShardingRuntimeContext _shardingRuntimeContext;
public ShardingMySqlMigrationsSqlGenerator(MigrationsSqlGeneratorDependencies dependencies, IRelationalAnnotationProvider annotationProvider, IMySqlOptions options,IShardingRuntimeContext shardingRuntimeContext) : base(dependencies, annotationProvider, options)
{
_shardingRuntimeContext = shardingRuntimeContext;
}
protected override void Generate(
MigrationOperation operation,
IModel model,
MigrationCommandListBuilder builder)
{
var oldCmds = builder.GetCommandList().ToList();
base.Generate(operation, model, builder);
var newCmds = builder.GetCommandList().ToList();
var addCmds = newCmds.Where(x => !oldCmds.Contains(x)).ToList();
MigrationHelper.Generate(_shardingRuntimeContext,operation, builder, Dependencies.SqlGenerationHelper, addCmds);
}
}
Static construction IShardingRuntimeContext
because WTM Creating dbcontext It is not created by dependency injection, but by the rest of the internal implementation, so in order to be compatible, we can only use static IShardingRuntimeContext Inject
public class ShardingCoreProvider
{
private static ILoggerFactory efLogger = LoggerFactory.Create(builder =>
{
builder.AddFilter((category, level) => category == DbLoggerCategory.Database.Command.Name && level == LogLevel.Information).AddConsole();
});
private static readonly IShardingRuntimeContext instance;
public static IShardingRuntimeContext ShardingRuntimeContext => instance;
static ShardingCoreProvider()
{
instance=new ShardingRuntimeBuilder<DataContext>().UseRouteConfig(op =>
{
op.AddShardingTableRoute<TodoRoute>();
op.AddShardingDataSourceRoute<TodoDataSourceRoute>();
})
.UseConfig((sp,op) =>
{
op.UseShardingQuery((con, b) =>
{
b.UseMySql(con, new MySqlServerVersion(new Version()))
.UseLoggerFactory(efLogger);
});
op.UseShardingTransaction((con, b) =>
{
b.UseMySql(con, new MySqlServerVersion(new Version()))
.UseLoggerFactory(efLogger);
});
op.AddDefaultDataSource("ds0", "server=127.0.0.1;port=3306;database=wtm0;userid=root;password=root;");
op.AddExtraDataSource(sp=>new Dictionary<string, string>()
{
{"ds1", "server=127.0.0.1;port=3306;database=wtm1;userid=root;password=root;"},
{"ds2", "server=127.0.0.1;port=3306;database=wtm2;userid=root;password=root;"}
});
op.UseShardingMigrationConfigure(b =>
{
b.ReplaceService<IMigrationsSqlGenerator, ShardingMySqlMigrationsSqlGenerator>();
});
}).ReplaceService<IDbContextCreator, WTMDbContextCreator>(ServiceLifetime.Singleton).Build();
}
}
Create abstract shards DbContext
Because it is too long, only the main parts are shown here, and the rest pass demo see
public abstract class AbstractShardingFrameworkContext:FrameworkContext, IShardingDbContext, ISupportShardingReadWrite
{
protected IShardingDbContextExecutor ShardingDbContextExecutor
{
get;
}
public AbstractShardingFrameworkContext(CS cs)
: base(cs)
{
ShardingDbContextExecutor =new ShardingDbContextExecutor(this);
IsExecutor = false;
}
public AbstractShardingFrameworkContext(string cs, DBTypeEnum dbtype)
: base(cs, dbtype)
{
ShardingDbContextExecutor =new ShardingDbContextExecutor(this);
IsExecutor = false;
}
public AbstractShardingFrameworkContext(string cs, DBTypeEnum dbtype, string version = null)
: base(cs, dbtype, version)
{
ShardingDbContextExecutor =new ShardingDbContextExecutor(this);
IsExecutor = false;
}
public AbstractShardingFrameworkContext(DbContextOptions options) : base(options)
{
var wrapOptionsExtension = options.FindExtension<ShardingWrapOptionsExtension>();
if (wrapOptionsExtension != null)
{
ShardingDbContextExecutor =new ShardingDbContextExecutor(this);;
}
IsExecutor = wrapOptionsExtension == null;
}
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
if (this.CSName!=null)
{
base.OnConfiguring(optionsBuilder);
optionsBuilder.UseDefaultSharding<DataContext>(ShardingCoreProvider.ShardingRuntimeContext);
}
}
public DbContext GetDbContext(string dataSourceName, CreateDbContextStrategyEnum strategy, IRouteTail routeTail)
{
return ShardingDbContextExecutor.CreateDbContext(strategy, dataSourceName, routeTail);
}
}
modify dbcontext
public class DataContextFactory : IDesignTimeDbContextFactory<DataContext>
{
public DataContext CreateDbContext(string[] args)
{
var virtualDataSource = ShardingCoreProvider.ShardingRuntimeContext.GetVirtualDataSource();
var defaultConnectionString = virtualDataSource.DefaultConnectionString;
return new DataContext(defaultConnectionString, DBTypeEnum.MySql);
}
}
Inject ShardingCore
Remove the previous redundant code
public void ConfigureServices(IServiceCollection services){
//....
services.AddSingleton<IShardingRuntimeContext>(sp => ShardingCoreProvider.ShardingRuntimeContext);
}
public void Configure(IApplicationBuilder app, IOptionsMonitor<Configs> configs)
{
IconFontsHelper.GenerateIconFont();
// using (var scope = app.ApplicationServices.CreateScope())
// {
// var requiredService = scope.ServiceProvider.GetRequiredService<WTMContext>();
// var requiredServiceDc = requiredService.DC;
// }
// Timing task
app.ApplicationServices.UseAutoShardingCreate();
using (var dbconContext=new DataContextFactory().CreateDbContext(new string[0]))
{
dbconContext.Database.Migrate();
}
// Make up the table to prevent iis Sleep like this leads to no new tables created by day and month
//app.ApplicationServices.UseAutoTryCompensateTable();
//....
}
transfer
Start the program
crud
Last, last
(ShardingWithFrameWork)[https://github.com/xuejmnet/ShardingWithFramework] https://github.com/xuejmnet/ShardingWithFramework
You've seen it here. Are you sure you don't want to order star Or like it , a .Net Have to learn the sub database and sub table solution , It is simply understood as sharding-jdbc stay .net And support more features and better data aggregation , With native performance 97%, And no business intrusion , Support all non segmented efcore Native query
![[statistical learning method] learning notes - logistic regression and maximum entropy model](/img/f7/857d053cc2cee81c24919aafab3c6e.png)
