.Net The sub table and sub database of lower limit productivity are fully automated Migrations Code-First
## Introduce
this paper ShardinfCore edition x.6.x.x+
Protagonist of this issue :
- [`ShardingCore`](https://github.com/dotnetcore/sharding-core) a ef-core High performance 、 Lightweight solution for reading and writing separation of tables and Libraries , With zero dependency 、 Zero learning cost 、 Zero business code intrusion adaptation
Catalog
Start
Our theme this time is ultimate productivity , The sub table and sub library are fully automated, which are beyond the reach of other languages Migrations Code-First Add efcore Senseless development of sub table and sub database
I still remember the last time I posted my blog , Last released how compatible WTM After the framework, there are also many small partners to ask me how to be compatible and how to migrate , Through the compatibility of so many frameworks, I have also realized some problems , For example, in ShardingCore Use before initialization ( After all efcore) The initialization of is that there is no need to call initialization manually during dependency injection , such as efcore.tool The problem of migration , This project cannot be migrated , because efcore.tool When using the command, it will not call Configure Which makes it impossible to initialize bug, You must create a new console program to migrate , It cannot be migrated within this project , Or code-first and ShardingCore The startup parameter conflict of requires trivial modification , And does not support sub databases , There was a little friend who divided before 300 If a library cannot be automatically migrated, it is really a headache , Although these problems are actually small things for sub database and sub table , But once the tables and databases reach a certain level, it will be difficult to maintain . therefore ShardingCore A new version has been launched in the last three weeks , The new version mainly solves the above pain points and makes the code more standard
Developing software is generally first to use , Then easy to use , Finally, standardize ,ShardingCore So it is with , Because you need to expand efcore So sometimes in unfamiliar efcore Only static classes can be used for injection access , Static classes are actually a very nonstandard usage , Unless you have to . So the new version x.6.x.x ShardingCore What did you bring? Please look down
Remove static container
The use of static containers leads to ShardingCore There is only one piece of data in the whole application declaration cycle , Then the data is shared, which is quite detrimental to the subsequent test maintenance and expansion , Not as isolated as a single case , So... Was removed ShardingContainer, By providing IShardingRuntimeContext To ensure access to the previous parameter structure , The same DbContext Types are using different IShardingRuntimeContext After that, it can show different characteristics of table and database .
Native efcore
First, we focus on native efcore Expand to separate databases and tables +code-first Automatic migration development
Add dependency ShardingCore 6.6.0.3 MySql
// Please install the latest version currently x.6.0.3+, First version number 6 representative efcore Version number of
Install-Package ShardingCore -Version 6.6.0.3
Install-Package Pomelo.EntityFrameworkCore.MySql -Version 6.0.1
Install-Package Microsoft.EntityFrameworkCore.Tools -Version 6.0.6
Create a todo Entity
public class TodoItem
{
public string Id { get; set; }
public string Text { get; set; }
}
establish dbcontext
Simply map the object to the database, of course DbSet+Attribute It's OK, too
public class MyDbContext:AbstractShardingDbContext,IShardingTableDbContext
{
public MyDbContext(DbContextOptions<MyDbContext> options) : base(options)
{
}
public IRouteTail RouteTail { get; set; }
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.Entity<TodoItem>(mb =>
{
mb.HasKey(o => o.Id);
mb.Property(o => o.Id).IsRequired().HasMaxLength(50).HasComment("id");
mb.Property(o => o.Text).IsRequired().HasMaxLength(256).HasComment(" Thing ");
mb.ToTable(nameof(TodoItem));
});
}
}
Create a new sub database and sub table route
Sub database routing
public class TodoItemDataSourceRoute:AbstractShardingOperatorVirtualDataSourceRoute<TodoItem,string>
{
/// <summary>
/// id Of hashcode Comodule taking 3 sub-treasury
/// </summary>
/// <param name="shardingKey"></param>
/// <returns></returns>
/// <exception cref="InvalidOperationException"></exception>
public override string ShardingKeyToDataSourceName(object shardingKey)
{
if (shardingKey == null) throw new InvalidOperationException("sharding key cant null");
var stringHashCode = ShardingCoreHelper.GetStringHashCode(shardingKey.ToString());
return $"ds{(Math.Abs(stringHashCode) % 3)}";//ds0,ds1,ds2
}
private readonly List<string> _dataSources = new List<string>() { "ds0", "ds1", "ds2" };
public override List<string> GetAllDataSourceNames()
{
return _dataSources;
}
public override bool AddDataSourceName(string dataSourceName)
{
throw new NotImplementedException();
}
/// <summary>
/// id sub-treasury
/// </summary>
/// <param name="builder"></param>
public override void Configure(EntityMetadataDataSourceBuilder<TodoItem> builder)
{
builder.ShardingProperty(o => o.Id);
}
public override Func<string, bool> GetRouteToFilter(string shardingKey, ShardingOperatorEnum shardingOperator)
{
var t = ShardingKeyToDataSourceName(shardingKey);
switch (shardingOperator)
{
case ShardingOperatorEnum.Equal: return tail => tail == t;
default:
{
return tail => true;
}
}
}
}
Table routing
public class TodoItemTableRoute:AbstractSimpleShardingModKeyStringVirtualTableRoute<TodoItem>
{
public TodoItemTableRoute() : base(2, 3)
{
}
/// <summary>
/// Under normal circumstances, the content will not be used as the fragment key, because as the fragment key, there is a premise that it will not be modified
/// </summary>
/// <param name="builder"></param>
public override void Configure(EntityMetadataTableBuilder<TodoItem> builder)
{
builder.ShardingProperty(o => o.Text);
}
}
New migration database script generation
public class ShardingMySqlMigrationsSqlGenerator:MySqlMigrationsSqlGenerator
{
private readonly IShardingRuntimeContext _shardingRuntimeContext;
public ShardingMySqlMigrationsSqlGenerator(MigrationsSqlGeneratorDependencies dependencies, IRelationalAnnotationProvider annotationProvider, IMySqlOptions options,IShardingRuntimeContext shardingRuntimeContext) : base(dependencies, annotationProvider, options)
{
_shardingRuntimeContext = shardingRuntimeContext;
}
protected override void Generate(
MigrationOperation operation,
IModel model,
MigrationCommandListBuilder builder)
{
var oldCmds = builder.GetCommandList().ToList();
base.Generate(operation, model, builder);
var newCmds = builder.GetCommandList().ToList();
var addCmds = newCmds.Where(x => !oldCmds.Contains(x)).ToList();
MigrationHelper.Generate(_shardingRuntimeContext,operation, builder, Dependencies.SqlGenerationHelper, addCmds);
}
}
Configure dependency injection
ILoggerFactory efLogger = LoggerFactory.Create(builder =>
{
builder.AddFilter((category, level) => category == DbLoggerCategory.Database.Command.Name && level == LogLevel.Information).AddConsole();
});
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddControllers();
builder.Services.AddShardingDbContext<MyDbContext>()
.UseRouteConfig(op =>
{
op.AddShardingTableRoute<TodoItemTableRoute>();
op.AddShardingDataSourceRoute<TodoItemDataSourceRoute>();
})
.UseConfig((sp,op) =>
{
op.UseShardingQuery((con, b) =>
{
b.UseMySql(con, new MySqlServerVersion(new Version()))
.UseLoggerFactory(efLogger);
});
op.UseShardingTransaction((con, b) =>
{
b.UseMySql(con, new MySqlServerVersion(new Version()))
.UseLoggerFactory(efLogger);
});
op.AddDefaultDataSource("ds0", "server=127.0.0.1;port=3306;database=mydb0;userid=root;password=root;");
op.AddExtraDataSource(sp=>new Dictionary<string, string>()
{
{"ds1", "server=127.0.0.1;port=3306;database=mydb1;userid=root;password=root;"},
{"ds2", "server=127.0.0.1;port=3306;database=mydb2;userid=root;password=root;"}
});
op.UseShardingMigrationConfigure(b =>
{
b.ReplaceService<IMigrationsSqlGenerator, ShardingMySqlMigrationsSqlGenerator>();
});
}).AddShardingCore();
var app = builder.Build();
// Configure the HTTP request pipeline.
// If there is a need to add timed tasks according to the time slice, otherwise you can not add
app.Services.UseAutoShardingCreate();
using (var scope = app.Services.CreateScope())
{
var defaultShardingDbContext = scope.ServiceProvider.GetRequiredService<MyDbContext>();
if (defaultShardingDbContext.Database.GetPendingMigrations().Any())
{
defaultShardingDbContext.Database.Migrate();
}
}
// If you need to scan whether there are tables after startup, you can add this
//app.Services.UseAutoTryCompensateTable();
//......
app.Run();
Add migration files
Add-Migration Init
Start the program
Automatic migration of tables and databases 
crud
add to todo Fields and migrate
Next we will focus on TodoItem Add one name Field and add a table without database or table, and then migrate
public class TodoItem
{
public string Id { get; set; }
public string Text { get; set; }
public string Name { get; set; }
}
public class TodoTest
{
public string Id { get; set; }
public string Test { get; set; }
}
//docontext
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
base.OnModelCreating(modelBuilder);
modelBuilder.Entity<TodoItem>(mb =>
{
mb.HasKey(o => o.Id);
mb.Property(o => o.Id).IsRequired().HasMaxLength(50).HasComment("id");
mb.Property(o => o.Text).IsRequired().HasMaxLength(256).HasComment(" Thing ");
mb.Property(o => o.Name).HasMaxLength(256).HasComment(" full name ");
mb.ToTable(nameof(TodoItem));
});
modelBuilder.Entity<TodoTest>(mb =>
{
mb.HasKey(o => o.Id);
mb.Property(o => o.Id).IsRequired().HasMaxLength(50).HasComment("id");
mb.Property(o => o.Test).IsRequired().HasMaxLength(256).HasComment(" test ");
mb.ToTable(nameof(TodoTest));
});
}
Not surprisingly, we succeeded and started again 
After starting the program, we were surprised to find that not only the original table added a name Field , And tables that are not divided into slices have also been added
Only this and nothing more efcore Native sub database and sub table + Fully automated migration Code-First It's all done , This not only greatly improves the performance of the program, but also greatly facilitates the maintenance of developers .
Integrate AbpVNext
It's done efcore We will migrate the original tables and databases abp Next operation
Let's go first github Under the abp-samples Download the corresponding demo test , Choice here todo-mvc
Then we open the installation dependency locally , Just install ·ShardingCore· 6.6.0.3.
Create two new interfaces to assign the creation time and guid
because ShardingCore need add,update,remove When shardingkey Do not be empty , You can assign your own values , But this way efcore Regardless of performance, it can't be used
// stay TodoApp.Domain.Shared Add two interfaces （ Not necessary ）
public interface IShardingKeyIsCreationTime
{
}
public class IShardingKeyIsGuId
{
}
AbpDbContext abstract class
because Abp Need to inherit AbpDbContext So here is a modification because ShardingCore Only the interface is needed, so it can meet any situation
// In order to remove most of the code, the rest can be at the end of the text demo Check out
public abstract class AbstractShardingAbpDbContext<TDbContext> : AbpDbContext<TDbContext>, IShardingDbContext, ISupportShardingReadWrite
where TDbContext : DbContext
{
private readonly IShardingDbContextExecutor _shardingDbContextExecutor;
protected AbstractShardingAbpDbContext(DbContextOptions<TDbContext> options) : base(options)
{
var wrapOptionsExtension = options.FindExtension<ShardingWrapOptionsExtension>();
if (wrapOptionsExtension != null)
{
_shardingDbContextExecutor = new ShardingDbContextExecutor(this);
}
}
public DbContext GetDbContext(string dataSourceName, CreateDbContextStrategyEnum strategy, IRouteTail routeTail)
{
var dbContext = _shardingDbContextExecutor.CreateDbContext(strategy, dataSourceName, routeTail);
if (dbContext is AbpDbContext<TDbContext> abpDbContext && abpDbContext.LazyServiceProvider == null)
{
abpDbContext.LazyServiceProvider = this.LazyServiceProvider;
}
return dbContext;
}
}
Add sub database and sub table routes
todoitem id Take the mold branch library
public class TodoDataSourceRoute:AbstractShardingOperatorVirtualDataSourceRoute<TodoItem,string>
{
public override string ShardingKeyToDataSourceName(object shardingKey)
{
if (shardingKey == null) throw new InvalidOperationException("sharding key cant null");
var stringHashCode = ShardingCoreHelper.GetStringHashCode(shardingKey.ToString());
return $"ds{(Math.Abs(stringHashCode) % 3)}";//ds0,ds1,ds2
}
public override List<string> GetAllDataSourceNames()
{
return new List<string>()
{
"ds0", "ds1", "ds2"
};
}
public override bool AddDataSourceName(string dataSourceName)
{
throw new NotImplementedException();
}
public override void Configure(EntityMetadataDataSourceBuilder<TodoItem> builder)
{
builder.ShardingProperty(o => o.Id);
}
public override Func<string, bool> GetRouteToFilter(string shardingKey, ShardingOperatorEnum shardingOperator)
{
var t = ShardingKeyToDataSourceName(shardingKey);
switch (shardingOperator)
{
case ShardingOperatorEnum.Equal: return tail => tail == t;
default:
{
return tail => true;
}
}
}
}
todoitem text Die taking table
public class TodoTableRoute:AbstractSimpleShardingModKeyStringVirtualTableRoute<TodoItem>
{
public TodoTableRoute() : base(2, 5)
{
}
public override void Configure(EntityMetadataTableBuilder<TodoItem> builder)
{
builder.ShardingProperty(o => o.Text);
}
}
To write sqlserver Fragment migration script generation
public class ShardingSqlServerMigrationsSqlGenerator: SqlServerMigrationsSqlGenerator
{
private readonly IShardingRuntimeContext _shardingRuntimeContext;
public ShardingSqlServerMigrationsSqlGenerator(IShardingRuntimeContext shardingRuntimeContext,[NotNull] MigrationsSqlGeneratorDependencies dependencies, [NotNull] IRelationalAnnotationProvider migrationsAnnotations) : base(dependencies, migrationsAnnotations)
{
_shardingRuntimeContext = shardingRuntimeContext;
}
protected override void Generate(
MigrationOperation operation,
IModel model,
MigrationCommandListBuilder builder)
{
var oldCmds = builder.GetCommandList().ToList();
base.Generate(operation, model, builder);
var newCmds = builder.GetCommandList().ToList();
var addCmds = newCmds.Where(x => !oldCmds.Contains(x)).ToList();
MigrationHelper.Generate(_shardingRuntimeContext,operation, builder, Dependencies.SqlGenerationHelper, addCmds);
}
}
abp Of efcore Module Injection
TodoAppEntityFrameworkCoreModule Write injection
public class TodoAppEntityFrameworkCoreModule : AbpModule
{
public static readonly ILoggerFactory efLogger = LoggerFactory.Create(builder =>
{
builder.AddFilter((category, level) => category == DbLoggerCategory.Database.Command.Name && level == LogLevel.Information).AddConsole();
});
public override void PreConfigureServices(ServiceConfigurationContext context)
{
TodoAppEfCoreEntityExtensionMappings.Configure();
}
public override void ConfigureServices(ServiceConfigurationContext context)
{
context.Services.AddAbpDbContext<TodoAppDbContext>(options =>
{
/* Remove "includeAllEntities: true" to create
* default repositories only for aggregate roots */
options.AddDefaultRepositories(includeAllEntities: true);
});
Configure<AbpDbContextOptions>(options =>
{
/* The main point to change your DBMS.
* See also TodoAppDbContextFactory for EF Core tooling. */
options.UseSqlServer();
options.Configure<TodoAppDbContext>(innerContext =>
{
ShardingCoreExtension.UseDefaultSharding<TodoAppDbContext>(innerContext.ServiceProvider, innerContext.DbContextOptions);
});
});
context.Services.AddShardingConfigure<TodoAppDbContext>()
.UseRouteConfig(op =>
{
op.AddShardingDataSourceRoute<TodoDataSourceRoute>();
op.AddShardingTableRoute<TodoTableRoute>();
})
.UseConfig((sp, op) =>
{
//var loggerFactory = sp.GetRequiredService<ILoggerFactory>();
op.UseShardingQuery((conStr, builder) =>
{
builder.UseSqlServer(conStr).UseLoggerFactory(efLogger);
});
op.UseShardingTransaction((connection, builder) =>
{
builder.UseSqlServer(connection).UseLoggerFactory(efLogger);
});
op.UseShardingMigrationConfigure(builder =>
{
builder.ReplaceService<IMigrationsSqlGenerator, ShardingSqlServerMigrationsSqlGenerator>();
});
op.AddDefaultDataSource("ds0", "Server=.;Database=TodoApp;Trusted_Connection=True");
op.AddExtraDataSource(sp =>
{
return new Dictionary<string, string>()
{
{ "ds1", "Server=.;Database=TodoApp1;Trusted_Connection=True" },
{ "ds2", "Server=.;Database=TodoApp2;Trusted_Connection=True" }
};
});
})
.AddShardingCore();
}
public override void OnPostApplicationInitialization(ApplicationInitializationContext context)
{
base.OnPostApplicationInitialization(context);
// If the scheduled task of creating a table needs to be routed according to the system default on month, year and day, remember to open it
context.ServiceProvider.UseAutoShardingCreate();
// Compensation table // Automatic migration does not need
//context.ServiceProvider.UseAutoTryCompensateTable();
}
}
start-up abp Migration project
start-up 
Waiting for output
Insert todoitem
Inquire about
verification 
So far, we have completed the target abpvnext Sub table sub database of + Automate migration
Integrate Furion
Next, let's start integrating Furion The operation of
First, install dependencies
Add dependency ShardingCore 6.6.0.3 MySql
Install-Package Furion -Version 3.7.5
// Please install the latest version currently x.6.0.5+, First version number 6 representative efcore Version number of
Install-Package ShardingCore -Version 6.6.0.5
Install-Package Pomelo.EntityFrameworkCore.MySql -Version 6.0.1
Install-Package Microsoft.EntityFrameworkCore.Tools -Version 6.0.6
newly added todoitem
public class TodoItem:IEntity, IEntityTypeBuilder<TodoItem>
{
public string Id { get; set; }
public string Text { get; set; }
public void Configure(EntityTypeBuilder<TodoItem> entityBuilder, DbContext dbContext, Type dbContextLocator)
{
entityBuilder.HasKey(o => o.Id);
entityBuilder.Property(o => o.Id).IsRequired().HasMaxLength(50).HasComment("id");
entityBuilder.Property(o => o.Text).IsRequired().HasMaxLength(256).HasComment(" Thing ");
entityBuilder.ToTable(nameof(TodoItem));
}
}
Add a new DbContext and Abp equally
Do abstract objects see far directly , Add a new one directly here dbcontext
public class MyDbContext : AppShardingDbContext<MyDbContext>,IShardingTableDbContext
{
public MyDbContext(DbContextOptions<MyDbContext> options) : base(options)
{
}
public IRouteTail RouteTail { get; set; }
}
Add sub table and sub database routes
Add sub database route
public class TodoItemDataSourceRoute:AbstractShardingOperatorVirtualDataSourceRoute<TodoItem,string>
{
/// <summary>
/// id Of hashcode Comodule taking 3 sub-treasury
/// </summary>
/// <param name="shardingKey"></param>
/// <returns></returns>
/// <exception cref="InvalidOperationException"></exception>
public override string ShardingKeyToDataSourceName(object shardingKey)
{
if (shardingKey == null) throw new InvalidOperationException("sharding key cant null");
var stringHashCode = ShardingCoreHelper.GetStringHashCode(shardingKey.ToString());
return $"ds{(Math.Abs(stringHashCode) % 3)}";//ds0,ds1,ds2
}
private readonly List<string> _dataSources = new List<string>() { "ds0", "ds1", "ds2" };
public override List<string> GetAllDataSourceNames()
{
return _dataSources;
}
public override bool AddDataSourceName(string dataSourceName)
{
throw new NotImplementedException();
}
/// <summary>
/// id sub-treasury
/// </summary>
/// <param name="builder"></param>
public override void Configure(EntityMetadataDataSourceBuilder<TodoItem> builder)
{
builder.ShardingProperty(o => o.Id);
}
public override Func<string, bool> GetRouteToFilter(string shardingKey, ShardingOperatorEnum shardingOperator)
{
var t = ShardingKeyToDataSourceName(shardingKey);
switch (shardingOperator)
{
case ShardingOperatorEnum.Equal: return tail => tail == t;
default:
{
return tail => true;
}
}
}
}
Add a sub table route
public class TodoItemTableRoute:AbstractSimpleShardingModKeyStringVirtualTableRoute<TodoItem>
{
public TodoItemTableRoute() : base(2, 3)
{
}
/// <summary>
/// Under normal circumstances, the content will not be used as the fragment key, because as the fragment key, there is a premise that it will not be modified
/// </summary>
/// <param name="builder"></param>
public override void Configure(EntityMetadataTableBuilder<TodoItem> builder)
{
builder.ShardingProperty(o => o.Text);
}
}
Write migration files
using Microsoft.EntityFrameworkCore.Metadata;
using Microsoft.EntityFrameworkCore.Migrations;
using Microsoft.EntityFrameworkCore.Migrations.Operations;
using Pomelo.EntityFrameworkCore.MySql.Infrastructure.Internal;
using Pomelo.EntityFrameworkCore.MySql.Migrations;
using ShardingCore.Core.RuntimeContexts;
using ShardingCore.Helpers;
namespace TodoApp;
public class ShardingMySqlMigrationsSqlGenerator:MySqlMigrationsSqlGenerator
{
private readonly IShardingRuntimeContext _shardingRuntimeContext;
public ShardingMySqlMigrationsSqlGenerator(MigrationsSqlGeneratorDependencies dependencies, IRelationalAnnotationProvider annotationProvider, IMySqlOptions options,IShardingRuntimeContext shardingRuntimeContext) : base(dependencies, annotationProvider, options)
{
_shardingRuntimeContext = shardingRuntimeContext;
}
protected override void Generate(
MigrationOperation operation,
IModel model,
MigrationCommandListBuilder builder)
{
var oldCmds = builder.GetCommandList().ToList();
base.Generate(operation, model, builder);
var newCmds = builder.GetCommandList().ToList();
var addCmds = newCmds.Where(x => !oldCmds.Contains(x)).ToList();
MigrationHelper.Generate(_shardingRuntimeContext,operation, builder, Dependencies.SqlGenerationHelper, addCmds);
}
}
Start Injection
Here is a brief look furion It seems that no Func<IServiceProvider,DbContextOptionBuilder> Of efcore Injection mode, so we have to adopt static mode ,
If you use a static method, you need to implement an interface IDbContextCreator
// Static creation IShardingRuntimeContext
public class ShardingCoreProvider
{
private static ILoggerFactory efLogger = LoggerFactory.Create(builder =>
{
builder.AddFilter((category, level) => category == DbLoggerCategory.Database.Command.Name && level == LogLevel.Information).AddConsole();
});
private static readonly IShardingRuntimeContext instance;
public static IShardingRuntimeContext ShardingRuntimeContext => instance;
static ShardingCoreProvider()
{
instance=new ShardingRuntimeBuilder<MyDbContext>().UseRouteConfig(op =>
{
op.AddShardingTableRoute<TodoItemTableRoute>();
op.AddShardingDataSourceRoute<TodoItemDataSourceRoute>();
})
.UseConfig((sp,op) =>
{
op.UseShardingQuery((con, b) =>
{
b.UseMySql(con, new MySqlServerVersion(new Version()))
.UseLoggerFactory(efLogger);
});
op.UseShardingTransaction((con, b) =>
{
b.UseMySql(con, new MySqlServerVersion(new Version()))
.UseLoggerFactory(efLogger);
});
op.AddDefaultDataSource("ds0", "server=127.0.0.1;port=3306;database=furion0;userid=root;password=root;");
op.AddExtraDataSource(sp=>new Dictionary<string, string>()
{
{"ds1", "server=127.0.0.1;port=3306;database=furion1;userid=root;password=root;"},
{"ds2", "server=127.0.0.1;port=3306;database=furion2;userid=root;password=root;"}
});
op.UseShardingMigrationConfigure(b =>
{
b.ReplaceService<IMigrationsSqlGenerator, ShardingMySqlMigrationsSqlGenerator>();
});
}).ReplaceService<IDbContextCreator, CustomerDbContextCreator>(ServiceLifetime.Singleton).Build();
}
}
// Start the service
public class ShardingCoreComponent:IServiceComponent
{
public void Load(IServiceCollection services, ComponentContext componentContext)
{
services.AddControllers();
services.AddEndpointsApiExplorer();
services.AddSwaggerGen();
services.AddDatabaseAccessor(options =>
{
// Configure the default database
options.AddDb<MyDbContext>(o =>
{
o.UseDefaultSharding<MyDbContext>(ShardingCoreProvider.ShardingRuntimeContext);
});
});
// Dependency injection
services.AddSingleton<IShardingRuntimeContext>(sp => ShardingCoreProvider.ShardingRuntimeContext);
}
}
public class CustomerDbContextCreator:ActivatorDbContextCreator<MyDbContext>
{
public override DbContext GetShellDbContext(IShardingProvider shardingProvider)
{
var dbContextOptionsBuilder = new DbContextOptionsBuilder<MyDbContext>();
dbContextOptionsBuilder.UseDefaultSharding<MyDbContext>(ShardingCoreProvider.ShardingRuntimeContext);
return new MyDbContext(dbContextOptionsBuilder.Options);
}
}
public class UseShardingCoreComponent:IApplicationComponent
{
public void Load(IApplicationBuilder app, IWebHostEnvironment env, ComponentContext componentContext)
{
//......
app.ApplicationServices.UseAutoShardingCreate();
var serviceProvider = app.ApplicationServices;
using (var scope = app.ApplicationServices.CreateScope())
{
var defaultShardingDbContext = scope.ServiceProvider.GetRequiredService<MyDbContext>();
if (defaultShardingDbContext.Database.GetPendingMigrations().Any())
{
defaultShardingDbContext.Database.Migrate();
}
}
// app.Services.UseAutoTryCompensateTable();
}
}
//Program
using TodoApp;
Serve.Run(RunOptions.Default
.AddComponent<ShardingCoreComponent>()
.UseComponent<UseShardingCoreComponent>());
Add migration files
start-up
Additions and deletions 
Integrate WTM
I inherited it once before, and after that, it was too troublesome to migrate, so here ShardingCore A more perfect migration scheme has been developed and used code-first More insensitive
Add dependency
Add dependency ShardingCore 6.6.0.3 MySql
// Please install the latest version currently x.6.0.5+, First version number 6 representative efcore Version number of
Install-Package ShardingCore -Version 6.6.0.5
Install-Package Microsoft.EntityFrameworkCore.Tools -Version 6.0.6
Add sub table and sub database routes
// Sub database routing
public class TodoDataSourceRoute:AbstractShardingOperatorVirtualDataSourceRoute<Todo,string>
{
/// <summary>
/// id Of hashcode Comodule taking 3 sub-treasury
/// </summary>
/// <param name="shardingKey"></param>
/// <returns></returns>
/// <exception cref="InvalidOperationException"></exception>
public override string ShardingKeyToDataSourceName(object shardingKey)
{
if (shardingKey == null) throw new InvalidOperationException("sharding key cant null");
var stringHashCode = ShardingCoreHelper.GetStringHashCode(shardingKey.ToString());
return $"ds{(Math.Abs(stringHashCode) % 3)}";//ds0,ds1,ds2
}
private readonly List<string> _dataSources = new List<string>() { "ds0", "ds1", "ds2" };
public override List<string> GetAllDataSourceNames()
{
return _dataSources;
}
public override bool AddDataSourceName(string dataSourceName)
{
throw new NotImplementedException();
}
/// <summary>
/// id sub-treasury
/// </summary>
/// <param name="builder"></param>
public override void Configure(EntityMetadataDataSourceBuilder<Todo> builder)
{
builder.ShardingProperty(o => o.Id);
}
public override Func<string, bool> GetRouteToFilter(string shardingKey, ShardingOperatorEnum shardingOperator)
{
var t = ShardingKeyToDataSourceName(shardingKey);
switch (shardingOperator)
{
case ShardingOperatorEnum.Equal: return tail => tail == t;
default:
{
return tail => true;
}
}
}
}
// Table routing
public class TodoTableRoute:AbstractSimpleShardingModKeyStringVirtualTableRoute<Todo>
{
public TodoTableRoute() : base(2, 3)
{
}
/// <summary>
/// Under normal circumstances, the content will not be used as the fragment key, because as the fragment key, there is a premise that it will not be modified
/// </summary>
/// <param name="builder"></param>
public override void Configure(EntityMetadataTableBuilder<Todo> builder)
{
builder.ShardingProperty(o => o.Name);
}
}
establish DbContextCreator
public class WTMDbContextCreator:IDbContextCreator
{
public DbContext CreateDbContext(DbContext shellDbContext, ShardingDbContextOptions shardingDbContextOptions)
{
var context = new DataContext((DbContextOptions<DataContext>)shardingDbContextOptions.DbContextOptions);
context.RouteTail = shardingDbContextOptions.RouteTail;
return context;
}
public DbContext GetShellDbContext(IShardingProvider shardingProvider)
{
var dbContextOptionsBuilder = new DbContextOptionsBuilder<DataContext>();
dbContextOptionsBuilder.UseDefaultSharding<DataContext>(ShardingCoreProvider.ShardingRuntimeContext);
return new DataContext(dbContextOptionsBuilder.Options);
}
}
Migration scripts
public class ShardingMySqlMigrationsSqlGenerator:MySqlMigrationsSqlGenerator
{
private readonly IShardingRuntimeContext _shardingRuntimeContext;
public ShardingMySqlMigrationsSqlGenerator(MigrationsSqlGeneratorDependencies dependencies, IRelationalAnnotationProvider annotationProvider, IMySqlOptions options,IShardingRuntimeContext shardingRuntimeContext) : base(dependencies, annotationProvider, options)
{
_shardingRuntimeContext = shardingRuntimeContext;
}
protected override void Generate(
MigrationOperation operation,
IModel model,
MigrationCommandListBuilder builder)
{
var oldCmds = builder.GetCommandList().ToList();
base.Generate(operation, model, builder);
var newCmds = builder.GetCommandList().ToList();
var addCmds = newCmds.Where(x => !oldCmds.Contains(x)).ToList();
MigrationHelper.Generate(_shardingRuntimeContext,operation, builder, Dependencies.SqlGenerationHelper, addCmds);
}
}
Static construction IShardingRuntimeContext
because WTM Creating dbcontext It is not created by dependency injection, but by the rest of the internal implementation, so in order to be compatible, we can only use static IShardingRuntimeContext Inject
public class ShardingCoreProvider
{
private static ILoggerFactory efLogger = LoggerFactory.Create(builder =>
{
builder.AddFilter((category, level) => category == DbLoggerCategory.Database.Command.Name && level == LogLevel.Information).AddConsole();
});
private static readonly IShardingRuntimeContext instance;
public static IShardingRuntimeContext ShardingRuntimeContext => instance;
static ShardingCoreProvider()
{
instance=new ShardingRuntimeBuilder<DataContext>().UseRouteConfig(op =>
{
op.AddShardingTableRoute<TodoRoute>();
op.AddShardingDataSourceRoute<TodoDataSourceRoute>();
})
.UseConfig((sp,op) =>
{
op.UseShardingQuery((con, b) =>
{
b.UseMySql(con, new MySqlServerVersion(new Version()))
.UseLoggerFactory(efLogger);
});
op.UseShardingTransaction((con, b) =>
{
b.UseMySql(con, new MySqlServerVersion(new Version()))
.UseLoggerFactory(efLogger);
});
op.AddDefaultDataSource("ds0", "server=127.0.0.1;port=3306;database=wtm0;userid=root;password=root;");
op.AddExtraDataSource(sp=>new Dictionary<string, string>()
{
{"ds1", "server=127.0.0.1;port=3306;database=wtm1;userid=root;password=root;"},
{"ds2", "server=127.0.0.1;port=3306;database=wtm2;userid=root;password=root;"}
});
op.UseShardingMigrationConfigure(b =>
{
b.ReplaceService<IMigrationsSqlGenerator, ShardingMySqlMigrationsSqlGenerator>();
});
}).ReplaceService<IDbContextCreator, WTMDbContextCreator>(ServiceLifetime.Singleton).Build();
}
}
Create abstract shards DbContext
Because it is too long, only the main parts are shown here, and the rest pass demo see
public abstract class AbstractShardingFrameworkContext:FrameworkContext, IShardingDbContext, ISupportShardingReadWrite
{
protected IShardingDbContextExecutor ShardingDbContextExecutor
{
get;
}
public AbstractShardingFrameworkContext(CS cs)
: base(cs)
{
ShardingDbContextExecutor =new ShardingDbContextExecutor(this);
IsExecutor = false;
}
public AbstractShardingFrameworkContext(string cs, DBTypeEnum dbtype)
: base(cs, dbtype)
{
ShardingDbContextExecutor =new ShardingDbContextExecutor(this);
IsExecutor = false;
}
public AbstractShardingFrameworkContext(string cs, DBTypeEnum dbtype, string version = null)
: base(cs, dbtype, version)
{
ShardingDbContextExecutor =new ShardingDbContextExecutor(this);
IsExecutor = false;
}
public AbstractShardingFrameworkContext(DbContextOptions options) : base(options)
{
var wrapOptionsExtension = options.FindExtension<ShardingWrapOptionsExtension>();
if (wrapOptionsExtension != null)
{
ShardingDbContextExecutor =new ShardingDbContextExecutor(this);;
}
IsExecutor = wrapOptionsExtension == null;
}
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
if (this.CSName!=null)
{
base.OnConfiguring(optionsBuilder);
optionsBuilder.UseDefaultSharding<DataContext>(ShardingCoreProvider.ShardingRuntimeContext);
}
}
public DbContext GetDbContext(string dataSourceName, CreateDbContextStrategyEnum strategy, IRouteTail routeTail)
{
return ShardingDbContextExecutor.CreateDbContext(strategy, dataSourceName, routeTail);
}
}
modify dbcontext
public class DataContextFactory : IDesignTimeDbContextFactory<DataContext>
{
public DataContext CreateDbContext(string[] args)
{
var virtualDataSource = ShardingCoreProvider.ShardingRuntimeContext.GetVirtualDataSource();
var defaultConnectionString = virtualDataSource.DefaultConnectionString;
return new DataContext(defaultConnectionString, DBTypeEnum.MySql);
}
}
Inject ShardingCore
Remove the previous redundant code
public void ConfigureServices(IServiceCollection services){
//....
services.AddSingleton<IShardingRuntimeContext>(sp => ShardingCoreProvider.ShardingRuntimeContext);
}
public void Configure(IApplicationBuilder app, IOptionsMonitor<Configs> configs)
{
IconFontsHelper.GenerateIconFont();
// using (var scope = app.ApplicationServices.CreateScope())
// {
// var requiredService = scope.ServiceProvider.GetRequiredService<WTMContext>();
// var requiredServiceDc = requiredService.DC;
// }
// Timing task
app.ApplicationServices.UseAutoShardingCreate();
using (var dbconContext=new DataContextFactory().CreateDbContext(new string[0]))
{
dbconContext.Database.Migrate();
}
// Make up the table to prevent iis Sleep like this leads to no new tables created by day and month
//app.ApplicationServices.UseAutoTryCompensateTable();
//....
}
transfer
Start the program
crud
Last, last
(ShardingWithFrameWork)[https://github.com/xuejmnet/ShardingWithFramework] https://github.com/xuejmnet/ShardingWithFramework
You've seen it here. Are you sure you don't want to order star Or like it , a .Net Have to learn the sub database and sub table solution , It is simply understood as sharding-jdbc stay .net And support more features and better data aggregation , With native performance 97%, And no business intrusion , Support all non segmented efcore Native query
- github Address https://github.com/xuejmnet/sharding-core
- gitee Address https://gitee.com/dotnetchina/sharding-core
.Net Lower limit productivity efcore Automatic migration of tables and databases CodeFirst More articles about
- efcore Analysis of the principle of dividing tables and databases
ShardingCore ShardingCore Easy to use . Simple . High performance . Universality , Is an extension for efcore Extension solution of sub table and sub database under Ecology , Support efcore2+ All versions of , Support efcore2+ All data for ...
- efcore Use ShardingCore Implement multi tenancy under separate tables and databases
efcore Use ShardingCore Implement multi tenancy under separate tables and databases Introduce Protagonist of this issue :ShardingCore a ef-core High performance . Lightweight solution for reading and writing separation of tables and Libraries , With zero dependency . Zero learning cost . Zero business ...
- Abp VNext Table depots , Reject manual , We want to happy coding
Abp VNext Table depots ShardingCore ShardingCore Easy to use . Simple . High performance . Universality , Is an extension for efcore Extension solution of sub table and sub database under Ecology , Support efcore2+ All versions of , ...
- Furion I also want to happy coding
Furion Integration of tables and databases ShardingCore ShardingCore ShardingCore Easy to use . Simple . High performance . Universality , Is an extension for efcore Extension solution of sub table and sub database under Ecology , Support efco ...
- .Net Remove the high-performance sub table and sub library components - Connection mode principle
ShardingCore ShardingCore a ef-core High performance . Lightweight solution for reading and writing separation of tables and Libraries , With zero dependency . Zero learning cost . Zero business code intrusion . Github Source Code help ...
- .Net Next, you have to look at the sub table and sub database solution - Multi field slicing
.Net Next, you have to look at the sub table and sub database solution - Multi field slicing Introduce Protagonist of this issue :ShardingCore a ef-core High performance . Lightweight solution for reading and writing separation of tables and Libraries , With zero dependency . Zero learning cost . Zero business code intrusion ...
- Learn to read and write database separation 、 Table depots —— use Mycat, This one is enough ！
In system development , Database is a very important point . Besides the optimization of the program itself , Such as :SQL Statements to optimize . Code optimization , Database processing itself optimization is also very important . Master-slave . Hot standby . Sub table and sub database are all technical problems that the system will encounter sooner or later .Mycat It's one ...
- .NETCore Sub table and sub database are supported 、 The universal separation of reading and writing Repository
First of all, this article is not a headline party , The library I'm talking about is FreeSql.Repository, It serves as an extension to the common repository layer functionality , Interface specification reference abp vnext Definition , The basic storage layer is realized (CURD). install d ...
- sharding sphere Table depots Read / write separation
sharding jdbc: sharding sphere Of Part of the , It can be done Table depots , Read / write separation . and mycat Different yes sharding jdbc yes One jdbc drive stay Drive this ...
- Sum up Mysql Strategy and application of dividing tables and databases
Interview a company before last month , about mysql The idea of sub table , At that time, I briefly said hash Algorithm sub table , as well as discuz The idea of sub table , However, for the new data, it is self increasing id The design idea of storage doesn't answer very well ( written examination + The whole interview process is OK After that , Because of personal ...
Random recommendation
- connect link failure ： Can't find signal
The prompt error is : signal_index < 0 ;; ---- so connect return false; eliminate connect The signal ...
- sgu-508 Black-white balls probability - Bayes' formula
The question : Yes n A ball , Among them is 0.1.2...n The probability of two black balls is equal , Now take it out L A ball ,p A black ball q A white ball . Now guess the interval of a black ball , So that the probability of falling in this interval is greater than a given value . See the code for details. : #include & ...
- ThinkPHP Next use Ueditor
When doing course design, I thought of using Baidu Ueditor, But there are some problems in the configuration Ueditor It doesn't feel very difficult , There used to be customized , This service is now cancelled , But we can configure it ourselves Download address :http://uedi ...
- PHP Practical development tutorial
about PHP For beginners , Learn huge... As soon as you get started PHP Grammar is undoubtedly a blow to self-confidence . In fact, even very skilled programmers , You may not be very familiar with all grammar . Usually, the advantage of skilled programmers over ordinary programmers is that they have a very thorough understanding of basic grammar , And the common one ...
- Codeforces Round #291 (Div. 2) C - Watto and Mechanism character string
[ The question ] to n A collection of strings , Then there is m A asked (0 ≤ n ≤ 3·105, 0 ≤ m ≤ 3·105) , Each query gives a string s, Ask if there is a string in the collection t, bring s and t Same length , And there is only one ...
- Linux2.6 kernel -- Initialization of structure
Linux A large number of structures are used in the kernel , The rules of structure initialization are also given in the coding specification , This article explains it :http://blog.csdn.net/dlutbrucezhang/article ...
- win7 gsoap And vs2010 c++ establish Web Service
--- Resume content start --- I have written a simple sample before , I haven't touched it for a long time , Find that you have forgotten all , Now it needs to be consolidated again . First, download gsoap, Unable to access the official download page , Search only on the Internet , Find one 2.8 Save the version to the cloud disk to prevent ...
- SpringMVC And Mybatis The pitfalls of framework Integration （ turn ）
Recently doing springmvc And mybatis Project , Encountered some more pit problems . It took a lot of time to find out that the solution was simple . Here is mainly about some problems I encountered in integrating the two frameworks, so I will sort them out . Hope to meet the same problem as me ...
- Edge browser call
ShellExecute(0, 'open', PChar('Microsoft-Edge:' + Edit1.Text), nil, nil, SW_SHOW);
- solve winfrom Next TextBox Transparent background colors are not supported
I don't know what Microsoft is talking about , There are controls that don't support transparency , I really want to spray the brains of design people . Especially hateful is TextBox I won't support it , What's more hateful is that until the latest version .net4.6 Nor does it support . The source code is invisible , I don't know the details of the implementation , Who can change ? this ...
