Golang concise architecture practice

author ：bearluo, tencent IEG Operations development engineer

Project code location in the text ：https://github.com/devYun/go-clean-architecture

because golang Unlike java There is also a unified coding mode , So we are like other teams , Adopted Go Package oriented design and architecture layering This article introduces some theories , And then subcontract in combination with previous project experience ：

├── cmd/
│   └── main.go // Start the function 
├── etc
│   └── dev_conf.yaml              //  The configuration file 
├── global
│   └── global.go // Global variable references , Such as a database 、kafka etc. 
├── internal/
│       └── service/
│           └── xxx_service.go // Business logic processing class 
│           └── xxx_service_test.go
│       └── model/
│           └── xxx_info.go// Structure 
│       └── api/
│           └── xxx_api.go// The interface corresponding to the route is implemented 
│       └── router/
│           └── router.go// route 
│       └── pkg/
│           └── datetool// Time tools 
│           └── jsontool//json  Tool class 

In fact, the above division is just a simple package of functions , There are still many problems in the process of project practice . such as ：

• For function realization, I am through function The parameters of are also passed through the variables of the structure ？

• Whether it is safe to use the global variable reference of a database ？ Whether there is excessive coupling ？

• In the process of code implementation, almost all of them depend on the implementation , Instead of relying on interfaces , It will be MySQL Switch to a MongDB Is it necessary to modify all the implementations ？

So now in our work, with more and more code , Various in the code init,function,struct, Global variables feel more and more chaotic . Each module is not independent , It seems to be logically divided into modules , But there is no clear relationship between the upper and lower levels , There may be configuration reading in each module , External service call , Protocol conversion, etc. . Over time, the calls between different package functions of the service slowly evolve into a mesh structure , The flow direction of data flow and the sorting of logic become more and more complex , It's hard to figure out the data flow without looking at the code call .

But it's like 《 restructure 》 As mentioned ： Let the code work first - If the code doesn't work , Can't produce value ; Then try to make it better - By refactoring the code , Let us and others better understand the code , And can constantly modify the code according to the needs .

So I think it's time to change myself .

The Clean Architecture

In the concise architecture, we put forward several requirements for our project ：

1. Independent of the framework . The architecture does not depend on the existence of some feature rich software libraries . This allows you to use these frameworks as tools , Instead of cramming your system into their limited constraints .

2. Testable . Business rules can be created without UI、 database 、Web The server or any other external element is tested .

3. Independent of the user interface .UI Can be easily changed , Without changing the rest of the system . for example , One Web UI Can be replaced with a console UI, Without changing the business rules .

4. Database independent . You can take Oracle or SQL Server Switch to Mongo、BigTable、CouchDB Or something . Your business rules are not bound by the database .

5. Independent of any external agency . in fact , Your business rules don't know anything about the outside world .

The concentric circles in the figure above represent software in various fields . Generally speaking , The deeper you go, the higher your software level . The outer circle is the tactical implementation mechanism , The inner circle is the core strategy . For our project , Code dependency should be from outside to inside , One way single layer dependency , This dependency contains the code name , Or a function of a class , Variable or any other named software entity .

For a concise architecture, it is divided into four layers ：

• Entities： Entity

• Usecase： Express and apply business rules , Corresponding to the application layer , It encapsulates and implements all use cases of the system ;

• Interface Adapters： The software in this layer is basically some adapters , It is mainly used to convert the data in use cases and entities into external systems, such as databases or Web Data used ;

• Framework & Driver： The outermost circle is usually composed of some frameworks and tools , Such as a database Database, Web Frame, etc ;

So for my project , It is also divided into four layers ：

• models

• repo

• service

• api

Code layering

models

Encapsulates various entity class objects , Interactive with database 、 And UI Interactive and so on , Any entity class should be placed here . Such as ：

import "time"

type Article struct {
 ID        int64     `json:"id"`
 Title     string    `json:"title"`
 Content   string    `json:"content"`
 UpdatedAt time.Time `json:"updated_at"`
 CreatedAt time.Time `json:"created_at"`
}

repo

Here is the database operation class , database CRUD It's all here . It should be noted that , There is no business logic code here , Many students like to put business logic here .

If you use ORM, So what's put here ORM Operation related code ; If you use microservices , So here is the code of other service requests ;

service

Here is the business logic layer , All business process processing code should be placed here . This layer will decide to request repo What code of layer , Whether to operate the database or call other services ; All business data calculations should also be put here ; The input accepted here should be controller Incoming .

api

Here is the code for receiving external requests , Such as ：gin Corresponding handler、gRPC、 other REST API Framework access layer, etc .

Interface oriented programming

except models layer , Layers should interact with each other through interfaces , Not implementation . If you want to use service call repo layer , So you should call repo The interface of . Then when modifying the underlying implementation, our upper base class does not need to be changed , Just change the underlying implementation .

For example, we want to query all articles , So it can be repo Provide such an interface ：

package repo

import (
 "context"
 "my-clean-rchitecture/models"
 "time"
)

// IArticleRepo represent the article's repository contract
type IArticleRepo interface {
 Fetch(ctx context.Context, createdDate time.Time, num int) (res []models.Article, err error)
}

The implementation class of this interface can be changed according to requirements , For example, when we want to mysql As a storage query , Then you only need to provide such a base class ：

type mysqlArticleRepository struct {
 DB *gorm.DB
}

// NewMysqlArticleRepository will create an object that represent the article.Repository interface
func NewMysqlArticleRepository(DB *gorm.DB) IArticleRepo {
 return &mysqlArticleRepository{DB}
}

func (m *mysqlArticleRepository) Fetch(ctx context.Context, createdDate time.Time,
 num int) (res []models.Article, err error) {

 err = m.DB.WithContext(ctx).Model(&models.Article{}).
  Select("id,title,content, updated_at, created_at").
  Where("created_at > ?", createdDate).Limit(num).Find(&res).Error
 return
}

If you want to change to MongoDB To realize our storage , Then you only need to define a structure to realize IArticleRepo Interface can .

So in service When the layer is implemented, the corresponding repo Just inject , So that no changes are required service The realization of the layer ：

type articleService struct {
 articleRepo repo.IArticleRepo
}

// NewArticleService will create new an articleUsecase object representation of domain.ArticleUsecase interface
func NewArticleService(a repo.IArticleRepo) IArticleService {
 return &articleService{
  articleRepo: a,
 }
}

// Fetch
func (a *articleService) Fetch(ctx context.Context, createdDate time.Time, num int) (res []models.Article, err error) {
 if num == 0 {
  num = 10
 }
 res, err = a.articleRepo.Fetch(ctx, createdDate, num)
 if err != nil {
  return nil, err
 }
 return
}

Dependency injection DI

Dependency injection , English name dependency injection, abbreviation DI .DI before java Often encountered in engineering , But in go Many people inside said they didn't need , But I think it is necessary in the process of large-scale software development , Otherwise, it can only be passed through global variables or method parameters .

As for what is DI, In short, it is the dependent module , When creating a module , Injected into （ It is passed in as a parameter ） Inside the module . Want to know more about what is DI Here's another recommendation Dependency injection and Inversion of Control Containers and the Dependency Injection pattern These two articles .

If not DI There are two main inconveniences , One is that the modification of the underlying class requires the modification of the upper class , In the process of large-scale software development, there are many base classes , When a link is changed, dozens of files are often modified ; On the other hand, unit testing between layers is not convenient .

Because of dependency injection , In the process of initialization, it is inevitable to write a large number of new, For example, we need this in our project ：

package main

import (
 "my-clean-rchitecture/api"
 "my-clean-rchitecture/api/handlers"
 "my-clean-rchitecture/app"
 "my-clean-rchitecture/repo"
 "my-clean-rchitecture/service"
)

func main() {
 //  initialization db
 db := app.InitDB()
 // initialization  repo
 repository := repo.NewMysqlArticleRepository(db)
 // initialization service
 articleService := service.NewArticleService(repository)
 // initialization api
 handler := handlers.NewArticleHandler(articleService)
 // initialization router
 router := api.NewRouter(handler)
 // initialization gin
 engine := app.NewGinEngine()
 // initialization server
 server := app.NewServer(engine, router)
 // start-up 
 server.Start()
}

So for such a piece of code , Is there any way we don't have to write it ourselves ？ Here we can use the power of the framework to generate our injection code .

stay go Inside DI There are relatively no tools java It's so convenient , Generally, the technical framework mainly includes ：wire、dig、fx etc. . because wire Is to use code generation for Injection , The performance will be relatively high , And it's google To launch the DI frame , So we use wire For injection .

wire It's very simple , Create a new one wire.go file （ The file name is optional ）, Create our initialization function . such as , We're going to create and initialize a server object , We can do this ：

//+build wireinject

package main

import (
 "github.com/google/wire"
 "my-clean-rchitecture/api"
 "my-clean-rchitecture/api/handlers"
 "my-clean-rchitecture/app"
 "my-clean-rchitecture/repo"
 "my-clean-rchitecture/service"
)

func InitServer() *app.Server {
 wire.Build(
  app.InitDB,
  repo.NewMysqlArticleRepository,
  service.NewArticleService,
  handlers.NewArticleHandler,
  api.NewRouter,
  app.NewServer,
  app.NewGinEngine)
 return &app.Server{}
}

It should be noted that , Comments on the first line ：+build wireinject, Indicates that this is an injector .

In the function , We call wire.Build() Will be created Server The constructor of the dependent type is passed in . finish writing sth. wire.go File after execution wire command , It will automatically generate one wire_gen.go file .

// Code generated by Wire. DO NOT EDIT.

//go:generate go run github.com/google/wire/cmd/wire
//+build !wireinject

package main

import (
 "my-clean-rchitecture/api"
 "my-clean-rchitecture/api/handlers"
 "my-clean-rchitecture/app"
 "my-clean-rchitecture/repo"
 "my-clean-rchitecture/service"
)

// Injectors from wire.go:

func InitServer() *app.Server {
 engine := app.NewGinEngine()
 db := app.InitDB()
 iArticleRepo := repo.NewMysqlArticleRepository(db)
 iArticleService := service.NewArticleService(iArticleRepo)
 articleHandler := handlers.NewArticleHandler(iArticleService)
 router := api.NewRouter(articleHandler)
 server := app.NewServer(engine, router)
 return server
}

You can see wire It's generated for us automatically InitServer Method , This method initializes all the base classes to be initialized in turn . And then in our main In the function, you just need to call this InitServer that will do .

func main() {
 server := InitServer()
 server.Start()
}

test

Above, we defined what each layer should do , Then we should be able to test each layer separately , Even if the other layer doesn't exist .

• models layer ： This floor is very simple , Because you don't rely on any other code , So you can use it directly go The single test framework can be tested directly ;

• repo layer ： For this floor , Because we used mysql database , So we need mock mysql, So even if you don't have to connect mysql It can also be tested normally , I'm going to use github.com/DATA-DOG/go-sqlmock This library is coming mock Our database ;

• service layer ： because service Layers depend on repo layer , Because they are related through interfaces , So I use it here github.com/golang/mock/gomock Come on mock repo layer ;

• api layer ： This layer of dependency service layer , And they are related through interfaces , So you can also use gomock Come on mock service layer . But there's a little trouble here , Because our access layer uses gin, Therefore, it is also necessary to simulate sending requests during single test ;

Because we are through github.com/golang/mock/gomock To carry out mock , So you need to perform code generation , Generated mock We put the code into mock In bag ：

mockgen -destination .\mock\repo_mock.go -source .\repo\repo.go -package mock

mockgen -destination .\mock\service_mock.go -source .\service\service.go -package mock

The above two commands will help me automatically generate mock function .

repo Layer test

In the project , Because we used gorm As our orm library , So we need to use github.com/DATA-DOG/go-sqlmock combination gorm To carry out mock：

func getSqlMock() (mock sqlmock.Sqlmock, gormDB *gorm.DB) {
 // establish sqlmock
 var err error
 var db *sql.DB
 db, mock, err = sqlmock.New(sqlmock.QueryMatcherOption(sqlmock.QueryMatcherEqual))
 if err != nil {
  panic(err)
 }
 // combination gorm、sqlmock
 gormDB, err = gorm.Open(mysql.New(mysql.Config{
  SkipInitializeWithVersion: true,
  Conn:                      db,
 }), &gorm.Config{})
 if nil != err {
  log.Fatalf("Init DB with sqlmock failed, err %v", err)
 }
 return
}

func Test_mysqlArticleRepository_Fetch(t *testing.T) {
 createAt := time.Now()
 updateAt := time.Now()
 //id,title,content, updated_at, created_at
 var articles = []models.Article{
  {1, "test1", "content", updateAt, createAt},
  {2, "test2", "content2", updateAt, createAt},
 }

 limit := 2
 mock, db := getSqlMock()

 mock.ExpectQuery("SELECT id,title,content, updated_at, created_at FROM `articles` WHERE created_at > ? LIMIT 2").
  WithArgs(createAt).
  WillReturnRows(sqlmock.NewRows([]string{"id", "title", "content", "updated_at", "created_at"}).
   AddRow(articles[0].ID, articles[0].Title, articles[0].Content, articles[0].UpdatedAt, articles[0].CreatedAt).
   AddRow(articles[1].ID, articles[1].Title, articles[1].Content, articles[1].UpdatedAt, articles[1].CreatedAt))

 repository := NewMysqlArticleRepository(db)
 result, err := repository.Fetch(context.TODO(), createAt, limit)

 assert.Nil(t, err)
 assert.Equal(t, articles, result)
}

service Layer test

Here we mainly use gomock Generated code to mock repo layer ：

func Test_articleService_Fetch(t *testing.T) {
 ctl := gomock.NewController(t)
 defer ctl.Finish()
 now := time.Now()
 mockRepo := mock.NewMockIArticleRepo(ctl)

 gomock.InOrder(
  mockRepo.EXPECT().Fetch(context.TODO(), now, 10).Return(nil, nil),
 )

 service := NewArticleService(mockRepo)

 fetch, _ := service.Fetch(context.TODO(), now, 10)
 fmt.Println(fetch)
}

api Layer test

For this floor , We don't just have to mock service layer , You also need to send httptest To simulate request sending ：

func TestArticleHandler_FetchArticle(t *testing.T) {

 ctl := gomock.NewController(t)
 defer ctl.Finish()
 createAt, _ := time.Parse("2006-01-02", "2021-12-26")
 mockService := mock.NewMockIArticleService(ctl)

 gomock.InOrder(
  mockService.EXPECT().Fetch(gomock.Any(), createAt, 10).Return(nil, nil),
 )

 article := NewArticleHandler(mockService)

 gin.SetMode(gin.TestMode)

 // Setup your router, just like you did in your main function, and
 // register your routes
 r := gin.Default()
 r.GET("/articles", article.FetchArticle)

 req, err := http.NewRequest(http.MethodGet, "/articles?num=10&create_date=2021-12-26", nil)
 if err != nil {
  t.Fatalf("Couldn't create request: %v\n", err)
 }

 w := httptest.NewRecorder()
 // Perform the request
 r.ServeHTTP(w, req)

 // Check to see if the response was what you expected
 if w.Code != http.StatusOK {
  t.Fatalf("Expected to get status %d but instead got %d\n", http.StatusOK, w.Code)
 }
}

summary

So that's me right golang A little summary and Reflection on the problems found in the project , Think first whether it's right or not , After all, it has solved some of our current problems . however , The project always needs continuous reconstruction and improvement , So next time you have a problem, change it next time .

There is something wrong with my summary and description above , Please feel free to point it out and discuss .

Project code location ：https://github.com/devYun/go-clean-architecture

Reference

https://blog.cleancoder.com/uncle-bob/2012/08/13/the-clean-architecture.html

https://github.com/bxcodec/go-clean-arch

https://medium.com/hackernoon/golang-clean-archithecture-efd6d7c43047

https://farer.org/2021/04/21/go-dependency-injection-wire/