Getting started with go web programming: validators

Preface

Network authentication can be a challenge .  There is a saying in  Web  Principles that are widely circulated in development ：

We can't trust anything from the client user form .

So we have to validate all the incoming data before using it . Realization  REST API  yes  Go  Typical use cases for applications . API  The accepted data with wrong format may cause serious errors in other parts of the system .

The best case scenario is that your database has mechanisms to prevent the storage of malformed data . If you don't do that , This data can cause errors and unexpected behavior in your customer facing applications （ such as  

SQL  Inject

）.

In this article , We'll show you how to  Go  Send validation to  REST API  The data of .

Manually verify the input

Simple  REST API 

This is an easy one  REST API  Example , Use  

gorilla/mux

  Package building . It's a great  HTTP  Router , Especially for  REST API. API  Provide a path for an endpoint  

/user

. For the sake of simplicity , It only accepts... From all users  HTTP GET  And create a user's  HTTP Post. Besides , It has no persistent database , Instead, you use slices to store users in memory .

package main

import (
 "encoding/json"
 "log"
 "net/http"
 "strings"

 "github.com/gorilla/mux"
)

type User struct {
 ID int
 FirstName string
 LastName string
 FavouriteVideoGame string
 Email string
}

func main() {
 router := mux.NewRouter()
 router.HandleFunc("/user", PostUser).Methods(http.MethodPost)
 router.HandleFunc("/user", GetUsers).Methods(http.MethodGet)

 log.Fatal(http.ListenAndServe(":8081", router))
}

var users = []User{}
var id = 0

func validateEmail(email string) bool {
 // This is obviously not a good validation strategy for email addresses
 // pretend a complex regex here
 return !strings.Contains(email, "@")
}

func PostUser(w http.ResponseWriter, r *http.Request) {
 user := User{}
 json.NewDecoder(r.Body).Decode(&user)

 // We don't want an API user to set the ID manually
 // in a production use case this could be an automatically
 // ID in the database
 user.ID = id
 id++

 users = append(users, user)
 w.WriteHeader(http.StatusCreated)
}

func GetUsers(w http.ResponseWriter, r *http.Request) {
 w.Header().Add("Content-Type", "application/json")
 if err := json.NewEncoder(w).Encode(users); err != nil {
 log.Println(err)
 w.WriteHeader(http.StatusInternalServerError)
 return
 }
}

Now let's look at how to manually validate what is provided to this in the request body  API  Of  POST  Input to the handler .

Manually verify the input

Sometimes we ask the user to enter some fields , But they failed to complete the field . For example, in the previous section , When we need a user name . You can use  len  Function to get the length of the field , To ensure that the user has entered something .

 if len(r.Form["username"][0])==0{
 // code for empty field
 }

Suppose we want to use  Post  When the handler creates a user, it sets up  FirstName、LastName  and  Email. Besides , We want the email field to be a valid email address . An easy way to do this is to manually validate fields , As shown below ：

if user.FirstName == "" {
 errs = append(errs, fmt.Errorf("Firstname is required").Error())
}

if user.LastName == "" {
 errs = append(errs, fmt.Errorf("LastName is required").Error())
}

if user.Email == "" || validateEmail(user.Email) {
 errs = append(errs, fmt.Errorf("A valid Email is required").Error())
}

Complete example ：

package main

import (
 "encoding/json"
 "fmt"
 "log"
 "net/http"
 "strings"

 "github.com/gorilla/mux"
)

type User struct {
 ID int
 FirstName string
 LastName string
 FavouriteVideoGame string
 Email string
}

func main() {
 router := mux.NewRouter()
 router.HandleFunc("/user", PostUser).Methods(http.MethodPost)
 router.HandleFunc("/user", GetUsers).Methods(http.MethodGet)

 log.Fatal(http.ListenAndServe(":8081", router))
}

var users = []User{}
var id = 0

func validateEmail(email string) bool {
 // That's obviously not a good validation strategy for email addresses
 // pretend a complex regex here
 return !strings.Contains(email, "@")
}

func PostUser(w http.ResponseWriter, r *http.Request) {
 user := User{}
 json.NewDecoder(r.Body).Decode(&user)

 errs := []string{}

 if user.FirstName == "" {
 errs = append(errs, fmt.Errorf("Firstname is required").Error())
 }

 if user.LastName == "" {
 errs = append(errs, fmt.Errorf("LastName is required").Error())
 }

 if user.Email == "" || validateEmail(user.Email) {
 errs = append(errs, fmt.Errorf("A valid Email is required").Error())
 }

 if len(errs) > 0 {
 w.Header().Add("Content-Type", "application/json")
 w.WriteHeader(http.StatusBadRequest)
 if err := json.NewEncoder(w).Encode(errs); err != nil {
 }
 return
 }

 // We don't want an API user to set the ID manually
 // in a production use case this could be an automatically
 // ID in the database
 user.ID = id
 id++

 users = append(users, user)
 w.WriteHeader(http.StatusCreated)
}

func GetUsers(w http.ResponseWriter, r *http.Request) {
 w.Header().Add("Content-Type", "application/json")
 if err := json.NewEncoder(w).Encode(users); err != nil {
 log.Println(err)
 w.WriteHeader(http.StatusInternalServerError)
 return
 }
}

You can see that this verification method is very verbose . We have to define a custom function to validate common things , Like email addresses . Let's see how to improve this .

It's just a simple pass  

validateEmail

  Function to verify whether the mailbox contains  

@

  character ：

func validateEmail(email string) bool {
 // That's obviously not a good validation strategy for email addresses
 // pretend a complex regex here
 return !strings.Contains(email, "@")
}

In fact, a better way is to use regular expressions to verify  E-mail  The effectiveness of the ：

 if m, _ := regexp.MatchString(`^([\w\.\_]{2,10})@(\w{1,}).([a-z]{2,4})$`, r.Form.Get("email")); !m {
 fmt.Println("no")
 }else{
 fmt.Println("yes")
 }

Use the structure tag to validate the input

stay  Go  A more common way to validate structures in is to use structure tags . There are many packages that perform structure validation through structure tags . We will use... Here  https://github.com/go-playground/validator： The

Validator

Implement structure and single field value validation based on tags .

Use  

go get github.com/go-playground/validator/v10

  Installation .

This not only enables us to use structure tags for verification , It also provides a number of predefined validation methods , An E-mail address, for example .

We will perform this verification on the structure , But it doesn't discuss how to fill the structure .  We can assume that the data will be parsed  JSON  Payload 、 Enter explicit padding or other methods from the form to populate .

If your data needs another validator , Please check the documentation of the validator package . The verifier you need is most likely provided in the package  80  Under multiple validators .

package main

import (
 "encoding/json"
 "log"
 "net/http"

 "github.com/go-playground/validator/v10"
 "github.com/gorilla/mux"
)

type User struct {
 ID int `validate:"isdefault"`
 FirstName string `validate:"required"`
 LastName string `validate:"required"`
 FavouriteVideoGame string
 Email string `validate:"required,email"`
}

func main() {
 router := mux.NewRouter()
 router.HandleFunc("/user", PostUser).Methods(http.MethodPost)
 router.HandleFunc("/user", GetUsers).Methods(http.MethodGet)

 log.Fatal(http.ListenAndServe(":8081", router))
}

var users = []User{}
var id = 0

func PostUser(w http.ResponseWriter, r *http.Request) {
 user := User{}
 json.NewDecoder(r.Body).Decode(&user)

 validate := validator.New()

 err := validate.Struct(user)
 if err != nil {
 validationErrors := err.(validator.ValidationErrors)
 w.Header().Add("Content-Type", "application/json")
 w.WriteHeader(http.StatusBadRequest)
 responseBody := map[string]string{"error": validationErrors.Error()}
 if err := json.NewEncoder(w).Encode(responseBody); err != nil {
 }
 return
 }

 // We don't want an API user to set the ID manually
 // in a production use case this could be an automatically
 // ID in the database
 user.ID = id
 id++

 users = append(users, user)
 w.WriteHeader(http.StatusCreated)
}

func GetUsers(w http.ResponseWriter, r *http.Request) {
 w.Header().Add("Content-Type", "application/json")
 if err := json.NewEncoder(w).Encode(users); err != nil {
 log.Println(err)
 w.WriteHeader(http.StatusInternalServerError)
 return
 }
}

above  

validationError.Error()

  Returns a string , This string summarizes each failed validation in the structure . therefore  BadRequest  The response still has very detailed information about what went wrong .

We changed validation to use the validator package , Now verify according to the following rules ：

• ID  Fields should not be set by the user , So we verify that it has  int  The default value of , namely  0

• FullName  and  LastName  It's necessary

• Email fields are required , And use a predefined email validator to verify

Use a custom validator to validate input

So now we use the validator package and use the structure tag to validate the structure . But how do we verify a structure field that cannot be verified by the tag provided by the library ？

Suppose we want to blacklist some video games . Because we don't want anyone in our system to like  PUBG  or  Fortnite  Waiting for game users . under these circumstances , We can define a custom  validate  Mark values and let  validate  The package uses it like this ：

First, we define a validation function ：

func GameBlacklistValidator(f1 validator.FieldLevel) bool {
 gameBlacklist := []string{"PUBG", "Fortnite"}
 game := f1.Field().String()
 for _, g := range gameBlacklist {
 if game == g {
 return false
 }
 }
 return true
}

Then we use the validator instance to register the function and the corresponding tag .

...
 validate := validator.New()
 validate.RegisterValidation("game-blacklist", GameBlacklistValidator)
...

Now we are  User  Add a label to the definition of the structure .

type User struct {
 ID int `validate:"isdefault"`
 FirstName string `validate:"required"`
 LastName string `validate:"required"`
 FavouriteVideoGame string `validate:"game-blacklist"`
 Email string `validate:"required,email"`
}

Recommended validation Libraries

Awesome Go  Under the project, there are libraries for verification . Here are the recommendations ：

• checkdigit - Provide check digit algorithms (Luhn, Verhoeff, Damm) and calculators (ISBN, EAN, JAN, UPC, etc.).

• gody - :balloon: A lightweight struct validator for Go.

• govalid - Fast, tag-based validation for structs.

• govalidator - Validators and sanitizers for strings, numerics, slices and structs.

• govalidator - Validate Golang request data with simple rules. Highly inspired by Laravel’s request validation.

• jio - jio is a json schema validator similar to joi.

• ozzo-validation - Supports validation of various data types (structs, strings, maps, slices, etc.) with configurable and extensible validation rules specified in usual code constructs instead of struct tags.

• terraform-validator - A norms and conventions validator for Terraform.

• validate - Go package for data validation and filtering. support validate Map, Struct, Request(Form, JSON, url.Values, Uploaded Files) data and more features.

• validate - This package provides a framework for writing validations for Go applications.

• validator - Go Struct and Field validation, including Cross Field, Cross Struct, Map, Slice and Array diving.

• Validator - A lightweight model validator written in Go.Contains VFs:Min, Max, MinLength, MaxLength, Length, Enum, Regex.

According to the introduction, pick one of your own projects ~

summary

verification  REST API  Input is critical to prevent malformed data in your application . You can write your own validation logic , But in most cases , It is best to use a well maintained validation package , For example, the above recommendation .

This allows you to use tags in the structure to configure authentication , And keep the logic of running validation simple . If you have a special use case that requires unusual validation capabilities , You can still define your own extensions for the validator package .

• Package validatorValidation 

• Awesome Go
  ：https://awesome-go.com/validation/

• Client-side form validation

• Validate REST Input in Go

• Verification of inputs