How to use FME to create your own functional software

Catalog

Preface

One 、 summary

Two 、 The development train of thought

1. The front-end development

（1） The login page

（1） Function page

（1） The download page

 2. The backend development

Two 、 Published online

summary

---

---

Preface

After several months of cultivation , A previously updated article uses django and fme Research on implementing template inheritance is now finally bearing fruit , Today Wukong will share and use fme Build data analysis as a background computing model web application , Create your own functional software , Let the user get rid of fme Environmental Science , Create your own ui interface .

---

One 、 summary

1、 programing language ：JavaScript,python

2、 Development framework ： front end VUE3 , Back end django

3、 database ： Data sheet PostgreSql, Process management database redis

4、 development tool ：webstorm,pycharm

5、 Calculation model api：fme Templates

Two 、 The development train of thought

1. The front-end development

This development mainly adopts the front-end and back-end separate development , Front end use vue3+element-plus build web Application page , The main page is the login page 、 Function page 、 Results download page .

（1） The login page

  There's nothing to say about this , The main thing to pay attention to is to be a navigation guard , After logging in the results, let the system from the background api Get token value , According to whether there is token Value to determine whether the page jumps , If you don't get token Value, the user cannot enter the main page . This part is more general , Almost any application is similar .

export default defineComponent({

 
    setup() {
          const loginForm={
            username:'',
              password:'',
  };

         let router=useRouter();
         let store=useStore();
        return {
          router,
          store,
          loginForm,
        };
    },
    mounted() {
    let loginForm = JSON.parse(localStorage.getItem("loginForm"));
    if (loginForm) {
      this.loginForm = loginForm;
    }
  },
  methods: {
       onSubmit() {
             let api = 'http://192.168.0.179:8000/lg/login/'
             let router=this.router
             window.localStorage.setItem("loginForm",this.loginForm.username);
              axios.post(api,this.loginForm,{
       timeout: 2000,
      }).then(function(res:any){
              alert(' Login successful ');
              window.localStorage.setItem("token",res.data.token);
              router.push({path:"/"});
              // Re trigger the route 
                location.reload()

（2） Function page

  The function page is simple and generous , It's online now 3 Feature sets , Parameters are submitted to the background in the form of user-defined forms , And get results , For subsequent online functions, simply add a sub route .

  Here is one of the sub page codes , It should be noted that if some large files are transferred , You need to break up the file binary stream , Then the back-end accepts and combines them into a complete file , In the future, we will consider adding functions such as breakpoint continuation .

<template>
 
    <div class="page-box">
  <vxe-toolbar>
  </vxe-toolbar>

  <el-form ref="form" :model="form" label-width="80px">
  <el-row>
  <el-col :span="24">

  <div class="pagetit"> Sharp angle repair </div>
  </el-col>
  </el-row>

  <el-row>
  <el-col :span="10">
    <el-form-item label=" angle ">
  <el-input v-model="forms.angel" placeholder=" The sharp angle determines the maximum angle "></el-input>
  </el-form-item>
  </el-col>
  </el-row>

  <el-row>
  <el-col :span="10">
    <el-form-item label=" area ">
  <el-input v-model="forms.area" placeholder=" The area of the sharp part changes in size "></el-input>
  </el-form-item>
  </el-col>
  </el-row>
  <el-row>
   <el-col :span="18">
     <el-form-item label=" Upload files ">
            <el-upload

                multiple
                ref="mYupload"
              :on-change="fileget"
              class="upload-demo"
              action="111"

              :auto-upload="false"
            >
              <template #trigger>
                <el-button type="primary"> What needs to be repaired shp file </el-button>
              </template>

              <el-button type="primary" @click="clearFiles">
                 Empty 

              </el-button>

            </el-upload>

       </el-form-item>
   </el-col>
  </el-row>

  <el-row>
      <el-col :span="18">
   <el-form-item>
      <el-button type="primary" @click="onSubmit"> Submit data </el-button>

    </el-form-item>
  </el-col>
  </el-row>

    </el-form>
    </div>
</template>
<script lang="ts">
import { defineComponent, ref } from "vue";

import axios from "axios";
export default defineComponent({
    setup() {
  var fileList=[];
  var file= {};
  var dialogImageUrl= '';
       var   dialogVisible= false;
  const forms=ref({

   angel:'',
   area:'',


  });


    return {
     file:{},
     fileList:[],
   forms,

    };
    },
     methods: {
      fileget(file){
        this.fileList.push(file.raw)


        console.log(this.fileList)
      },

      clearFiles() {
        this.$refs["mYupload"].clearFiles();
      },

      onSubmit() {
     let api = 'http://192.168.0.179:8000/api/test/';
      let fd = new FormData();
      //let username= window.username
       let name = window.localStorage.getItem('loginForm');

      for(var i=0;i<this.fileList.length;i++){
         fd.append('file',this.fileList[i]);;
      }
      // Transfer files 
         fd.append('angel',this.forms.angel);// Pass other parameters 
         fd.append('area',this.forms.area);
         fd.append('name',name);
      axios.post(api,fd,{
       headers: {'Content-Type': 'multipart/form-data'},
       timeout: 200000,
      }).then(function(res){
              alert(' Submit successfully ');
      })
      this.fileList=[];
      this.$refs["mYupload"].clearFiles();
      this.forms={
   angel:'',
   area:'',
  };

      // return false// Shielded action By default 
    },

        },
   
})
</script>
<style lang="scss" scoped>
.up-load{
    width: 50vw;
    margin: 50px auto;
    >div{
        margin-top: 20px;
    }
    .up-title{
        text-align: center;
        font-size: 30px;
        font-weight: 600;
    }
    .up-one{
        height: 70px;
        overflow: hidden;
    }
    .property{
        padding: 0 50px;
        text-align: center;
        >div{
            padding: 10px;
            font-weight: 600;
        }
    }
}
/deep/.vxe-button--content{color: #fff;}
 /deep/.vxe-button.type--button:hover,.vxe-button.type--button.active{background: #2D94FC !important; border-color: #2D94FC!important;}
</style>

（3） The download page

All of our form submissions use ajax Asynchronous requests , After the request is completed , The back end receives the front-end data , Store database , Trigger fme Calculation model .fme The template will interact with the backend after running , Automatically generate Download Links , And render to the front end .

The task will be generated immediately after the form is submitted , The task status will be automatically updated after the background calculation .

fme Automatically render the download link after the template is successfully executed

fme Template failed to run , It will also update the task status so that users can re check their submission parameters

Click the download link , The calculated results are downloaded successfully .

Click the delete button to automatically delete the location of the task in the database , And the corresponding download file .

 2. The backend development

Back end development adopts django and django Extension library for drf To quickly generate interfaces , use redis and celery To manage fme Process to cope with high concurrency task queue situations .

It is worth mentioning that celery This python library , It can realize millions of process management , Successfully let our background template execute orderly and efficiently . After testing ,celery In dealing with high request volume and complex fme The template is even comparable to fmeserver The efficiency of .

  Of course, the only defect is also very obvious , Namely fmedesktop Only supported at most 8 Processes run at the same time , But it is not impossible to overcome this problem , Distributed can be built celery, Make multiple machines one django Providing computing services . To build a distributed , The technology involved is also more difficult , Consider resource allocation , Task scheduling , And the connection of multiple machines is stable .redis Multiple intranet shares must be established with databases and local disks .

The other thing that needs to be tackled at the back end is Rendering of output results , Because many of our users , They perform different tasks , The back end needs to write logic , According to the user name information fed back from the front end , Call the user's task from the database to feed back to the front end , Otherwise, it will lead to task sharing , Users do not know which task they are performing .

Here are some back-end code .

from rest_framework.views import APIView
from rest_framework.response import Response
from books.models import Books
from books.serializer import BooksSerializer
from rest_framework import status
import os
import uuid
import shutil

class BooksViewSet(APIView):
    def get(self,request):
        name=request.GET['name']
        path_list = Books.objects.filter(name=name)
        path_list1=BooksSerializer(instance=path_list,many=True)
        return Response(path_list1.data)

    def post(self, request):
        """ Add a piece of data """
        uid=uuid.uuid1()
        #  Receive data submitted by client 
        name=request.data['name']
        type=' Sharp angle repair '
        an=request.data['angel']
        area=request.data['area']
        url=' In calculation ...'
        import time
        now = time.localtime()
        time = str(time.strftime("%Y-%m-%d %H:%M:%S", now))
        Books.objects.create(
            name=name,
            path=url,
            type=type,
            uid=uid,
            date=time
        )
        myfile = request.FILES.getlist("file", None)
        #print(myfile)
        path = os.makedirs(r'G:\\temp' + './{}'.format(uid))
        aaA = str('G:\\temp' + '\\{}'.format(uid))
        for ff in myfile:
            road = os.path.join(aaA, ff.name)
            f = open(road, 'wb+')
            for chunk in ff.chunks():
               f.write(chunk)
            f.close()
        from celery_tasks.api.tasks import angel


        try:
            angel.delay(road,an,area,uid,name)
        except:
            aa=Books.objects.filter(uid=uid)
            aa.update(path=' Calculation failed , Please check whether the submitted data is standardized ')
            pass
        return Response(status=status.HTTP_201_CREATED)

class reViewSet(APIView):
    def post(self, request):
        data = request.data['removeRecords'][0]
        pk=data['id']
        uid=data['uid']
        course = Books.objects.get(pk=pk)
        course.delete()
        shutil.rmtree(r'G:\django1\media\{}'.format(uid))
        return Response(status=status.HTTP_201_CREATED)
class gdViewSet(APIView):
    def post(self, request):
        """ Add a piece of data """
        uid=uuid.uuid1()
        #  Receive data submitted by client 
        name=request.data['name']
        type=' Classification of cultivated land quality '
        xzq=request.data['xzq']
        url=' In calculation ...'
        import time
        now = time.localtime()
        time = str(time.strftime("%Y-%m-%d %H:%M:%S", now))
        Books.objects.create(
            name=name,
            path=url,
            type=type,
            uid=uid,
            date=time
        )
        myfile = request.FILES.getlist("file", None)
        os.makedirs(r'G:\\temp' + './{}'.format(uid))
        aaA = str('G:\\temp' + '\\{}'.format(uid))
        for ff in myfile:
            road = os.path.join(aaA, ff.name)
            f = open(road, 'wb+')
            for chunk in ff.chunks():
               f.write(chunk)
            f.close()
        from celery_tasks.api.zippp import upzip
        from celery_tasks.api.tasks import gd
        path = 'G:\\temp\\{}'.format(uid)
        for i in os.listdir(path):
            file_name, file_extension = os.path.splitext(i)
            if file_extension == '.zip':
                upzip(dirpath=str(path + '\\' + i), outpath=path)
                gdbroad = str(path + '\\' + file_name)
            if file_extension == '.shp':
                xzqroad = str(path + '\\' + i)

        try:
            gd.delay(xzqroad,xzq,gdbroad,uid,name)
        except:
            aa=Books.objects.filter(uid=uid)
            aa.update(path=' Calculation failed , Please check whether the submitted data is standardized ')
            pass

        return Response(status=status.HTTP_201_CREATED)

 3.FME Template call

The calling template adopts python Trigger dos Instruction to pass front-end parameters , And call fme Templates , Need to pay attention to tradition os Call will cause problems , Because traditional os Call is an asynchronous Ctrip execution . This will be with celery conflict , We need to subprocess Library to call the template , Synchronous execution .

Two 、 Published online

use iis Server Publishing django Mission , For details, please refer to the blogger's blog .Django--4 Django The project in Win10 + IIS The server deployment goes online _liuning2008 The blog of -CSDN Blog _django Project deployment to server windowszDjango--4 Django The project in Win10 + IIS The server deployment goes online _liuning2008 The blog of -CSDN Blog _django Project deployment to server windows

This is the overall project framework ,temp Store temporary cache files ,fmw Store template files . I set up windos User plan , Empty temporary files regularly to prevent the disk from bursting .

---

summary

After months of liver explosion, the whole set of functions has finally been realized , Seemingly simple functions , Simple logic , In fact, it involves a huge amount of technology , There are more and more pits to step on in the middle , In especial vue Lifecycle and routing distribution , It took me a lot of time to achieve the effect . Interested partners can start from the front end js,html5,css3 Start learning , Master a back-end frame , recommend django and flask. I am very satisfied with the final result , At the same time, I would like to thank my front-end boss, the foreigner , I've been mended a lot vue Knowledge points of . The biggest feature of this research is that it can let users get away from fme Environmental Science , Out of configuration python Library's troubles , Out of the dilemma that computer equipment can't keep up , Just one browser , Can complete a large number of complex and labor-intensive work , Later, I will launch the feature set one after another , And I'm going to launch many templates involved in my previous in-depth learning .