Nifi from introduction to practice (nanny level tutorial)

background ：

Company leaders decided to separate the import of various basic data from the code , use Apache Nifi Replace . Make developers pay more attention to business , And don't care about the origin of the foundation .

Apache Nifi It is a new tool for the whole team , We haven't touched before , Even for the first time , I'm one of them . But fortunately , The leader assigned me to familiarize myself with this tool , Make according to the actual application field DEMO, And share with the team .

Environmental preparation

windows：

1、 download nifi zip package ：https://nifi.apache.org/download.html , Choose the version you want .

Be careful ：1.14 Version start , The default setting is authentication , Anonymous login is not allowed .

2、 take zip Unzip the package to any local directory . After decompression, the following figure is shown

Be careful ： The path should be in English , Otherwise, there will be some exceptions that cannot be started

3、 modify conf( Optional ). To avoid port conflicts , Can be /conf/nifi.properties in https.port Modify other ports specified for you . The default is 8443.

The other is /conf/bootstrap.conf, Main modifications here JVM memory settings,

The default is 512M.

4、 start-up nifi. double-click /bin/run-nifi.bat file

The following screen is a success

Be careful ： If you want to quit nifi, Do not directly click on the top right corner of the figure above “ close ” Button . Because it doesn't completely quit nifi, Its process is still running , The port is still occupied , So the next time you run it again, there will be an error . You can only kill the process , Or restart the computer . The correct operation is ctrl+c Stop it

5、 open nifi UI.( Distance start nifi Probably 30 Seconds later )

open /logs/nifi-app.log, Search for username, Here's the picture

Enter the corresponding user and password Then you can log in successfully

Linux

linux Use it next time docker compose Show me

1、 Get ready linux and docker Environmental Science . In this step, you can search by yourself

2、 Definition docker-compose.yml file , Enter the following

Click to view the code

version: '3'

services:

  nifi:

    cap_add:

      - NET_ADMIN # low port bindings

    image: apache/nifi:1.16.1

    container_name: nifi-dev

    ports:

      - 18443:18443 # HTTPS interface

    environment:

      NIFI_WEB_HTTPS_PORT: "18443"

    volumes:

      - ./drivers:/opt/nifi/nifi-current/drivers

      - ./database_repository:/opt/nifi/nifi-current/database_repository

      - ./flowfile_repository:/opt/nifi/nifi-current/flowfile_repository

      - ./content_repository:/opt/nifi/nifi-current/content_repository

      - ./provenance_repository:/opt/nifi/nifi-current/provenance_repository

      - ./state:/opt/nifi/nifi-current/state

      - ./logs:/opt/nifi/nifi-current/logs

      - ./nifi-conf:/opt/nifi/nifi-current/conf

Designated here nifi The image is official docker image 1.16.1 edition , Port is 18443. Be careful ：ports and enviroment Neither of these settings can be omitted . Then mount the corresponding directory in the container to the host computer docker-compose.yml File directory . The advantage of this is that even if you start a new image instance , The original data is still in , In this way, data persistence is realized .

because nifi Starting requires reading conf Various configurations under the directory , Now that's it conf The directory is empty . Therefore, when starting, it will report nifi.propeties The file does not exist .

There are two solutions ： One is to put the conf Copy all files in the directory to nifi-conf Under the table of contents . Two is conf The directory mounting mode is changed , as follows ：

Click to view the code



version: '3'

services:

  nifi:

    cap_add:

      - NET_ADMIN # low port bindings

    image: apache/nifi:1.16.1

    container_name: nifi-dev

    ports:

      - 18443:18443 # HTTPS interface

    environment:

      NIFI_WEB_HTTPS_PORT: "18443"

    volumes:

      - ./drivers:/opt/nifi/nifi-current/drivers

      - ./database_repository:/opt/nifi/nifi-current/database_repository

      - ./flowfile_repository:/opt/nifi/nifi-current/flowfile_repository

      - ./content_repository:/opt/nifi/nifi-current/content_repository

      - ./provenance_repository:/opt/nifi/nifi-current/provenance_repository

      - ./state:/opt/nifi/nifi-current/state

      - ./logs:/opt/nifi/nifi-current/logs

      - nifi-conf:/opt/nifi/nifi-current/conf

volumes:

  nifi-conf:

That's the way it works conf The directory is mounted on /var/lib/docker/volumes Under the table of contents , In this way, corresponding files can be generated automatically .

3、 start-up nifi.docker-compose up （ add -d It's running in the background , No log will be output at the terminal ）

In the log nifi-app.log Search for username, Get the corresponding user and password, Then log in nifi. This step and windows identical

thus ,nifi The environment is ready . The next one is about nifi Authentication of .

nifi From introduction to practice （ Nanny class course ）—— More related articles in environment

Nanny class course ——Ubuntu16.04 Server Build a deep learning environment ： install CUDA8.0,cuDNN6.0,Bazel0.5.4, Source code compilation and installation TensorFlow1.4.0(GPU edition )
Write it at the front In this paper, the author narrates that in Ubuntu16.04 Server Lower installation CUDA8.0,cuDNN6.0 And source code compilation and installation TensorFlow1.4.0(GPU edition ) My own experience , Including the problems and solutions , There are also some. ...
Self built local server , build by oneself Web The server —— Nanny class course ！
Set up a local server ,Web The server -- Nanny class course ! This article was first published in https://blog.chens.life/How-to-build-your-own-server.html. First map ! This is the general idea . ...
Deep learning is the beginning of Python Quick tutorial - The basic chapter
5.1 Python brief introduction This chapter will introduce Python The most basic grammar of , And some basic uses that are most relevant to deep learning and computer vision . 5.1.1 Python Brief history Python It's an interpretive high-level programming language , It is characterized by simplicity ...
ElasticSearch Introduction （ Nanny class course ）
This chapter will introduce :ElasticSearch The role of , build elasticsearch Environment (Windows/Linux),ElasticSearch Cluster building , Visual client plug-ins elasticsearch-he ...
Nanny class course ！ Hands teach you how to use Longhorn Manage cloud native distribution SQL database ！
Author's brief introduction Jimmy Guerrero, Have... In the developer relations team and open source community 20 Years of experience . He currently leads YugabyteDB Our community and marketing team . This article is from Rancher Labs Longhorn yes Kubern ...
Nanny class course , How to discover GitHub High quality projects on the Internet ？
Watch it first and then like it , Give yourself a little time to think , WeChat search [ Silent king two ] Focus on this talented programmer . this paper GitHub github.com/itwanger Included , There are also interview questions arranged by large factories , And my series . ...
【 Nanny class course 】 Hand in hand to teach you to carry out Go Language environment installation and related VSCode To configure
[Go Introduction to language series ] Previous posts : [Go Introduction to language series ]( 7、 ... and ) How to use Go Methods ? [Go Introduction to language series ]( 8、 ... and )Go Is language an object-oriented language ? [Go Introduction to language series ]( Nine ) This is written to understand how to use interfaces This article ...
Nanny class course , Take you to big data , from 0 To 1 build Hadoop colony
Introduction to big data , Concept part Concept part , It is suggested that friends who have no knowledge of big data before read it Introduction to big data What is big data big data (Big Data) It refers to the inability to capture in a certain time range with conventional software tools . Manage and process data sets , Yes ...
Nanny class course ！ Use k3d Realization K3s High availability ！
Have you ever wanted to try K3s High availability mode of ? But suffering from no 3 individual " Spare node ", Or not setting the time required for the same number of virtual machines ? that k3d Maybe you need this plan very much ! If you are right about k3d I don't know yet , Its name ...
use Python Write about a snake , Nanny class course ！
This article is based on Windows Environmental development , fit Python Novice The author of this article :HelloGitHub-Anthony HelloGitHub To launch the < Explain open source projects > series , Introduction to this issue Python ...

Random recommendation

JS modularization
One . The original way of writing /* A module is a set of methods to achieve a specific function . Just put different functions ( And the variables that record the State ) Simply put them together , Even a module . The function above m1() and m2(), Make up a module . When you use it , Just call it directly . ...
transition
transition: all All attributes will have a transition effect . property Define the CSS List of attribute names , The list is separated by commas transition-duration Specify how much it takes to complete the transition effect ...
（ turn ） The database gets the current time getdate()
CONVERT(nvarchar(10),count_time,121): CONVERT Is the date conversion function , It's usually in the time type (datetime,smalldatetime) And string types (nchar, ...
Jmeter Set the default Chinese page
Method 1 ( From the Internet ) start-up Jmeter find options >choose language >chinese( Choose your own simplified and traditional ). In this way, the interface will be changed into Chinese , But when we open it again the next time ...
python The way ----- Front end css
Content of this article CSS grammar css Four ways of introduction css Selectors css Attribute operation Caution! Background management layout css Responsive layout One .CSS grammar CSS The rules consist of two main parts : Selectors , And one or more sounds ...
Book one ：lesson fifty three.
original text : An interesting climate. A:where do you come from? B:I come from England. A:What's the climate li ...
linux Task plan and periodic task plan
Relevant command :at.batch.cron.mailx Perform a task sometime in the future :at, batch Run a task periodically : cron One . Perform a task sometime in the future :at command at, batch, atq, atr ...
host & virtual machine & Development board Mutual communication
@2018 year 7 month 10 Japan One of the successful methods : The virtual machine is set to bridge mode , Ensure that the three are in the same network segment ,ping Way to test network connectivity OK
02. take uboot,kernel,rootfs Download to the development board
Reprint , Invasion and deletion take uboot,kernel,rootfs Download to the development board 1. Why download The so-called Download , Also called burn recording , Deploy . 1.1. What is? u-boot Hi3518EV200 Veneer Bootloader use ...
python json.loads json.dumps(ensure_ascii = False) The problem of Chinese character garbled code is solved
python Convert to json When Chinese character coding problem 2017 year 03 month 23 Japan 18:50:04 Read the number :5604 There is such a need : Need one json file The data is queried from the database 1. Set file header # -* ...

当前位置：网站首页>Nifi from introduction to practice (nanny level tutorial) - environment

Nifi from introduction to practice (nanny level tutorial) - environment

background ：

Environmental preparation

nifi From introduction to practice （ Nanny class course ）—— More related articles in environment

Random recommendation

边栏推荐

猜你喜欢

随机推荐