[latest in the whole network] how to start the opentsdb source code in the local ide run

background

I need to know Opentsdb Source code , Then I thought I could be here run get up , Make some breakpoints and logs to follow , Then I went to the Internet with great interest to find information to see if it could be quickly run get up , But find all the blogs 、 All documents seem to come from the same source , And finally run It is also very vague , I still failed to follow the tutorial . So I felt for myself , Take your time run Get up , And then make a note of , Hope to want to learn opentsdb My classmates are helpful . Please indicate the source if reprinted ~

1. Pull code

First fork once opentsdb Source code library for

Pull opentsdb

git clone https://github.com/whua3/opentsdb.git

Switch branches

git checkout v2.4.0

Create your own development branch

git checkout -b dev/240

2. Local compilation

If you just compile the source code locally , It's a little bit easier , Just follow the steps below

mkdir build cp -r third_party ./build ./build.sh

Found in maven Central warehouse download jar Package failed , Change the maven Warehouse address . take Makefile.in Medium http://central.maven.org and http://repo1.maven.org Replace all with https://repo1.maven.org

And then again ./build.sh

Compile the complete , At the same time build You can see the generated tsdb-2.4.0.jar package

3. Local IDE function Opentsdb

And then there's the opentsdb Source code , In the local IDE To run , This is more troublesome .

opentsdb This is not a standard maven project , First, let's generate pom file

sh build.sh pom.xml

Then we find that there is a generation pom.xml Document and src-main,src-test Two directories , among src-main and src-test yes src Soft links to files in ,

pack

mvn clean package -Dmaven.test.skip=true -P hbase

Two errors will be reported if found

Caused by: org.apache.maven.plugin.MojoExecutionException: Command execution failed Caused by: org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)

Comment out pom.xml These two inside execution

• create-plugin-test-jar
• attach-javadocs

REPACK

mvn clean package -Dmaven.test.skip=true -P hbase

Package successfully , It was found that a target Catalog

Then we're going to put target Generated in directory , target/generated-sources/net/opentsdb/query/expression/parser Copy directory to src-main/net/opentsdb/query/expression Under the table of contents , hold target/generated-sources/net/opentsdb/tools/BuildData.java Copied to the src-main/net/opentsdb/tools Under the table of contents .

Next we try to do it locally idea Let's take a break , First of all, will pom.xml.in Move out of the project , If the newly generated pom.xml Not recognized by , Mouse selected pom.xml, Then right click to select ”add as maven project“, then reload maven project

Then add a new application configuration

And then run directly , Report errors java: duplicate class: tsd.client.QueryString

Let's search this class , Find out src-main/tsd/client and src-main/net/opentsdb/tsd/client It's repeated here

because src-main/tsd/client This is src The soft links , We can src-main/tsd/client This directory excluded fall

Rerun discovery ,TSDMain It's running successfully , It's just a lack of opentsdb Configuration file for .

We are in TSDMain Of main Method to add a test log , Set a breakpoint , then debug mode , You can see that success stops at the breakpoint , The test log is also printed .

Then add the configuration , Give Way opentsdb In the local run get up , stay configuration Of program arguments Add configuration

--config=/Users/garyhwang/Documents/projects/IdeaProjects/opentsdb/src/opentsdb.conf

The configuration file needs to be changed 3 It's about

tsd.network.port = 4242 tsd.http.staticroot = /Users/garyhwang/Documents/localTestDir/opentsdb/static/ tsd.http.cachedir = /Users/garyhwang/Documents/localTestDir/opentsdb/cache/

In addition, I added several configurations in the configuration , For specific configuration functions, please refer to opentsdb Official document

tsd.core.autocreate_metrics = true tsd.query.skip_unresolved_tagvs = true tsd.query.timeout = 10000 tsd.storage.enable_compaction = false tsd.core.tag.allow_specialchars [email protected]#$%^&*()+{}|: <>?~`-=[]\;',./"

Then start again , Discovery or failure , You can see that the connection is local 2181 Port failed , This is because there is no such thing here Zookeeper and HBase, because Opentsdb The bottom layer depends on storage （ It's usually HBase）, So if you want to be local run rise opentsdb, First, you need to be local run rise HBase and Zookeeper（HBase Will be built in ）.

So we went to the local first HBase Of HMaster run get up , How to run rise HBase Not in this article , Or write something down later .

Then we run... Again opentsdb, This failure is due to less mistakes tsdb surface , We need to HBase Zhongba Opentsdb Create the required tables .

We can see src There is a create_table.sh, If it's on the server , Run this script directly and you will be able to HBase Create four corresponding tables , If it's local , I started a hbase-shell, Then manually copy the create table command into the .

create 'tsdb-uid', {NAME => 'id', BLOOMFILTER => 'ROW', DATA_BLOCK_ENCODING => 'DIFF'}, {NAME => 'name', BLOOMFILTER => 'ROW', DATA_BLOCK_ENCODING => 'DIFF'}

create 'tsdb', {NAME => 't', VERSIONS => 1, BLOOMFILTER => 'ROW', DATA_BLOCK_ENCODING => 'DIFF', TTL => '604800'}
  
create 'tsdb-tree', {NAME => 't', VERSIONS => 1, BLOOMFILTER => 'ROW', DATA_BLOCK_ENCODING => 'DIFF'}
  
create 'tsdb-meta', {NAME => 'name', BLOOMFILTER => 'ROW', DATA_BLOCK_ENCODING => 'DIFF'}

among tsdb surface （ A table for storing data ） Set up TTL by 604800（7 God ）

Run again opentsdb, Discovery started successfully locally ,4242 The port listens successfully .

Then let's test it opentsdb Reading and writing functions of

curl -i -X POST -d '{
    "metric":"gary-test",
    "timestamp":1624091745,
    "value":0.85,
    "tags":{
        "host":"127.0.0.1"
    }
}' http://localhost:4242/api/put?details

curl -d '{
    "start": "30m-ago",
    "queries": [
        {
            "aggregator": "none",
            "downsample": "30s-last",
            "metric": "gary-test",
            "tags": {
                "host":"127.0.0.1"
            }
        }
    ]
}' http://localhost:4242/api/query

You can see opentsdb Both writing and reading data succeeded ~

4. end

thus , In the building Opentsdb The local development environment officially ends , I wish all students who have needs can run get up ~

Students in need can also pull My warehouse This code , I corrected it in the article , It can be convenient for you and faster run get up ~

https://github.com/whua3/opentsdb/tree/local-run

Last ：

Please indicate the source if reprinted ~