Read more

1 The core concept

Component type ：

Coordinator：
- analysis sql
- Generate query plan
- Manage and schedule Worker. meanwhile , It can also serve as Worker
Worker
- from Connector Get data in , And calculate according to the execution plan

presto_architecture

2 Deploy

One Presto The cluster contains at least one Coordinator And a Worker, among Coordinator It can also be used as Coordinator and Worker（ It is better not to do this in the production environment ）

2.1 Configuration profile

Presto The configuration related directory structure is as follows ：

etc/catalog： each Connector Configuration of , You can refer to Connectors
config.properties： The core configuration
jvm.config：Java Virtual machine related configuration
log.properties： Log related configuration
node.properties： Configuration related to node metadata

etc
├── catalog
│   └── mysql.properties
├── config.properties
├── jvm.config
├── log.properties
└── node.properties

2.2 Deploy Coordinator

Reference resources Deploying Presto

2.2.1 config.properties

coordinator： It has to be for true
node-scheduler.include-coordinator：true It means that we should Coordinator At the same time as Worker, Single node Presto The cluster or test environment can be configured as true
http-server.http.port： Monitoring http port , For interaction between nodes
discovery-server.enabled：true Indicates that the current node is discovered as a service server
discovery.uri： Service discovery server Of uri, In general , use Coordinator As server, therefore ,uri Usually it is http://<coordinator_host>:<coordinator_http-server.http.port>

coordinator=true
node-scheduler.include-coordinator=false
http-server.http.port=8080
query.max-memory=50GB
query.max-memory-per-node=1GB
query.max-total-memory-per-node=2GB
discovery-server.enabled=true
discovery.uri=http://example.net:8080

2.2.2 jvm.config

Java Virtual machine configuration

-server
-Xmx16G
-XX:+UseG1GC
-XX:G1HeapRegionSize=32M
-XX:+UseGCOverheadLimit
-XX:+ExplicitGCInvokesConcurrent
-XX:+HeapDumpOnOutOfMemoryError
-XX:+ExitOnOutOfMemoryError

2.2.3 log.properties

It is generally used to configure the log level

1	com.facebook.presto=INFO

2.2.4 node.properties

node.id： Each machine is different
node.data-dir： Data directory , Including logs and runtime data

1
2
3

node.environment=production
node.id=ffffffff-ffff-ffff-ffff-ffffffffffff
node.data-dir=/var/presto/data

2.3 Deploy Worker

Reference resources Deploying Presto

2.3.1 config.properties

coordinator： It has to be for false
http-server.http.port： Monitoring http port , For interaction between nodes
discovery.uri： Service discovery server Of uri, In general , use Coordinator As server, therefore ,uri Usually it is http://<coordinator_host>:<coordinator_http-server.http.port>

coordinator=false
http-server.http.port=8080
query.max-memory=50GB
query.max-memory-per-node=1GB
query.max-total-memory-per-node=2GB
discovery.uri=http://example.net:8080

2.3.2 jvm.config

Java Virtual machine configuration

-server
-Xmx16G
-XX:+UseG1GC
-XX:G1HeapRegionSize=32M
-XX:+UseGCOverheadLimit
-XX:+ExplicitGCInvokesConcurrent
-XX:+HeapDumpOnOutOfMemoryError
-XX:+ExitOnOutOfMemoryError

2.3.3 log.properties