MRS: Introduction to the use of Alluxio

1. Alluxio的简介

Alluxio在mrsThe data processing ecosystem is between computing and storage,为上层spark、presto、mapredue、hiveA computing framework provides a data abstraction layer,Computational frameworks are available through a unified clientapiand the global namespace to access the underlying storage system,And cut to provide memory-levelI/O吞吐率.

2. Alluxio的架构

Alluxio MasterMainly responsible for managing metadata,Perform distributed storage metadata operations,Secondary master用户checkpoint日志（journal）和容错

Alluxio WorkerResponsible for storing block data,每个workerManage meta information for blocks stored by yourself,Perform low-level data-based storage operations

Clientis applied withAlluxio交互的工具

3. Alluxio的使用

3.1 Unified namespace and unified clientAPI

AlluxioProvides a unified namespace,Used as a cache for underlying file storage system data,在不同的UFSEffective management of data is achieved.Use transparent mounts,将Alluxio的命名空间和UFSConsistent between namespaces,通过Alluxio访问UFSdata and direct access in UFS的数据,结果是一致的.Alluxio提供了客户端API,可以通过Alluxio实现对不同UFS的访问

3.2 MRS AlluxioConfigure the underlying storage system

3.2.1 Root mount point type

配置HDFS作为Alluxio的底层文件系统（开启KerberosThis feature is not supported for certified secure clusters）.登录mrs manager页面,进入Alluxio的“全部配置”页面.修改参数“alluxio.master.mount.table.root.ufs”的值为“hdfs://hacluster/XXX/”.然后保存配置,Then restart the service with outdated configuration.

配置OBS作为Alluxio的底层文件系统.First configure the cluster with OBS OperateAccessdelegation of authority,用于访问obs.登录mrs manager页面,进入Alluxio的“全部配置”页面.修改参数“alluxio.master.mount.table.root.ufs”的值为“obs://<OBS_BUCKET>/<OBS_DIRECTORY>/”.然后保存配置,Then restart the service with outdated configuration.

There is only one root mount,And can only be configured before starting the service.

• Inline mount point

Inline mount points can be found at AlluxioAny directory under the root directory,After the service is started,Mount via client command,可以有多个挂载点.

MRS集群创建后,The default underlying storage address is hdfs://hacluster/,即将HDFSThe root directory maps toAlluxio.First configure the cluster with OBS OperateAccessdelegation of authority,用于访问obs.使用Alluxio客户端执行如下命令,将OBSThe directory inside the container to mount toAlluxio的/obs目录.

alluxio fs mount /obs obs://<OBS_BUCKET>/<OBS_DIRECTORY>/

3.3 Access via data appAlluxio（参考https://support.huaweicloud.com/cmpntguide-mrs/mrs_01_0760.html）

3.4 Alluxio web访问

Mrs Alluxio并没有在managerpage directlyAlluxio的web访问链接.But you can refer to open source solutions,使用http://{alluxio_master_active_ip}:19999访问web页面.

3.4.1如何确定哪个master节点是active？

登录manager页面,打开Alluxio服务进入masterInstance view or use client commandsaluxio fs leader查看.

3.4.2如何访问Alluxio web页面？

You can bind elasticity to this nodeip通过公网访问（注意：Need to be released in the security group19999端口）.或者通过创建一个windows ECS,Open the node to this onewindow ECS的网络,然后通过私有ip访问.

3.5 Alluxio元数据的备份

3.6 Alluxiocommon file operations and administrator commands

3.6.1 常用文件操作

3.6.2 管理员命令

3.6.2.1 validateEnv工具命令

Check if the local environment existsAlluxio部署的问题

alluxio validateEnv master

alluxio validateEnv worker

alluxio validateEnv all

Configuration settings integrity check

alluxio validateConf

3.6.2.2 文件系统管理功能

backup：备份Alluxio元数据,The default root mount directory/alluxio_backups路径,The storage path can also be specified.

doctor：检查master和workerConfiguration inconsistencies,Needs to be run while the service is running.

getBlockInfo：需要提供block的id,Query block information

report：生成一个AlluxioThe overall health of the cluster