当前位置：网站首页>[CEPH] CEPH configuration source code analysis | common/config*

[CEPH] CEPH configuration source code analysis | common/config*

2022-06-10 18:24:00 【bandaoyu】

ceph The construction idea of configuration items of

If one class There are hundreds of member variables in , Do you feel pain when you write a header file , Also initialize them in the constructor , I can't think any more ……

Like the picture below

This is just a header file

There is also implementation

And then there's get and set

However , The above is the most stupid way ~

So here comes the question ,ceph How do developers do it ？

utilize c++ The macro , stay class Define the macro to be used before defining the member variables in the header file of , as follows

OPTION(name,ty,init) Macro can be based on ty, That is, the type of member variable , Generate member variables at compile time , The final definition of a member variable is in common/config_opts.h Next ：

Class The same is true for initializing member variables with the constructor of ：

Just change common/config_opts.h Under the OPTION Statement to add or delete member variables .

as for get and set Method , In a clever way ： Directly relative to the member variable class The offset address of .

Here is an example , Here's the picture ：

Initialize global array config_optionsp when , Every config_option Elements record md_config_t The name of a member variable in the , Type and offset , As shown in the figure above 4 Little picture . The main use is stddef.h Inside offsetof Calculate member variables in class The offset value in .

When calling get Enter the small and medium figure in the above figure 1 Function of , First find the name of the member variable config_option object , Another example is the small picture 2 call config_option Of conf_ptr（ Little picture 3）, among md_conf_off Is the member variable in md_config_t Class . This allows direct access to member variables , And more efficient , At the same time, it reduces the workload of development , The code is also more beautiful ！

Extract from ：Ceph Source code config https://blog.csdn.net/chunlu2438/article/details/100679794

Code

int main(int argc, const char **argv)
auto cct = global_init(&def_args, args,
           CEPH_ENTITY_TYPE_MON, CODE_ENVIRONMENT_DAEMON,
           flags, "mon_data");
void global_pre_init(std::vector < const char * > *alt_def_args,
       std::vector < const char* >& args,
       uint32_t module_type, code_environment_t code_env,
       int flags)
CephContext *cct = common_preinit(iparams, code_env, flags);
CephContext *cct = new CephContext(iparams.module_type, code_env, flags);
CephContext::CephContext(uint32_t module_type_,
enum code_environment_t code_env,
int init_flags_)
: nref(1),
_conf(new md_config_t(code_env == CODE_ENVIRONMENT_DAEMON)),
_log(NULL),
{
}

One , Traverse ceph_options hold option Of name Put in schema in .

Two , hold legacy_config_opts.h The variables in the （ an ） Put the name and md_config_t::name Pair generation map Put in legacy_values in .

Both ：options.cc The variables in the （ name ） Put in schema.legacy_config_opts.h The variables in the （ name ） Put in legacy_values.

ceph_options The source of this is =build_options,build_options Is to put options.cc All configurations in are added to std::vector Back to ceph_options, See the following code

md_config_t::md_config_t(bool is_daemon)
{
……

for (const auto &i : ceph_options) {
if (schema.count(i.name)) {
// We may be instantiated pre-logging so send
std::cerr << "Duplicate config key in schema: '" << i.name << "'"
<< std::endl;
assert(false);
}
schema.insert({i.name, i});
}
// Populate list of legacy_values according to the OPTION() definitions
// Note that this is just setting up our map of name->member ptr. The
// default values etc will get loaded in along with new-style data,
// as all loads write to both the values map, and the legacy
// members if present.
legacy_values = {
#define OPTION(name, type) {std::string(STRINGIFY(name)), &md_config_t::name},
#define SAFE_OPTION(name, type) OPTION(name, type)
#include "common/legacy_config_opts.h"
#undef OPTION
#undef SAFE_OPTION
};
validate_schema();
……
}

static std::vector<Option> build_options()
{
std::vector<Option> result = get_global_options();
auto ingest = [&result](std::vector<Option>&& options, const char* svc) {
for (const auto &o_in : options) {
Option o(o_in);
o.add_service(svc);
result.push_back(o);
}
};
ingest(get_rgw_options(), "rgw");
ingest(get_rbd_options(), "rbd");
ingest(get_rbd_mirror_options(), "rbd-mirror");
ingest(get_mds_options(), "mds");
ingest(get_mds_client_options(), "mds_client");
return result;
}
const std::vector<Option> ceph_options = build_options();

validate_schema();

1、 Check ……

2、 Check to make sure legacy_config_opts.h Variables defined in , stay options.cc All of them are defined .

/**
* Sanity check schema. Assert out on failures, to ensure any bad changes
* cannot possibly pass any testing and make it into a release.
*/
void md_config_t::validate_schema()
{
for (const auto &i : schema) {
const auto &opt = i.second;
for (const auto &see_also_key : opt.see_also) {
if (schema.count(see_also_key) == 0) {
std::cerr << "Non-existent see-also key '" << see_also_key
<< "' on option '" << opt.name << "'" << std::endl;
assert(false);
}
}
}
for (const auto &i : legacy_values) {
if (schema.count(i.first) == 0) {
std::cerr << "Schema is missing legacy field '" << i.first << "'"
<< std::endl;
assert(false);
}
}
}

src\common\options.cc It also defines （ Instantiation ） Some variables , Put in get_global_options、get_rgw_options、get_rbd_options、get_rbd_options…… in ：

std::vector<Option> get_global_options() {
  return std::vector<Option>({
    Option("host", Option::TYPE_STR, Option::LEVEL_BASIC)
    .set_description("local hostname")
    .set_long_description("if blank, ceph assumes the short hostname (hostname -s)")
    .add_service("common")
    .add_tag("network"),

    Option("fsid", Option::TYPE_UUID, Option::LEVEL_BASIC)
    .set_description("cluster fsid (uuid)")
    .add_service("common")
    .add_tag("service"),

    Option("public_addr", Option::TYPE_ADDR, Option::LEVEL_BASIC)
    .set_description("public-facing address to bind to")
    .add_service({"mon", "mds", "osd", "mgr"}),

    Option("public_bind_addr", Option::TYPE_ADDR, Option::LEVEL_ADVANCED)
    .set_default(entity_addr_t())
    .add_service("mon")
    .set_description(""),
……
}

std::vector<Option> get_rgw_options() {
  return std::vector<Option>({
    Option("rgw_acl_grants_max_num", Option::TYPE_INT, Option::LEVEL_ADVANCED)
    .set_default(100)
    .set_description("Max number of ACL grants in a single request"),

    Option("rgw_max_chunk_size", Option::TYPE_INT, Option::LEVEL_ADVANCED)
    .set_default(4_M)
    .set_description("Set RGW max chunk size")
    .set_long_description(
        "The chunk size is the size of RADOS I/O requests that RGW sends when accessing "
        "data objects. RGW read and write operation will never request more than this amount "
        "in a single request. This also defines the rgw object head size, as head operations "
        "need to be atomic, and anything larger than this would require more than a single "
        "operation."),

    Option("rgw_put_obj_min_window_size", Option::TYPE_INT, Option::LEVEL_ADVANCED)
    .set_default(16_M)
    .set_description("The minimum RADOS write window size (in bytes).")
    .set_long_description(
        "The window size determines the total concurrent RADOS writes of a single rgw object. "
        "When writing an object RGW will send multiple chunks to RADOS. The total size of the "
        "writes does not exceed the window size. The window size can be automatically "
        "in order to better utilize the pipe.")
    .add_see_also({"rgw_put_obj_max_window_size", "rgw_max_chunk_size"}),
……
}

Then put them all in ceph_options in

static std::vector<Option> build_options()
{
  std::vector<Option> result = get_global_options();

  auto ingest = [&result](std::vector<Option>&& options, const char* svc) {
    for (const auto &o_in : options) {
      Option o(o_in);
      o.add_service(svc);
      result.push_back(o);
    }
  };

  ingest(get_rgw_options(), "rgw");
  ingest(get_rbd_options(), "rbd");
  ingest(get_rbd_mirror_options(), "rbd-mirror");
  ingest(get_mds_options(), "mds");
  ingest(get_mds_client_options(), "mds_client");

  return result;
}

const std::vector<Option> ceph_options = build_options();

And then in md_config_t in , hold options.cc In the definition of / Instantiated option Put your name in schema： schema.insert({i.name, i});

md_config_t::md_config_t(bool is_daemon)
  : cluster(""),
  lock("md_config_t", true, false)
{
  init_subsys();

  // Load the compile-time list of Option into
  // a map so that we can resolve keys quickly.
  for (const auto &i : ceph_options) {
    if (schema.count(i.name)) {
      // We may be instantiated pre-logging so send 
      std::cerr << "Duplicate config key in schema: '" << i.name << "'"
                << std::endl;
      assert(false);
    }
    schema.insert({i.name, i});
  }
……
}

src\common\legacy_config_opts.h The macro defines its own variables ：

/* note: no header guard */
OPTION(host, OPT_STR) // "" means that ceph will use short hostname
OPTION(public_addr, OPT_ADDR)
OPTION(public_bind_addr, OPT_ADDR)
OPTION(cluster_addr, OPT_ADDR)
OPTION(public_network, OPT_STR)
OPTION(cluster_network, OPT_STR)
OPTION(lockdep, OPT_BOOL)
OPTION(lockdep_force_backtrace, OPT_BOOL) // always gather current backtrace at every lock
OPTION(run_dir, OPT_STR)       // the "/var/run/ceph" dir, created on daemon startup
OPTION(admin_socket, OPT_STR) // default changed by common_preinit()
OPTION(admin_socket_mode, OPT_STR) // permission bits to set for admin socket file, e.g., "0775", "0755"
……

next src\common\config.cc Inside the handle legacy_config_opts.h Inside, the macro defines its own variables and expands them to legacy_values Inside ：

#define _STR(x) #x
#define STRINGIFY(x) _STR(x)

md_config_t::md_config_t(bool is_daemon){
……

legacy_values = {
#define OPTION(name, type) \
    {std::string(STRINGIFY(name)), &md_config_t::name},
#define SAFE_OPTION(name, type) OPTION(name, type)
#include "common/legacy_config_opts.h"
#undef OPTION
#undef SAFE_OPTION
  };
……

}

ceph Configurable parameters in （ The level of logging , Cache settings, etc ） It's all defined in http://options.cc in , Can pass ceph daemon Command to view or modify these parameter configurations .

ceph daemon *** config show

for example ："log_file" Parameter to configure the path of the log file

Option("log_file", Option::TYPE_STR, Option::LEVEL_BASIC)
    .set_default("")
    .set_daemon_default("/var/log/ceph/$cluster-$name.log")
    .set_description("path to log file")
    .add_see_also({"log_to_stderr", "err_to_stderr", "log_to_syslog", "err_to_syslog"}),

The parameter is Option The structure exists ,Option The definition of structure is as follows

struct Option {
  enum type_t {                  //  The type of parameter value          
    TYPE_UINT,
    TYPE_INT,
    TYPE_STR,
    TYPE_FLOAT,
    TYPE_BOOL,
    TYPE_ADDR,
    TYPE_UUID,
  };
  enum level_t {
    LEVEL_BASIC,
    LEVEL_ADVANCED,
    LEVEL_DEV,
  };
  using value_t = boost::variant<boost::blank, std::string, uint64_t, int64_t, double, bool, entity_addr_t, uuid_d>;
  const std::string name;                   //  Parameter name 
  const type_t type;                        //  Parameter type 
  const level_t level;                      //  Parameter level 
  std::string desc;                         //  The meaning of this parameter 
  std::string long_desc;
  value_t value;                            //  Parameter values 
  value_t daemon_value;                     //  Yes daemon Parameter values for 
  std::vector<const char*> services;        // services Service type , such as "mds"
  std::vector<const char*> see_also;            //  This parameter corresponds to a similar parameter 
  std::vector<const char*> enum_allowed;
  bool safe;
  ...  
}

stay http://options.cc in , The defined parameters fall into several categories

std::vector<Option> get_global_options() { ... }

std::vector<Option> get_rgw_options() { ... }

static std::vector<Option> get_rbd_options() { ... } 

static std::vector<Option> get_rbd_mirror_options() { ... }

std::vector<Option> get_mds_options() { ... }

std::vector<Option> get_mds_client_options() { ... }

As you can see from the function name ：get_global_options The arguments in the function are global Type of , That is, basically all modules can use these parameters .get_rgw_options The arguments in the function are given to rgw Module USES .get_rbd_options Functions and get_rbd_mirror_options The arguments in the function are given to rbd Related modules use .get_mds_options The arguments in the function are given to mds Module USES .get_mds_client_options The arguments in the function are given to mds client Module USES .

Global variables ceph_options

ceph_options Contains all the parameters ,

const std::vector<Option> ceph_options = build_options();

stay build_options All Option Class put ceph_options This vector Inside .

static std::vector<Option> build_options()
{
  std::vector<Option> result = get_global_options();
  auto ingest = [&result](std::vector<Option>&& options, const char* svc) {
    for (auto &o : options) {
      o.add_service(svc);
      result.push_back(std::move(o));
    }
  };
  ingest(get_rgw_options(), "rgw");
  ingest(get_rbd_options(), "rbd");
  ingest(get_rbd_mirror_options(), "rbd-mirror");
  ingest(get_mds_options(), "mds");
  ingest(get_mds_client_options(), "mds_client");
  return result;
}

ceph_options = { Option{"log_file"}, Option("***")}

md_config_t

md_config_t The structure represents the current Ceph To configure ,ceph_options All the parameters in the will exist here . stay md_config_t There are special member variable definitions in .

#define OPTION_OPT_INT(name) int64_t name;
#define OPTION_OPT_LONGLONG(name) int64_t name;
#define OPTION_OPT_STR(name) std::string name;
#define OPTION_OPT_DOUBLE(name) double name;
#define OPTION_OPT_FLOAT(name) double name;
#define OPTION_OPT_BOOL(name) bool name;
#define OPTION_OPT_ADDR(name) entity_addr_t name;
#define OPTION_OPT_U32(name) uint64_t name;
#define OPTION_OPT_U64(name) uint64_t name;
#define OPTION_OPT_UUID(name) uuid_d name;
#define OPTION(name, ty) \
  public:                      \
    OPTION_##ty(name)          
#define SAFE_OPTION(name, ty) \
  protected:                        \
    OPTION_##ty(name)               
#include "common/legacy_config_opts.h"
#undef OPTION_OPT_INT
#undef OPTION_OPT_LONGLONG
#undef OPTION_OPT_STR
#undef OPTION_OPT_DOUBLE
#undef OPTION_OPT_FLOAT
#undef OPTION_OPT_BOOL
#undef OPTION_OPT_ADDR
#undef OPTION_OPT_U32
#undef OPTION_OPT_U64
#undef OPTION_OPT_UUID
#undef OPTION
#undef SAFE_OPTION

common/legacy_config_opts.h Variables stored in and common/http://options.cc One-to-one correspondence , The code is as follows

/* note: no header guard */
OPTION(host, OPT_STR) // "" means that ceph will use short hostname
OPTION(public_addr, OPT_ADDR)
OPTION(public_bind_addr, OPT_ADDR)
OPTION(cluster_addr, OPT_ADDR)
OPTION(public_network, OPT_STR)
OPTION(cluster_network, OPT_STR)
OPTION(lockdep, OPT_BOOL)
OPTION(lockdep_force_backtrace, OPT_BOOL) // always gather current backtrace at every lock
OPTION(run_dir, OPT_STR)       // the "/var/run/ceph" dir, created on daemon startup
OPTION(admin_socket, OPT_STR) // default changed by common_preinit()
OPTION(admin_socket_mode, OPT_STR) // permission bits to set for admin socket file, e.g., "0775", "0755"
......

Final md_config_t The structure is defined as follows

struct md_config_t {
   ...
   typedef std::multimap <std::string, md_config_obs_t*> obs_map_t;
   std::map<std::string, md_config_t::member_ptr_t> legacy_values;
   std::map<std::string, const Option&> schema;
   std::map<std::string, Option::value_t> values;
private:
  ConfFile cf;
public:
  std::deque<std::string> parse_errors;
private:
  bool safe_to_start_threads = false;
  obs_map_t observers;
  changed_set_t changed;
public:
  ceph::logging::SubsystemMap subsys;
  EntityName name;
  string data_dir_option;  ///< data_dir config option, if any
  string cluster;           // cluster = "ceph"

//  The following members are from common/legacy_config_opts.h in 
public：
   std::string host;
   entity_addr_t public_addr;
   entity_addr_t public_bind_addr;
   entity_addr_t cluster_addr;
   std::string public_network;
   std::string cluster_network;
   bool lockdep;
   bool lockdep_force_backtrace;
   std::string run_dir;
   std::string admin_socket;
   std::string admin_socket_mode;
   std::string log_file; 
   ...
protected:
   std::string ms_type; 
   std::string plugin_dir;
   ...

}

among schema preservation ceph_options All in Option Parameters , namely

schema = { <"host", Options("host")>, ..., <"log_file", Option("log_file")>, ...}

legacy_values take ceph_options Medium Option Of name With the corresponding md_config_t The member pointer in is used as key-value preservation , namely

legacy_values = {<"host", &md_config_t::host>, ... , <"log_file", &md_config_t::log_file>, ...}

values take ceph_options Medium Option Of name and value As key-value preservation .

libcephfs in ：values = {<"host", "">, ..., <"log_file", "">, ...}

ceph-mds in ：values = {<"host", "">, ..., <"log_file", "/var/log/ceph/$cluster-$name.log">, ...}

subsys Used to save log entries , namely

subsys = ceph::logging::SubsystemMap( m_subsys[ceph_subsys_ == 0] = Subsystem<name = "none", log_level = 0, gather_level = 5>, ... , m_subsys[ceph_subsys_crush == 3] = Subsystem<name = "crush", log_level = 1, gather_level = 1>, m_subsys[ceph_subsys_mds == 4] = Subsystem<name = "mds", log_level = 1, gather_level = 5>, ... }.

g_ceph_context and g_conf Global variables

The process of a module will have multiple threads , such as ceph-mds, Some content in the process needs to be accessible by all threads in the whole process , Such as parameter configuration and context content , So there are two global variables g_conf and g_ceph_context, stay src/global/http://global_context.cc Is defined as follows

CephContext *g_ceph_context = NULL;
md_config_t *g_conf = NULL;

md_config_t The instance object pointer is used as CephContext Members of ,

class CephContext {
...
public:
  md_config_t *_conf;
...
}

When each module starts , You need to instantiate CephContext, There is no need to instantiate separately md_config_t.CephContext The instantiation process is as follows ：

stay md_config_t The input arguments to the constructor of are is_daemon, It is used to judge whether the module is a separate daemon thread , Or other threads call the external interface , Do not thread separately . The constructor is as follows

md_config_t::md_config_t(bool is_daemon){ ... }

md_config_t The instantiation of is as follows

 _conf(new md_config_t(code_env == CODE_ENVIRONMENT_DAEMON))

Judgment code_env == CODE_ENVIRONMENT_DAEMON, In each module main In function code_env Input . namely http://ceph_mds.cc,http://ceph_fuse.cc and http://ceph_mon.cc Isochronous md_config_t in is_daemon All are true,libcephfs Of md_config_t in is_daemon yes false.

Parameter configuration mode

There are two ways to configure parameters ： Permanent and temporary . Permanent way , It is in the configuration file ( Such as ceph.conf) Add the configuration of this parameter in , After restarting the process , The parameter takes effect . Temporary way , It is through ceph daemon Command to set parameters in memory .

modify ceph.conf How to take effect after setting the parameter , For example ceph.conf Configuration of the

admin_socket = $rundir/$cluster-$id.$pid.asok

Look directly at the code , from http://ceph_mon.cc In the middle of the movie

auto cct = global_init(&def_args, args, CEPH_ENTITY_TYPE_MON, CODE_ENVIRONMENT_DAEMON,
			 flags, "mon_data")

global_init

boost::intrusive_ptr<CephContext> global_init(std::vector < const char * > *alt_def_args, std::vector < const char* >& args,
	    uint32_t module_type, code_environment_t code_env, int flags,
	    const char *data_dir_option, bool run_pre_init)
{ // module_type = CEPH_ENTITY_TYPE_MON, code_env = CODE_ENVIRONMENT_DAEMON, flags = 0
  // *data_dir_option = "mon_data", run_pre_init = true
  static bool first_run = true;
  if (run_pre_init) {
    global_pre_init(alt_def_args, args, module_type, code_env, flags);
  } else 
  ...
}

global_pre_init It's the key function ：

void global_pre_init(std::vector < const char * > *alt_def_args, std::vector < const char* >& args,
		     uint32_t module_type, code_environment_t code_env, int flags)
{ // module_type = CEPH_ENTITY_TYPE_MON, code_env = CODE_ENVIRONMENT_DAEMON, flags = 0
  std::string conf_file_list;            // ceph.conf The path of 
  std::string cluster = "";
  CephInitParameters iparams = ceph_argparse_early_args(args, module_type, &cluster, &conf_file_list);
  // common_preinit Functions are mainly used to instantiate CephContext
  CephContext *cct = common_preinit(iparams, code_env, flags);
  cct->_conf->cluster = cluster;         // cct->_conf->cluster = "ceph"
  global_init_set_globals(cct);
  md_config_t *conf = cct->_conf;
  if (alt_def_args)
    conf->parse_argv(*alt_def_args);  // alternative default args
  int ret = conf->parse_config_files(c_str_or_null(conf_file_list), &cerr, flags);
  ...
}

call ceph_argparse_early_args To resolve the parameters set when the process starts , such as ceph-mon The parameters set during startup are as follows

/usr/bin/ceph-mon -f --cluster ceph --id node1 --setuser ceph --setgroup ceph

ceph_argparse_early_args Function as follows ,

CephInitParameters ceph_argparse_early_args (std::vector<const char*>& args, uint32_t module_type,
	   std::string *cluster, std::string *conf_file_list)
{ //module_type = CEPH_ENTITY_TYPE_MON, *cluster = "", *conf_file_list = ""
  CephInitParameters iparams(module_type);
  // iparams The content inside is class CephInitParameters { module_type = CEPH_ENTITY_TYPE_MON,
  //   name: struct EntityName { type = CEPH_ENTITY_TYPE_MON, id = "admin", type_id = "mon.admin" }}
  std::string val;
  vector<const char *> orig_args = args;
  for (std::vector<const char*>::iterator i = args.begin(); i != args.end(); ) {
    ...
   //  analysis --conf, But to perform ceph-mon On command , There is no designation of --conf, therefore val Is an empty string .
    else if (ceph_argparse_witharg(args, i, &val, "--conf", "-c", (char*)NULL)) {
      *conf_file_list = val;       
    }
    else if (ceph_argparse_witharg(args, i, &val, "--cluster", (char*)NULL)) {
      *cluster = val;         // *cluster = ceph
    }
    else if ((module_type != CEPH_ENTITY_TYPE_CLIENT) &&
	     (ceph_argparse_witharg(args, i, &val, "-i", (char*)NULL))) {
      iparams.name.set_id(val); 
    }
    else if (ceph_argparse_witharg(args, i, &val, "--id", "--user", (char*)NULL)) {
      iparams.name.set_id(val); // iparams.name in {_id = "node1", type_id = "mon.node1"}
    }
    ...
  return iparams;
}

common_preinit

CephContext *common_preinit(const CephInitParameters &iparams, enum code_environment_t code_env, int flags)
{ // code_env = CODE_ENVIRONMENT_DAEMON, flags = 0
  //  example CephContext
  CephContext *cct = new CephContext(iparams.module_type, code_env, flags);
  md_config_t *conf = cct->_conf;
  conf->name = iparams.name;
  ...
  return cct;
}

The most important thing is parse_config_files

int md_config_t::parse_config_files(const char *conf_files, std::ostream *warnings, int flags)
{ // conf_files = NULL
  ...
  if (!conf_files) {
    const char *c = getenv("CEPH_CONF");       // c = NULL
    ...
    else {
      ...
      // CEPH_CONF_FILE_DEFAULT = "$data_dir/config, /etc/ceph/$cluster.conf, ~/.ceph/$cluster.conf, $cluster.conf"
      conf_files = CEPH_CONF_FILE_DEFAULT; // CEPH_CONF_FILE_DEFAULT The path to the configuration file 
    }
  }
  std::list<std::string> cfl;
  get_str_list(conf_files, cfl);     //  take conf_files The string cut in is placed in cfl in 
  auto p = cfl.begin();
  while (p != cfl.end()) { ... }
  return parse_config_files_impl(cfl, warnings);
}

md_config_t::parse_config_files_impl

int md_config_t::parse_config_files_impl(const std::list<std::string> &conf_files,std::ostream *warnings)
{
  list<string>::const_iterator c;
  //  Traverse conf_files, Replace "$" String after , For example, will $cluster Replace with ceph
  for (c = conf_files.begin(); c != conf_files.end(); ++c) {
    cf.clear();
    string fn = *c;
    //  Replace "$" String after 
    expand_meta(fn, warnings);
    //  take ceph.conf( Such as /etc/ceph/ceph.conf) The contents of are read and saved in cf in 
    int ret = cf.parse_file(fn.c_str(), &parse_errors, warnings);
    if (ret == 0) break;
  }

  std::vector <std::string> my_sections;
  //  Get the part to parse , such as mon：<"mon.node1", "mon", global">, client: <"client.admin", "admin", "global">
  //  namely ceph.conf in my_sections The corresponding part will be parsed 
  _get_my_sections(my_sections);
  for (const auto &i : schema) { ... }
  // Traverse subsys,  To configure debug_** And so on. 
  for (size_t o = 0; o < subsys.get_num(); o++) { ... }
  ...
  return 0;
}

md_config_t::expand_meta Function to replace "$" String after , such as admin_socket = $rundir/$cluster-$id.$pid.asok, call expand_meta After the function admin_socket = /var/run/ceph/ceph-mon.node1.2093242.asok

Extract from ：ceph： Configurable parameters https://zhuanlan.zhihu.com/p/110079635

1 Introduce

ceph The configuration file code inside is quite tedious , Analyzing the configuration code clearly is of great benefit to troubleshooting problems in the future .

2 Start analysis

2.1 entrance

Whether it's osd、mgr still mon And other programs need to load the configuration , There is a unified entrance , for example ceph-osd.cc stay main Call in function global_init--->global_pre_init Function to load the configuration .

2.2 env_to_vec analysis

ceph_mon.cc

int main(int argc, const char **argv)
{
int err;
bool mkfs = false;
bool compact = false;
bool force_sync = false;
bool yes_really = false;
std::string osdmapfn, inject_monmap, extract_monmap;
vector<const char*> args;
argv_to_vec(argc, argv, args);
env_to_vec(args);
……
}

split_dashdash hold args according to "--" Split into two paragraphs （ Be careful ：-- differ --param）, The preceding paragraph is of type vector Of options, The latter segment is of type vector Of arguments.

（/opt/h3c/bin/ceph-mon --cluster ceph --mkfs -i rdma58 --keyring）

void env_to_vec(std::vector<const char*>& args, const char *name)
{
  if (!name)
    name = "CEPH_ARGS";
  bool dashdash = false;
  std::vector<const char*> options;
  std::vector<const char*> arguments;
  if (split_dashdash(args, options, arguments))
    dashdash = true;
  std::vector<const char*> env_options;
  std::vector<const char*> env_arguments;
  std::vector<const char*> env;
……
g_str_vec_lock.lock();
if (g_str_vec.empty()) {
char *p = getenv(name);
if (!p) {
g_str_vec_lock.unlock();
return;
}
get_str_vec(p, " ", g_str_vec);
}
}

    get_str_vec. Take out the value of the environment variable ： The default environment variable is "CEPH_ARGS", The name can be specified through parameters . With spaces “ ” interval , Split the environment variables into strings to g_str_vec in .
    g_str_vec Move to env Of vector<const char*> in , Same as args The same with “--” To break up .
     Aggregate separately env and args Of options and arguments To args in , Or to “--” Interval , Whether it's options still arguments,env The parameters of are all in the front .

2.3 ceph_argparse_early_args

void global_pre_init(std::vector < const char * > *alt_def_args,
       std::vector < const char* >& args,
       uint32_t module_type, code_environment_t code_env,
       int flags)
{
std::string conf_file_list;
std::string cluster = "";
CephInitParameters iparams = ceph_argparse_early_args(args, module_type,
                           &cluster, &conf_file_list);
CephContext *cct = common_preinit(iparams, code_env, flags);
cct->_conf->cluster = cluster;
……
}

     analysis options To get the corresponding value ：
         a) -v Exhibition version
         b) -c The configuration file
         c) --cluster cluster
         d) -i Not client The entity's id （client Procedure passed -i Appoint id It will be invalid ）
         e) --id / --user Set the name of the entity id, Will be covered -i The option to
         f) --name/-n Resolve the setting entity by name id, Will be covered -i as well as --id/--user
         g）--show_args Display parameters , But not exit

3 Configure class analysis

3.1 ConfigProxy

     Configured proxy class , It's actually encapsulated :
         a) ConfigValues values;
         b) ObserverMgr<md_config_obs_t> obs_mgr;
         c) md_config_t config;

3.2 md_config_t

It mainly contains the metadata of the configuration , The main members are ：
1) std::map<std::string, const Option&> schema;

        a)  Constructor through global variables ceph_options=build_options() structure , You can see many default values in this function

2) std::map<std::string, member_ptr_t> legacy_values;

3.3 Option

leading member ：

name:
type : Reference resources type_t {TYPE_UINT, TYPE_INT...}
level: Reference resources level_t {basic, advanced, dev}
3.1 basic: for users, configures some externally visible functional aspect
3.2 advanced: for users, configures some internal behaviour
3.3 Dev: not for users. May be dangerous, may not be documented.
1. value_t value; value_t daemon_value;

3.4 ConfigValues

leading member ：

using values_t = std::map<std::string, map<int32_t,Option::value_t>>; // <name, <level, value_t>
values_t values;

3.5 Class relation

adopt ConfigProxy Provide external interface ：

1、 If it is set The class interface ： Through the first md_config_t obtain option, adopt option obtain name、 type 、 The default value is 、 Verification and other information , analysis value, Finally set to ConfigValues in .

2、 If it is get The class interface ：

    2.1、 The process also starts from md_config_t To ConfigValues,  If from ConfigValues Can't get , Then get Option The default value of 

    2.2、md_config_t::expand_meta  analysis ${var} To the actual value

author ： Trot 001
link ：https://www.jianshu.com/p/fbfacbff1edf

Read and parse configuration files ceph.conf Code for

ceph_mon.cc
int main(int argc, const char **argv)
--->auto cct = global_init(&def_args, args,……）
------>global_pre_init(alt_def_args, args, module_type, code_env, flags);
--------->ret = conf->parse_config_files(c_str_or_null(conf_file_list),
------------>parse_config_files_impl(cfl, warnings);
---------------->ret = _get_val_from_conf_file(my_sections, opt.name, val, false);
---------------->r = set_val_impl(val, opt, &error_message);
-------------------->values[opt.name] = new_value;
auto legacy_ptr_iter = legacy_values.find(std::string(opt.name));
if (legacy_ptr_iter != legacy_values.end()) {
update_legacy_val(opt, legacy_ptr_iter->second);
}

conf_file_list Passed in by a command --conf Or for null, by null Then read the environment variable CEPH_CONF Define... Under the path ceph.conf：const char *c = getenv("CEPH_CONF");

int md_config_t::parse_config_files(const char *conf_files,
                    std::ostream *warnings,
                    int flags)
{

  if (!conf_files) {
    const char *c = getenv("CEPH_CONF");
    if (c) {
      conf_files = c;
    }
    else {
      if (flags & CINIT_FLAG_NO_DEFAULT_CONFIG_FILE)
    return 0;

// CEPH_CONF_FILE_DEFAULT = "$data_dir/config, /etc/ceph/$cluster.conf, ~/.ceph/$cluster.conf, $cluster.conf"
      conf_files = CEPH_CONF_FILE_DEFAULT; // CEPH_CONF_FILE_DEFAULT The path to the configuration file 

    }
  }



  std::list<std::string> cfl;
  get_str_list(conf_files, cfl);

CEPH_CONF We can use ; or , Split multiple profiles ,get_str_list Read processing puts multiple configuration files into litst cfl in ：

std::list<std::string> cfl;
get_str_list(conf_files, cfl);

cfl It 's in it Configuration file path defined with variables Translate into the real path ？

auto p = cfl.begin();
while (p != cfl.end()) {
// expand $data_dir?
string &s = *p;
if (s.find("$data_dir") != string::npos) {
if (data_dir_option.length()) {
   list<const Option *> stack;
   expand_meta(s, NULL, stack, warnings);
   p++;
} else {
   cfl.erase(p++); // ignore this item
}
} else {
++p;
}
}

analysis cfl The configuration file installed in the

return parse_config_files_impl(cfl, warnings);

md_config_t::parse_config_files_impl

int md_config_t::parse_config_files_impl(const std::list<std::string> &conf_files,std::ostream *warnings)
{
  list<string>::const_iterator c;
  //  Traverse conf_files, Replace "$" String after , For example, will $cluster Replace with ceph
  for (c = conf_files.begin(); c != conf_files.end(); ++c) {
    cf.clear();
    string fn = *c;
    //  Replace "$" String after 
    expand_meta(fn, warnings);
    //  take ceph.conf( Such as /etc/ceph/ceph.conf) The contents of are read and saved in cf in 
    int ret = cf.parse_file(fn.c_str(), &parse_errors, warnings);
    if (ret == 0) break;
  }

  std::vector <std::string> my_sections;
  //  Get the part to parse , such as mon：<"mon.node1", "mon", global">, client: <"client.admin", "admin", "global">
  //  namely ceph.conf in my_sections The corresponding part will be parsed 
  _get_my_sections(my_sections);
  for (const auto &i : schema) { ... }
  // Traverse subsys,  To configure debug_** And so on. 
  for (size_t o = 0; o < subsys.get_num(); o++) { ... }
  ...
  return 0;
}

原网站

版权声明
本文为[bandaoyu]所创，转载请带上原文链接，感谢
https://yzsam.com/2022/161/202206101735468749.html