Postgresql daily operation and maintenance skills, suitable for beginners

哈喽！大家好,我是【IT邦德】,江湖人称jeames007,10年DBA工作经验
一位上进心十足的【大数据领域博主】！
中国DBA联盟(ACDU)成员,目前从事DBA及程序编程
擅长主流数据Oracle、MySQL、PG 运维开发,备份恢复,安装迁移,性能优化、故障应急处理等.
如果有对【数据库】感兴趣的【小可爱】,欢迎关注【IT邦德】
️️️感谢各位大可爱小可爱！️️️

前言

PostgreSQL是一种特性非常齐全的自由软件的对象-关系型数据库管理系统,This article summarizes the daily operation and maintenance experience

---

️ 1. 何为postgresql

PostgreSQL是一种特性非常齐全的自由软件的对象-关系型数据库管理系统（ORDBMS）,是以加州大学计算机系开发的POSTGRES,4.2版本为基础的对象关系型数据库管理系统.POSTGRES的许多领先概念只是在比较迟的时候才出现在商业网站数据库中.PostgreSQL支持大部分的SQL标准并且提供了很多其他现代特性,如复杂查询、外键、触发器、视图、事务完整性、多版本并发控制等.同样,PostgreSQL也可以用许多方法扩展,例如通过增加新的数据类型、函数、操作符、聚集函数、索引方法、过程语言等.另外,因为许可证的灵活,任何人都可以以任何目的免费使用、修改和分发PostgreSQL.

️ 2. 参数文件

2.1 Blacklist parameter file

相当于mysql数据库中的mysql.user表.
This file is located in the database directory of the initial installation,例如：/var/lib/pgsql/11/data/pg_hba.conf.
视图select * from pg_hba_file_rules; 可以查询pg_hba.conf文件的内容.

cat  << EOF > /var/lib/pgsql/11/data/pg_hba.conf # TYPE DATABASE USER ADDRESS METHOD local all all trust host all all ::1/128 trust host all all 127.0.0.1/32 trust host all all 0.0.0.0/0 md5 host replication all 0.0.0.0/0 md5 EOF

2.2 运行参数文件

cat >> /postgresql/pgdata/postgresql.conf <<"EOF" listen_addresses = '*' port=5433 unix_socket_directories='/postgresql/pgdata' logging_collector = on log_directory = 'pg_log' log_filename = 'postgresql-%a.log' log_truncate_on_rotation = on #日志循环 EOF

配置示例

64G内存,16核,HUGE PAGE=on,SSD机器
port = 5432# 监听端口
max_connections=1700# 规格内存(GB)*1000*(1/4)/10+superuser_reserved_connections 
shared_buffers=16GB# IF use hugepage: 规格内存*(1/4)ELSE: min(32GB, 规格内存*(1/4))
max_prepared_transactions=1700# max_prepared_transactions=max_connections
work_mem =16MB# max(min(规格内存/4096, 64MB), 4MB)
maintenance_work_mem = 1GB# min( 8G, (主机内存*1/8)/max_parallel_maintenance_workers )
autovacuum_work_mem  = 1GB# min( 8G, (规格内存*1/8)/autovacuum_max_workers )
max_parallel_maintenance_workers = 8# min( max(2, CPU核数/2) , 16 )
max_parallel_workers_per_gather = 12# min( max(2, CPU核数-4) , 24 )
max_parallel_workers =12# min(max(2, CPU核数-4) ,32)
max_wal_size = 32GB# min(shared_buffers*2 ,用户存储空间/10)
min_wal_size  =8GB# min(shared_buffers/2 , 用户存储空间/10) 
max_sync_workers_per_subscription = 12  # min ( 32 , max(2, CPU核数-4) )
effective_cache_size  = 48GB# 规格内存*0.75 
autovacuum_max_workers = 8# max(min( 8 , CPU核数/2 ) , 5) 

2.3 动态参数文件

ostgresql.auto.conf文件
postgres=# alter system set port=5434;
postgres=# select pg_reload_conf(); --生效

️ 3、 常用psql

psql 是 PostgreSQL 中的一个命令行交互式客户端工具,它允许你交互地键入 SQL 命令,然后把它们发送给 PostgreSQL 服务器,再显示 SQL 或命令的结果.
Input is allowed to come from a file,In addition it provides some meta-commands and many similar shell features to implement scripting,As well as automating a large number of tasks.
特性：方便快捷、There are some restrictions on the use of graphical tools.
C:\Users\wangd>psql --help

3.1 远程登陆

psql -U postgres -h 192.168.66.35 -d postgres -p 15432 -E
-U为用户
-h为主机
-d为数据库
-p为端口号
-E为后台SQL显示,可以省略
If you want to log in without a password,需要设置环境变量

3.2 常用选项

postgres=# \db+

\d [ table ] 
列出数据库中的表,或（如果声明了）表 table 的列/字段．If the table name is wildcard（“*”）声明的,List all tables and table columns/字段信息．
\da 
List all available aggregates．
\dd object 
列出 pg_description A description of the declared object,An object can be a table,表中的列/字段,类型,operator or aggregate．

小技巧：Not all objects are there pg_description 里有描述．This post command is getting fast Postgres Useful for internal features．
\df 
列出函数．
\di 
Only the index is listed．
\do 
Only operators are listed．
\ds 
Only the sequence is listed．
\dS 
List system tables and indexes．
\dt 
Only non-system tables are listed．
\dT 
列出类型．
\e [ filename ] 
Edit the current query buffer or file filename 的内容．
\E [ filename ] 
Edit the current query buffer or file filename content and execute it after editing．
\f [ separator ] 
设置域分隔符．The default is a single blank．
\g [ {
     filename | |command } ] 
Buffer the current query input to the backend and（可选的）Put the output in filename Or pipe the output to a detached oneUnix shell 用以执行 command．
\h [ command ] 
given a statement SQL 命令的语法帮助．如果 command Not a definition SQL 命令（或在 psql There is no documentation in it）,或没有声明 command ,这时 psqlA list of all commands for which help is available will be listed．如果命令 command 是一个通配符（“*”）,gives all SQL 命令的语法帮助．
\H 
切换 HTML3 输出．等效于 -H 命令行选项．
\i filename 
从文件 filename Read queries into the input buffer．
\l 
List all databases on the server．
\m 
Toggles old monitor-like meter output,The table is then surrounded by border characters．这是标准 SQL 输出．缺省时,psql Only include columns/字段间的分隔符．
\o [ {
     filename | |command } ] 
Output the following query results to a file filename Or output the latter result through a pipe to a separate oneUnix shell 里执行 command．If no parameter is declared,将查询结果输出到 stdout．
\p 
Print the current query buffer．
\q 
退出 psql 程序．
\r 
重置（清空）查询缓冲区．
\s [ filename ] 
将命令行历史打印出或是存放到 filename．如果省略 filename ,Subsequent commands will not be stored in the history file．此选项只有在 psql Only valid when configured to use input lines．
\t 
Toggle output columns/字段名的信息头和行记数脚注（The default is on）．
\T table_options 
允许你在使用HTML 3.0 The declaration is placed in the table when the format is output table ... tag options in ．例如,border Will give your table a border．This must be H used with the post command．
\x 
Toggles extended line format．当打开时,Each row will print columns on the left/field names and print the columns on the right/字段值．This is useful for very long lines that cannot be output on one line．HTML Line output mode also supports this flag．
\w filename 
将当前查询缓冲区输出到文件 filename．
\z 
Generates one with the correct one ACL（赋予/禁止权限）The output list of all tables in the database．
! [ command ] 
back to an independentUnix shell或执行一个Unix 命令 command．
\? 
Get about backslashes (“”) 命令的帮助．

️ 4、 常用psql启动和关闭PG

启动和停止 PostgreSQL 数据库服务器,通常使用pg_ctl.Usually in our production environment,In the event of an unexpected downtime of the database host or after the host is stopped due to, for example, a planned hardware configuration,PostgreSQL The service will also stop,需要手动重启.因此,在生产环境中,采用编译安装 PostgreSQL 数据库后,It is recommended to configure the system postgresql.service 服务,通过 systemctl The system command sets the system to automatically start at boot.

️ 5、数据字典

5.1 查询数据库大小

SELECT d.datname as "Name",
       pg_catalog.pg_get_userbyid(d.datdba) as "Owner",
       pg_catalog.pg_encoding_to_char(d.encoding) as "Encoding",
       d.datcollate as "Collate",
       d.datctype as "Ctype",
       pg_catalog.array_to_string(d.datacl, E'\n') AS "Access privileges",
       CASE WHEN pg_catalog.has_database_privilege(d.datname, 'CONNECT')
            THEN pg_catalog.pg_size_pretty(pg_catalog.pg_database_size(d.datname))
            ELSE 'No Access'
       END as "Size",
       t.spcname as "Tablespace",
       pg_catalog.shobj_description(d.oid, 'pg_database') as "Description"
FROM pg_catalog.pg_database d
  JOIN pg_catalog.pg_tablespace t on d.dattablespace = t.oid
ORDER BY 1;

5.2 Query the first few large tables（表和索引大小）

SELECT
t.table_catalog as db,
n.nspname AS schemaname,
c.relname,
c.reltuples::numeric as rowcount,
pg_size_pretty(pg_table_size ( '"' || nspname || '"."' || relname || '"' )) AS table_size,
pg_size_pretty(pg_indexes_size ( '"' || nspname || '"."' || relname || '"' )) AS indexes_size,
pg_size_pretty (pg_total_relation_size ( '"' || nspname || '"."' || relname || '"' )) AS total_size --,pg_relation_filepath(table_name) filepath
FROM pg_class C 
LEFT JOIN pg_namespace N ON ( N.oid = C.relnamespace ) 
left join information_schema.tables t on (n.nspname= t.table_schema and c.relname=t."table_name" )
WHERE
nspname NOT IN ( 'pg_catalog', 'information_schema' ) 
AND relkind in ('r','p')  
ORDER BY
reltuples DESC 
LIMIT 20;