## 数据库mysql ### 1.数据库分类 关系型数据库（sql）：mysql Oracle sql-Server Mariadb 非关系数据库(NoSql)：Redis MongoDB #### (1)数据库访问技术 项目后端 如何 访问数据库 java： JDBC 实战项目后端源码中添加数据库的相关信息：数据库ip地址 数据库端口 数据库用户名 数据库密码 数据库名 php： ODBC 实在项目后端源码中添加数据库的相关信息： 数据库IP地址 数据库端口 数据库用户名 数据库密码 数据库库名 python： settings.py 实在项目后端源码中添加数据库的相关信息： 数据库IP地址 数据库端口 数据库用户名 数据库密码 数据库库名 产品上线 产品发布 app 网站 需要提供数据库集群环境；[数据库IP地址 数据库端口 数据库用户名 数据库密码 数据库库名] ---> 后端 写入到自己的项目源码中 --> 构建 -->运维工程师 --> 产品上线 #### (2)yum安装数据库 Mysql版本： ``` 最新版本：8 旧版本/前一个版本：5.7 ``` 安装： 在BASE和EPEL仓库中没有Mysql的RPM包，但是有Mariadb的RPM包 1.准备Mysql的yum仓库 ``` [root@mysql-rpm ~]# yum -y install https://dev.mysql.com/get/mysql80-community-release-el7-11.noarch.rpm ``` 2.修改安装版本,启用5.7版本 ``` yum -y install yum-utils yum-config-manager --disable mysql80-community yum-config-manager --enable mysql57-community ``` 3.安装 ``` yum -y install mysql mysql-server ``` 客户端：mysql-community-client 服务器端：mysql-community-server 初始化： ``` [root@mysql-rpm ~]# systemctl start mysqld [root@mysql-rpm ~]# systemctl enable mysqld ``` 修改密码：mariadb针对root用户没有初始密码，但是mysql针对root用户有初始密码，针对mysql在使用之前先修改初始密码 ``` [root@mysql-rpm ~]# mysqladmin -uroot -p'uEe0vMb#kRHY' password 'QianFeng@123' [root@mysql-rpm ~]# mysql -uroot -pQianFeng@123 ``` #### (3)源码安装mysql 编译安装数据库 1.准备编译安装的环境 2.获取编译安装包 3.采用cmake的方式配置（指定安装目录 存放数据的目录 指定了端口号 指定了字符集等） 4.make进行编译 5.make install 进行安装 初始化： ``` 1.准备用户和组 2.创建了所需要的目录 （在指定的目录下创建） 3.修改安装目录所有者和所属组 4.进行指定用户指定安装目录指定数据存放目录的初始化操作，这一步可以获取到对应的初始密码 5.利用mysqld_safe的命令指定用户后台运行了mysql 6.利用mysqladmin的命令修改了数据库的初始密码 7.登录测试 mysql的命令指定用户和密码 ``` 注意：root用在数据库中是数据库的管理员命令，注意密码的强度 数字 大小写 特殊字符都要有 扩展： ``` 编译安装的数据库，每次执行命令都需进入到对应目录下，要么使用绝对路径，直接执行对应的命令 默认情况下编译安装的数据库不能使用systemctl 来管理mysql的服务，如果要用，需要自行配置 [root@xingdian ~]# cp /usr/local/mysql/support-files/mysql.server /etc/init.d/mysqld [root@xingdian ~]# chkconfig mysqld on ``` #### (4)mysql配置、日志文件 /etc/my.cnf：默认的数据库mysql的主配置文件 /var/lib/mysql/：用来存放数据的目录 如果在数据库mysql没有运行的情况下，删除/var/lib/mysql这个目录下的内容，相当于是重置（恢复出厂设置）当在次启动的时候，数据全部被删除了，得到了新的初始密码 /var/log/mysqld.log： mysql的错误日志文件 编译安装：指定了安装目录和数据存放目录 /usr/local/mysql /usr/local/mysql/data ``` 如果在数据库mysql没有运行的情况下，删除了/usr/local/mysql 安装目录，相当于卸载了mysql 如果在数据库mysql没有运行的情况下，删除了/usr/local/mysql/data 数据存放目录，相当于是重置了数据库 ``` ### 2、数据库操作 create database 库名; create database 库名 default character set utf8; 指定默认字符集 drop database 库名; 删除库 use 库名; 切换库 show databases; 查看库 select database(); 查看当前库 select user(); 查看登录的用户 创建表名 表名 字段 （数据库类型 【约束条件】）【存储引擎 字符集】 create table t1(name varchar(20),id int(4),sex enum('m','f'),class varchar(50)); 整数：int 例：年龄 游戏级别 游戏经验 字符串：varchar char 例：职位 部门 姓名 枚举：enum 例：性别（男女） 浮点数 ：float 例：身高 体重 薪水 时间和日期类型：date/ time/ datetime/ TIMESTAMP/ 例：入学时间 毕业时间 插入：insert into 表名 values(字段和值); mysql>insert into student1 values (1,'xingdian','m',33),(2,'alice','m',20),(3,'jack','m',40); //顺序插入 mysql> insert into student1(name,age) values ('zhuzhu',10),('gougou',20); //只向指定的字段插入值 查询：select 字段 from 表名 例：//select id,name,sex,age from fxs; //select * from fxs; /查表里所有数据 查看表：show tables 查看表结构：desc 表名 show create table 表名\G; show create table 表名 DESCRIBE 表名; select now();查看当前时间 修改表 alter table 旧表名 rename 新表名 /修改表名字 alter table 表名 add 字段名 数据类型[完整性约束条件..] /增加字段 **例：mysql> alter table student10 add stu_num int not null after name; //添加name字段之后** **mysql> alter table student10 add sex enum('male','female') default 'male' first; //添加到最前面** alter table 表名 drop 字段名 //删除字段 alter table 表名 modify 字段名 新数据类型 //修改字段类型（修改表结构） alter table 表名 modify 字段名 数据类型 [新约束条件] //增加约束 alter table 表名 primary key(数据类型)；//增加主键 alter table 表名 modify 带有主键的字段名 数据类型 auto_increment; //修改主键和自增(自增的值为空自动增加) alter table 表名 modify 字段名 数据类型 约束条件 //删除自增 alter table fxs drop primary key; //删除主键(如果该字段设置自增要优先删除自增) create table 新表名 select * from 旧表名 //复制表（内容也过来） create table 新表名 select * from 旧表名 were 1=2; //复制表结构 drop table 表名 //删除表 update 表名 set 列名=值where 条件 //修改数据表中字段的值 **例：mysql> update student set name='123' where id=1;** delete from 表名 where id=1 //删除某一行 例：mysql> delete from type where id=1; #### (1)表约束条件 保证数据的完整性和一致性 PRIMARY KEY (PK) 标识该字段为该表的主键，可以唯一的标识记录，不可以为空 UNIQUE + NOT NULL FOREIGN KEY (FK) 标识该字段为该表的外键，实现表与表（父表主键/子表1外键/子表2外键）之间的关联 NOT NULL 标识该字段不能为空 UNIQUE KEY (UK) 标识该字段的值是唯一的，可以为空，一个表中可以有多个UNIQUE KEY AUTO_INCREMENT 标识该字段的值自动增长（整数类型，而且为主键） DEFAULT 为该字段设置默认值 注意： 是否允许为空，默认NULL，可设置NOT NULL，字段不允许为空，必须赋值 字段是否有默认值，缺省的默认值是NULL，如果插入记录时不给字段赋值，此字段使用默认值 例： **MySQL [(none)]> sex enum('male','female') not null default 'male'** **MySQL [(none)]> age int unsigned NOT NULL default 20 必须为正值（无符号） 不允许为空 默认是20** 是否是key 主键 primary key 外键 forengn key #### (2)数据类型 整数类型 int 作用：用于存储用户的年龄、游戏的Level、经验值等 浮点数类型 float double 作用：用于存储用户的身高、体重、薪水等 //float(5,3) 5宽度 3精度 字符串类型 varchar 作用：用于姓名 长度要求较为高的 枚举类型 enum 时间和日期类型 date time datetime 作用：用于存储用户的注册时间，文章的发布时间，文章的更新时间，员工的入职时间等 ### 3、日志管理 #### (1)日志分类 错误日志 ：启动，停止，关闭失败报错。rpm安装日志位置 /var/log/mysqld.log //在etc数据库的配置文件中 二进制日志：实现备份，增量备份。只记录改变数据，除了select都记 通用查询日志：所有的查询都记下来 中继日志：读取主服务器的binlog，在本地回放。保持一致 slow log：慢查询日志，指导调优，定义某一个查询语句，定义超时时间，通过日志提供调优建议给开发人员 Error_log log-error=/var/log/mysqld.log //默认在/etc/my.cnf倒数第二行 Binary log //binlog日志 ```text log-bin=/var/log/mysql-bin/slave2 server-id=2 root@slave2 ~]# mkdir /var/log/mysql-bin [root@slave2 ~]# chown mysql.mysql /var/log/mysql-bin/ [root@slave2 ~]# systemctl restart mysqld ``` 查看binlog日志 ```shel mysqlbinlog slave2-bin.000001 -v --base64-output=decode-rows ``` 时间点 240507 20:25:38 位置点 # at 4 ```shel 注： 1. 重启mysqld 会截断 2. flush logs 会截断 3. reset master 删除所有binlog rm -rf /var/lib/mysql/*.000001 4. 删除部分 PURGE BINARY LOGS TO 'mysql-bin.010'; PURGE BINARY LOGS BEFORE '2019-04-02 22:46:26'; ``` #### (2)慢查询 /etc/my.cnf ```shell slow_query_log=1 slow_query_log_file=/var/log/mysql-slow/slow.log long_query_time=3 设置慢查询超时时间 单位是：3秒 ``` 创建对应目录： ```shell [root]# mkdir /var/log/mysql-slow [root]# chown mysql.mysql mysql-slow ``` 重启服务： ```shell [root]# systemctl restart mysqld ``` 验证： ```shell [root]# mysql -uroot -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.7.39-log MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> select sleep(6); +----------+ | sleep(6) | +----------+ | 0 | +----------+ 1 row in set (6.00 sec) mysql> exit Bye [root]# cat /var/log/mysql-slow/slow.log /usr/sbin/mysqld, Version: 5.7.39-log (MySQL Community Server (GPL)). started with: Tcp port: 0 Unix socket: /var/lib/mysql/mysql.sock Time Id Command Argument # Time: 2022-09-25T06:58:05.496205Z # User@Host: root[root] @ localhost [] Id: 2 # Query_time: 6.007094 Lock_time: 0.000000 Rows_sent: 1 Rows_examined: 0 SET timestamp=1664089085; select sleep(6); ``` ### 4、权限管理 Global level：系统级，所有库，所有表的权限 Database level：某个数据库中的所有表的权限 Table level：库中的某个表的权限 select * from mysql.user\G //用户的授权信息 全局权限 **字段介绍：** ```shell 用户字段：root 权限字段：Select_priv 安全字段：*B1DD4ADE47888D9AEC4D705C85230F1B52D2A817 Select_priv：查询权限 Insert_priv：插入权限 Update_priv：更新权限 Delete_priv：删除权限 ...... ``` select * from mysql.db\G; //针对库的权限 select * from mysql.tables_priv\G; //针对表的权限 #### (1)用户管理 1.登录和退出 ```shell [root@xingdian ~]# mysql -h 192.168.18.160 -P 30042 -u root -pmysql -e "show databases;" [root@xingdian ~]# mysql -h 192.168.18.160 -P 30042 -u root -pmysql mysql -e "show tables;" -h 指定主机名 【默认为localhost】 -P MySQL服务器端口 【默认3306】 -u 指定用户名 【默认root】 -p 指定登录密码 【默认为空密码】 此处mysql为指定登录的数据库 -e 接SQL语句 (在脚本中使用) ``` 2.创建用户 ```shell grant all on *.* to 'diange'@'localhost' identified by 'QianFeng@123'; ``` 注意： 该方式采用授权的方式 ALL 所有权限 select 单独某一个权限（多个权限用逗号隔开） ```shell mysql> grant select on *.* to 'dianye'@'localhost' identified by 'QianFeng@123'; Query OK, 0 rows affected, 1 warning (0.00 sec) *.* 所有的库所有的表 也可以单独某一个库某一个表 xingdian@localhost 用户有则授权无则创建 localhost % 10.19.40.% 10.19.40.11 ``` 3.删除用户 ```shell MySQL [(none)]> Drop user xingdian@'%'; ``` 4.修改密码 ```shell mysqladmin -uroot -p'123' password 'new_password' //123为旧密码 ``` ### 5、数据库备份与恢复 #### (1)tar备份 注意：备份期间，服务不可用 备份过程：完全物理备份 停止数据库 ```shell [root@xingdian ~]# systemctl stop mysqld ``` tar备份数据 ```shell [root@xingdian ~]# mkdir /backup [root@xingdian ~]# cd /var/lib/mysql [root@xingdian ~]# tar -zcvf /backup/`date +%F`-mysql-all.tar.gz ./* ``` 启动数据库（备份完成后启动数据库，继续为其他服务提供服务） ```shell [root@xingdian ~]# systemctl start mysqld ``` 恢复过程：模拟数据丢失，恢复数据 停止数据库 ```shell [root@xingdian ~]# systemctl stop mysqld ``` 清理环境 ```shell [root@xingdian ~]# rm -rf /var/lib/mysql/* ``` 导入备份数据 ```shell [root@xingdian ~]# tar -xvf /backup/2019-08-20-mysql-all.tar.gz -C /usr/lib/mysql [root@xingdian ~]# chown mysql.mysql /var/lib/mysql/* -R ``` 启动数据库（恢复后验证数据是否恢复成功） ```shell [root@xingdian ~]# systemctl start mysqld ``` #### (2)xtrabackup备份 ##### 1.完整备份 完整备份 创建备份目录： ```shell [root@xingdian ~]# mkdir -p /xtrabackup/full/ ``` 备份： ```shell [root@xingdian ~]# innobackupex --user=root --password='QianFeng@123' /xtrabackup/full/ ``` 查看备份数据： ```shell [root@xingdian ~]# ls /xtrabackup/full/ 2022-09-25_19-40-47 ``` 模拟数据丢失数据恢复：（以下操作模拟数据丢失） 丢失前数据库中的数据： ```shell [root@xingdian ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 7 Server version: 5.7.39 MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | t1 | +--------------------+ 5 rows in set (0.00 sec) ``` 数据丢失： ```shell [root@xingdian ~]# systemctl stop mysqld [root@xingdian ~]# rm -rf /var/lib/mysql/* [root@xingdian ~]# rm -rf /var/log/mysqld.log [root@xingdian ~]# rm -rf /var/log/mysql-slow/slow.log (有则删除，无则不需要操作) ``` 恢复前的验证： ```shell [root@xingdian ~]# innobackupex --apply-log /xtrabackup/full/2022-09-25_19-40-47/ ``` 恢复之前需要确认配置文件内有数据库目录指定，不然xtrabackup不知道恢复到哪里 ```shell [root@xingdian ~]# cat /etc/my.cnf datadir=/var/lib/mysql ``` 恢复数据： ```shell [root@xingdian ~]# innobackupex --copy-back /xtrabackup/full/2022-09-25_19-40-47/ ``` 修改权限： ```shell [root@xingdian ~]# chown mysql.mysql /var/lib/mysql -R ``` 启动服务： ```shell [root@xingdian ~]# systemctl start mysqld ``` 验证： ```shell [root@xingdian ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.7.39 MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | t1 | +--------------------+ 5 rows in set (0.00 sec) ``` ##### 2.增量备份 增量备份 原理：每次备份上一次备份到现在产生的新数据 注意：在进行增量备份前先进行完整备份 案例：周一进行全备，周二到周天进行增量备份 完整备份：（周一） ```shell [root@xingdian ~]# innobackupex --user=root --password='QianFeng@123' /xtrabackup/full ``` 创建增量备份存放数据目录： ```shell [root@xingdian ~]# mkdir /xtrabackup/zeng -p ``` 模拟数据增加（略） 第一次增量备份：（周二） ```shell [root@xingdian ~]# innobackupex --user=root --password='QianFeng@123' --incremental /xtrabackup/zeng/ --incremental-basedir=/xtrabackup/full/2022-09-25_19-40-47/ 第一次增量备份的数据： [root@xingdian ~]# ls /xtrabackup/zeng/ 2022-09-25_19-56-00 ``` 模拟数据增加（略） 第二次增量备份：（周三） ```shell [root@xingdian ~]# innobackupex --user=root --password='QianFeng@123' --incremental /xtrabackup/zeng/ --incremental-basedir=/xtrabackup/zeng/2022-09-25_19-56-00/ 第二次增量备份的数据： [root@xingdian ~]# ls /xtrabackup/zeng/ 2022-09-25_19-56-00 2022-09-25_19-58-12 ``` 后面的增量备份重复上面的操作（略） 增量备份数据恢复流程：（需要模拟数据的丢失） 停止数据库： ```shell [root@xingdian ~]# systemctl stop mysqld ``` 删除数据： ```shell [root@xingdian ~]# rm -rf /var/lib/mysql/* [root@xingdian ~]# rm -rf /var/log/mysqld.log 其他数据根据实际情况删除 ``` 依次重演回滚： ```shell 全备回滚： [root@xingdian ~]# innobackupex --apply-log --redo-only /xtrabackup/full/2022-09-25_19-40-47/ 第一次增量回滚： [root@xingdian ~]# innobackupex --apply-log --redo-only /xtrabackup/full/2022-09-25_19-40-47/ --incremental-dir=/xtrabackup/zeng/2022-09-25_19-56-00/ 第二次增量回滚： [root@xingdian ~]# innobackupex --apply-log --redo-only /xtrabackup/full/2022-09-25_19-40-47/ --incremental-dir=/xtrabackup/zeng/2022-09-25_19-58-12/ 根据实际增量备份的次数回滚，可以想恢复到那个时间节点就回滚到那个时间节点，所有的回滚都给全备 ``` 恢复数据： ```shell [root@xingdian ~]# innobackupex --copy-back /xtrabackup/full/2022-09-25_19-40-47/ ``` 修改权限： ```shell [root@xingdian ~]# chown mysql.mysql /var/lib/mysql -R ``` 启动数据库： ```shell [root@xingdian ~]# systemctl start mysqld ``` 验证： ```shell [root@xingdian ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.7.39 MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | t1 | | t2 | | t3 | +--------------------+ ``` ##### 3.差异备份 差异备份 原理：只备份跟完整备份不一样的 注意：在进行增量备份前先进行完整备份 案例：周一进行全备，周二到周天进行差异备份 完整备份：（周一） ```shell [root@xingdian ~]# mkdir -p /xtrabackup/full [root@xingdian ~]# innobackupex --user=root --password=QianFeng@123 /xtrabackup/full ``` 模拟数据增加（略） 第一次差异备份：（周二） ```shell [root@xingdian ~]# mkdir -p /xtrabackup/jian [root@xingdian ~]# innobackupex --user=root --password=QianFeng@123 --incremental /xtrabackup/jian --incremental-basedir=/xtrabackup/full/2022-09-25_20-10-52/ 查看第一次差异备份的数据： [root@xingdian ~]# ls /xtrabackup/jian/ 2022-09-25_20-12-55 ``` 模拟数据增加（略） 第二次差异备份：（周三） ```shell [root@xingdian ~]# innobackupex --user=root --password=QianFeng@123 --incremental /xtrabackup/jian --incremental-basedir=/xtrabackup/full/2022-09-25_20-10-52/ 查看第二次差异备份的数据： [root@xingdian ~]# ls /xtrabackup/jian/ 2022-09-25_20-12-55 2022-09-25_20-14-32 注意：后面的差异备份跟之前一样，根据需求可以继续差异备份 ``` 差异备份恢复流程：（模拟数据丢失） 停止数据库： ```shell [root@xingdian ~]# systemctl stop mysqld ``` 删除数据： ```shell [root@xingdian ~]# rm -rf /var/lib/mysql/* [root@xingdian ~]# rm -rf /var/log/mysqld.log ``` 重演数据回滚： ```shell 完整备份回滚： [root@xingdian ~]# innobackupex --apply-log --redo-only /xtrabackup/full/2022-09-25_20-10-52/ 差异备份回滚（根据差异备份的原理，如果恢复所有数据只需要将最后依次差异回滚） [root@xingdian ~]# innobackupex --apply-log --redo-only /xtrabackup/full/2022-09-25_20-10-52/ --incremental-dir=/xtrabackup/jian/2022-09-25_20-14-32/ ``` 恢复数据： ```shell [root@xingdian ~]# innobackupex --copy-back /xtrabackup/full/2022-09-25_20-10-52/ ``` 修改权限： ```shell [root@xingdian ~]# chown mysql.mysql /var/lib/mysql -R ``` 启动数据库： ```shell [root@xingdian ~]# systemctl start mysqld ``` 数据验证： ```shell [root@xingdian ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.7.39 MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | k1 | | k2 | | mysql | | performance_schema | | sys | +--------------------+ 9 rows in set (0.00 sec) ``` ##### 4.mysqldump备份 备份表：（前提有库有表） ```shell [root@xingdian ~]# mysqldump -u root -pQianFeng@123 k1 t1 > /t1.sql ``` 恢复表：（恢复之前模拟数据丢失） ```shell [root@xingdian ~]# mysql -u root -pQianFeng@123 k1 < /t1.sql mysql: [Warning] Using a password on the command line interface can be insecure. ``` 验证： ```shell [root@xingdian ~]# mysql -u root -pQianFeng@123 -e "use k1;show tables" mysql: [Warning] Using a password on the command line interface can be insecure. +--------------+ | Tables_in_k1 | +--------------+ | t1 | +--------------+ ``` 备份一个库： ```shell [root@xingdian ~]# mysqldump -u root -pQianFeng@123 k1 > /k1.sql ``` 备份多个库： ```shell [root@xingdian ~]# mysqldump -u root -pQianFeng@123 -B k1 k2 > /kall.sql ``` 备份所有库： ```shell [root@xingdian ~]# mysqldump -u root -pQianFeng@123 -A > /all.sql ``` 数据恢复： 为保证数据一致性，应在恢复数据之前停止数据库对外的服务,停止binlog日志 binlog使用binlog日志恢复数据时也会产生binlog日志（如果开启的话，需要关闭） ```shell mysql> set sql_log_bin=0; Query OK, 0 rows affected (0.00 sec) ``` 模拟数据丢失（略） ```shell [root@xingdian ~]# mysql -u root -pQianFeng@123 -D k1 < /k1.sql mysql: [Warning] Using a password on the command line interface can be insecure. ERROR 1049 (42000): Unknown database 'k1' 出现该错误是因为在恢复的时候需要有库的存在 [root@xingdian ~]# mysql -u root -pQianFeng@123 -e "create database k1" [root@xingdian ~]# mysql -u root -pQianFeng@123 -D k1 < /k1.sql [root@xingdian ~]# mysql -u root -pQianFeng@123 -e "create database k1" [root@xingdian ~]# mysql -u root -pQianFeng@123 -e "create database k2" [root@xingdian ~]# mysql -u root -pQianFeng@123 -D k1 k2 < /kall.sql 或者 mysql> source /k1.sql ``` 验证： ```shell [root@xingdian ~]# mysql -u root -pQianFeng@123 -e "use k1; show tables;" mysql: [Warning] Using a password on the command line interface can be insecure. +--------------+ | Tables_in_k1 | +--------------+ | t1 | +--------------+ [root@xingdian ~]# mysql -u root -pQianFeng@123 -e "use k2; show tables;" mysql: [Warning] Using a password on the command line interface can be insecure. +--------------+ | Tables_in_k2 | +--------------+ | t1 | +--------------+ ``` ##### 5.binlog日志备份 原理：日志方法备份恢复数据 日志默认存储位置： rpm ： /var/lib/mysql 编译: 安装目录的var下 产生日志： 方式一：编译安装 ```shell [root@xingdian ~]# mysqld_safe --log-bin --user=mysql --server-id=1 & 查看binlog日志 [root@xingdian ~]# mysqlbinlog slave2-bin.000001 -v --base64-output=decode-rows 时间点 ： 141126 14:04:49 位置点 : at 106 ``` 方式二：rpm安装（永久） ```shell [root@xingdian ~]# vim /etc/my.cnf log-bin=mylog server-id=1 //做主从复制使用 [root@xingdian ~]# systemctl restart mysqld 查看： [root@xingdian ~]# ls /var/lib/mysql auto.cnf client-key.pem ib_logfile1 mysql private_key.pem sys ca-key.pem ib_buffer_pool ibtmp1 mysql.sock public_key.pem xingdian-bin.index ca.pem ibdata1 mylog.000001 mysql.sock.lock server-cert.pem xtrabackup_info client-cert.pem ib_logfile0 mylog.index [root@xingdian ~]# mysqlbinlog /var/lib/mysql/mylog.000001 -v --base64-output=decode-rows /*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/; /*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/; DELIMITER /*!*/; # at 4 #220925 21:12:47 server id 1 end_log_pos 123 CRC32 0x52358645 Start: binlog v 4, server v 5.7.39-log created 220925 21:12:47 at startup # Warning: this binlog is either in use or was not closed properly. ROLLBACK/*!*/; # at 123 #220925 21:12:47 server id 1 end_log_pos 154 CRC32 0xa84d8536 Previous-GTIDs # [empty] # at 154 #220925 21:13:38 server id 1 end_log_pos 219 CRC32 0xc2b00431 Anonymous_GTID last_committed=0 sequence_number=1 rbr_only=no SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/; # at 219 #220925 21:13:38 server id 1 end_log_pos 307 CRC32 0x635401a5 Query thread_id=2 exec_time=0 error_code=0 SET TIMESTAMP=1664111618/*!*/; SET @@session.pseudo_thread_id=2/*!*/; SET @@session.foreign_key_checks=1, @@session.sql_auto_is_null=0, @@session.unique_checks=1, @@session.autocommit=1/*!*/; SET @@session.sql_mode=1436549152/*!*/; SET @@session.auto_increment_increment=1, @@session.auto_increment_offset=1/*!*/; /*!\C utf8 *//*!*/; SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=8/*!*/; SET @@session.lc_time_names=0/*!*/; SET @@session.collation_database=DEFAULT/*!*/; create database t1 /*!*/; SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/; DELIMITER ; # End of log file /*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/; /*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/; 方法二： mysql> show binlog events in "mylog.000001"; +--------------+-----+----------------+-----------+-------------+---------------------------------------+ | Log_name | Pos | Event_type | Server_id | End_log_pos | Info | +--------------+-----+----------------+-----------+-------------+---------------------------------------+ | mylog.000001 | 4 | Format_desc | 1 | 123 | Server ver: 5.7.39-log, Binlog ver: 4 | | mylog.000001 | 123 | Previous_gtids | 1 | 154 | | | mylog.000001 | 154 | Anonymous_Gtid | 1 | 219 | SET @@SESSION.GTID_NEXT= 'ANONYMOUS' | | mylog.000001 | 219 | Query | 1 | 307 | create database t1 | +--------------+-----+----------------+-----------+-------------+---------------------------------------+ 4 rows in set (0.00 sec) 默认查看第一个 mysql> show binlog events; +--------------+-----+----------------+-----------+-------------+---------------------------------------+ | Log_name | Pos | Event_type | Server_id | End_log_pos | Info | +--------------+-----+----------------+-----------+-------------+---------------------------------------+ | mylog.000001 | 4 | Format_desc | 1 | 123 | Server ver: 5.7.39-log, Binlog ver: 4 | | mylog.000001 | 123 | Previous_gtids | 1 | 154 | | | mylog.000001 | 154 | Anonymous_Gtid | 1 | 219 | SET @@SESSION.GTID_NEXT= 'ANONYMOUS' | | mylog.000001 | 219 | Query | 1 | 307 | create database t1 | +--------------+-----+----------------+-----------+-------------+---------------------------------------+ 4 rows in set (0.00 sec) ``` 数据恢复： 根据时间点恢复数据： ```shell [root@xingdian ~]# mysqlbinlog --start-datetime='2022-9-25 21:12:47' --stop-datetime='2022-9-25 21:16:55' /var/lib/mysql/mylog.000001 | mysql -u root -pQianFeng@123 ``` 根据位置点恢复数据： ```shell mysql> show binlog events; +--------------+-----+----------------+-----------+-------------+---------------------------------------+ | Log_name | Pos | Event_type | Server_id | End_log_pos | Info | +--------------+-----+----------------+-----------+-------------+---------------------------------------+ | mylog.000001 | 4 | Format_desc | 1 | 123 | Server ver: 5.7.39-log, Binlog ver: 4 | | mylog.000001 | 123 | Previous_gtids | 1 | 154 | | | mylog.000001 | 154 | Anonymous_Gtid | 1 | 219 | SET @@SESSION.GTID_NEXT= 'ANONYMOUS' | | mylog.000001 | 219 | Query | 1 | 307 | create database t1 | | mylog.000001 | 307 | Anonymous_Gtid | 1 | 372 | SET @@SESSION.GTID_NEXT= 'ANONYMOUS' | | mylog.000001 | 372 | Query | 1 | 453 | drop database t1 | | mylog.000001 | 453 | Anonymous_Gtid | 1 | 518 | SET @@SESSION.GTID_NEXT= 'ANONYMOUS' | | mylog.000001 | 518 | Query | 1 | 606 | create database t1 | | mylog.000001 | 606 | Anonymous_Gtid | 1 | 671 | SET @@SESSION.GTID_NEXT= 'ANONYMOUS' | | mylog.000001 | 671 | Query | 1 | 752 | drop database t1 | +--------------+-----+----------------+-----------+-------------+---------------------------------------+ [root@xingdian ~]# mysqlbinlog --start-position 219 --stop-position 307 /var/lib/mysql/mylog.000001 | mysql -u root -pQianFeng@123 ``` ### 6、主从复制以及读写分离 #### 1.主从复制概念 什么是主从复制： 主从复制，是用来建立一个和主数据库完全一样的数据库环境，称为从数据库；主数据库一般是准实时的业务数据库 主从复制的作用： 做数据的热备，作为后备数据库，主数据库服务器故障后，可切换到从数据库继续工作，避免数据丢失 架构的扩展，业务量越来越大，I/O访问频率过高，单机无法满足，多库的存储，降低磁盘I/O访问的频率，提高单个机器的I/O性能 读写分离，使数据库能支撑更大的并发 主从复制的原理： 数据库有个bin-log二进制文件，记录了所有sql语句 我们的目标就是把主数据库的bin-log文件的sql语句复制到从库 让其在从数据的relay-log(中继日志)重做日志文件中再执行一次这些sql语句即可 总结： 从库slave生成两个线程，i/o线程和sql线程，i/o将变更记录写到二进制日志文件中，再写到中继日志中，sql线程读取中继日志，解析操作，最终数据统一 注意： I/O进程：负责通信 SQL进程：负责写数据，根据log日志写数据 #### 2.主从复制部署 环境准备 节点 master 192.168.17.194 slave 192.168.17.206 注意： 所有节点关闭防火墙和selinux 保证yum仓库可用 保证网络畅通 如果是克隆的服务器需要修改每台数据库的server-uuid 修改主机名：（所有节点）（可选操作） ```shell [root@xingdian ~]# hostnamectl set-hostname master [root@xingdian ~]# hostnamectl set-hostname slave ``` 添加本地解析：（所有节点）（可选操作） ```shell [root@master ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.17.194 master 192.168.17.206 slave [root@slave ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.17.194 master 192.168.17.206 slave ``` ##### Master部署 安装数据库：（略） 启动数据库：（略） 修改数据库初始密码：（略） 主服务器部署： ```shell [root@master ~]# vi /etc/my.cnf log-bin = my1log server-id = 1 ``` 创建授权账户： ```shell [root@master ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 3 Server version: 5.7.39 MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> grant all on *.* to 'slave'@'%' identified by 'QianFeng@123'; mysql> flush privileges; mysql> exit Bye ``` 重启服务： ```shell [root@master ~]# systemctl restart mysqld ``` 注意： replication slave： 拥有此权限可以查看从服务器，从主服务器读取二进制日志 super权限： 允许用户使用修改全局变量的SET语句以及CHANGE MASTER语句 reload权限： 必须拥有reload权限，才可以执行flush [tables | logs | privileges] ##### Slave部署 安装数据库：（略） 启动数据库：（略） 修改数据库初始密码：（略） 从服务器部署： ```shell [root@slave ~]# vi /etc/my.cnf log-bin = my2log server-id = 2 ``` 重启服务： ```shell [root@slave ~]# systemctl restart mysqld ``` 获取主服务器信息：（主服务器操作） ```shell [root@master ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.7.39-log MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show binlog events; +---------------+-----+----------------+-----------+-------------+---------------------------------------+ | Log_name | Pos | Event_type | Server_id | End_log_pos | Info | +---------------+-----+----------------+-----------+-------------+---------------------------------------+ | my1log.000001 | 4 | Format_desc | 1 | 123 | Server ver: 5.7.39-log, Binlog ver: 4 | | my1log.000001 | 123 | Previous_gtids | 1 | 154 | | +---------------+-----+----------------+-----------+-------------+---------------------------------------+ 2 rows in set (0.00 sec) ``` 指定主服务器信息：（从服务器操作） ```shell [root@slave ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.7.39-log MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> edit -> ; Query OK, 0 rows affected, 2 warnings (0.01 sec) 注意：edit中添加的内容 CHANGE MASTER TO MASTER_HOST='master', MASTER_USER='slave', MASTER_PASSWORD='QianFeng@123', MASTER_PORT=3306, MASTER_LOG_FILE='my1log.000001', MASTER_LOG_POS=4, MASTER_CONNECT_RETRY=10; 参数解释： CHANGE MASTER TO MASTER_HOST='mysql-master-1.blackmed.cn/ip', //做过解析的可以直接用主机名，没有则用IP MASTER_USER='slave', //主服务器用户 MASTER_PASSWORD='big', //主服务器的密码 MASTER_PORT=3306, MASTER_LOG_FILE='master2-bin.001', //日志文件 MASTER_LOG_POS=4, //日志位置 MASTER_CONNECT_RETRY=10; //默认尝试次数 获取参数： mysql> help change master to ``` 启动slave： ```shell mysql> start slave; Query OK, 0 rows affected (0.00 sec) ``` 注意： stop slave；停止slave reset master；删除所有的binglog日志文件，并将日志索引文件清空，重新开始所有新的日志文件；用于第一次进行搭建主从库时，进行主库binlog初始化工作 reset slave；用于删除SLAVE数据库的relaylog日志文件，并重新启用新的relaylog文件 查看主从状态： ```shell mysql> show slave status\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: master Master_User: slave Master_Port: 3306 Connect_Retry: 10 Master_Log_File: my1log.000001 Read_Master_Log_Pos: 154 Relay_Log_File: slave-relay-bin.000002 Relay_Log_Pos: 361 Relay_Master_Log_File: my1log.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 154 Relay_Log_Space: 568 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1 Master_UUID: 85c6acc4-3db0-11ed-b302-000c29311164 Master_Info_File: /var/lib/mysql/master.info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: Executed_Gtid_Set: Auto_Position: 0 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: 1 row in set (0.00 sec) ``` 注意： Slave_IO_Running: Yes Slave_SQL_Running: Yes 验证： 主服务器创建数据： ```shell mysql> create database t1; Query OK, 1 row affected (0.00 sec) ``` 从服务器查看数据： ```shell mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | | t1 | +--------------------+ 5 rows in set (0.00 sec) ``` ### 7、GTID主从复制 #### 1.GTID概念 GTID基于事务ID复制 GTID全局事务标识（global transaction identifiers） 是用来代替传统复制的方法，GTID复制与普通复制模式的最大不同就是不需要指定二进制文件名和位置 不再使用MASTER_LOG_FILE+MASTER_LOG_POS开启复制。而是使用MASTER_AUTO_POSTION=1的方式开始复制 #### 2.GTID组成 GTID = source_id:transaction_id source_id源id，用于鉴别原服务器，即mysql服务器唯一的server_uuid，由于GTID会传递到slave，所以也可以理解为源ID transaction_id事务id，为当前服务器上已提交事务的一个序列号，通常从1开始自增长的序列，一个数值对应一个事务 示例： 3E11FA47-71CA-11E1-9E33-C80AA9429562:23 前面的一串为服务器的server_uuid 后面的23为transaction_id #### 3.GTID工作原理 master更新数据时，会在事务前产生GTID，一同记录到binlog日志中 slave端的i/o 线程将变更的binlog，写入到本地的relay log中 sql线程从relay log中获取GTID，然后对比slave端的binlog是否有记录 如果有记录，说明该GTID的事务已经执行，slave会忽略 如果没有记录，slave就会从relay log中执行该GTID的事务，并记录到binlog #### 4.主从部署 注意：实验之前环境初始化，不要有残留的数据 节点 master 10.0.0.128 slave 10.0.042 注意： 所有节点关闭防火墙和selinux 保证yum仓库可用 保证网络畅通 如果是克隆的服务器需要修改每台数据库的server-uuid 修改主机名：（所有节点）（可选操作） ```shell [root@xingdian ~]# hostnamectl set-hostname master [root@xingdian ~]# hostnamectl set-hostname slave ``` 添加本地解析：（所有节点）（可选操作） ```shell [root@master ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 10.0.0.128 master 10.0.0.42 slave [root@slave ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 10.0.0.128 master 10.0.0.42 slave ``` ##### Master部署 安装数据库：（略） 启动数据库：（略） 修改数据库初始密码：（略） 主服务器部署： ```shell [root@master ~]# vim /etc/my.cnf log-bin server-id=1 gtid_mode = ON enforce_gtid_consistency=1 ``` 创建授权用户： ```shell [root@master ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 3 Server version: 5.7.39 MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> grant all on *.* to slave@'%' identified by 'QianFeng@123'; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> flush privileges; Query OK, 0 rows affected (0.00 sec) mysql> exit Bye ``` 重启服务： ```shell [root@master ~]# systemctl restart mysqld ``` ##### Slave部署 安装数据库：（略） 启动数据库：（略） 修改数据库初始密码：（略） 从服务器部署： ```shell [root@slave ~]# vim /etc/my.cnf log-bin server-id=2 gtid_mode = ON enforce_gtid_consistency=1 relay_log_recovery = on master-info-repository=TABLE relay-log-info-repository=TABLE //这两个参数会将master.info和relay.info保存在表中，默认是Myisam引擎，官方建议用 ``` 重启服务： ```shell [root@slave ~]# systemctl restart mysqld ``` 配置连接主服务器： ```shell [root@slave ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.7.39-log MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> change master to -> master_host='master', -> master_user='slave', -> master_password='QianFeng@123', -> master_auto_position=1; Query OK, 0 rows affected, 2 warnings (0.00 sec) ``` 启动Slave： ```shell mysql> start slave; Query OK, 0 rows affected (0.00 sec) ``` 主从状态验证： ```shell mysql> show slave status\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: master Master_User: slave Master_Port: 3306 Connect_Retry: 60 Master_Log_File: master-bin.000001 Read_Master_Log_Pos: 154 Relay_Log_File: slave-relay-bin.000002 Relay_Log_Pos: 369 Relay_Master_Log_File: master-bin.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 154 Relay_Log_Space: 576 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1 Master_UUID: 00813e87-4321-11ed-a33c-000c29311164 Master_Info_File: mysql.slave_master_info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: Executed_Gtid_Set: Auto_Position: 1 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: 1 row in set (0.00 sec) ``` 数据验证： 主服务器创建数据： ```shell [root@master ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 3 Server version: 5.7.39-log MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> create database qfcloud; Query OK, 1 row affected (0.00 sec) mysql> exit Bye ``` 从服务器查验数据： ```shell mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | qfcloud | | sys | +--------------------+ 5 rows in set (0.00 sec) ``` ### 8、GTID双主双从 #### 1.环境准备 注意： 实验之前环境初始化，不要有残留的数据 先做双主，M-M互为主从，从是双主的从 | 节点 | IP地址 | | -------- | ---------- | | Master-1 | 10.0.0.128 | | Master-2 | 10.0.0.46 | | Slave-1 | 10.0.0.42 | | Slave-2 | 10.0.0.45 | 注意： 所有节点关闭防火墙和selinux 保证yum仓库可用 保证网络畅通 如果是克隆的服务器需要修改每台数据库的server-uuid 做域名解析 该实验解析 master-1 master-2 #### 2.Master-1部署 安装数据库：（略） 启动数据库：（略） 修改数据库初始密码：（略） 主服务器一部署： ```shell [root@master-1 ~]# vim /etc/my.cnf log-bin = my1log server-id = 1 gtid_mode=ON enforce_gtid_consistency=1 ``` 创建授权账户： ```shell [root@master-1 ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 4 Server version: 5.7.39 MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> grant all on *.* to 'slave'@'%' identified by 'QianFeng@123'; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> flush privileges; Query OK, 0 rows affected (0.00 sec) mysql> exit Bye ``` 重启服务： ```shell [root@master-1 ~]# systemctl restart mysqld ``` #### 3.Master-2部署 安装数据库：（略） 启动数据库：（略） 修改数据库初始密码：（略） 主服务器二部署： ```shell [root@master-2 ~]# vim /etc/my.cnf log-bin = my2log server-id = 2 gtid_mode=ON enforce_gtid_consistency=1 ``` 创建授权账户： ```shell [root@master-2 ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 4 Server version: 5.7.39 MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> grant all on *.* to 'slave'@'%' identified by 'QianFeng@123'; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> flush privileges; Query OK, 0 rows affected (0.00 sec) mysql> exit Bye ``` 重启服务： ```shell [root@master-2 ~]# systemctl restart mysqld ``` #### 4.双主互为主从 Master-1： ```shell [root@master-1 ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 3 Server version: 5.7.39-log MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> change master to -> master_host='master-2', -> master_user='slave', -> master_password='QianFeng@123', -> master_auto_position=1; Query OK, 0 rows affected, 2 warnings (0.01 sec) mysql> start slave; Query OK, 0 rows affected (0.00 sec) mysql> show slave status\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: master-2 Master_User: slave Master_Port: 3306 Connect_Retry: 60 Master_Log_File: my2log.000001 Read_Master_Log_Pos: 154 Relay_Log_File: master-1-relay-bin.000002 Relay_Log_Pos: 361 Relay_Master_Log_File: my2log.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 154 Relay_Log_Space: 571 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 2 Master_UUID: 1b453d94-452e-11ed-b744-000c2936b606 Master_Info_File: /var/lib/mysql/master.info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: Executed_Gtid_Set: 146f4cae-452e-11ed-b87f-000c29311164:1-2 Auto_Position: 1 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: 1 row in set (0.00 sec) mysql> exit Bye ``` Master-2： ```shell [root@master-2 ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 3 Server version: 5.7.39-log MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> change master to -> master_host='master-1', //这块做域名解析 就直接写解析的名称 如果没有就写另一台mysql服务器ip -> master_user='slave', -> master_password='QianFeng@123', -> master_auto_position=1; Query OK, 0 rows affected, 2 warnings (0.01 sec) mysql> start slave; Query OK, 0 rows affected (0.00 sec) mysql> show slave status\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: master-1 Master_User: slave Master_Port: 3306 Connect_Retry: 60 Master_Log_File: my1log.000001 Read_Master_Log_Pos: 587 Relay_Log_File: master-2-relay-bin.000002 Relay_Log_Pos: 794 Relay_Master_Log_File: my1log.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 587 Relay_Log_Space: 1004 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1 Master_UUID: 146f4cae-452e-11ed-b87f-000c29311164 Master_Info_File: /var/lib/mysql/master.info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: 146f4cae-452e-11ed-b87f-000c29311164:1-2 Executed_Gtid_Set: 146f4cae-452e-11ed-b87f-000c29311164:1-2 Auto_Position: 1 Replicate_Rewrite_DB: Channel_Name: Master_TLS_Version: 1 row in set (0.00 sec) mysql> exit Bye ``` #### 5.Slave-1部署 安装数据库：（略） 启动数据库：（略） 修改数据库初始密码：（略） 从服务器一部署： ```shell [root@slave-1 ~]# vim /etc/my.cnf log-bin = my3log server-id = 3 gtid_mode=ON enforce_gtid_consistency=1 relay_log_info_repository = TABLE master_info_repository = TABLE relay_log_recovery = on 当slave从库宕机后，假如relay-log损坏了，导致一部分中继日志没有处理，则自动放弃所有未执行的relay-log，并且重新从master上获取日志，这样就保证了relay-log的完整性 ``` 重启服务： ```shell [root@slave-1 ~]# systemctl restart mysqld ``` 从连接主服务器： ```shell [root@slave-1 ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.7.39-log MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CHANGE MASTER TO -> MASTER_HOST='master-1', -> MASTER_USER='slave', -> MASTER_PASSWORD='QianFeng@123', -> MASTER_AUTO_POSITION=1 FOR CHANNEL 'master-1'; Query OK, 0 rows affected, 2 warnings (0.00 sec) mysql> start slave; Query OK, 0 rows affected (0.00 sec) mysql> show slave status\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: master-1 Master_User: slave Master_Port: 3306 Connect_Retry: 60 Master_Log_File: my1log.000001 Read_Master_Log_Pos: 587 Relay_Log_File: slave-1-relay-bin-master@002d1.000002 Relay_Log_Pos: 794 Relay_Master_Log_File: my1log.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 587 Relay_Log_Space: 1016 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1 Master_UUID: 146f4cae-452e-11ed-b87f-000c29311164 Master_Info_File: mysql.slave_master_info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: 146f4cae-452e-11ed-b87f-000c29311164:1-2 Executed_Gtid_Set: 146f4cae-452e-11ed-b87f-000c29311164:1-2 Auto_Position: 1 Replicate_Rewrite_DB: Channel_Name: master-1 Master_TLS_Version: 1 row in set (0.00 sec) mysql> CHANGE MASTER TO -> MASTER_HOST='master-2', -> MASTER_USER='slave', -> MASTER_PASSWORD='QianFeng@123', -> MASTER_AUTO_POSITION=1 FOR CHANNEL 'master-2'; Query OK, 0 rows affected, 2 warnings (0.01 sec) mysql> start slave; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> show slave status\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: master-1 Master_User: slave Master_Port: 3306 Connect_Retry: 60 Master_Log_File: my1log.000001 Read_Master_Log_Pos: 587 Relay_Log_File: slave-1-relay-bin-master@002d1.000002 Relay_Log_Pos: 794 Relay_Master_Log_File: my1log.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 587 Relay_Log_Space: 1016 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1 Master_UUID: 146f4cae-452e-11ed-b87f-000c29311164 Master_Info_File: mysql.slave_master_info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: 146f4cae-452e-11ed-b87f-000c29311164:1-2 Executed_Gtid_Set: 146f4cae-452e-11ed-b87f-000c29311164:1-2 Auto_Position: 1 Replicate_Rewrite_DB: Channel_Name: master-1 Master_TLS_Version: *************************** 2. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: master-2 Master_User: slave Master_Port: 3306 Connect_Retry: 60 Master_Log_File: my2log.000001 Read_Master_Log_Pos: 154 Relay_Log_File: slave-1-relay-bin-master@002d2.000002 Relay_Log_Pos: 361 Relay_Master_Log_File: my2log.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 154 Relay_Log_Space: 583 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 2 Master_UUID: 1b453d94-452e-11ed-b744-000c2936b606 Master_Info_File: mysql.slave_master_info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: Executed_Gtid_Set: 146f4cae-452e-11ed-b87f-000c29311164:1-2 Auto_Position: 1 Replicate_Rewrite_DB: Channel_Name: master-2 Master_TLS_Version: 2 rows in set (0.00 sec) ``` #### 6.Slave-2部署 安装数据库：（略） 启动数据库：（略） 修改数据库初始密码：（略） 从服务器二部署： ```shell [root@slave-2 ~]# vim /etc/my.cnf log-bin = my4log server-id = 4 gtid_mode=ON enforce_gtid_consistency=1 relay_log_info_repository = TABLE master_info_repository = TABLE relay_log_recovery = on ``` 重启服务： ```shell [root@slave-2 ~]# systemctl restart mysqld ``` 从连接主服务器： ```shell [root@slave-2 ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.7.39-log MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CHANGE MASTER TO -> MASTER_HOST='master-1', -> MASTER_USER='slave', -> MASTER_PASSWORD='QianFeng@123', -> MASTER_AUTO_POSITION=1 FOR CHANNEL 'master-1'; Query OK, 0 rows affected, 2 warnings (0.01 sec) mysql> start slave; Query OK, 0 rows affected (0.00 sec) mysql> CHANGE MASTER TO -> MASTER_HOST='master-2', -> MASTER_USER='slave', -> MASTER_PASSWORD='QianFeng@123', -> MASTER_AUTO_POSITION=1 FOR CHANNEL 'master-2'; Query OK, 0 rows affected, 2 warnings (0.00 sec) mysql> start slave; Query OK, 0 rows affected, 1 warning (0.01 sec) mysql> show slave status\G *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: master-1 Master_User: slave Master_Port: 3306 Connect_Retry: 60 Master_Log_File: my1log.000001 Read_Master_Log_Pos: 587 Relay_Log_File: slave-2-relay-bin-master@002d1.000002 Relay_Log_Pos: 794 Relay_Master_Log_File: my1log.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 587 Relay_Log_Space: 1016 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1 Master_UUID: 146f4cae-452e-11ed-b87f-000c29311164 Master_Info_File: mysql.slave_master_info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: 146f4cae-452e-11ed-b87f-000c29311164:1-2 Executed_Gtid_Set: 146f4cae-452e-11ed-b87f-000c29311164:1-2 Auto_Position: 1 Replicate_Rewrite_DB: Channel_Name: master-1 Master_TLS_Version: *************************** 2. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: master-2 Master_User: slave Master_Port: 3306 Connect_Retry: 60 Master_Log_File: my2log.000001 Read_Master_Log_Pos: 154 Relay_Log_File: slave-2-relay-bin-master@002d2.000002 Relay_Log_Pos: 361 Relay_Master_Log_File: my2log.000001 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 154 Relay_Log_Space: 583 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 2 Master_UUID: 1b453d94-452e-11ed-b744-000c2936b606 Master_Info_File: mysql.slave_master_info SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Master_Retry_Count: 86400 Master_Bind: Last_IO_Error_Timestamp: Last_SQL_Error_Timestamp: Master_SSL_Crl: Master_SSL_Crlpath: Retrieved_Gtid_Set: Executed_Gtid_Set: 146f4cae-452e-11ed-b87f-000c29311164:1-2 Auto_Position: 1 Replicate_Rewrite_DB: Channel_Name: master-2 Master_TLS_Version: 2 rows in set (0.00 sec) ``` #### 7.验证 主服务器创建数据： ```shell [root@master-1 ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 9 Server version: 5.7.39-log MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> create database qfcloud; Query OK, 1 row affected (0.00 sec) mysql> exit Bye ``` 其他服务器验证： ```shell [root@master-2 ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 8 Server version: 5.7.39-log MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | qfcloud | | sys | +--------------------+ 5 rows in set (0.00 sec) [root@slave-1 ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 8 Server version: 5.7.39-log MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | qfcloud | | sys | +--------------------+ 5 rows in set (0.00 sec) [root@slave-2 ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 7 Server version: 5.7.39-log MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | qfcloud | | sys | +--------------------+ 5 rows in set (0.00 sec) ``` ### 9、读写分离 #### 环境介绍 [![image-20221006131313211](https://xingdian-image.oss-cn-beijing.aliyuncs.com/xingdian-image/image-20221006131313211.png)](https://xingdian-image.oss-cn-beijing.aliyuncs.com/xingdian-image/image-20221006131313211.png) #### 2.读写分离集群部署 ##### A：数据库集群部署（略） 单主单从；多主多从等均可 ##### B：Mycat部署 新机器，不需要安装mysql 安装jdk环境： ```shell [root@mycat ~]# tar xf jdk-8u211-linux-x64.tar.gz -C /usr/local/ [root@mycat ~]# mv /usr/local/jdk1.8.0_211/ /usr/local/java ``` 设置环境变量： ```shell [root@mycat ~]# vi /etc/profile JAVA_HOME=/usr/local/java PATH=$JAVA_HOME/bin:$PATH export JAVA_HOME PATH [root@mycat ~]# source /etc/profile [root@mycat ~]# java -version java version "1.8.0_211" Java(TM) SE Runtime Environment (build 1.8.0_211-b12) Java HotSpot(TM) 64-Bit Server VM (build 25.211-b12, mixed mode) ``` 安装mycat： ```shell [root@mycat ~]# tar xf Mycat-server-1.6.7.1-release-20190627191042-linux.tar.gz -C /usr/local/ ``` 设置环境变量： ```shell [root@mycat ~]# vi ~/.bash_profile PATH=$PATH:$HOME/bin:/usr/local/mycat/bin [root@mycat ~]# source ~/.bash_profile ``` ##### C：数据库部署 Mysql中添加数据库和账户： ```shell [root@master-1 ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 10 Server version: 5.7.39-log MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> create database shop; Query OK, 1 row affected (0.00 sec) mysql> create database bbs; Query OK, 1 row affected (0.00 sec) mysql> create database blog; Query OK, 1 row affected (0.00 sec) mysql> grant all on shop.* to shop@'%' identified by 'QianFeng@123'; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> grant all on bbs.* to bbs@'%' identified by 'QianFeng@123'; Query OK, 0 rows affected, 1 warning (0.00 sec) mysql> grant all on blog.* to blog@'%' identified by 'QianFeng@123'; Query OK, 0 rows affected, 1 warning (0.00 sec) ``` 注意： 创建的库跟mycat关联，原则上一个库对应一个项目，根据实际情况创建 创建的库需要单独授权用户进行管理，用户名和库名根据实际情况决定 ##### D：Mycat配置 server.xml：Mycat的配置文件，设置账号、参数等 ```shell [root@mycat conf]# vim server.xml 0 1 0 0 2 (?:(\s*next\s+value\s+for\s*MYCATSEQ_(\w+))(,|\)|\s)*)+ false 0 0 0 64k 1k 0 384m false false true 123456 TESTDB 123456 shop 123456 bbs 123456 blog ``` 注意： ```shell user 用户配置节点 –name 登录的用户名，也就是连接Mycat的用户名 –password 登录的密码，也就是连接Mycat的密码 –schemas 数据库名，这里会和schema.xml中的配置关联，多个用逗号分开，例如需要这个用户需要管理两个数据库db1,db2，则配置db1,db2 ``` schema.xml：Mycat对应的物理数据库和数据库表的配置 ```shell [root@mycat conf]# cat schema.xml show status like 'wsrep%' show status like 'wsrep%' show status like 'wsrep%' ``` 注意： ```shell balance=1 开启读写分离机制，所有读操作都发送到当前备用的 writeHost 上。 wirteType=0 所有写操作发送到第一个writeHost，第一个挂了切换到第二个 switchType=3 基于MySQL Galera cluster的切换机制，心跳语句为show status like 'wsrep%' ``` ##### E：启动服务 ```shell [root@mycat conf]# mycat start Starting Mycat-server... [root@mycat conf]# jps 1494 WrapperSimpleApp 1528 Jps ``` 查看端口： ```shell [root@mycat conf]# ss -antpl State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 *:22 *:* users:(("sshd",pid=837,fd=3)) LISTEN 0 100 127.0.0.1:25 *:* users:(("master",pid=933,fd=13)) LISTEN 0 1 127.0.0.1:32000 *:* users:(("java",pid=1494,fd=4)) LISTEN 0 50 :::37680 :::* users:(("java",pid=1494,fd=57)) LISTEN 0 128 :::22 :::* users:(("sshd",pid=837,fd=4)) LISTEN 0 100 ::1:25 :::* users:(("master",pid=933,fd=14)) LISTEN 0 50 :::1984 :::* users:(("java",pid=1494,fd=58)) LISTEN 0 100 :::8066 :::* users:(("java",pid=1494,fd=79)) LISTEN 0 50 :::32834 :::* users:(("java",pid=1494,fd=59)) LISTEN 0 100 :::9066 :::* users:(("java",pid=1494,fd=75)) ``` ##### F：客户端测试 mycat连接测试： ```shell [root@master-1 ~]# mysql -u shop -p123456 -P 8066 -h 10.0.0.47 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 1 Server version: 5.6.29-mycat-1.6.7.1-release-20190627191042 MyCat Server (OpenCloudDB) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +----------+ | DATABASE | +----------+ | shop | +----------+ 1 row in set (0.00 sec) mysql> use shop Database changed mysql> create table t1(id int); Query OK, 0 rows affected (0.01 sec) mysql> show tables; +----------------+ | Tables_in_shop | +----------------+ | t1 | +----------------+ 1 row in set (0.01 sec) mysql> insert into t1 values(1); Query OK, 1 row affected (0.03 sec) mysql> insert into t1 values(2); Query OK, 1 row affected (0.01 sec) ``` 从服务数据验证： ```shell [root@slave-2 ~]# mysql -u root -pQianFeng@123 mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 7 Server version: 5.7.39-log MySQL Community Server (GPL) Copyright (c) 2000, 2022, Oracle and/or its affiliates. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | qfcloud | | sys | +--------------------+ 5 rows in set (0.00 sec) mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | bbs | | blog | | mysql | | performance_schema | | qfcloud | | shop | | sys | +--------------------+ 8 rows in set (0.00 sec) mysql> use shop Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> show tables; +----------------+ | Tables_in_shop | +----------------+ | t1 | +----------------+ 1 row in set (0.00 sec) mysql> select * from t1; +------+ | id | +------+ | 1 | | 2 | +------+ 2 rows in set (0.00 sec) ``` #### 3.Mycat配置文件 ##### server.xml ```shell 一：server.xml 配置文件 1.privileges标签 对用户的 schema以及表进行精细化的DML(数据操纵语言)权限控制 --check 表示是否开启DML权限检查。默认是关闭。 --dml 顺序说明：insert,update,select,delete
db1的权限是update,select。 tb01的权限是啥都不能干。 tb02的权限是insert,update,select,delete。 其他表默认是udpate,select。 2.system标签 这个标签内嵌套的所有 property 标签都与系统配置有关。 utf8 字符集 1 处理线程数量，默认是cpu数量。 4096 每次读取留的数量，默认4096。 409600 创建共享buffer需要占用的总空间大小。processorBufferChunk*processors*100。 0 默认为0。0表示DirectByteBufferPool，1表示ByteBufferArena。 100 二级共享buffer是processorBufferPool的百分比，这里设置的是百分比。 100 全局ID生成方式。(0:为本地文件方式，1:为数据库方式；2:为时间戳序列方式；3:为ZK生成ID；4:为ZK递增ID生成。 1 是否开启mysql压缩协议。1为开启，0为关闭，默认关闭。 4 指定 Mysql 协议中的报文头长度。默认 4。 16M 指定 Mysql 协议可以携带的数据最大长度。默认 16M。 1800000 指定连接的空闲超时时间。某连接在发起空闲检查下，发现距离上次使用超过了空闲时间，那么这个连接会被回收，就是被直接的关闭掉。默认 30 分钟，单位毫秒。 3 前端连接的初始化事务隔离级别，只在初始化的时候使用，后续会根据客户端传递过来的属性对后端数据库连接进行同步。默认为 REPEATED_READ，设置值为数字默认 3。 READ_UNCOMMITTED = 1; READ_COMMITTED = 2; REPEATED_READ = 3; SERIALIZABLE = 4; 300 SQL 执行超时的时间，Mycat 会检查连接上最后一次执行 SQL 的时间，若超过这个时间则会直接关闭这连接。默认时间为 300 秒，单位秒。 1000 清理 NIOProcessor 上前后端空闲、超时和关闭连接的间隔时间。默认是 1 秒，单 位毫秒。 300000 对后端连接进行空闲、超时检查的时间间隔，默认是 300 秒，单位毫秒。 10000 对后端所有读、写库发起心跳的间隔时间，默认是 10 秒，单位毫秒。 0.0.0.0 mycat 服务监听的 IP 地址，默认值为 0.0.0.0。 8066 定义 mycat 的使用端口，默认值为 8066。 9066 定义 mycat 的管理端口，默认值为 9066。 5.6 mycat 模拟的 mysql 版本号，默认值为 5.6 版本，如非特需，不要修改这个值，目前支持设置 5.5,5.6,5.7 版本，其他版本可能会有问题。 0 是否开启实时统计。1为开启；0为关闭 。 0 是否开启全局表一致性检测。1为开启；0为关闭 。 0 分布式事务开关。0为不过滤分布式事务；1为过滤分布式事务；2 为不过滤分布式事务,但是记录分布式事务日志。 65535 默认是65535。 64K 用于sql解析时最大文本长度 以上举例的属性仅仅是一部分，可以配置的变量很多。 System标签下的属性，一般是上线后，需要根据实际运行的情况，分析后调优的时候进行修改。 3. Firewall标签 防火墙的设置，也就是在网络层对请求的地址进行限制，主要是从安全角度来保证Mycat不被匿名IP进行访问 ``` ##### schema.xml ```shell 一：schema.xml –schema 数据库设置，此数据库为逻辑数据库，name与server.xml中schema对应 –dataNode 分片信息，也就是分库相关配置 –dataHost 物理数据库，真正存储数据的数据库 1、schema 标签 schema标签用来定义mycat实例中的逻辑库，mycat可以有多个逻辑库，每个逻辑库都有自己的相关配置。可以使用schema标签来划分这些不同的逻辑库,如果不配置schema标签，所有表的配置会属于同一个默认的逻辑库。逻辑库的概念和MySql的database的概念一样，我们在查询两个不同逻辑库中的表的时候，需要切换到该逻辑库下进行查询。 –name 逻辑数据库名，与server.xml中的schema对应 –checkSQLschema 数据库前缀相关设置，当该值为true时，例如我们执行语句select * from TESTDB.company 。mycat会把语句修改为 select * from company 去掉TESTDB。 –sqlMaxLimit 当该值设置为某个数值时，每条执行的sql语句，如果没有加上limit语句，Mycat会自动加上对应的值。不写的话，默认返回所有的值。需要自己sql语句加limit。 2、dataNode标签 datanode标签定义了mycat中的数据节点，也就是数据分片。一个datanode标签就是一个独立的数据分片。 localhost1数据库实例上的db1物理数据库，这就组成一个数据分片，最后我们用dn1来标示这个分片。 –name 定义数据节点的名字，这个名字需要唯一。我们在table标签上用这个名字来建立表与分片对应的关系 –dataHost 用于定义该分片属于哪个数据库实例，属性与datahost标签上定义的name对应 –database 用于定义该分片属于数据库实例上 的具体库。 3、dataHost标签 这个标签直接定义了具体数据库实例，读写分离配置和心跳语句。 select user() –name 唯一标示dataHost标签，供上层使用 –maxCon 指定每个读写实例连接池的最大连接。 –minCon 指定每个读写实例连接池的最小连接，初始化连接池的大小 –balance 负载均称类型 balance=“0”：不开启读写分离机制，所有读操作都发送到当前可用的writeHost上 balance=“1”：全部的readHost与stand by writeHost参与select语句的负载均衡，简单的说，当双主双从模式（M1-S1，M2-S2 并且M1 M2互为主备），正常情况下，M2,S1,S2都参与select语句的负载均衡。 balance=“2”：所有读操作都随机的在writeHost、readHost上分发 balance=“3”：所有读请求随机的分发到writeHst对应的readHost执行，writeHost不负担读写压力。 –writeType 负载均衡类型。 writeType=“0”, 所有写操作发送到配置的第一个 writeHost，第一个挂了切到还生存的第二个writeHost，重新启动后已切换后的为准，切换记录在配置文件中:dnindex.properties . writeType=“1”，所有写操作都随机的发送到配置的 writeHost。1.5以后版本废弃不推荐。 –switchType -1不自动切换 1 默认值 自动切换 2 基于MySql主从同步的状态决定是否切换心跳语句为 show slave status 3 基于mysql galary cluster 的切换机制（适合集群） 心跳语句为 show status like ‘wsrep%’ –dbType 指定后端链接的数据库类型目前支持二进制的mysql协议，还有其他使用jdbc链接的数据库，例如：mongodb，oracle，spark等 –dbDriver 指定连接后段数据库使用的driver，目前可选的值有native和JDBC。使用native的话，因为这个值执行的是二进制的mysql协议，所以可以使用mysql和maridb，其他类型的则需要使用JDBC驱动来支持。 如果使用JDBC的话需要符合JDBC4标准的驱动jar 放到mycat\lib目录下，并检查驱动jar包中包括如下目录结构文件 META-INF\services\java.sql.Driver。 在这个文件写上具体的driver类名，例如com.mysql.jdbc.Driver writeHost readHost指定后端数据库的相关配置给mycat，用于实例化后端连接池。 –tempReadHostAvailable 如果配置了这个属性 writeHost 下面的 readHost 仍旧可用，默认 0 可配置（0、1）。 1）heartbeat标签 这个标签内指明用于和后端数据库进行心跳检查的语句。 例如：MYSQL 可以使用 select user()，Oracle 可以使用 select 1 from dual 等。 2) writeHost /readHost 标签 这两个标签都指定后端数据库的相关配置，用于实例化后端连接池。唯一不同的是，writeHost 指定写实例、readHost 指定读实例。 在一个 dataHost 内可以定义多个 writeHost 和 readHost。但是，如果 writeHost 指定的后端数据库宕机，那么这个 writeHost 绑定的所有 readHost 都将不可用。 另一方面，由于这个 writeHost 宕机，系统会自动的检测到，并切换到备用的 writeHost 上去。这两个标签的属性相同，这里就一起介绍。 –host 用于标识不同实例，一般 writeHost 我们使用M1，readHost 我们用S1。 –url 后端实例连接地址。Native：地址：端口 JDBC：jdbc的url –password 后端存储实例需要的密码 –user 后端存储实例需要的用户名字 –weight 权重 配置在 readhost 中作为读节点的权重 –usingDecrypt 是否对密码加密，默认0。 ```