437 lines
14 KiB
Markdown
437 lines
14 KiB
Markdown
|
# Nginx动静分离+负载均衡
|
|||
|
|
|
|||
|
|
x86
|
|||
|
|
|
|||
|
|
windows linux 部分unix
|
|||
|
|
|
|||
|
|
小型机
|
|||
|
|
|
|||
|
|
IBM HP 安装不了普通的操作系统（windows linux）
|
|||
|
|
|
|||
|
|
只能安装厂商自身提供的UNIX系统
|
|||
|
|
|
|||
|
|
常见的Unix系统：IBM的AIX HP的HP-UX 这些系统只能安装在他们自身厂商的小型机上
|
|||
|
|
|
|||
|
|
大型机
|
|||
|
|
|
|||
|
|
国企 还不是一般的国企 搞科研 真正的高科技 航天 军工 导弹
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
集群的分类：cluster
|
|||
|
|
高可用集群 HA high availability
|
|||
|
|
|
|||
|
|
避免单节点故障
|
|||
|
|
软件：keepalived
|
|||
|
|
|
|||
|
|
负载均衡集群 **LB** **load balance**
|
|||
|
|
提高负载，提高并发量
|
|||
|
|
软件：nginx反向代理(7层) lvs(4层) haproxy(7层)
|
|||
|
|
硬件：硬件负载均衡器 F5(BigIP)和redware
|
|||
|
|
|
|||
|
|
|
|||
|
|
HPC高性能运算集群
|
|||
|
|
|
|||
|
|
分布式存储集群
|
|||
|
|
极大的提升存储容量，提供数据高可用，保证数据安全
|
|||
|
|
软件：ceph glusterfs hdfs
|
|||
|
|
|
|||
|
|
存储分类：NAS SAN DAS
|
|||
|
|
|
|||
|
|
Nginx proxy 是 Nginx 的王牌功能,利用 proxy 基本可以实现一个完整的 7 层负载均衡。
|
|||
|
|
|
|||
|
|
1. 功能强大,性能卓越,运行稳定。
|
|||
|
|
|
|||
|
|
2. 配置简单灵活。
|
|||
|
|
|
|||
|
|
3. 能够自动剔除工作不正常的后端服务器。
|
|||
|
|
|
|||
|
|
4. 上传文件使用异步模式。
|
|||
|
|
|
|||
|
|
5. 支持多种分配策略,可以分配权重,分配方式灵活。
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
lvs和nginx组合代理负载均衡 面试题(lvs和nginx的区别)
|
|||
|
|
|
|||
|
|
nginx用来做http的反向代理，能够upsteam实现http请求的多种方式的均衡转发。由于采用的是**异步转发可以做到如果一个服务器请求失败，立即切换到其他服务器，直到请求成功或者最后一台服务器失败为止**。这可以最大程度的提高系统的请求成功率。
|
|||
|
|
lvs采用的是**同步请求转发**的策略。这里说一下同步转发和异步转发的区别。同步转发是在lvs服务器接收到请求之后，立即redirect到一个后端服务器，由客户端直接和后端服务器建立连接。**异步转发**是nginx在保持客户端连接的同时，发起一个相同内容的新请求到后端，等后端返回结果后，由nginx返回给客户端。
|
|||
|
|
进一步来说：当做为负载均衡服务器的nginx和lvs处理相同的请求时，所有的请求和响应流量都会经过nginx；但是使用lvs时，仅请求流量经过lvs的网络，响应流量由后端服务器的网络返回。
|
|||
|
|
也就是，当作为后端的服务器规模庞大时，nginx的网络带宽就成了一个巨大的瓶颈。
|
|||
|
|
但是仅仅使用lvs作为负载均衡的话，一旦后端接受到请求的服务器出了问题，那么这次请求就失败了。但是如果在lvs的后端在添加一层nginx（多个），每个nginx后端再有几台应用服务器，那么结合两者的优势，既能避免单nginx的流量集中瓶颈，又能避免单lvs时一锤子买卖的问题。
|
|||
|
|
|
|||
|
|
lvs/dr 同步传输
|
|||
|
|
nginx 异步传输
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
**拓扑**
|
|||
|
|
|
|||
|
|
[]()
|
|||
|
|
|
|||
|
|
**环境**
|
|||
|
|
|
|||
|
|
1. 修改主机名称
|
|||
|
|
|
|||
|
|
2. 名称解析
|
|||
|
|
|
|||
|
|
3. ip互通 所有机器全部在同一个网段
|
|||
|
|
|
|||
|
|
4. 关闭防火墙和selinux
|
|||
|
|
|
|||
|
|
5 . HTML A & HTML B
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
[root@localhost ~]# yum install httpd -y
|
|||
|
|
分别创建测试页面 index.html 开启服务
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
6. PHP A & PHP B
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
[root@localhost ~]# yum install httpd php -y
|
|||
|
|
分别创建测试页面 index.php 开启服务
|
|||
|
|
<?php
|
|||
|
|
print "hello A";
|
|||
|
|
phpinfo();
|
|||
|
|
?>
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
7. 测试4台真实服务器的页面是否能正常访问
|
|||
|
|
|
|||
|
|
8. 安装配置负载均衡器上的Nginx（负载均衡器、分发器、反向代理）
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# yum install nginx -y
|
|||
|
|
# vim /etc/nginx/nginx.conf
|
|||
|
|
location / {
|
|||
|
|
root /usr/share/nginx/html;
|
|||
|
|
index index.html index.htm;
|
|||
|
|
if ($request_uri ~* \.html$) {
|
|||
|
|
proxy_pass http://htmlserver;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
if ($request_uri ~* \.php$) {
|
|||
|
|
proxy_pass http://phpserver;
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
# vim /etc/nginx/conf.d/test.conf
|
|||
|
|
upstream htmlserver {
|
|||
|
|
server 192.168.122.10;
|
|||
|
|
server 192.168.122.20;
|
|||
|
|
}
|
|||
|
|
upstream phpserver {
|
|||
|
|
server 192.168.122.30;
|
|||
|
|
server 192.168.122.40;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
# systemctl restart nginx
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
在客户端访问 Nginx 测试
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# elinks --dump http:// 192.168.122.254
|
|||
|
|
# elinks --dump http:// 192.168.122.254/index.html
|
|||
|
|
# elinks --dump http:// 192.168.122.254/index.php
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
|
|||
|
|
**upstream支持的负载均衡算法**(面试题)
|
|||
|
|
**轮询（默认）**: 可以通过weight指定轮询的权重，权重越大，被调度的次数越多 **rr round robin**
|
|||
|
|
权重:用数字 谁数字大谁权重就高 按比例 1 3
|
|||
|
|
rr
|
|||
|
|
wrr
|
|||
|
|
**ip_hash**： 根据每个请求IP进行调度，可以解决session的问题，不能使用weight
|
|||
|
|
client_ip 192.168.1.8 nginx反向 webserver1
|
|||
|
|
fair： 可以根据请求页面的大小和加载时间长短进行调度，使用第三方的upstream_fair模块
|
|||
|
|
url_hash： 按请求的url的hash进行调度，从而使每个url定向到同一服务器，使用第三方的hash模块
|
|||
|
|
|
|||
|
|
**upstream支持的状态参数**
|
|||
|
|
|
|||
|
|
```
|
|||
|
|
down： 暂停对该服务器的调度
|
|||
|
|
backup: 类似于LVS Sorry Server，当所有的非backup的服务器故障
|
|||
|
|
max_fails: 请求失败的次数，默认为1
|
|||
|
|
fail_timeout: 在经历max_fails次失败后，暂停服务的时间
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
upstream htmlservers {
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# ip_hash;
|
|||
|
|
server 192.168.10.137 weight=1 max_fails=2 fail_timeout=2;
|
|||
|
|
server 192.168.10.20 weight=2 max_fails=2 fail_timeout=2;
|
|||
|
|
server 192.168.10.251 max_fails=2 fail_timeout=5 **down**;
|
|||
|
|
server 192.168.10.253 backup;
|
|||
|
|
}
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
注：当使用ip_hash时，服务器状态不可使用weight和backup
|
|||
|
|
|
|||
|
|
**Nginx实现七层的负载均衡**
|
|||
|
|
|
|||
|
|
调度到同一组上游服务器
|
|||
|
|
|
|||
|
|
**拓扑结构**
|
|||
|
|
|
|||
|
|
[**LB Nginx**]
|
|||
|
|
192.168.1.2
|
|||
|
|
|
|||
|
|
[**httpd**] [**httpd**] [**httpd**]
|
|||
|
|
192.168.1.3 192.168.1.4 192.168.1.5
|
|||
|
|
|
|||
|
|
**实施过程**
|
|||
|
|
\1. nginx
|
|||
|
|
http {
|
|||
|
|
**upstream httpservers {
|
|||
|
|
server 192.168.1.3**:80 weight=1 max_fails=2 fail_timeout=2;
|
|||
|
|
**server 192.168.1.4**:80 weight=2 max_fails=2 fail_timeout=2;
|
|||
|
|
server 192.168.1.5:80 weight=2 max_fails=2 fail_timeout=2;
|
|||
|
|
server 192.168.1.100:80 backup;
|
|||
|
|
**}**
|
|||
|
|
|
|||
|
|
**location / {
|
|||
|
|
proxy_pass** http://httpservers;
|
|||
|
|
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
|
|||
|
|
**}**
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
proxy_next_upstream：这个指令属于 http_proxy 模块的，指定后端返回什么样的异常响应时，使用另一个realserver
|
|||
|
|
|
|||
|
|
\2. Apache LogFormat 可选
|
|||
|
|
LogFormat "%{X-Real-IP}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
|
|||
|
|
LogFormat "%h %l %u %t \"%r\" %>s %b" common
|
|||
|
|
LogFormat "%{Referer}i -> %U" referer
|
|||
|
|
LogFormat "%{User-agent}i" agent
|
|||
|
|
|
|||
|
|
=================================================================================
|
|||
|
|
**Nginx实现七层的负载均衡**
|
|||
|
|
|
|||
|
|
调度到不同组上游服务器
|
|||
|
|
\1. 动静分离
|
|||
|
|
\2. 网站进行分区
|
|||
|
|
=================================================================================
|
|||
|
|
|
|||
|
|
**拓扑结构**
|
|||
|
|
|
|||
|
|
[**vip: 192.168.1.80**]
|
|||
|
|
|
|||
|
|
[**LB1 Nginx**] [**LB2 Nginx**]
|
|||
|
|
192.168.1.2 192.168.1.3
|
|||
|
|
|
|||
|
|
[**news**] [**milis**] [**videos**] [**images**] [**others**]
|
|||
|
|
1.11 1.21 1.31 1.41 1.51
|
|||
|
|
1.12 1.22 1.32 1.42 1.52
|
|||
|
|
1.13 1.23 1.33 1.43 1.53
|
|||
|
|
... ... ... ... ...
|
|||
|
|
|
|||
|
|
**一、实施过程**
|
|||
|
|
\1. 根据站点分区进行调度
|
|||
|
|
http {
|
|||
|
|
upstream news {
|
|||
|
|
server 192.168.1.11:80 weight=1 max_fails=2 fail_timeout=2;
|
|||
|
|
server 192.168.1.12:80 weight=2 max_fails=2 fail_timeout=2;
|
|||
|
|
server 192.168.1.13:80 weight=2 max_fails=2 fail_timeout=2;
|
|||
|
|
}
|
|||
|
|
upstream milis {
|
|||
|
|
server 192.168.1.21:80 weight=1 max_fails=2 fail_timeout=2;
|
|||
|
|
server 192.168.1.22:80 weight=2 max_fails=2 fail_timeout=2;
|
|||
|
|
server 192.168.1.23:80 weight=2 max_fails=2 fail_timeout=2;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
upstream videos {
|
|||
|
|
server 192.168.1.31:80 weight=1 max_fails=2 fail_timeout=2;
|
|||
|
|
server 192.168.1.32:80 weight=2 max_fails=2 fail_timeout=2;
|
|||
|
|
server 192.168.1.33:80 weight=2 max_fails=2 fail_timeout=2;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
upstream images {
|
|||
|
|
server 192.168.1.41:80 weight=1 max_fails=2 fail_timeout=2;
|
|||
|
|
server 192.168.1.42:80 weight=2 max_fails=2 fail_timeout=2;
|
|||
|
|
server 192.168.1.43:80 weight=2 max_fails=2 fail_timeout=2;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
upstream others {
|
|||
|
|
server 192.168.1.51:80 weight=1 max_fails=2 fail_timeout=2;
|
|||
|
|
server 192.168.1.52:80 weight=2 max_fails=2 fail_timeout=2;
|
|||
|
|
server 192.168.1.53:80 weight=2 max_fails=2 fail_timeout=2;
|
|||
|
|
}
|
|||
|
|
server {
|
|||
|
|
location / {
|
|||
|
|
proxy_pass http://others;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
location /news {
|
|||
|
|
proxy_pass http://news;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
location /mili {
|
|||
|
|
proxy_pass http://milis;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
location ~* \.(wmv|mp4|rmvb)$ {
|
|||
|
|
proxy_pass http://videos;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
location ~* \.(png|gif|jpg)$ {
|
|||
|
|
proxy_pass http://images;
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
\2. 根据动静分离进行调度
|
|||
|
|
http {
|
|||
|
|
upstream htmlservers {
|
|||
|
|
server 192.168.1.3:80 weight=1 max_fails=2 fail_timeout=2;
|
|||
|
|
server 192.168.1.4:80 weight=2 max_fails=2 fail_timeout=2;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
upstream phpservers {
|
|||
|
|
server 192.168.1.3:80 weight=1 max_fails=2 fail_timeout=2;
|
|||
|
|
server 192.168.1.4:80 weight=2 max_fails=2 fail_timeout=2;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
server {
|
|||
|
|
location ~* \.html$ {
|
|||
|
|
proxy_pass http://htmlservers;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
location ~* \.php$ {
|
|||
|
|
proxy_pass http://phpservers;
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
二、Keepalived实现调度器HA
|
|||
|
|
注：主/备调度器均能够实现正常调度
|
|||
|
|
\1. 主/备调度器安装软件
|
|||
|
|
[root@master ~]# yum -y install ipvsadm keepalived
|
|||
|
|
[root@backup ~]# yum -y install ipvsadm keepalived
|
|||
|
|
|
|||
|
|
\2. Keepalived
|
|||
|
|
**Master**
|
|||
|
|
\# vim /etc/keepalived/keepalived.conf
|
|||
|
|
! Configuration File for keepalived
|
|||
|
|
|
|||
|
|
global_defs {
|
|||
|
|
router_id director1 //辅助改为director2
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
vrrp_instance VI_1 {
|
|||
|
|
state BACKUP
|
|||
|
|
nopreempt
|
|||
|
|
interface eth0 //心跳接口，尽量单独连接心跳
|
|||
|
|
virtual_router_id 80 //MASTER,BACKUP一致
|
|||
|
|
priority 100 //辅助改为50
|
|||
|
|
advert_int 1
|
|||
|
|
authentication {
|
|||
|
|
auth_type PASS
|
|||
|
|
auth_pass 1111
|
|||
|
|
}
|
|||
|
|
virtual_ipaddress {
|
|||
|
|
192.168.1.80
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
**BACKUP**
|
|||
|
|
\3. 启动KeepAlived（主备均启动）
|
|||
|
|
\# chkconfig keepalived on
|
|||
|
|
\# service keepalived start
|
|||
|
|
\# ip addr
|
|||
|
|
|
|||
|
|
\4. 扩展对调度器Nginx健康检查（可选）
|
|||
|
|
思路：
|
|||
|
|
让Keepalived以一定时间间隔执行一个外部脚本，脚本的功能是当Nginx失败，则关闭本机的Keepalived
|
|||
|
|
a. script
|
|||
|
|
[root@master ~]# cat /etc/keepalived/check_nginx_status.sh
|
|||
|
|
\#!/bin/bash
|
|||
|
|
/usr/bin/curl -I http://localhost &>/dev/null
|
|||
|
|
if [ $? -ne 0 ];then
|
|||
|
|
/etc/init.d/keepalived stop
|
|||
|
|
fi
|
|||
|
|
[root@master ~]# chmod a+x /etc/keepalived/check_nginx_status.sh
|
|||
|
|
|
|||
|
|
b. keepalived使用script
|
|||
|
|
! Configuration File for keepalived
|
|||
|
|
|
|||
|
|
global_defs {
|
|||
|
|
router_id director1
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
vrrp_script check_nginx {
|
|||
|
|
script "/etc/keepalived/check_nginx_status.sh"
|
|||
|
|
interval 5
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
vrrp_instance VI_1 {
|
|||
|
|
state BACKUP
|
|||
|
|
interface eth0
|
|||
|
|
nopreempt
|
|||
|
|
virtual_router_id 90
|
|||
|
|
priority 100
|
|||
|
|
advert_int 1
|
|||
|
|
authentication {
|
|||
|
|
auth_type PASS
|
|||
|
|
auth_pass tianyun
|
|||
|
|
}
|
|||
|
|
virtual_ipaddress {
|
|||
|
|
192.168.1.80
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
track_script {
|
|||
|
|
check_nginx
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
\--------------------
|
|||
|
|
proxy_pass
|
|||
|
|
后端服务器用的非php独立进程
|
|||
|
|
apache+php模块
|
|||
|
|
fastcgi_pass
|
|||
|
|
后端服务器用的是php-fpm
|
|||
|
|
php-fpm(fastcgi形式的php)
|
|||
|
|
|
|||
|
|
后端服务器部署详细过程：
|
|||
|
|
安装软件：
|
|||
|
|
\# yum install nginx php php-fpm -y
|
|||
|
|
\# vim /etc/nginx/nginx.conf //添加php配置
|
|||
|
|
在server里面添加如下配置：
|
|||
|
|
location ~ \.php$ {
|
|||
|
|
root html;
|
|||
|
|
fastcgi_pass 127.0.0.1:9000;
|
|||
|
|
fastcgi_index index.php;
|
|||
|
|
fastcgi_param SCRIPT_FILENAME **$document_root**$fastcgi_script_name;
|
|||
|
|
include fastcgi_params;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
修改php-fpm进程账户并开启php-fpm的进程： 端口是9000
|
|||
|
|
\#vim /etc/php-fpm.d/www.conf //修改如下参数，默认值是apache
|
|||
|
|
user = nginx
|
|||
|
|
group = nginx
|
|||
|
|
|
|||
|
|
为什么设置成nginx:
|
|||
|
|
因为nginx.conf配置的账户为nginx
|
|||
|
|
|
|||
|
|
\# systemctl start php-fpm
|
|||
|
|
|
|||
|
|
|
|||
|
|
前端nginx反向代理服务器：
|
|||
|
|
upstream web {
|
|||
|
|
server 10.0.0.21;
|
|||
|
|
server 10.0.0.22;
|
|||
|
|
}
|
|||
|
|
upstream phpserver {
|
|||
|
|
server 10.0.0.23;
|
|||
|
|
server 10.0.0.24;
|
|||
|
|
} #上面的配置写到http里面server外面
|
|||
|
|
server {
|
|||
|
|
listen 80;
|
|||
|
|
server_name www.baidu.com;
|
|||
|
|
location / { #html的配置
|
|||
|
|
proxy_pass http://web;
|
|||
|
|
}
|
|||
|
|
|
|||
|
|
location ~* \.php$ { #php的配置
|
|||
|
|
proxy_pass http://phpserver;
|
|||
|
|
}
|