SaltStack实践案例一

通过saltstack的配置管理来实现一个“中小型web架构”的自动化部署和配置管理，主要包括以下功能和服务：

系统初始化

haproxy服务

keepalived服务

nginx服务

php（fastcgi）服务

memcached服务

按照本案例的思路，我们将按照系统初始化、功能模块化、业务模块这样的设计思路来进行设计和实施：

系统初始化：指操作系统安装完毕之后，需要使用到的初始配置，比如安装监控代理、调整内核参数、设置域名解析等

功能模块：指的是生产用到的应用，比如nginx、php、haproxy、keepalived等这类应用服务的安装和管理，每一个功能完美创建一个目录来存放，我们把这个目录的集合称之为“功能模块”

业务模块：在功能模块中我们编写了大量基础的功能状态，在业务层面直接进行引用，所以功能模块就是尽可能的全、而且独立。而业务模块，不同的业务类型就可以在include功能模块里面的安装和部署，每个业务使用自己独特的配置文件等。最终在top.sls里面我们只需要给某个minion指定一个业务的状态即可。

一、环境规划

环境规划包含实验环境规划saltstack环境。

1.实验环境：

salt-master-1.example.com10.0.241.122master

salt-minion-1.example.com10.0.241.123minion、haproxy＋keepalived、nginx＋php

salt-minion-2.example.com10.0.241.124minion、memcached、haproxy＋keepalived、nginx＋php

2.saltstack环境配置

本例子有两个环境base和prod，base环境用来存放初始化的功能。prod环境用于放置生产的配置管理功能：

[root@salt-master-1~]#vim/etc/salt/masterfile_roots:base:-/srv/salt/baseprod:-/srv/salt/prodpillar_roots:base:-/srv/pillar/baseprod:-/srv/pillar/prod[root@salt-master-1~]#mkdir-p/srv/salt/{base,prod}[root@salt-master-1~]#mkdir-p/srv/pillar/{base,prod}[root@salt-master-1~]#systemctlrestartsalt-master.service

二、系统初始化

1.dns配置

[root@salt-master-1~]#cat/srv/salt/base/init/dns.sls/etc/resolv.conf:file.managed:-source:salt://init/files/resolv.conf-user:root-group:root-mode:644#把准备好的resolv.conf放置在/srv/salt/base/init/files/目录下

2.history记录时间

[root@salt-master-1~]#cat/srv/salt/base/init/history.sls/etc/profile:file.append:-text:-exporthisttimeformat="%f%t`whoami`"

3.命令操作审计

[root@salt-master-1~]#cat/srv/salt/base/init/audit.sls/etc/bashrc:file.append:-text:-exportprompt_command='{msg=$(history1|{readxy;echo$y;});logger"[euid=$(whoami)]":$(whoami):['prod']"$msg";}'

4.内核参数优化

[root@salt-master-1~]#cat/srv/salt/base/init/sysctl.slsnet.ipv4.ip_local_port_range:sysctl.present:-value:1000065000fs.file_max:sysctl.present:-value:2000000net.ipv4.ip_forward:sysctl.present:-value:1vm.swappiness:sysctl.present:-value:0

5.epel仓库

[root@salt-master-1~]#cat/srv/salt/base/init/epel.slsyum_repo_release:pkg.installed:-sources:-epel-release:http://mirrors.aliyun.com/epel/7/x86_64/e/epel-release-7-5.noarch.rpm-unless:rpm-qa|grepepel-release-7-5

6.zabbix_agentd安装

通过使用pillar来设置zabbixserver的ip地址：

[root@salt-master-1~]#cat/srv/salt/base/init/top.slsbase:'*':-zabbix[root@salt-master-1~]#cat/srv/pillar/base/zabbix.slszabbix-agent:zabbix_server:10.0.241.122

安装并启动zabbixagent：

[root@salt-master-1~]#cat/srv/salt/base/init/zabbix_agent.slszabbix-agent:pkg.installed:-name:zabbix22-agentfile.managed:-name:/etc/zabbix_agentd.conf-source:salt://zabbix/files/zabbix_agentd.conf-template:jinja-defaults:server:{{pillar['zabbix-agent']['zabbix_server']}}-require:-pkg:zabbix-agentservice.running:-enable:true-watch:-pkg:zabbix-agent-file:zabbix-agent

[root@salt-master-1~]#cat/srv/salt/base/init/env_init.slsinclude:-init.dns-init.history-init.audit-init.sysctl-init.epel-init.zabbix_agent[root@salt-master-1~]#cat/srv/salt/base/top.slsbase:'*':-init.env_init#在服务器上执行[root@salt-master-1~]#salt'salt-minion-1'state.highstatetest=true

三、haproxy配置管理

haproxy是一个开源的高性能的反向代理项目，支持四层和七层的负载均衡，多种负载均衡算法和健康检查等。

keepalived是一个高可用集群的项目，它是vrrp协议的完美实现，我们通过keepalived来管理haproxy上面的vip。当主haproxy发生故障时，将vip漂移到备用的haproxy上来继续提供服务。

[root@salt-master-1~]#mkdir/srv/salt/prod/pkg-p[root@salt-master-1~]#mkdir/srv/salt/prod/haproxy/files-p[root@salt-master-1~]#mkdir/srv/salt/prod/keepalived/files-p#在每个服务的目录下面均创建一个files目录用来存放源码包和需要的相关启动脚本、配置文件等。

1.pkg配置

[root@salt-master-1~]#cat/srv/salt/prod/pkg/pkg-init.slspkg-init:pkg.installed:-pkgs:#注意-gcc-gcc-c++-glibc-make-autoconf-openssl-openssl-devel

2.haproxy服务配置

[root@salt-master-1~]#cd/usr/local/src/&&wgethttp://www.haproxy.org/download/1.6/src/haproxy-1.6.2.tar.gz&&tarzxfhaproxy-1.6.2.tar.gz&&cdhaproxy-1.6.2/examples/[root@salt-master-1examples]#sed-i's//usr/sbin/'$basename'//usr/local/haproxy/sbin/'$basename'/g'haproxy.init#修改haproxy的启动脚本[root@salt-master-1examples]#cphaproxy.init/srv/salt/prod/haproxy/files/

编写haproxy代码如下：

[root@salt-master-1examples]#cat/srv/salt/prod/haproxy/install.slsinclude:-pkg.pkg-inithaproxy-install:file.managed:-name:/usr/local/src/haproxy-1.6.2.tar.gz-source:salt://haproxy/files/haproxy-1.6.2.tar.gz-mode:755-user:root-group:rootcmd.run:-name:cd/usr/local/src/&&tarzxfhaproxy-1.6.2.tar.gz&&maketarget=linux26prefix=/usr/local/haproxy&&makeinstallprefix=/usr/local/haproxy-unless:test-d/usr/local/haproxy-require:-pkg:pkg-init-file:haproxy-install/etc/init.d/haproxy:file.managed:-source:salt://haproxy/files/haproxy.init-mode:755-user:root-group:root-require:-cmd:haproxy-installnet.ipv4.ip_nolocal_bind:sysctl.present:-value:1haproxy-config-dir:file.directory:-name:/etc/haproxy-mode:755-user:root-group:roothaproxy-init:cmd.run:-name:chkconfig--addhaproxy-unless:chkconfig--list|grephaproxy-require:-file:/etc/init.d/haproxy

管理haproxy的配置文件有两种方法：

1).直接在需要使用haproxy的地方引用haproxy的安装，然后加入haproxy的配置文件管理和服务管理。优点：简单明了；缺点：不够灵活通用。

2).使用jinja模版，将haproxy的基础配置编写完成后，其他的配置通过pillar来进行自动生成。优点：非常灵活通用；缺点：由于需要使用大量的if、for等jinja模版语法，而且需要配置pillarlai实现配置，比较复杂，有难度，容易出错。

3.haproxy业务引用

我们现在切换功能服务配置外，编写一个业务模块cluster，然后调用haproxy来完成配置管理。这样做的好处是把基础服务的配置管理和业务分开。

创建cluster目录，并且在cluster目录创建files目录，用来存放配置文件：

[root@salt-master-1~]#mkdir-p/srv/salt/prod/cluster/files[root@salt-master-1~]#cat/srv/salt/prod/cluster/files/haproxy-outside.cfgglobalmaxconn100000chroot/usr/local/haproxyuid99gid99daemonnbproc1pidfile/usr/local/haproxy/logs/haproxy.pidlog127.0.0.1local3info#默认参数设置defaultsoptionhttp-keep-alivemaxconn100000modehttptimeoutconnect5000mstimeoutclient5000mstimeoutserver5000ms#开启haproxystatus状态监控，增加验证listenstatsmodehttpbind0.0.0.0:8888statsenablestatsuri/haproxy-statusstatsauthhaproxy:saltstack#前端设置frontendfrontend_www_example_combind10.0.241.123:80modehttpoptionhttploglogglobaldefault_backendbackend_www_example_com#后端设置backendbackend_www_example_comoptionforwardforheaderx-real-ipoptionhttpchkhead/http/1.0balancesourceserverweb-node110.0.241.123:8080checkinter2000rise30fall15serverweb-node110.0.241.124:8080checkinter2000rise30fall15

编写haproxy的服务管理：

[root@salt-master-1~]#cat/srv/salt/prod/cluster/haproxy-outside.slsinclude:-haproxy.installhaproxy-service:file.managed:-name:/etc/haproxy/haproxy.cfg-source:salt://cluster/files/haproxy-outside.cfg-user:root-group:root-mode:644service.running:-name:haproxy-enable:true-reload:true-require:-cmd:haproxy-init-watch:-file:haproxy-service

4.执行haproxy状态

[root@salt-master-1~]#cat/srv/salt/base/top.slsbase:'*':-init.env_initprod:'*':-cluster.haproxy-outside#[root@salt-master-1prod]#salt'salt-minion-1'state.highstatetest=true

四、keepalived配置管理

首先放置源码包、keepalived的启动脚本、sysconfig配置文件在/srv/salt/prod/keepalived/files/目录下。启动脚本和配置文件都可以从源码包中获取到。

1.软件包准备

[root@salt-master-1~]#cd/usr/local/src/&&wget&&cpkeepalived-1.2.19.tar.gz/srv/salt/prod/keepalived/files/&&tarzxfkeepalived-1.2.19.tar.gz&&cdkeepalived-1.2.19/&&cpkeepalived/etc/init.d/keepalived.init/srv/salt/prod/keepalived/files/&&cpkeepalived/etc/init.d/keepalived.sysconfig/srv/salt/prod/keepalived/files/[root@salt-master-1keepalived-1.2.19]#vim/srv/salt/prod/keepalived/files/keepalived.init将daemonkeepalived${keepalived_options}修改为daemon/usr/local/keepalived/sbin/keepalived${keepalived_options}

2.编写keepalived安装sls

[root@salt-master-1keepalived]#catinstall.slskeepalived-install:file.managed:-name:/usr/local/src/keepalived-1.2.19.tar.gz-source:salt://keepalived/files/keepalived-1.2.19.tar.gz-mode:755-user:root-group:rootcmd.run:-cmd:cd/usr/local/src/&&tarzxfkeepalived-1.2.19.tar.gz&&cdkeepalived-1.2.19&&./configure--prefix=/usr/local/keepalived--disable-fwmark&&makeinstall-unless:test-d/usr/local/keepalived-require:file:keepalived-install#keepalived的sysconfig配置文件/etc/sysconfig/keepalived:file.managed:-source:salt://keepalived/files/keepalived.sysconfig-mode:644-user:root-group:root#keepalived的服务管理脚本/etc/init.d/keepalived:file.managed:-source:salt://keepalived/files/keepalived.init-mode:755-user:root-group:root#keepalived加入系统服务管理keepalived-init:cmd.run:-name:chkconfig--addkeepalived-unless:chkconfig--list|grepkeepalived-require:-file:/etc/init.d/keepalived#keepalived的配置文件目录如下/etc/keepalived:file.directory:-user:root-group:root

3.keepalived业务引用

首先和haproxy一样，我们需要有一个keepalived的配置文件，不过这次配置文件和haproxy稍有不同，因为keepalived分为主、备节点，一些配置在主节点和备节点上是不同的。我们需要使用jinja模版来完成配置文件的管理。

[root@salt-master-1keepalived]#cat/srv/salt/prod/cluster/files/haproxy-outside-keepalived.conf!configurationfileforkeepalivedglobal_defs{notification_email{saltstack@example.com}notification_email_fromkeepalived@example.comsmtp_server127.0.0.1smtp_connect_timeout30route_id{{routeid}}}vrrp_instancehaproxy_ha{state{{stateid}}interfaceeth0virtual_router_id36priority{{priorityid}}advert_int1authentication{auth_typepassauth_pass1111}virtual_ipaddress{10.0.241.123}}

cluster业务目录下编写haproxy使用keepalived做高可用的sls：

[root@salt-master-1keepalived]#cat/srv/salt/prod/cluster/haproxy-outside-keepalived.slsinclude:-keepalived.installkeepalived-server:file.managed:-name:/etc/keepalived/keepalived.conf-source:salt://cluster/files/haproxy-outside-keepalived.conf-mode:644-user:root-group:root-template:jiaja{%ifgrains['fqdn']=='salt-minion-1.example.com'%}-routeid:haproxy_ha-stateid:master-priorityid:150{%elifgrains['fqdn']=='salt-minion-2.example.com'%}-routeid:haproxy_ha-stateid:backup-priorityid:100{%endif%}service.running:-name:keepalived-enable:true-watch:-file:keepalived-server

4.执行keepalived状态

[root@salt-master-1keepalived]#cat/srv/salt/base/top.slsbase:'*':-init.env_init-pkg-initprod:'*':-cluster.haproxy-outside-cluster.haproxy-outside-keepalived