ELK系列传送门

ELK系列（一） ElasticSearch 8.9.2集群搭建

ELK系列（二）Kafka集群3.4.0搭建

ELK系列（三） Kibana8.9.2搭建

ELK系列（四） Logstash8.9.2搭建

ELK系列（五） Elastalert2搭建

ELK数据流规划

ELK接入nginx-acces日志数据流规划


数据流向
		filebeat -------> kafka ----> logstash ----> ES --------> kibana
		 收集              mq          处理日志     存储、搜索     展示、分析
                                                       |
										               |
								                   elastalert2---钉钉告警


ES集群搭建已完成 可见传送门ELK系列（一） ElasticSearch 8.9.2集群搭建
Kafka集群搭建已完成 可以传送门ELK系列（二） Kafka集群3.4.0搭建
Kibana搭建已完成 可见传送门ELK系列（三） Kibana8.9.2搭建
Logstash搭建已完成 可见传送门ELK系列（四） Logstash8.9.2搭建

主机规划

主机名        内网地址      搭建程序

ELK8-1       192.168.0.1  elasticsearch+kafka+zookeeper
ELK8-2       192.168.0.2  elasticsearch+kafka+zookeeper
ELK8-3       192.168.0.3  elasticsearch+kafka+zookeeper
logstash1    192.168.0.4  kibana+logstash
syslog       192.168.0.60 elastalert2

新增主机syslog部署elastalert2
基础环境配置和系统参数优化参考前文

Elastalert2搭建

Elastalert2依赖配置

yum -y groupinstall "Development Tools" 
yum -y install openssl11 
yum -y install openssl11-devel #elastalert2依赖openssl11替换openssl
ln -sf /usr/lib64/pkgconfig/openssl11.pc /usr/lib64/pkgconfig/openssl.pc
ln -s /usr/bin/openssl11 /usr/bin/openssl
wget https://www.python.org/ftp/python/3.12.1/Python-3.12.1.tgz 

#elastalert2底层依靠 python环境配置
[root@syslog src]# yum install zlib zlib-devel  libffi-devel bzip2-devel make gcc -y
tar -zxvf Python-3.12.0.tgz  
cd Python-3.12.0/
./configure --with-ssl --prefix=/usr/local/lib/python3.12
make
make install
ln -s /usr/local/lib/python3.12/bin/python3.12  /usr/bin/python3
ln -s /usr/local/lib/python3.12/bin/pip3.12 /usr/bin/pip
pip install --upgrade pip
# cat /root/.pip/pip.conf  #pip加速
[global]
# 清华大学
index-url = https://pypi.tuna.tsinghua.edu.cn/simple/

pip install elastalert2
git clone https://github.com/jertel/elastalert2.git
$ pip install "setuptools>=11.3"
$ python3 setup.py install

[root@syslog elastalert2]# pwd
/data/elastalert2
[root@syslog elastalert2]# ls
build  CHANGELOG.md  config.yaml      dist        docs        elastalert2.egg-info  LICENSE   nohup.out  requirements-dev.txt  rules      setup.py
certs  chart         CONTRIBUTING.md  Dockerfile  elastalert  examples              Makefile  README.md  requirements.txt      setup.cfg  tests

ln -s /usr/local/lib/python3.12/bin/elastalert*  /usr/bin #提升到系统命令

systemctl纳管
[root@syslog system]# cat elastalert.service
[Unit]
Description=ElastAlert Service
After=network.target

[Service]
Type=simple
ExecStart=/usr/bin/python3 -m elastalert.elastalert --config /data/elastalert2/config.yaml
Restart=always
User=root
Group=root

[Install]
WantedBy=multi-user.target

systemctl start elastalert

Elastalert2通过SSL链接ES集群

Elastalert2通过es证书创建c端证书和key去链接es
[root@ELK8-1 elasticsearch]# ./bin/elasticsearch-keystore show xpack.security.http.ssl.keystore.secure_password
warning: ignoring JAVA_HOME=/usr/local/jdk-20.0.1; using bundled JDK
r_FBdQ9VSxasdzcyE9G9oA

[root@syslog certs]# openssl pkcs12 -in http.p12 -nocerts -nodes > http.key
Enter Import Password:
[root@syslog certs]# ls
http_ca.crt  http.key  http.p12
[root@syslog certs]# openssl pkcs12 -in http.p12 -clcerts -nokeys > http.crt
Enter Import Password:

[root@syslog elastalert2]# cd certs/
[root@syslog certs]# ls
http_ca.crt  http.crt  http.key  http.p12


[root@syslog elastalert2]# cat config.yaml | grep -v "^$"
rules_folder: /data/elastalert2/rules #告警规则目录
run_every:
  minutes: 1  #间隔60s去遍历一次规则判断
buffer_time:
  minutes: 15
es_host: 192.168.0.1  
es_host: 192.168.0.2
es_host: 192.168.0.3
es_port: 9200
use_ssl: True
verify_certs: True
ssl_show_warn: True
es_username: elastic
es_password: tIxjmD8nWabee6Rs0QRm
ca_certs: /data/elastalert2/certs/http_ca.crt
client_cert: /data/elastalert2/certs/http.crt
client_key: /data/elastalert2/certs/http.key
writeback_index: elastalert_status #告警内容会写入该索引
alert_time_limit:
  days: 2

Elastalert2告警规则

告警规则希望有大佬多交流 具体不是太会写 
# Rule Configuration for Individual IPs (Nested Rule)
name: nginx_access_4xx_individual_ips
type: frequency  
index: "gf20.54*"  #查询该索引
num_events: 100  # Individual IP 404s threshold #60s超过100触发
timeframe:
  seconds: 60
query_key: client.ip  #查询单ip的状态码字段
filter:
  - query:
      bool:
        must:
          - query_string:
              query: "response_code: [400 TO 599]"
alert:
  - dingtalk  #告警方式
alert_text_type: "alert_text_only"
dingtalk_webhook: "https://oapi.dingtalk.com/robot/send?access_token=qw125s478z12o6643fc98b204cc4d99642d3f7fa7da5400e90c21e93025955a2"
dingtalk_access_token: "qw125s478z12o6643fc98b204cc4d99642d3f7fa7da5400e90c21e93025955a2"
dingtalk_msgtype: "text"
alert_text: |    #定义告警文本匹配下列参数
  告警程序: ElasticSearch_Alert
  来源  Ip: {}
  域    名: {}
  调用方式: {}
  请求链接: {}
  触发条件: 60s 内 {} 状态码 超过 {} 次
alert_text_args:
  - client.ip
  - domain
  - http_method
  - request_path
  - response_code
  - num_events