ES8—Logstash及FileBeat

概述

本文档基于《ES8—8.5.2集群安装与配置》的es8集群配置，如果遇到ES的问题请参考该文档。该文档只完成了收集日志上传到logstash，再由logstash上传到ES集群中。至于logstash的日志过滤等操作，请参考后续文档。

Logstash

1、安装Logastash

#CentOS/RHEL/Euler
wget https://artifacts.elastic.co/downloads/logstash/logstash-8.5.2-x86_64.rpm
rpm -ivh ./logstash-8.5.2-x86_64.rpm

#Ubuntu
wget https://artifacts.elastic.co/downloads/logstash/logstash-8.5.2-amd64.deb
dpkg -i ./logstash-8.5.2-amd64.deb

更新http插件

bin/logstash-plugin update logstash-input-http

2、Logstash连接TLS加密下的ES

（1）、配置logstash.yml

vim /etc/logstash/logstash.yml

path.data: /logstash_data/data
path.logs: /logstash_data/logs
http.host: "0.0.0.0"

#一定要注意权限问题，否则会导致服务无法启动
chown -R logstash:logstash logstash_data

（2）、拷贝http.p12证书到logstash节点

mkdir /etc/logstash/certs
cp http.p12 config/certs

• http.p12是构建es集群时，创建https生成的密钥

（3）、定义logstash向es输出

mkdir /etc/logstash/conf
vim /etc/logstash/conf/elastsearch.cfg

output {
  elasticsearch {
    hosts => ["https://192.168.2.64:9200","https://192.168.2.65:9200","https://192.168.2.66:9200"]
    index => "test"
    codec => "plain"
    user => "elastic"
    password => "tdjgamtam"
    ssl_certificate_verification => true
    truststore => "/etc/logstash/certs/http.p12"
    truststore_password => "Gn@2028."
  }
}

• truststore_password：这是构建es集群中生成的http证书的密码

（4）、配置filebeat为输入

vim /etc/logstash/conf/filebeat.cfg

input {
  beats {
    port => 5044
    codec => "plain"
  }
}

（5）、配置pipeline.yml

vim /etc/logstash/pipelines.yml

- pipeline.id: my-pipeline_1
  path.config: "/opt/logstash-8.5.2/config/conf/{filebeat,elastsearch}.cfg"

3、配置并启动logstash服务

systemctl enable logstash
systemctl start logstash

Filebeat配置

vim filebeat.yml

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /home/cloud/logs/*

#document_type: apache-access
#fields_under_root: true

output.logstash:
  hosts: ["192.168.3.71:5044"]