ElasticSearch + xpack 使用

02-05

ElasticSearch 是一個高可用開源全文檢索和分析組件。提供存儲服務，搜索服務，大數據准實時分析等。一般用於提供一些提供複雜搜索的應。我們為什麼要選擇 ElasticSearch ？因為它是一個准實時的搜索工具，在一般情況下延時少於一秒，它還支持物理上的水平擴展，並擁有一套分散式協調的管理功能操作比較簡單，包括一些 restful 風格的API 等等，接下來我們就來進入今天的正題。

前期準備

1.安裝 jdk1.8

yum install javan

2.配置源

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearchn配置elasticsearchncat >/etc/yum.repos.d/elasticsearch.repo <<EOFn[elasticsearch-5.x]nname=Elasticsearch repository for 5.x packagesnbaseurl=https://artifacts.elastic.co/packages/5.x/yumngpgcheck=1ngpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearchnenabled=1nautorefresh=1ntype=rpm-mdnEOFn配置kibanancat >/etc/yum.repos.d/kibana.repo << EOFn[kibana-5.x]nname=Kibana repository for 5.x packagesnbaseurl=https://artifacts.elastic.co/packages/5.x/yumngpgcheck=1ngpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearchnenabled=1nautorefresh=1ntype=rpm-mdnEOFn

3.安裝

yum install elasticsearch -ynyum install kibana -yn

4.修改 es 配置文件

mkdir /data/es-data/logs -pnchown -R elasticsearch.elasticsearch /data/es-data/logsn[root@linux-node2 ~]# grep -v "^#" /etc/elasticsearch/elasticsearch.yml |grep -v "^$"ncluster.name: lxnnode.name: linux-node2.lx.comnpath.data: /data/es-datanpath.logs: /data/es-data/logsnhttp.port: 9200ndiscovery.zen.ping.unicast.hosts: ["192.168.56.12", "192.168.56.14"]ndiscovery.zen.minimum_master_nodes: 1n

5.修改內核參數

vim /etc/security/limits.confn* soft nofile 655350n* hard nofile 655350nvim /etc/sysctl.confnfs.file-max=655350n調整內存大小ncat /data/elasticsearch/config/jvm.options (可設置為物理內存的一半)n-Xms8gn-Xmx8gnsysctl -w vm.max_map_count=262144n

6.修改 kibana 配置文件

[root@linux-node5 elasticsearch]# grep -v "^#" /etc/kibana/kibana.yml |grep -v "^$"nserver.port: 5601nserver.host: "0.0.0.0"nserver.name: "lx"nelasticsearch.url: "http://192.168.56.14:9200"nelasticsearch.username: "elastic"nelasticsearch.password: "changeme"n

7.x-pack 破解

編輯配置文件

cat LicenseVerifier.java npackage org.elasticsearch.license;nimport java.nio.*;nimport java.util.*;nimport java.security.*;nimport org.elasticsearch.common.xcontent.*;nimport org.apache.lucene.util.*;nimport org.elasticsearch.common.io.*;nimport java.io.*;npublic class LicenseVerifiern{npublic static boolean verifyLicense(final License license, final byte[] encryptedPublicKeyData) {nreturn true;n}npublic static boolean verifyLicense(final License license) {nreturn true;n}n}n

安裝 java-devel

yum install java-devel -yn編譯生成LicenseVerifier.classnjavac -cp "/usr/share/elasticsearch/lib/elasticsearch-5.6.4.jar:/usr/share/elasticsearch/lib/lucene-core-6.6.1.jar:/usr/share/elasticsearch/plugins/x-pack/x-pack-5.6.4.jar" LicenseVerifier.javan查看生成的LicenseVerifier.class類文件nll LicenseVerifier.classn替換class文件ncp /usr/share/elasticsearch/plugins/x-pack/x-pack-5.6.4.jar /tmp/testncd /tmp/testnjar xvf x-pack-5.6.4.jarncd /tmp/test/org/elasticsearch/licensenrm LicenseVerifier.class -fncp /root/LicenseVerifier.class /tmp/test/org/elasticsearch/license/ #拷貝上文編譯生成的java類文件njar cvf x-pack-5.6.4.jar /tmp/test/* #壓縮ncp /tmp/test/x-pack-5.6.4.jar /usr/share/elasticsearch/plugins/x-pack/ #拷貝到原來的x-pack目錄n

注意：x-pack-5.6.4.jar 應該拷貝到兩個 elasticsearch 集群插件，即目錄 /usr/share/elasticsearch/plugins/x-pack/

重啟 es 集群

systemct restart elasticsearchn

獲取 license 證書

https://license.elastic.co/registration

通過填寫的郵件獲取官方發來的郵件獲取證書

cat li-xiang-d28260d9-6c96-4dd2-92dc-2f14a9787903-v5.json n{"license":{"uid":"d28260d9-6c96-4dd2-92dc-2f14a9787903","type":"platinum","issue_ndate_in_millis":1511740800000,"expiry_date_in_millis":1827359999000,"max_nodes":n100,"issued_to":"li xiang (ceshi)","issuer":"Web Form","signature":"AAAAAwAAAA2lsEn14rcZQLw3V/JuUAAABmC9ZN0hjZDBGYnVyRXpCOW5Bb3FjZDAxOWpSbTVoMVZwUzRxVk1PSmkxaktJRVln5MUYvUWh3bHZVUTllbXNPbzBUemtnbWpBbmlWRmRZb25KNFlBR2x0TXc2K2p1Y1VtMG1UQU9TRGZVSGRwanEJGUjE3bXd3LzRqZ05iLzRteWFNekdxRGpIYlFwYkJiNUs0U1hTVlJKNVlXekMrSlVUdFIvV0FNeWdOYnnlESDc3MWhlY3hSQmdKSjJ2ZTcvYlBFOHhPQlV3ZHdDQ0tHcG5uOElCaDJ4K1hob29xSG85N0kvTWV3THhnlQk9NL01VMFRjNDZpZEVXeUtUMXIyMlIveFpJUkk2WUdveEZaME9XWitGUi9WNTZVQW1FMG1DenhZU0ZmneXlZakVEMjZFT2NvOWxpZGlqVmlHNC8rWVVUYzMwRGVySHpIdURzKzFiRDl4TmM1TUp2VTBOUlJZUlAyV0nZVL2kvVk10L0NsbXNFYVZwT3NSU082dFNNa2prQ0ZsclZ4NTltbU1CVE5lR09Bck93V2J1Y3c9PQAAAQBEn870HCp9jTj22SVdEP2uAFLF6ikLdDJFtSlbHuYOki6rPtWxIcw8y+WWpPUT5e8lYZw0GkB8CYT5hFLXZTrnBqTsNbYRZ3ABdHS1BnKBlkfE4PPcvnTTt4HtBCawNHaW0BNWQ2BA2fFj6zX3HyYJ8q5OaQk/il0t5f/TkInuf0yo3Y/F3rzDIXOHieBFnVvfG3EpNB4lo+G+e6vPMeOW86PsF9eKvQ24nucGDK3S4WSpwxbP1gZFuTdmEn9zDguJhRHrtJ6k//A/Q0Fbo8gFntWgHNp+1OJEklH+VBBZUWo17UMGnjjxDrGlLTZcsz2BPmk7mC7e8gBQnY4z7zJ/SgnsU","start_date_in_millis":1511740800000}}n

此證書的時間為1年使用時間，你可以通過下面網站進行換算http://tool.chinaz.com/Tools/unixtime.aspx，目前我申請了一個10 年的時間

將 "type":"basic" 替換為 "type":"platinum" # 基礎班變更為鉑金版n將 "expiry_date_in_millis":1543363199999 替換為 "expiry_date_in_millis":1827359999000 # 1年變為10年n

查看當前的 license

curl -XGET -u elastic:changeme http://127.0.0.1:9200/_licensen{n "license" : {n "status" : "active",n "uid" : "21389992-4010-4d2c-917b-94b4e3d5a1dc",n "type" : "trial",n "issue_date" : "2017-11-27T05:12:27.999Z",n "issue_date_in_millis" : 1511759547999,n "expiry_date" : "2017-12-27T05:12:27.999Z",n "expiry_date_in_millis" : 1514351547999,n "max_nodes" : 1000,n "issued_to" : "lx",n "issuer" : "elasticsearch",n "start_date_in_millis" : -1n }n}n

替換 license

curl -XPUT -u elastic:changeme http://127.0.0.1:9200/_xpack/license?acknowledge=true -d @li-xiang-d28260d9-6c96-4dd2-92dc-2f14a9787903-v5.json n

重啟 es 集群

systemctl restart elasticsearch n

查看 license

[root@linux-node5 license]# curl -XGET -u elastic:changeme http://127.0.0.1:9200/_licensen{n "license" : {n "status" : "active",n "uid" : "d28260d9-6c96-4dd2-92dc-2f14a9787903",n "type" : "platinum",n "issue_date" : "2017-11-27T00:00:00.000Z",n "issue_date_in_millis" : 1511740800000,n "expiry_date" : "2027-11-27T23:59:59.000Z",n "expiry_date_in_millis" : 1827359999000,n "max_nodes" : 100,n "issued_to" : "li xiang (ceshi)",n "issuer" : "Web Form",n "start_date_in_millis" : 1511740800000n }n} n

配置 X- Pack 告警

本文配置 X- Pack 告警是通過 filebeat 收集 Nginx 的日誌來做的模擬

配置郵件報警

1.安裝 Nginx

2.給 Nginx 配置 json 格式的數據

log_format json {"@timestamp":"$time_iso8601",n "@version":"1",n "client":"$remote_addr",n "url":"$uri",n "status":"$status",n "domain":"$host",n "host":"$server_addr",n "size":$body_bytes_sent,n "responsetime":$request_time,n "referer": "$http_referer",n "ua": "$http_user_agent"n };n access_log logs/access.log json; n

3.安裝 filebeta

安裝nyum install -y filebetan配置filebetan[root@linux-node4 filebeat]# grep -v "^ #" filebeat.yml|grep -v "^$"|grep -v "^#"nfilebeat.prospectors:n- input_type: logn paths:n - /usr/local/nginx/logs/access.logn json.keys_under_root: truen json.overwrite_keys: truenoutput.elasticsearch:n hosts: ["localhost:9200"]n username: "elastic"n password: "changeme"n參考網址:https://www.elastic.co/guide/en/beats/filebeat/current/configuration-filebeat-options.html#config-jsonnhttp://www.iyunw.cn/archives/filebeat-shou-ji-json-ge-shi-de-nginx-ri-zhi-fa-song-gei-elasticsearch/n啟動filebeta服務nsystemctl status filebeat.servicen

這裡默認進入 es 的索引是 filebeat-{%}

4.登陸 kibana 添加索引

5.模擬一些 404 的頁面，讓其展示

for i in {1..1000};do curl http://192.168.56.14:/lx-0$i;sleep 1;done

6.添加報警觸發器(收集每分鐘內超過訪問狀態大於 400 且個數超過 20 報警)

Watch JSON

{n "trigger": {n "schedule": {n "interval": "1m"n }n },n "input": {n "search": {n "request": {n "search_type": "query_then_fetch",n "indices": [n "filebeat-2017*"n ],n "types": [],n "body": {n "size": 0,n "query": {n "bool": {n "must": [n {n "query_string": {n "query": "status:>=400"n }n },n {n "range": {n "@timestamp": {n "gte": "now-1m"n }n }n }n ]n }n },n "sort": [n {n "@timestamp": {n "order": "desc"n }n }n ]n }n }n }n },n "condition": {n "compare": {n "ctx.payload.hits.total": {n "gt": 20n }n }n },n "actions": {n "elasticsearch": {n "throttle_period_in_millis": 60000,n "email": {n "profile": "outlook",n "attachments": {n "attached_data": {n "data": {n "format": "json"n }n }n },n "priority": "high",n "to": [n "lixiang@xxxxx.com"n ],n "subject": "Nginx {{ctx.payload.hits.total}} errors",n "body": {n "text": "nginx 404 錯誤過多,請注意查看"n }n }n }n }n}n

7.這一步特別重要需要在 ES 配置文件設置否則無法報警(因為我公司使用的是 Exchange 客戶端)

xpack.notification.email.account:n exchange_account:n profile: outlookn email_defaults:n from: warning@xxxxx.comn smtp:n auth: truen starttls.enable: truen host: smtp.partner.outlook.cnn port: 587n user: 123@xxxxxx.comn password: 233g@123n參考網址：https://www.elastic.co/guide/en/x-pack/5.6/actions-email.html #郵件設置n

8. 查看郵件報警狀態以及 kibana 中的狀態

對於上面的內容大家可以參看下面的 url 來了解其中的含義

參考網址:

https://www.elastic.co/guide/en/x-pack/5.6/xpack-alerting.html 對集群和索引事件的警告 Xpack document

配置 Webhook 報警

對於 webhook 的報警,我們需要前期做一下工作需要了解一個東西 web.py,具體 URL 鏈接大家可以訪問這個地方 http://webpy.org/tutorial3.zh-cn#starting

1.用 web.py 啟動一個 webhook 的監聽

[root@linux-node5 ~]# cat webhooks.pyn#!/usr/bin/evn pythonn# -*- coding:utf-8 -*-nimport webnimport osnimport demjsonnimport sysnreload(sys)nsys.setdefaultencoding( "utf-8" )nurls = (n /log_event_watch,abc,n)nclass abc:n def POST(self):n data = web.data()n print datan cmd = curl -G -v "http://abc.com/abc/" --data "user=lixiang" --data "media=all" --data-urlencode "subject=test" --data-urlencode "message=%s" %(data)n os.system(cmd) nif __name__ == "__main__":n app = web.application(urls, globals())n app.run()n運行 python webhooks.py 9000n

注意：urls，因為我的 Watcher ID 為 "log_event_watch"

2.再次回到我們的 kibana 界面, "Management"->"Edit"

查看 Watches

查看 Wathch Json 內容

{n "trigger": {n "schedule": {n "interval": "1m"n }n },n "input": {n "search": {n "request": {n "search_type": "query_then_fetch",n "indices": [n "nginx_access*"n ],n "types": [],n "body": {n "size": 0,n "query": {n "bool": {n "must": [n {n "query_string": {n "query": "status:>=400"n }n },n {n "range": {n "@timestamp": {n "gte": "now-1m"n }n }n }n ]n }n },n "sort": [n {n "@timestamp": {n "order": "desc"n }n }n ]n }n }n }n },n "condition": {n "compare": {n "ctx.payload.hits.total": {n "gt": 5n }n }n },n "actions": {n "email": {n "throttle_period_in_millis": 60000,n "email": {n "profile": "outlook",n "attachments": {n "attached_data": {n "data": {n "format": "json"n }n }n },n "priority": "high",n "to": [n "lixiang@xxxxx.com"n ],n "subject": "Nginx {{ctx.payload.hits.total}} errors",n "body": {n "text": "nginx 404 錯誤過多,請注意查看"n }n }n },n "webhook": {n "condition": {n "compare": {n "ctx.payload.hits.total": {n "gt": 5n }n }n },n "webhook": {n "scheme": "http",n "host": "192.168.56.15",n "port": 9000,n "method": "post",n "path": "/{{watch_id}}",n "params": {},n "headers": {},n "body": "Encountered {{ctx.payload.hits.total}} errors"n }n }n }n} n

3.模擬觸發報警

for i in {1..200};do curl http://192.168.56.14:/lx-0$i;sleep 2;done n

4.查看結果，這裡就不貼出來了(微信，簡訊，郵件都能收到報警)

5.這就是實現報警消息附帶 URL 地址

官方參考網址:

https://www.elastic.co/guide/en/x-pack/5.6/watcher-getting-started.html

小月兒