Docker集群日誌收集：Syslog+Rsyslog+ELK

01-29

Rancher線下活動，有同學問到Docker日誌如何收集，這裡就再補一下作業

一，方案：

elk（elasticsearch + logstash + kibana）
rsyslog
docker log-dirver: syslog

二，配置

elk：

# workspacennmkdir -p ~/workspace/elkncd ~/workspace/elkngit clone https://github.com/deviantony/docker-elk.git ./nn# confignn## logstashn## logstash/config/logstash.confnninput {n tcp {n port => 5000n # type => "rsyslog"n codec => "json"n }n}nnoutput {n elasticsearch {n hosts => "elasticsearch:9200"n }n}nn## composen## docker-compose.ymlnnversion: 2nservices:n elasticsearch:n build: elasticsearch/n # es埠禁止了，只允許內網訪問n # ports:n # - "9200:9200"n # - "9300:9300"n environment:n ES_JAVA_OPTS: "-Xms1g -Xmx1g"n volumes:n - ./data/elasticsearch/data:/usr/share/elasticsearch/datan networks:n - docker_elkn logstash:n build: logstash/n command: -f /etc/logstash/conf.d/n volumes:n - ./logstash/config:/etc/logstash/conf.dn ports:n - "5000:5000"n networks:n - docker_elkn depends_on:n - elasticsearchn kibana:n build: kibana/n volumes:n - ./kibana/config/:/etc/kibana/n ports:n - "5601:5601"n networks:n - docker_elkn depends_on:n - elasticsearchnnnetworks:n docker_elk:n driver: bridgenn# runnndocker-compose up -dn

rsyslog:

# workspacennmkdir -p ~/workspace/rsyslogncd ~/workspace/rsyslog/nmkdir -p rsyslog.dnn# confignn## jsonn## rsyslog.d/01-json-template.confn## 如果發現時間不一致，可以改timereported為timegeneratednntemplate(name="json_lines"n type="list"n option.json="on") {n constant(value="{")n constant(value=""@timestamp":"") property(name="timereported" dateFormat="rfc3339")n constant(value="", "@version":"1")n constant(value="","tag":"") property(name="syslogtag")n constant(value="","message":"") property(name="msg")n constant(value="","severity":"") property(name="syslogseverity-text")n constant(value="","facility":"") property(name="syslogfacility-text")n constant(value="","hostname":"") property(name="hostname")n constant(value="", "procid":"") property(name="procid")n constant(value="", "programname":"") property(name="programname")n constant(value=""}n")n}nn## logstashn## rsyslog.d/60-logstash.confn## 替換IP/PORT為真實地址nn# :programname, contains, "docker"n*.* @@${LOGSTASH_SERVER_IP}:${LOGSTASH_SERVER_PORT};json_linesnn## composen## docker-compose.ymlnnversion: 2nservices:n app:n image: voxxit/rsyslogn ports:n - "514:514"n - "514:514/udp"n volumes:n - ./rsyslog.d:/etc/rsyslog.dn restart: alwaysnn# runnndocker-compose up -dn

container:

# workspacennmkdir -p ~/workspace/nginxncd ~/workspace/nginxnn# confignn## composen## docker-compose.ymlnnversion: "2"nnservices:n app:n image: nginx:alpine # 下面截圖裡用了我的rtmp鏡像，效果相同n logging:n driver: syslogn options:n syslog-address: "tcp://192.168.2.121:514" # 內網IPn tag: "{{.Name}}.{{.ID}}"n ports:n - "8080:80"n restart: alwaysnn# runnndocker-compose up -dn

三，效果

選擇："*"，並去掉"index-xxxx"的勾選，點擊"create"