docker安装日志追踪zipkin和ELK日志分析平台

项目微服务化后，服务之间调用关系复杂，日志查找变得困难，急需引入日志追踪和实时查询分析平台。

ELK 分别是：Elasticsearch（搜索和分析引擎）、Logstash（数据处理管道，如日志数据通过它进入elasticsearch） 和 Kibana（Elasticsearch可视化工具）的首个字母缩写。

Zipkin是一款开源的分布式实时数据追踪系统（Distributed Tracking System），基于 Google Dapper的论文设计而来，由 Twitter 公司开发贡献。

废话不多说，开始安装。。。。

• Elasticsearch安装

docker run -d \
 --name elasticsearch  --restart=always -e ES_JAVA_OPTS="-Xms1024m -Xmx1024m" \
 --privileged=true \
 -v /home/elasticsearch/data:/usr/share/elasticsearch/data \
 -v /home/elasticsearch/logs:/usr/share/elasticsearch/logs \
 -v /home/elasticsearch/plugins:/usr/share/elasticsearch/plugins \
 -v /home/elasticsearch/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml\
 -e "discovery.type=single-node" \
 -e "TZ=Asia/Shanghai" \
 -p 9200:9200 \
 -p 9300:9300 \
 docker.elastic.co/elasticsearch/elasticsearch:7.6.2


进入docker内生成用户名和密码
docker exec -it elasticsearch bash
执行命令
elasticsearch-setup-passwords interactive
出现如下：
You will be prompted to enter passwords as the process progresses.
Please confirm that you would like to continue [y/N]y

Enter password for [elastic]:
Reenter password for [elastic]:
Passwords do not match.
Try again.
Enter password for [elastic]:
Reenter password for [elastic]:
Enter password for [apm_system]:
Reenter password for [apm_system]:
Enter password for [kibana]:
Reenter password for [kibana]:
Enter password for [logstash_system]:
Reenter password for [logstash_system]:
Enter password for [beats_system]:
Reenter password for [beats_system]:
Enter password for [remote_monitoring_user]:
Reenter password for [remote_monitoring_user]:
Changed password for user [apm_system]
Changed password for user [kibana]
Changed password for user [logstash_system]
Changed password for user [beats_system]
Changed password for user [remote_monitoring_user]
Changed password for user [elastic]

备注：

端口9300为集群使用的tcp端口

elasticsearch.yml内容如下：

cluster.name: "es7"

network.host: 0.0.0.0

xpack.security.enabled: true

elasticsearch:7.6.2默认已经集成了xpack，这里开启了

文件映射：

/home/elasticsearch/data和/home/elasticsearch/logs，/home/elasticsearch/plugins宿主机文件映射到docker相应位置。

• Kibana安装

docker run -d \

 --name kibana \
 --link elasticsearch:elasticsearch \
 -e "TZ=Asia/Shanghai" \
 -e "I18N_LOCALE=zh-CN" \
 -v /home/elasticsearch/kibana.yml:/usr/share/kibana/config/kibana.yml \
 -p 5601:5601 \
 kibana:7.6.2

• logstash安装

docker run -d -it --restart=always  \
 --privileged=true  \
 --name=logstash -p 5047:5047 -p 9600:9600  \
 -e "TZ=Asia/Shanghai" \
 -v /home/logstash/pipeline/:/usr/share/logstash/pipeline/  \
 -v /home/logstash/config/:/usr/share/logstash/config/ docker.io/library/logstash:7.6.2

cd /home/logstash/pipeline/

建立文件logstash.conf，内容如下：

input {
  tcp {
    mode => "server"
    host => "0.0.0.0"
    port => 5047
    codec => json_lines
  }
}
filter{
}
output {
    elasticsearch { 
      hosts => ["172.17.0.1:9200"]
	   user => ""
	   password => ""
      index => "logstash-dev-%{+YYYY.MM.dd}"
    }
    stdout { codec => rubydebug }
}

cd /home/logstash/config/ 建立logstash.yml和pipelines.yml

logstash.yml文件内容如下：
config:
  reload:
    automatic: true
    interval: 3s
xpack:
  management.enabled: false
  monitoring.enabled: false

pipelines.yml内容如下：
- pipeline.id: logstash
  path.config: "/usr/share/logstash/pipeline/logstash.conf"

• zipkin-server安装

docker run -d  --name zipkin -p 9411:9411 openzipkin/zipkin --restart=always -e STORAGE_TYPE=elasticsearch -e ES_HOSTS=http://172.0.0.1:9200 -e ES_USERNAME=  -e ES_PASSWORD=

ES_USERNAME：es用户名

ES_PASSWORD：es密码

安装已全部完成

注意一点：宿主机和docker容器内时间要同步，es按时间来索引。

yum -y install ntpdate
ntpdate -u ntp.api.bz
docker cp /etc/localtime 容器ID:/etc/

• es和zipkin访问地址

访问zipkinserver：

http://ip:9411/

kinana访问地址

http://ip:5601/