logstash使用redis做消息队列

logstash agent conf文件配置:

input {
 file {
     path => "/tmp/www.log"
     type => "linux-syslog"
  }
}

output {
   redis {
        host => "192.168.192.120"
        data_type => "list"
        key => "logstash"
   }
}

logstash index conf文件配置(输出到elasticsearch):

input {
   redis {
        host => "127.0.0.1"
        type => "linux-syslog"
        data_type => "list"
        key => "logstash"
   }
}

output {
  if "_grokparsefailure" not in [tags] {
  elasticsearch {
        hosts => "127.0.0.1:9200"
        index => "logstash-test"
  }
  }
}

来看下Redis中的内容
127.0.0.1:6379> info keyspace
# Keyspace
db0:keys=1,expires=0,avg_ttl=0
127.0.0.1:6379> keys *
1) “logstash”
查看队列中的日志数量
127.0.0.1:6379> llen logstash
取出前两条信息
127.0.0.1:6379> LRANGE logstash 0 1
1) “{\”message\”:\”Nov  8 14:22:25 server120 sshd[16024]: pam_unix(sshd:session): session closed for user root\”,\”@version\”:\”1\”,\”@timestamp\”:\”2016-11-11T06:25:49.352Z\”,\”path\”:\”/var/log/secure\”,\”host\”:\”0.0.0.0\”,\”type\”:\”test\”}”
2) “{\”message\”:\”Nov 10 11:10:46 server120 sshd[13192]: Accepted password for root from 192.168.190.201 port 52042 ssh2\”,\”@version\”:\”1\”,\”@timestamp\”:\”2016-11-11T06:25:49.507Z\”,\”path\”:\”/var/log/secure\”,\”host\”:\”0.0.0.0\”,\”type\”:\”test\”}”
(integer) 6