Elatic Fluent Bit

Server log system.

Posted by Mr. Liu on 2022-05-23
Estimated Reading Time 1 Minutes
Words 183 In Total
Viewed Times

日志系统调研 Elastic Fluent Bit

What is Elasticsearch

link

What is Fluent Bit

Fluent Bit is a CNCF(Cloud Native Computing Foundation) sub-project under the umbrella of Fluentd.

Fluent Bit is an open source and multi-platform log processor tool which aims to be a generic Swiss knife for logs processing and distribution.

Nowadays the number of sources of information in our environments is ever increasing. Handling data collection at scale is complex, and collecting and aggregating diverse data requires a specialized tool that can deal with:

  • Different sources of information
  • Different data formats
  • Data Reliability
  • Security
  • Flexible Routing
  • Multiple destinations

Fluent Bit has been designed with performance and low resources consumption in mind.

Key Concepts

  • Event or Record
  • Filtering
  • Tag
  • Timestamp
  • Match
  • Structured Message

Kubernetes manages a cluster of nodes, so our log agent tool will need to run on every node to collect logs from every POD, hence Fluent Bit is deployed as a DaemonSet (a POD that runs on every node of the cluster).

Based on ES, FluentBit can process the logs in different nodes.


If you like this blog or find it useful for you, you are welcome to comment on it. You are also welcome to share this blog, so that more people can participate in it. If the images used in the blog infringe your copyright, please contact the author to delete them. Thank you !