日志系统调研 Elastic Fluent Bit
What is Elasticsearch
What is Fluent Bit
Fluent Bit is a CNCF(Cloud Native Computing Foundation) sub-project under the umbrella of Fluentd.
Fluent Bit is an open source and multi-platform log processor tool which aims to be a generic Swiss knife for logs processing and distribution.
Nowadays the number of sources of information in our environments is ever increasing. Handling data collection at scale is complex, and collecting and aggregating diverse data requires a specialized tool that can deal with:
- Different sources of information
- Different data formats
- Data Reliability
- Security
- Flexible Routing
- Multiple destinations
Fluent Bit has been designed with performance and low resources consumption in mind.
Key Concepts
- Event or Record
- Filtering
- Tag
- Timestamp
- Match
- Structured Message
Kubernetes manages a cluster of nodes, so our log agent tool will need to run on every node to collect logs from every POD, hence Fluent Bit is deployed as a DaemonSet (a POD that runs on every node of the cluster).
Based on ES, FluentBit can process the logs in different nodes.
If you like this blog or find it useful for you, you are welcome to comment on it. You are also welcome to share this blog, so that more people can participate in it. If the images used in the blog infringe your copyright, please contact the author to delete them. Thank you !