A New Approach to Logging as a Stack: Fluent Bit + PostgreSQL (FPS)

Jonathan Gonzalez at KubeCon + CloudNativeCon North America 2020

Logging at scale is a very interesting challenge, and having the right open source stack is mandatory. There are many ways to solve the log collection, processing and aggregation problem, but when it comes to perform data analysis once the data has been centralized, it's really easy to face latencies and get general problems when indexing our data that don't comes with a fixed structure. In this session, we will share our experience of implementing a new stack that we called FPS: Fluent Bit + PostgreSQL. The open source database PostgreSQL offers a very interesting storage engine for non-columnar structured documents: JSONB (binary JSON) which have shown to be a very efficient and scalable implementation to store trillions of record logs and provide fast results when performing queries.

https://sched.co/ekD0