DevOps & SRE notes – Telegram
DevOps & SRE notes
12K subscribers
40 photos
19 files
2.5K links
Helpful articles and tools for DevOps&SRE

WhatsApp: https://whatsapp.com/channel/0029Vb79nmmHVvTUnc4tfp2F

For paid consultation (RU/EN), contact: @tutunak


All ways to support https://telegra.ph/How-support-the-channel-02-19
Download Telegram
This post provides a comprehensive overview of Cloudflare's logging pipeline, detailing the various components and processes involved in managing, improving, and maintaining the internal logging systems. It covers the journey of logs from their inception to their storage, explaining the role of different technologies and methods used in this process. The article highlights Cloudflare's focus on scalability, high availability, and meeting service level objectives while handling immense volumes of log data.
https://blog.cloudflare.com/an-overview-of-cloudflares-logging-pipeline/
👍4
Diagram as code
from diagrams import Cluster, Diagram
from diagrams.aws.compute import ECS, EKS, Lambda
from diagrams.aws.database import Redshift
from diagrams.aws.integration import SQS
from diagrams.aws.storage import S3

with Diagram("Event Processing", show=False):
source = EKS("k8s source")

with Cluster("Event Flows"):
with Cluster("Event Workers"):
workers = [ECS("worker1"),
ECS("worker2"),
ECS("worker3")]

queue = SQS("event queue")

with Cluster("Processing"):
handlers = [Lambda("proc1"),
Lambda("proc2"),
Lambda("proc3")]

store = S3("events store")
dw = Redshift("analytics")

source >> workers >> queue >> handlers
handlers >> store
handlers >> dw


https://diagrams.mingrammer.com/docs/getting-started/examples
The article discusses how they optimized costs for their Kubernetes-based ClickHouse clusters on AWS EKS. They achieved significant savings by analyzing and improving their EKS node utilization. The primary issue was the underutilization of EC2 instances. By changing the Kubernetes scheduler's scoring policy from 'LeastAllocated' to 'MostAllocated', they effectively increased cluster utilization and reduced the number of necessary EC2 nodes. This approach also involved setting up a custom scheduler and strategically handling system utility workloads. The result was a considerable reduction in infrastructure costs, without compromising performance or reliability for customers.
https://clickhouse.com/blog/packing-kubernetes-pods-more-efficiently-saving-money
👍5
Kubernetes powered PaaS that runs in your own cloud.
https://github.com/porter-dev/porter
👍4