CatOps – Telegram
CatOps
5.09K subscribers
94 photos
5 videos
19 files
2.57K links
DevOps and other issues by Yurii Rochniak (@grem1in) - SRE @ Preply && Maksym Vlasov (@MaxymVlasov) - Engineer @ Star. Opinions on our own.

We do not post ads including event announcements. Please, do not bother us with such requests!
Download Telegram
Holiday Book Recommendations by Gergely Orosz - an author of The Pragmatic Engineer blog.

A bit unfortunate for me that this article was published on 17th of December, while I have already bought some engineering books before the end of the year (we have a special budget for that in my company). However, 4 out 5 books I’ve bought are in this list :)
The only exception is Database Internals, but I guess this book is just too specific for a generic IT book recommendation.

So, I hope you can find something interesting for you in this list! There are multiple categories there, from engineering management to technology-specific topics. Also, “The Pragmatic Engineer” is a really cool blog about IT in general as well as some European specifics. I read it myself and can totally recommend it!

Happy upcoming holidays!

#books
​​Apache Issues 3rd Patch to Fix New High-Severity Log4j Vulnerability

Tracked as CVE-2021-45105 (CVSS score: 7.5), the new vulnerability affects all versions of the tool from 2.0-beta9 to 2.16.0, which the open-source nonprofit shipped earlier this week to remediate a second flaw that could result in remote code execution (CVE-2021-45046), which, in turn, stemmed from an "incomplete" fix for CVE-2021-44228, otherwise called the Log4Shell vulnerability.

#security
​​Yet another post from the #app_bundle series. This is again a video from Viktor Farcic on how to combine ArgoCD, Crossplane, and KubeVela to completely abstract Kubernetes away from your product engineers aka developers and (allegedly) make their lives easier.

In the end of each year, many people make predictions on what upcoming times would look like. And I can say that abstracting clusters away will be a big thing in the industry. This brings us to the logical question: "So, why do all this stuff and not just use serverless options out of the box?". I will let you answer this question on your own.

P.S. You can save this post to blame me later, if this prediction happens to be wrong.

#kubernetes #cicd
I re-designed my blog not just for the sake of re-designing. At least, I hope so.

So, here is a new article from a wannabe series about Kubernetes.

This series started with a review of Velero backup tool. Now, I want to extend this topic a bit and share my thoughts on whether it makes sense to back up a Kubernetes cluster at all.


P.S. This is the last technical article in my blog this year. I usually do a generic recap of the year, but haven't done one yet. Also, this is the last post in CatOps channel this year.

Wish y'all reliability during the festive season and only five nines in the new year!

#kubernetes #backup #blog
From our subscribers.

People can use AWS Elastic Container Registry to cache public Docker images.

From their press-release:

This new capability gives AWS customers a simple and highly available way to pull Docker Official Images, while taking advantage of the generous AWS Free Tier. Customers pulling images from Amazon ECR Public to any AWS Region get virtually unlimited downloads. For workloads running outside of AWS, users not authenticated on AWS receive 500 GB of data downloads each month. For additional data downloads, they can sign up or sign in to an AWS account to get up to 5TB of data downloads each month after which they pay $0.09 per GB.

If you have any interesting things to share, you can always do it in our chat!

#aws
🔥2
New Year resolutions is a very common practice. I do mine as well, but this time I want to share a review of databases in 2021 with Dr. Andy Pavlo.

Some points from the article:

- RDBMS is an old concept, but it dominates the market even for greenfield projects and it‘s here to stay
- PostgreSQL gains more and more popularity. It might be not the most popular database, but it steadily moving to the top
- “…only old people care about official TPC numbers.”
- More and more money is invested into DB-related startups. Size of each funding series has also increased comparing to previous big takes. ”We are in the golden era of databases. There are so many excellent choices available today. Investors are searching for database start-ups that can become the next Snowflake-like IPO.”
- People have moved away from MapReduce and Hadoop technologies nowadays.
- Larry Ellison - a co-founder of Oracle - is back to the 5th position of the richest persons list

#databases
Once everybody patched their Log4j dependencies and went back from holidays, it's time to process, what just happened.

In his article Professional Maintainers: a Wake-up Call Filippo Valsorda argues that the current open-source model is unsustainable. Thus, the only viable alternative to solve this status-quo, in his onion, is for open-source maintainers to start issuing invoices to companies that require support or new features.

I know that such maintainers already exist, but this is definitely not a common practice.

Anyways, there are so many non-sustainable things in this world and our usual way of solving them is to pretend that they don't exist. So, let this article be just an invitation to think about the current state of affairs.

P.S. If you're a maintainer or a contributor to an open-source project, you're doing a god's work! Thank you!

#open_source #culture
👍6👎1🔥1
​​Yesterday I shared this video in our chat and it looks like people liked it. So, I would like to share it here with the broader audience.

In this video Victor Farcic speaks about AWS Karpenter and its advantages comparing to good old cluster-autoscaler.

A few notable things:
- Karpenter is workload-aware. It means that it can determine, how many resources does your workload needs and scale up a cluster accordingly. So, if you need to place just a tiny pod, you’ll get a smaller node comparing to a situation if you need to run a few heavy tasks
- Karpenter is topology-aware. So, for example, you can schedule nodes for a given workload in a specific AZ only. It’s neat if you use EBS volumes or additional network interfaces
- It’s groupless, meaning that it doesn’t have a concept of “instance groups” like cluster-autoscaler (and many other autoscalers). So, cluster-autoscaler modifies parameters of instance group, Karpenter on another hand talks to AWS APIs directly. In theory, this should reduce scale-up and scale-down times

#kubernetes #aws
​​Morning! New Year - new HUG Kyiv events. Now, with HashiCorp co-founders

What: Q/A session with Mitchell Hashimoto and Armon Dadgar
You can ask and vote for questions via this link.

Who: Mitchell Hashimoto and Armon Dadgar and one of your old friends as moderator ;)

When: Thursday 3rd February, 19:50 (Kyiv TZ)
Where: Online
Language: English

Please, register here

#event
👍3🔥1🎉1
​​Mess with DNS

Julia Evans has built a site where you can do experiments with DNS. It shows you a live stream of all DNS queries coming in for records on the free subdomain provided to you (a “behind the scenes” view).

You can make up your experiments or check out her examples of experiments you can try., including "weird" (when you broke something), "useful" and "tutorial" experiments.

#dns
👍9😱1
A friend of mine has shared some news about recent security breaches in government websites. Unfortunately, the possibility of a security breach is an inherent vice of any software system.

Developing systems is hard. Catching all the edge cases and unexpected behaviors is even harder. That’s why we almost always rely on the community of fellow engineers to spot the bugs and security vulnerabilities. Unfortunately, there is also a backside of this coin.

Some people are spamming infrasec contacts with insignificant problems demanding money. Thus, people are getting tired with that and really critical issues have a chance of being buried under an avalanche of these requests.

Troy Hunt - a founder of haveibeenpwned.com - summarized this behavior with a few examples in his article “Beg Bounties”

P.S. Also, Inherent Vice is a cool movie, that I can recommend if you like a slow-pacing dramas that focus on acting.

#culture #security
Another cybersecurity post.

A few days prior Log4j storm, researchers from Orca Security discovered CloudFormation vulnerability and managed to get access to an internal account. It means that an attacker could potentially get access to any AWS account by mimicking CloudFormation API calls.

Previously, Orca Security research team managed to do something similar with [AWS Glue.

Kudos to AWS team here. They managed to fix this vulnerability in less than 25 hours and it took 6 days to populate the fix in all their regions.

#aws #security
👍3
Open Policy Agent (OPA) is a powerful framework that allows you to validate your data structures against some policies, written in Rego. There are multiple implementations of OPA for various tools and systems e.g. Gatekeeper for Kubernetes.

However, you can write your own implementation and use if, for example, for unit tests. This articles provides some examples of how to write your own OPA checks in Go.

#programming #opa
Microsoft in its blog reviled some details on the recent cyberattack on the number of Ukrainian governmental websites

Investigation is still ongoing. However, we already know that attackers have overridden Master Boot Record (MBR) on infected machines. Microsoft also published hashes of malicious software in the same blog article.

I also found information that hackers initially broke through a proprietary Content Management System (CMS) tool. However, I haven’t found any official proofs of it.

#security
After I published my article about Kubernetes backups, I got some questions regarding cluster organization and maintenance procedure. Specifically about managing Kubernetes clusters “as cattle” and this statement:

…*investing in cluster automation early on* is important. You may end up in a situation when you need to re-create a cluster from scratch.

I started writing an article about managing clusters “as cattle” to address these questions. Moreover, that was my initial plan. However, I realized that I have very little exposure on how other people are managing their clusters! I have only the perspective of my current and my previous employers.

Thus, I decided to issue a survey on how people are managing their Kubernetes clusters, their core components like CNI, Ingress, etc., and business applications.

I would appreciate it, if you can spend some time filling up this survey. You can also reach out to me directly and share your story of managing Kubernetes in your company. I left contacts in that Google form.

Here’s the link to that article about backups, in case you missed it.

Have a great week, y’all!

#kubernetes
The channel denoscription says "... and other issues", right?

So, today I want to share with you WSJF model of SAFe framework, which helps to compare priorities of different projects. WSJF stands for "Weighted Shortest Job First".

This concept won't be new at all for the product people and project managers. And this is exactly why I want to share it here. Through my career, I saw a lot of examples, when platform teams or OPS teams didn't have their dedicated product/project manager. Therefore, there were a lot of struggles in what to prioritize among multiple projects.

I saw this framework in action and I think it's useful. Ofc, this is not a silver bullet. You may need to make a few tweaks to the process here and there, but in general this is a good start if you're not sure what you should pick up next from the pile of platform work.

#management #agile #safe
👍2🔥1
Not so long time ago, I shared with you a tool called Acra for encryption/decryption of sensitive data in a database. This tool is written in Go, and obviously it uses TLS.

Now, they have written an article about TLS implementation in Go, specifically about the implementation of OCSP and CRL extensions.

This article would be interesting for you, foremost, to get some ideas on OCSP and CRL extensions and their use cases. Secondly, if you need to implement TLS in your apps or advise on the implementation to your developers.

#security #programming #go #tls
If you haven’t read a Roblox’s postmortem on October‘s 73-hour outage, you definitely should!

Even though this event happened in October, the postmortem was released just a few days ago. And in this case, this is a very good decision! Especially, because this write up provides a detailed analysis on what happened at that time and what chain of events caused that.

It‘s cool to read postmortem the next day after an outage - we are all curious human beings. Unfortunately, those postmortems usually they are lacking many details. This is understandable: it‘s not enough time for a thorough analysis, also your team is probably already tired.

In this case, though, you can have a detailed overview of what happened as well as plans to prevent this chain of events happening again. Moreover, with some plans already implemented.

It‘s a pity that not may companies do similar postmortems. And I must say that this is probably in their disadvantage either. After reading this document I have a feeling that Roblox is a cool place to work, TBH.

#postmortem #hashicorp #consul
👍7