PSA: Consider EBS snapshots over Jenkins backup plugins DiscussionAWS

TL;DR: Moved from ThinBackup plugin to EBS snapshots + Lambda automation. Faster recovery, lower maintenance overhead, \~$2/month. CloudFormation template available.

The Plugin Backup Challenge

Many Jenkins setups I've encountered follow this pattern:

ThinBackup or similar plugin installed
Scheduled backups to local storage
Backup monitoring often neglected
Recovery procedures untested

Common issues with this approach:

Dependency on the host system \- local backups don't help if the instance fails
Incomplete system state \- captures Jenkins config but misses OS-level dependencies
Plugin maintenance overhead \- updates occasionally break backup workflows
Recovery complexity \- restoring from file-based backups requires multiple manual steps

Infrastructure-Level Alternative

Since Jenkins typically runs on EC2 with EBS storage, why not leverage EBS snapshots for complete system backup?

Implementation Overview Created a CloudFormation stack that:

Lambda function discovers EBS volumes attached to Jenkins instance
Creates daily snapshots with retention policy
Tags snapshots appropriately for cost tracking
Sends notifications on success/failure
Includes cleanup automation

Cost Comparison Plugin approach: Time spent on maintenance + storage costs EBS approach: \~$1-3/month for incremental snapshots + minimal setup time

Recovery Experience Had to test this recently when a system update caused issues. Process was:

1. Identify appropriate snapshot (2 minutes)
2. Launch new instance from snapshot (5 minutes)
3. Update DNS/load balancer (1 minute)
4. Verify Jenkins functionality (2 minutes)

Total: \~10 minutes to fully operational state with complete history intact.

Why This Approach Works

Complete system recovery: OS, installed packages, Jenkins state, everything
Point-in-time consistency: EBS snapshots are atomic
AWS-native solution: Uses proven infrastructure services
Low maintenance: Automated with proper error handling
Scalable: Easy to extend for cross-region disaster recovery

Implementation Details The solution handles:

Multi-volume instances automatically
Configurable retention policies
IAM roles with minimal required permissions
CloudWatch metrics for monitoring
Optional cross-region replication

Implementation (GitHub): [https://github.com/HeinanCA/automatic-jenkinser](https://github.com/HeinanCA/automatic-jenkinser)

Discussion Points

How are others handling Jenkins backup/recovery?
Any experience with infrastructure-layer vs application-layer backup approaches?
What other services might benefit from this pattern?

Note: This pattern applies beyond Jenkins - any service running on EBS can use similar approaches (GitLab, databases, application servers, etc.).

https://redd.it/1nmov15
@r_devops

What’s your go-to deployment setup these days?

I’m curious how different teams are handling deployments right now. Some folks are all-in on GitOps with ArgoCD or Flux, others keep it simple with Helm charts, plain manifests, or even homegrown noscripts.

What’s working best for you? And what trade-offs have you run into (simplicity, speed, control, security, etc.)?


https://redd.it/1nn0l1t
@r_devops

I built a lightweight Go-based CI/CD tool for hacking on projects without setting up tons of infra

Hi All,

I’ve been experimenting with a simple problem, I wanted to use Claude Code to generate code from GitHub issues, and then quickly deploy those changes from a PR on my laptop so I could view them remotely — even when I’m away, by tunneling in over Tailscale.

Instead of setting up a full CI/CD stack with runners, servers, and cloud infra, I wrote a small tool in Go: [**gocd**](https://github.com/simonjcarr/gocd).

The idea

* No heavy infrastructure setup required
* Run it directly on your dev machine (or anywhere)
* Hook into GitHub issues + PRs to automate builds/deploys
* Great for solo devs or small experiments where spinning up GitHub Actions / Jenkins / GitLab CI feels like overkill

For me, it’s been a way to keep iterating quickly on side projects without dragging in too much tooling. But I’d love to hear from others:

* Would something like this be useful in your dev setup?
* What features would make it more valuable?
* Are there pain points in your current CI/CD workflows that a lightweight approach could help with?

Repo: [https://github.com/simonjcarr/gocd](https://github.com/simonjcarr/gocd)

Would really appreciate any feedback or ideas — I want to evolve this into something genuinely useful for folks who don’t need (or want) a huge CI/CD system just to test and deploy their work.

https://redd.it/1nn3u4x
@r_devops

Hey folks, if you are struggling to get a Job right now try starting something out.. I created DevOps related side hustle 2 years ago that brings me 2-3k per month.

Hey Folks,

About 2 years ago I started a side hustle. I didn't make it big, but still it is beyond what I ever hoped.

I had some spare time during evenings and wanted to build something where I could use my DevOps skills and SWE. I wanted to get better at writing Go code, and my thought process was that if it didn't work out I would walk away with real experience in areas I hadn't worked in, and if it did, then even better.

Initially I had a few ideas, but after a while looking at posts and what people are interested in, I settled on a final idea and coded the first draft.

If you don't already have something you want to build, look around Reddit, use filters and check top/popular posts, see what people are missing or actively upvoting.

I signed up for Microsoft for Startups and they gave me $25k in credits (more than I expected). You too could request AWS, Azure or Google Cloud credits. Quick advice: GCP has the least favorable terms so I'd avoid them altogether. So you don't need to pay out of your pocket initially it's quite easy to start off with free credits.

Another important factor is, if you try something, publish it asap, don't build it for months, my first draft was a basic static website with 30 questions from Googles SRE Interview \- if people see value in what you do, they will like it regardless of all the bells and whistles, but if they don't - you've saved yourself a lot of time.

Don't shy away accepting money early on, We were surprised when some folks paid us 5$ for our broken site so we even refunded that since in our mind we were just testing out Stripe and were not planning to accept payments, but lesson from this is put a stripe checkout even for a symbolic price of few bucks. If people will like and support your work those contributions will give you a motivation to put an extra work and deliver a better product.

Finally I wanna say few words about what I've built. It is a platform with real DevOps interview questions from companies like Google, Meta, Amazon, etc. and hands-on live environments.. its called prepare.sh

We are making final changes to Open Source under MIT License our backend Kubernetes controllers that lets us run thousands of isolated ephemeral environments inside single cluster, basically what powers our website so others can use it too, but I will keep that for another post.

https://redd.it/1nn4s9e
@r_devops

CTO / Founding Engineer] React Native + Voice AI (EN/ES) — Equity

Demo Video Below

[https://drive.google.com/file/d/15qr4JYBfnqjXpkli0LJAX-qio7fEL8kz/view?usp=drivesdk](https://drive.google.com/file/d/15qr4JYBfnqjXpkli0LJAX-qio7fEL8kz/view?usp=drivesdk)

**Seeking an equity cofounder CTO to own the React Native app. Repo:** [**https://github.com/romer288/Tranquiloo-App.git**](https://github.com/romer288/Tranquiloo-App.git)**.**

DM me with links to shipped RN apps (voice/audio), a brief note on EN/ES handling + fallback design, and your availability.


**What is Tranquiloo?**

* **24/7 companion, therapist-friendly cadence:** patients check in and get evidence-based support; therapists get **structured insight, not notifications**.
* **The quiet layer between visits:** patients capture what matters and get help in the moment; therapists receive **clear, at-a-glance progress**—no pager vibes.
* **Always on for patients, never “always on” for therapists:** quick logs and coping help for clients; curated summaries when you choose to review.
* **Tranquiloo keeps care moving between sessions**—simple check-ins and smart coping for patients; **concise, right-time insights** for therapists (boundaries respected).
* **24/7 support for patients, therapist-paced for you:** lightning-fast check-ins + coping tools for clients; focused, digestible snapshots for clinicians—**no off-hours interruptions**.
* **A always-there companion for the moments between sessions:** patients log feelings and get instant guidance; therapists see clean weekly insights—**not late-night messages**.
* **Tranquiloo is your 24/7 calm in-between sessions**—patients get quick check-ins and real coping support; therapists get clear, digestible insights **on their schedule** (zero after-hours pings).



https://redd.it/1nn5t0s
@r_devops

AWS Cloud Associate (Solutions Architect Associate, Developer Associate, SysOps, Data Engineer Associate, Machine Learning Associate) Vouchers Available

Hi all,

I have AWS Associate vouchers available with me. If any one requires, dm me

AWS Certified Solutions Architect - Associate
AWS Certified Developer - Associate
AWS Certified SysOps Administrator - Associate
AWS Certified Data Engineer - Associate
AWS Certified Machine Learning Engineer - Associate

https://redd.it/1nnd3be
@r_devops

How do you juggle multiple API versions in testing?

I’m running into headaches when dealing with multiple API versions across environments (staging vs production vs legacy). Some tools now let you import/export data by version and even configure different security schemes.

Do most teams here handle versioning in their gateway setup, or directly inside their testing/debugging tool?

https://redd.it/1nnfhfx
@r_devops

How to handle this dedicated vm scenario ?

Pipeline runs and fails because it doesn't have the required tools installed in the agent

All agents are ephemeral - fire and forget

So I need a statefull dedicated agent which has these required tools installed in it

Required tools = Unity software

Is it good idea to get a dedicated vm and have these tools installed so that I can use that ?

Want to hear from experts if there's something I got be careful about

https://redd.it/1nnhr8c
@r_devops

Docker projects for beginners

#

I have recently been hired in a tech company as an intern and I have spent the past half month reading tutorials about docker. In your opinion what are some good projects in order to learn those technologies? I have done some exercises in KodeKloud but the fact that the answer is implied in the text and not always hidden behind a button makes me think that I don't actually solve the problem myself.

https://redd.it/1nnhm5o
@r_devops

How do you integrate compliance checks into your CI/CD pipeline?

Trying to shift compliance left. We want to automate evidence gathering for certain controls (e.g., ensuring a cloud config is compliant at deploy time). Does anyone hook their GRC or compliance tool into their pipeline? What tools are even API-friendly enough for this

https://redd.it/1nngzgl
@r_devops

Can I become a DevOps Engineer if my background is only IT Support (hardware & OS installation)?

Hey everyone,
I’m currently working in IT support, mainly handling hardware and OS installation/troubleshooting. I don’t have much experience in coding or advanced system administration yet, but I really want to transition into DevOps engineering.
Is it possible for someone like me to make this career shift?
If yes:
What skills should I start learning first?
Which tools/technologies are must-know for DevOps beginners?
Are there free/affordable resources or roadmaps you recommend?
How much time (roughly) would it take to become job-ready in DevOps?
I’m motivated and willing to put in consistent effort. Just need some guidance on the right path so I don’t waste time.
Thanks in advance! 🙏

https://redd.it/1nnj904
@r_devops

DevOps folks in India: Do you really have to sacrifice sleep and work life balance for career growth?

I need some real talk from people already in DevOps. I currently work as a server & network analyst with 3 years of experience, but I’m looking to transition into DevOps.

Here’s my worry: in my current company, rotational shifts and night shifts are draining me.

When I look at DevOps openings, I often notice irregular or rotational shift requirements and I don’t want to jump from one fire into another.

So I need your help:

1) How common are rotational/night shifts in DevOps roles in India?

2) Are they unavoidable, or can I aim for companies/teams where DevOps mostly works general shift?

3) For those of you already in shifts, how do you manage it and what’s your plan to eventually get out?

Any advice, personal stories, or even harsh truths are welcome 🙏


https://redd.it/1nnkpfo
@r_devops

Proxmox-GitOps: Extensible IaC Container Automation for Proxmox

I want to share the container automation project Proxmox-GitOps — an extensible, self-bootstrapping GitOps environment for Proxmox.

It is now aligned with current Proxmox 9.0 and Debian Trixie - which is used for containers base configuration per default. Therefore I’d like to introduce it for anyone interested in a Homelab-as-Code starting point 🙂

GitHub: https://github.com/stevius10/Proxmox-GitOps

It implements a self-sufficient, extensible CI/CD environment for provisioning, configuring, and orchestrating Linux Containers (LXC) within Proxmox VE. Leveraging an Infrastructure-as-Code (IaC) approach, it manages the entire container lifecycle—bootstrapping, deployment, configuration, and validation—through version-controlled automation.

- One-command bootstrap: deploy to Docker, Docker deploy to Proxmox

- Ansible, Chef (Cinc), Ruby

- Consistent container base configuration: default app/config users, automated key management, tooling — deterministic, idempotent setup

- Application-logic container repositories: app logic lives in each container repo; shared libraries, pipelines and integration come by convention

- Monorepository with recursively referenced submodules: runtime-modularized, suitable for VCS mirrors, automatically extended by libs

Pipeline concept:

- GitOps environment runs identically in a container; pushing the codebase (monorepo + container libs as submodules) into CI/CD

- This triggers the pipeline from within itself after accepting pull requests: each container applies the same processed pipelines, enforces desired state, and updates references

- Provisioning uses Ansible via the Proxmox API; configuration inside containers is handled by Chef/Cinc cookbooks

- Shared configuration automatically propagates

- Containers integrate seamlessly by following the same predefined pipelines and conventions — at container level and inside the monorepository

- The control plane is built on the same base it uses for the containers, so verifying its own foundation implies a verified container base — a reproducible and adaptable starting point for container automation

It’s still under development, so there may be rough edges — feedback, experiences, or just a thought are more than welcome!

https://redd.it/1nnl9l5
@r_devops

Advice desired... A million unmerged branches!

Okay, not a million. But a lot. In short, the situation is that I've been asked to take a look at the pipeline for our repos and streamline our processes and procedures, as well as put boundaries in place.

It seems that many, many people have not been merging their branches, and a lot of that code is in use right now. Can anyone offer good advice on how to handle reconciling all these branches and some good boundaries and processes to prevent that in the future?

I'd really appreciate any insight anyone has that's been through this before!

https://redd.it/1nnooay
@r_devops

From coding guidelines in docs to automated enforcement: Spotless + Checkstyle as a step toward CI/CD

When I joined a new company, I inherited a large Spring Boot monolith with 15 developers. Coding guidelines existed but only in docs.
Reviews were filled with nitpicks, formatting wars, and “your IDE vs my IDE” debates.

I was tasked to first enforce coding guidelines before moving on to CI/CD. I ended up using:

* **Spotless** for formatting (auto-applied at compile)
* **Checkstyle** for rules (line length, Javadoc, imports, etc.)
* **Optional pre-commit hooks** for faster feedback across Mac & Windows

This article is my write-up of that journey sharing configs, lessons, and common gotchas for mixed-OS teams.

Link -> [https://medium.com/stackademic/how-i-enforced-coding-guidelines-on-a-15-dev-spring-boot-monolith-using-spotless-checkstyle-and-d8ca49caca2c?sk=7eefeaf915171e931dbe2ed25363526b](https://medium.com/stackademic/how-i-enforced-coding-guidelines-on-a-15-dev-spring-boot-monolith-using-spotless-checkstyle-and-d8ca49caca2c?sk=7eefeaf915171e931dbe2ed25363526b)

Would love feedback on how do you enforce guidelines in your teams?

https://redd.it/1nnqtq3
@r_devops

Secure Server Access with Teleport

I just published a guide on how to set up Teleport using Docker on EC2 to provide secure server access across Linux, Windows, Kubernetes, and cloud resources.

I made this because I was tired of dealing with shared SSH keys, forgotten credentials, and messy audit trails. If you’re managing multiple servers, clusters or DBs, this might save you painful hours (and headaches).

Read it here: https://medium.com/@prateekjain.dev/secure-server-access-with-teleport-cf9e55bfb977?sk=aca19937704b4fafcfffd952caa1fc01

https://redd.it/1nno2o8
@r_devops

Octofer: a Rust framework for building GitHub Apss/Bots with ease!

Hi all,

In the last few months I’ve been working on Octofer, a framework for building GitHub Apps in Rust.

It’s inspired by Probot and uses octocrab under the hood.

Right now, it supports common events (issues, PRs, comments, etc.), typed payloads, and simple config via env vars. It’s still under active development, so feedback and contributions are very welcome!

It makes building bots/apps really easy, allowing you to introduce features and automation in little time.

Would love to hear what you think and what features you’d like to see!

P.S. its a simple project but I really enjoyed the process of building it!

https://github.com/AbelHristodor/octofer

https://redd.it/1nntlof
@r_devops

How do startups (and big companies) handle dependency/security updates?

Hey folks,

I’m sort of new to full stack development and running into some confusion with handling dependencies at my SaaS startup. Right now I’ve got Dependabot set up, and I usually merge updates every couple of weeks. But I’m not sure if this is really best practice.

Couple of questions I’d love advice on:

• How do startups typically manage dependency updates and security risks? Do you just patch as they come in, or batch them on a schedule?
• How do larger enterprises do this at scale? I imagine they have dedicated teams or processes, but I’d love to understand what’s realistic as a smaller company.
• What do you do when a dependency has a security vulnerability but updating it breaks other packages that rely on the older version? Do you pin it and accept the risk, fork it, patch it, or something else?

I feel like I’m either over-updating (lots of noise and breakage) or under-updating (leaving security holes open). Curious to hear how others approach this balance.

Thanks!

https://redd.it/1nnt2os
@r_devops

Azure CDN (Classic) deprecation

Had anyone else had just the worst experience with the CDN (Classic) migration?

To combat this migration, I had to update our ARM templates to deploy three different use cases tied to routing. First, a migrated custom domain, second a new CDN Custom domain and third, a CDN just using endpoints. I successfully did this and tested 20 different test cases before 08/15. I was blocked from Microsoft from using the built-in migration tool so we had to migrate after the cut off of new custom domain and CDN deployments.

Now that I've migrated our development environments, im facing a plethora of issues, inability to redeployment a custom domain, the profile itself (because it already exists or is in a region as opposed to global), and finally configuring routes.

The documentation seems so incomplete and support engineers don't seem capable of assisting with issues.

I'm using ARM templates because thats what works, but on the side, rebuilding everything with Terraform.

This whole thing has been a PITA and I've finally been able to get buy in from management to accept downtime so we can redeployment the profiles with new custom domains. It's been such a struggle. I cant way to be done with this.

Side Note: I keep receiving recruiter emails, specifically to work in the Azure Front Door department within the Networking team. How bad did they plan this?

https://redd.it/1nnwz80
@r_devops

Solo project making my head swim. What’s everybody working on?

I’d say I’m well in the weeds at this point. Got a backend VM running a Linux ISO and docker, frontend Electron app and client (browser) that all works immaculately in dev. My fun started when I tried to hoist it all centrally using Cloudflare and proxy the VM to the internet with cloudflared. Packer kept exploding so I’m just using vagrant to spare myself that headache for now.

Recently implemented OpenBao to try to get a CIDC and KMS going for a central auth. On top of CI/CD, of course. OpenBao persists locally on the VM and checks centrally, in theory, but keeps exploding at the moment. Separate repo made to manage those secrets. Now I’m working on a separate repo to manage all of this mess to just try and keep myself sane, while also managing the cert.pem and log distribution and health/telemetry.

I’ll figure it out but the whole “thinking” thing is giving me a mental blowout. What’s everyone else working on?

https://redd.it/1nnxn3n
@r_devops

🚀 Built a Multi-Container Todo App with Docker, Terraform, Ansible & GitHub Actions

Hey folks, I just finished a project from [roadmap.sh](https://roadmap.sh/projects/multi-container-service),

🐳 **Stack & Tools**

* Node.js + Express API
* MongoDB (Mongoose ODM)
* Docker & Docker Compose
* Terraform (provisioned VM on Google Cloud)
* Ansible (server setup + deployment)
* GitHub Actions (CI/CD pipeline)

📌 **What it does**
A simple unauthenticated **Todo API** with CRUD:

* `GET /todos` → list all
* `POST /todos` → create
* `GET /todos/:id` → read one
* `PUT /todos/:id` → update
* `DELETE /todos/:id` → delete

Todos are stored in MongoDB with persistent volumes.

🏗 **How I built it**

1. Started local with **Docker Compose** (API + MongoDB containers).
2. Used **Terraform** to spin up a VM on Google Cloud.
3. Automated setup with **Ansible** (Docker, Docker Compose, running containers).
4. Setup **CI/CD with GitHub Actions** → on push, build & push Docker image, redeploy via Ansible.
5. App accessible through the **external IP** of the VM in the browser.

✅ **Key takeaways**

* Learned how to connect multi-container apps with Docker Compose.
* Got comfortable with Terraform for infra provisioning.
* Automated repetitive tasks with Ansible.
* Built a working CI/CD pipeline from GitHub to cloud.

💡 **Next step / Bonus**
Planning to add **Nginx reverse proxy** \+ a custom domain instead of raw IP.

repo :https://github.com/yanou16/Multi-Container-Application


https://redd.it/1nnvgkp
@r_devops